Specs

check_solr_schema(binary() | atom()) :: {:ok, map()} | {:error, Hui.Error.t()}

Check a Solr core/collection to ensure it has the required Lastfm data fields.

The check currently inspects Solr schema for a list of Lastfm fields and returns error if one or more of the fields are missing. See LastfmArchive.Transform.transform/3 for the list of fields.

Example

  LastfmArchive.Load.check_solr_schema("http://solr_url...")
  LastfmArchive.Load.check_solr_schema(:lastfm_archive) # ping a configured endpoint

See ping_solr/1 for more details on URL configuration.

Specs

load_solr(Hui.URL.t(), binary(), binary()) ::
  {:ok, Hui.Http.t()} | {:error, :enoent}

Load a TSV file data from the archive into Solr for a Lastfm user.

The function reads and converts scrobbles in a TSV file from the file archive into a list of maps. The maps are sent to Solr for ingestion. Use Hui.URL.t/0 struct to specify the Solr endpoint.

Example

  # define a Solr endpoint with %Hui.URL{} struct
  headers = [{"Content-type", "application/json"}]
  url = %Hui.URL{url: "http://localhost:8983/solr/lastfm_archive", handler: "update", headers: headers}

  # ingest data scrobbled in 2018
  LastfmArchive.Load.load_solr(url, "a_lastfm_user", "tsv/2018.tsv.gz")

TSV files must be pre-created by transforming raw JSON Lastfm data - see LastfmArchive.transform_archive/2.

Specs

ping_solr(binary() | atom()) :: {:ok, map()} | {:error, Hui.Error.t()}

Ping a Solr core/collection endpoint to check if it is running.

The endpoint can either be a URL string or an atom referring to an endpoint in configuration. The library uses Hui to interact with Solr, an endpoint can be specified as below:

Example

  LastfmArchive.Load.ping_solr("http://solr_url...")
  LastfmArchive.Load.ping_solr(:lastfm_archive) # check a configured endpoint

:lastfm_archive refers to the following Solr update endpoint in configuration:

  config :hui, :lastfm_archive,
    url: "http://solr_url..",
    handler: "update",
    headers: [{"Content-type", "application/json"}]

See Hui.URL module for more details.

Specs

read(binary(), binary()) :: {:ok, [binary()]} | {:error, :file.posix()}

Read and parse a TSV file from the archive for a Lastfm user.

TSV files generated by transforming raw JSON Lastfm data - see LastfmArchive.transform_archive/2. The file is parsed into a list of scrobbles.

Example

  LastfmArchive.Load.read "a_lastfm_user", "tsv/2007.tsv.gz"