View Source Scidata.IMDBReviews (Scidata v0.1.11)

Module for downloading the Large Movie Review Dataset.

Link to this section Summary

Functions

Downloads the IMDB reviews training dataset or fetches it locally.

Downloads the IMDB reviews test dataset or fetches it locally.

Link to this section Types

@type test_sentiment() :: :pos | :neg
@type train_sentiment() :: :pos | :neg | :unsup

Link to this section Functions

@spec download([{:example_types, [train_sentiment()]}]) :: %{
  review: [binary(), ...],
  sentiment: [1 | 0 | nil]
}

Downloads the IMDB reviews training dataset or fetches it locally.

example_types specifies which examples in the dataset should be returned according to each example's label: :pos for positive examples, :neg for negative examples, and :unsup for unlabeled examples. If no example_types are provided, :pos and :neg examples are fetched.

options

Options.

  • :base_url - Dataset base URL.

    Defaults to "http://ai.stanford.edu/~amaas/data/sentiment/"

  • :dataset_file - Dataset filename.

    Defaults to "aclImdb_v1.tar.gz"

  • :cache_dir - Cache directory.

    Defaults to System.tmp_dir!()

Link to this function

download_test(opts \\ [])

View Source
@spec download_test([{:example_types, [test_sentiment()]}]) :: %{
  review: [binary(), ...],
  sentiment: [1 | 0]
}

Downloads the IMDB reviews test dataset or fetches it locally.

example_types is the same as in download/1, but :unsup is unavailable because all unlabeled examples are in the training set.

Accepts the same options as download/1.