Specs

delete(term()) :: {:error, term()} | :ok

Deletes a given spider from the SimpleStorage

Specs

get(term()) :: {:error, term()} | {:ok, String.t()}

Get spider YML from the storage

Arguments:

• spider_name: The name of the spider configuration to store. This can be any Erlang term.

Returns:

• {:ok, spider_yml} if the storage operation was successful.
• {:error, reason} if the storage operation failed for some reason. reason can be any term.

Example usage:

Crawly.Models.YMLSpider.get(Books)

Specs

list() :: [term()]

List all spiders in SimpleStorage

Specs

load() :: :ok

Iterates through a list of spider configurations, loads them.

Specs

load(binary()) :: {term(), Code.binding()}

Loads a YAML binary as Spider module

Args:

yml_binary: A binary representation of a YAML spider.

Raises an error if there is a problem parsing the YAML document or evaluating the Elixir code template. Example: iex> yml_binary = "name: my_spider" iex> load(yml_binary) {:ok, %{name: "my_spider"}, []}

Specs

new(spider_name, yml_binary) :: :ok | {:error, term()}
when spider_name: term(), yml_binary: String.t()

Stores the YAML binary data for a spider configuration in a persistent storage backend.

Arguments:

• spider_name: The name of the spider configuration to store. This can be any Erlang term.
• yml_binary: A binary string containing the YAML data for the spider configuration.

Returns:

• :ok if the storage operation was successful.
• {:error, reason} if the storage operation failed for some reason. reason can be any term.

Example usage: Crawly.Models.YMLSpider.new(Books, "YML spider code")