image
image
v0.33.0

View Source Image.Classification (image v0.33.0)

Implements image classification functions using Axon machine learning models managed by Bumblebee.

Image classification refers to the task of extracting information from an image. In this module, the information extracted is one or more labels desribing the image. Typically something like "sports car" or "Blenheim spaniel". The labels returns depend on the machine learning model used.

configuration

 Configuration

The machine learning model to be used is configurable. The :model and :featurizer may be any model supported by Bumblebee. The :name is the name given to the classification service process.

The default configuration is:

# runtime.exs
config :image, :classifier,
  model: {:hf, "microsoft/resnet-50"},
  featurizer:  {:hf, "microsoft/resnet-50"},
  name: Image.Classification.Server,
  autostart: true

autostart

 Autostart

If autostart: true is configured (the default) then a process is started under a supervisor to execute the classification requests. If running the process under an application supervision tree is desired, set autostart: false. In that case the function classifer/1 can be used to return a Supervisor.child_spec/0.

adding-a-classification-server-to-an-application-supervision-tree

 Adding a classification server to an application supervision tree

To add image classification to an application supervision tree, use Image.Classification.classifier/1 to return a child spec: For example:

# Application.ex
def start(_type, _args) do
  children = [
    # default classifier configuration
    Image.Classification.classifier()

    # custom classifier configuration
    Image.Classification.classifier(model: {:hf, "google/vit-base-patch16-224"},
      featurizer: {:hf, "google/vit-base-patch16-224"})
  ]

  Supervisor.start_link(
    children,
    strategy: :one_for_one
  )
end

note

 Note

This module is only available if the optional dependency Bumblebee is configured in mix.exs.

Link to this section Summary

Functions

classifier(classifier \\ Application.get_env(:image, :classifier, []))

Returns a child spec suitable for starting an image classification process as part of a supervision tree.

classify(image, options \\ [])

Classify an image using a machine learning model.

labels(image, options \\ [])

Classify an image using a machine learning model and return the labels that meet a minimum score.

Link to this section Functions

Link to this function

classifier(classifier \\ Application.get_env(:image, :classifier, []))

View Source 
@spec classifier(configuration :: Keyword.t()) :: {Nx.Serving, Keyword.t()}

Returns a child spec suitable for starting an image classification process as part of a supervision tree.

arguments

 Arguments

  • configuration is a keyword list.The default is Application.get_env(:image, :classifier, []).

configuration-keys

 Configuration keys

  • :model is any supported machine learning model for image classification supported by Bumblebee.

  • :featurizer is any supported machine learning model for image featurization supported by Bumblebee.

  • :name is the name given to the classification process when it is started.

default-configuration

 Default configuration

The default configuration is:

[
  model: {:hf, "microsoft/resnet-50"},
  featurizer: {:hf, "microsoft/resnet-50"},
  name: Image.Classification.Server
]
Link to this function

classify(image, options \\ [])

View Source (since 0.18.0) 
@spec classify(image :: Vix.Vips.Image.t(), Keyword.t()) ::
  %{predictions: [%{label: String.t(), score: float()}]}
  | {:error, Image.error_message()}

Classify an image using a machine learning model.

arguments

 Arguments

options

 Options

  • :backend is any valid Nx backend. The default is Nx.default_backend/0.

  • :server is the name of the process performing the classification service. The default is Image.Classification.Server.

returns

 Returns

  • A map containing the predictions of the image classification.

example

 Example 

iex> puppy = Image.open!("./test/support/images/puppy.webp")
iex> %{predictions: [%{label: "Blenheim spaniel", score: _} | _rest]} =
...>   Image.Classification.classify(puppy)
Link to this function

labels(image, options \\ [])

View Source (since 0.18.0) 
@spec labels(image :: Vix.Vips.Image.t(), options :: Keyword.t()) ::
  [String.t()] | {:error, Image.error_message()}

Classify an image using a machine learning model and return the labels that meet a minimum score.

arguments

 Arguments

options

 Options

  • :backend is any valid Nx backend. The default is Nx.default_backend/0.

  • :min_score is the minimum score, a float between 0 and 1, which a label must match in order to be returned.

returns

 Returns

  • A list of labels. The list may be empty if there are no predictions that exceed the :min_score.

  • {:error, reason}

example

 Example 

iex> {:ok, image} = Image.open ("./test/support/images/lamborghini-forsennato-concept.jpg")
iex> Image.Classification.labels(image)
["sports car", "sport car"]