View Source Image.Classification (image v0.33.0)
Implements image classification functions using Axon machine learning models managed by Bumblebee.
Image classification refers to the task of extracting information from an image. In this module, the information extracted is one or more labels desribing the image. Typically something like "sports car" or "Blenheim spaniel". The labels returns depend on the machine learning model used.
configuration
Configuration
The machine learning model to be used is configurable.
The :model
and :featurizer
may be any model supported by Bumblebee. The
:name
is the name given to the classification service process.
The default configuration is:
# runtime.exs
config :image, :classifier,
model: {:hf, "microsoft/resnet-50"},
featurizer: {:hf, "microsoft/resnet-50"},
name: Image.Classification.Server,
autostart: true
autostart
Autostart
If autostart: true
is configured (the default) then a process
is started under a supervisor to execute the classification
requests. If running the process under an application
supervision tree is desired, set autostart: false
. In that
case the function classifer/1
can be
used to return a Supervisor.child_spec/0
.
adding-a-classification-server-to-an-application-supervision-tree
Adding a classification server to an application supervision tree
To add image classification to an application supervision tree,
use Image.Classification.classifier/1
to return a child spec:
For example:
# Application.ex
def start(_type, _args) do
children = [
# default classifier configuration
Image.Classification.classifier()
# custom classifier configuration
Image.Classification.classifier(model: {:hf, "google/vit-base-patch16-224"},
featurizer: {:hf, "google/vit-base-patch16-224"})
]
Supervisor.start_link(
children,
strategy: :one_for_one
)
end
note
Note
This module is only available if the optional dependency
Bumblebee is configured in
mix.exs
.
Link to this section Summary
Functions
Returns a child spec suitable for starting an image classification process as part of a supervision tree.
Classify an image using a machine learning model.
Classify an image using a machine learning model and return the labels that meet a minimum score.
Link to this section Functions
classifier(classifier \\ Application.get_env(:image, :classifier, []))
View SourceReturns a child spec suitable for starting an image classification process as part of a supervision tree.
arguments
Arguments
configuration
is a keyword list.The default isApplication.get_env(:image, :classifier, [])
.
configuration-keys
Configuration keys
:model
is any supported machine learning model for image classification supported by Bumblebee.:featurizer
is any supported machine learning model for image featurization supported by Bumblebee.:name
is the name given to the classification process when it is started.
default-configuration
Default configuration
The default configuration is:
[
model: {:hf, "microsoft/resnet-50"},
featurizer: {:hf, "microsoft/resnet-50"},
name: Image.Classification.Server
]
@spec classify(image :: Vix.Vips.Image.t(), Keyword.t()) :: %{predictions: [%{label: String.t(), score: float()}]} | {:error, Image.error_message()}
Classify an image using a machine learning model.
arguments
Arguments
image
is anyVix.Vips.Image.t/0
.options
is a keyword list of options
options
Options
:backend
is any validNx
backend. The default isNx.default_backend/0
.:server
is the name of the process performing the classification service. The default isImage.Classification.Server
.
returns
Returns
- A map containing the predictions of the image classification.
example
Example
iex> puppy = Image.open!("./test/support/images/puppy.webp")
iex> %{predictions: [%{label: "Blenheim spaniel", score: _} | _rest]} =
...> Image.Classification.classify(puppy)
@spec labels(image :: Vix.Vips.Image.t(), options :: Keyword.t()) :: [String.t()] | {:error, Image.error_message()}
Classify an image using a machine learning model and return the labels that meet a minimum score.
arguments
Arguments
image
is anyVix.Vips.Image.t/0
.options
is a keyword list of options.
options
Options
:backend
is any validNx
backend. The default isNx.default_backend/0
.:min_score
is the minimum score, a float between0
and1
, which a label must match in order to be returned.
returns
Returns
A list of labels. The list may be empty if there are no predictions that exceed the
:min_score
.{:error, reason}
example
Example
iex> {:ok, image} = Image.open ("./test/support/images/lamborghini-forsennato-concept.jpg")
iex> Image.Classification.labels(image)
["sports car", "sport car"]