Ecto (Ecto v3.5.5) View Source

Ecto is split into 4 main components:

  • Ecto.Repo - repositories are wrappers around the data store. Via the repository, we can create, update, destroy and query existing entries. A repository needs an adapter and credentials to communicate to the database

  • Ecto.Schema - schemas are used to map any data source into an Elixir struct. We will often use them to map tables into Elixir data but that's one of their use cases and not a requirement for using Ecto

  • Ecto.Changeset - changesets provide a way for developers to filter and cast external parameters, as well as a mechanism to track and validate changes before they are applied to your data

  • Ecto.Query - written in Elixir syntax, queries are used to retrieve information from a given repository. Queries in Ecto are secure, avoiding common problems like SQL Injection, while still being composable, allowing developers to build queries piece by piece instead of all at once

Besides the four components above, most developers use Ecto to interact with SQL databases, such as Postgres and MySQL via the ecto_sql project. ecto_sql provides many conveniences for working with SQL databases as well as the ability to version how your database changes through time via database migrations.

If you want to quickly check a sample application using Ecto, please check the getting started guide and the accompanying sample application. Ecto's README also links to other resources.

In the following sections, we will provide an overview of those components and how they interact with each other. Feel free to access their respective module documentation for more specific examples, options and configuration.

Repositories

Ecto.Repo is a wrapper around the database. We can define a repository as follows:

defmodule Repo do
  use Ecto.Repo,
    otp_app: :my_app,
    adapter: Ecto.Adapters.Postgres
end

Where the configuration for the Repo must be in your application environment, usually defined in your config/config.exs:

config :my_app, Repo,
  database: "ecto_simple",
  username: "postgres",
  password: "postgres",
  hostname: "localhost",
  # OR use a URL to connect instead
  url: "postgres://postgres:postgres@localhost/ecto_simple"

Each repository in Ecto defines a start_link/0 function that needs to be invoked before using the repository. In general, this function is not called directly, but used as part of your application supervision tree.

If your application was generated with a supervisor (by passing --sup to mix new) you will have a lib/my_app/application.ex file containing the application start callback that defines and starts your supervisor. You just need to edit the start/2 function to start the repo as a supervisor on your application's supervisor:

def start(_type, _args) do
  children = [
    {MyApp.Repo, []}
  ]

  opts = [strategy: :one_for_one, name: MyApp.Supervisor]
  Supervisor.start_link(children, opts)
end

Schema

Schemas allow developers to define the shape of their data. Let's see an example:

defmodule Weather do
  use Ecto.Schema

  # weather is the DB table
  schema "weather" do
    field :city,    :string
    field :temp_lo, :integer
    field :temp_hi, :integer
    field :prcp,    :float, default: 0.0
  end
end

By defining a schema, Ecto automatically defines a struct with the schema fields:

iex> weather = %Weather{temp_lo: 30}
iex> weather.temp_lo
30

The schema also allows us to interact with a repository:

iex> weather = %Weather{temp_lo: 0, temp_hi: 23}
iex> Repo.insert!(weather)
%Weather{...}

After persisting weather to the database, it will return a new copy of %Weather{} with the primary key (the id) set. We can use this value to read a struct back from the repository:

# Get the struct back
iex> weather = Repo.get Weather, 1
%Weather{id: 1, ...}

# Delete it
iex> Repo.delete!(weather)
%Weather{...}

NOTE: by using Ecto.Schema, an :id field with type :id (:id means :integer) is generated by default, which is the primary key of the Schema. If you want to use a different primary key, you can declare custom @primary_key before the schema/2 call. Consult the Ecto.Schema documentation for more information.

Notice how the storage (repository) and the data are decoupled. This provides two main benefits:

  • By having structs as data, we guarantee they are light-weight, serializable structures. In many languages, the data is often represented by large, complex objects, with entwined state transactions, which makes serialization, maintenance and understanding hard;

  • You do not need to define schemas in order to interact with repositories, operations like all, insert_all and so on allow developers to directly access and modify the data, keeping the database at your fingertips when necessary;

Changesets

Although in the example above we have directly inserted and deleted the struct in the repository, operations on top of schemas are done through changesets so Ecto can efficiently track changes.

Changesets allow developers to filter, cast, and validate changes before we apply them to the data. Imagine the given schema:

defmodule User do
  use Ecto.Schema

  import Ecto.Changeset

  schema "users" do
    field :name
    field :email
    field :age, :integer
  end

  def changeset(user, params \\ %{}) do
    user
    |> cast(params, [:name, :email, :age])
    |> validate_required([:name, :email])
    |> validate_format(:email, ~r/@/)
    |> validate_inclusion(:age, 18..100)
  end
end

The changeset/2 function first invokes Ecto.Changeset.cast/4 with the struct, the parameters and a list of allowed fields; this returns a changeset. The parameters is a map with binary keys and values that will be cast based on the type defined on the schema.

Any parameter that was not explicitly listed in the fields list will be ignored.

After casting, the changeset is given to many Ecto.Changeset.validate_* functions that validate only the changed fields. In other words: if a field was not given as a parameter, it won't be validated at all. For example, if the params map contain only the "name" and "email" keys, the "age" validation won't run.

Once a changeset is built, it can be given to functions like insert and update in the repository that will return an :ok or :error tuple:

case Repo.update(changeset) do
  {:ok, user} ->
    # user updated
  {:error, changeset} ->
    # an error occurred
end

The benefit of having explicit changesets is that we can easily provide different changesets for different use cases. For example, one could easily provide specific changesets for registering and updating users:

def registration_changeset(user, params) do
  # Changeset on create
end

def update_changeset(user, params) do
  # Changeset on update
end

Changesets are also capable of transforming database constraints, like unique indexes and foreign key checks, into errors. Allowing developers to keep their database consistent while still providing proper feedback to end users. Check Ecto.Changeset.unique_constraint/3 for some examples as well as the other _constraint functions.

Query

Last but not least, Ecto allows you to write queries in Elixir and send them to the repository, which translates them to the underlying database. Let's see an example:

import Ecto.Query, only: [from: 2]

query = from u in User,
          where: u.age > 18 or is_nil(u.email),
          select: u

# Returns %User{} structs matching the query
Repo.all(query)

In the example above we relied on our schema but queries can also be made directly against a table by giving the table name as a string. In such cases, the data to be fetched must be explicitly outlined:

query = from u in "users",
          where: u.age > 18 or is_nil(u.email),
          select: %{name: u.name, age: u.age}

# Returns maps as defined in select
Repo.all(query)

Queries are defined and extended with the from macro. The supported keywords are:

  • :distinct
  • :where
  • :order_by
  • :offset
  • :limit
  • :lock
  • :group_by
  • :having
  • :join
  • :select
  • :preload

Examples and detailed documentation for each of those are available in the Ecto.Query module. Functions supported in queries are listed in Ecto.Query.API.

When writing a query, you are inside Ecto's query syntax. In order to access params values or invoke Elixir functions, you need to use the ^ operator, which is overloaded by Ecto:

def min_age(min) do
  from u in User, where: u.age > ^min
end

Besides Repo.all/1 which returns all entries, repositories also provide Repo.one/1 which returns one entry or nil, Repo.one!/1 which returns one entry or raises, Repo.get/2 which fetches entries for a particular ID and more.

Finally, if you need an escape hatch, Ecto provides fragments (see Ecto.Query.API.fragment/1) to inject SQL (and non-SQL) fragments into queries. Also, most adapters provide direct APIs for queries, like Ecto.Adapters.SQL.query/4, allowing developers to completely bypass Ecto queries.

Other topics

Associations

Ecto supports defining associations on schemas:

defmodule Post do
  use Ecto.Schema

  schema "posts" do
    has_many :comments, Comment
  end
end

defmodule Comment do
  use Ecto.Schema

  schema "comments" do
    field :title, :string
    belongs_to :post, Post
  end
end

When an association is defined, Ecto also defines a field in the schema with the association name. By default, associations are not loaded into this field:

iex> post = Repo.get(Post, 42)
iex> post.comments
#Ecto.Association.NotLoaded<...>

However, developers can use the preload functionality in queries to automatically pre-populate the field:

Repo.all from p in Post, preload: [:comments]

Preloading can also be done with a pre-defined join value:

Repo.all from p in Post,
          join: c in assoc(p, :comments),
          where: c.votes > p.votes,
          preload: [comments: c]

Finally, for the simple cases, preloading can also be done after a collection was fetched:

posts = Repo.all(Post) |> Repo.preload(:comments)

The Ecto module also provides conveniences for working with associations. For example, Ecto.assoc/2 returns a query with all associated data to a given struct:

import Ecto

# Get all comments for the given post
Repo.all assoc(post, :comments)

# Or build a query on top of the associated comments
query = from c in assoc(post, :comments), where: not is_nil(c.title)
Repo.all(query)

Another function in Ecto is build_assoc/3, which allows someone to build an associated struct with the proper fields:

Repo.transaction fn ->
  post = Repo.insert!(%Post{title: "Hello", body: "world"})

  # Build a comment from post
  comment = Ecto.build_assoc(post, :comments, body: "Excellent!")

  Repo.insert!(comment)
end

In the example above, Ecto.build_assoc/3 is equivalent to:

%Comment{post_id: post.id, body: "Excellent!"}

You can find more information about defining associations and each respective association module in Ecto.Schema docs.

NOTE: Ecto does not lazy load associations. While lazily loading associations may sound convenient at first, in the long run it becomes a source of confusion and performance issues.

Embeds

Ecto also supports embeds. While associations keep parent and child entries in different tables, embeds stores the child along side the parent.

Databases like MongoDB have native support for embeds. Databases like PostgreSQL uses a mixture of JSONB (embeds_one/3) and ARRAY columns to provide this functionality.

Check Ecto.Schema.embeds_one/3 and Ecto.Schema.embeds_many/3 for more information.

Mix tasks and generators

Ecto provides many tasks to help your workflow as well as code generators. You can find all available tasks by typing mix help inside a project with Ecto listed as a dependency.

Ecto generators will automatically open the generated files if you have ECTO_EDITOR set in your environment variable.

Repo resolution

Ecto requires developers to specify the key :ecto_repos in their application configuration before using tasks like ecto.create and ecto.migrate. For example:

config :my_app, :ecto_repos, [MyApp.Repo]

config :my_app, MyApp.Repo,
  database: "ecto_simple",
  username: "postgres",
  password: "postgres",
  hostname: "localhost"

Link to this section Summary

Functions

Builds a query for the association in the given struct or structs.

Checks if an association is loaded.

Builds a struct from the given assoc in struct.

Dumps the given struct defined by an embedded schema.

Loads previously dumped data in the given format into a schema.

Gets the metadata from the given struct.

Returns the schema primary keys as a keyword list.

Returns the schema primary keys as a keyword list.

Returns a new struct with updated metadata.

Link to this section Functions

Link to this function

assoc(struct_or_structs, assocs)

View Source

Builds a query for the association in the given struct or structs.

Examples

In the example below, we get all comments associated to the given post:

post = Repo.get Post, 1
Repo.all Ecto.assoc(post, :comments)

assoc/2 can also receive a list of posts, as long as the posts are not empty:

posts = Repo.all from p in Post, where: is_nil(p.published_at)
Repo.all Ecto.assoc(posts, :comments)

This function can also be used to dynamically load through associations by giving it a list. For example, to get all authors for all comments for the given posts, do:

posts = Repo.all from p in Post, where: is_nil(p.published_at)
Repo.all Ecto.assoc(posts, [:comments, :author])

Checks if an association is loaded.

Examples

iex> post = Repo.get(Post, 1)
iex> Ecto.assoc_loaded?(post.comments)
false
iex> post = post |> Repo.preload(:comments)
iex> Ecto.assoc_loaded?(post.comments)
true
Link to this function

build_assoc(struct, assoc, attributes \\ %{})

View Source

Builds a struct from the given assoc in struct.

Examples

If the relationship is a has_one or has_many and the primary key is set in the parent struct, the key will automatically be set in the built association:

iex> post = Repo.get(Post, 13)
%Post{id: 13}
iex> build_assoc(post, :comments)
%Comment{id: nil, post_id: 13}

Note though it doesn't happen with belongs_to cases, as the key is often the primary key and such is usually generated dynamically:

iex> comment = Repo.get(Comment, 13)
%Comment{id: 13, post_id: 25}
iex> build_assoc(comment, :post)
%Post{id: nil}

You can also pass the attributes, which can be a map or a keyword list, to set the struct's fields except the association key.

iex> build_assoc(post, :comments, text: "cool")
%Comment{id: nil, post_id: 13, text: "cool"}

iex> build_assoc(post, :comments, %{text: "cool"})
%Comment{id: nil, post_id: 13, text: "cool"}

iex> build_assoc(post, :comments, post_id: 1)
%Comment{id: nil, post_id: 13}

The given attributes are expected to be structured data. If you want to build an association with external data, such as a request parameters, you can use Ecto.Changeset.cast/3 after build_assoc/3:

parent
|> Ecto.build_assoc(:child)
|> Ecto.Changeset.cast(params, [:field1, :field2])
Link to this function

embedded_dump(data, format)

View Source

Specs

embedded_dump(Ecto.Schema.t(), format :: atom()) :: map()

Dumps the given struct defined by an embedded schema.

This converts the given embedded schema to a map to be serialized with the given format. For example:

iex> Ecto.embedded_dump(%Post{}, :json)
%{title: "hello"}
Link to this function

embedded_load(schema_or_types, data, format)

View Source

Specs

embedded_load(
  module_or_map :: module() | map(),
  data :: map(),
  format :: atom()
) :: Ecto.Schema.t() | map()

Loads previously dumped data in the given format into a schema.

The first argument can be a an embedded schema module, or a map (of types) and determines the return value: a struct or a map, respectively.

The second argument data specifies fields and values that are to be loaded. It can be a map, a keyword list, or a {fields, values} tuple. Fields can be atoms or strings.

The third argument format is the format the data has been dumped as. For example, databases may dump embedded to :json, this function allows such dumped data to be put back into the schemas.

Fields that are not present in the schema (or types map) are ignored. If any of the values has invalid type, an error is raised.

Note that if you want to load data into a non-embedded schema that was directly persisted into a given repository, then use Ecto.Repo.load/2.

Examples

iex> result = Ecto.Adapters.SQL.query!(MyRepo, "SELECT users.settings FROM users", [])
iex> Enum.map(result.rows, fn [settings] -> Ecto.embedded_load(Setting, Jason.decode!(settings), :json) end)
[%Setting{...}, ...]

Gets the metadata from the given struct.

Specs

primary_key(Ecto.Schema.t()) :: Keyword.t()

Returns the schema primary keys as a keyword list.

Specs

primary_key!(Ecto.Schema.t()) :: Keyword.t()

Returns the schema primary keys as a keyword list.

Raises Ecto.NoPrimaryKeyFieldError if the schema has no primary key field.

Specs

put_meta(Ecto.Schema.schema(), meta) :: Ecto.Schema.schema()
when meta: [
       source: Ecto.Schema.source(),
       prefix: Ecto.Schema.prefix(),
       context: Ecto.Schema.Metadata.context(),
       state: Ecto.Schema.Metadata.state()
     ]

Returns a new struct with updated metadata.

It is possible to set:

  • :source - changes the struct query source
  • :prefix - changes the struct query prefix
  • :context - changes the struct meta context
  • :state - changes the struct state

Please refer to the Ecto.Schema.Metadata module for more information.