Ecto v1.1.0 Ecto.Schema

Defines a schema for a model.

A schema is a struct with associated metadata that is persisted to a repository. Every schema model is also a struct, that means that you work with models just like you would work with structs.

Example

defmodule User do
  use Ecto.Schema

  schema "users" do
    field :name, :string
    field :age, :integer, default: 0
    has_many :posts, Post
  end
end

By default, a schema will generate a primary key named id of type :integer and belongs_to associations in the schema will generate foreign keys of type :integer. Those setting can be configured below.

Schema attributes

The schema supports some attributes to be set before hand, configuring the defined schema.

Those attributes are:

  • @primary_key - configures the schema primary key. It expects a tuple with the primary key name, type (:id or :binary_id) and options. Defaults to {:id, :id, autogenerate: true}. When set to false, does not define a primary key in the model;

  • @foreign_key_type - configures the default foreign key type used by belongs_to associations. Defaults to :integer;

  • @timestamps_opts - configures the default timestamps type used by timestamps. Defaults to [type: Ecto.DateTime, usec: false];

  • @derive - the same as @derive available in Kernel.defstruct/1 as the schema defines a struct behind the scenes;

The advantage of configuring the schema via those attributes is that they can be set with a macro to configure application wide defaults.

For example, if your database does not support autoincrementing primary keys and requires something like UUID or a RecordID, you configure and use:binary_id as your primary key type as follows:

# Define a module to be used as base
defmodule MyApp.Schema do
  defmacro __using__(_) do
    quote do
      use Ecto.Schema
      @primary_key {:id, :binary_id, autogenerate: true}
      @foreign_key_type :binary_id
    end
  end
end

# Now use MyApp.Schema to define new models
defmodule MyApp.Comment do
  use MyApp.Schema

  schema "comments" do
    belongs_to :post, MyApp.Post
  end
end

Any models using MyApp.Model will get the :id field with type :binary_id as primary key. We explain what the :binary_id type entails in the next section.

The belongs_to association on MyApp.Comment will also define a :post_id field with :binary_id type that references the :id field of the MyApp.Post model.

Primary keys

Ecto supports two ID types, called :id and :binary_id which are often used as the type for primary keys and associations.

The :id type is used when the primary key is an integer while the :binary_id is used when the primary key is in binary format, which may be Ecto.UUID for databases like PostgreSQL and MySQL, or some specific ObjectID or RecordID often imposed by NoSQL databases.

In both cases, both types have their semantics specified by the underlying adapter/database. For example, if you use the :id type with :autogenerate, it means the database will be responsible for auto-generation the id if it supports it.

Similarly, the :binary_id type may be generated in the adapter for cases like UUID but it may also be handled by the database if required. In any case, both scenarios are handled transparently by Ecto.

Besides :id and :binary_id, which are often used by primary and foreign keys, Ecto provides a huge variety of types to be used by the remaining columns.

Types and casting

When defining the schema, types need to be given. Types are split in two categories, primitive types and custom types.

Primitive types

The primitive types are:

Ecto typeElixir typeLiteral syntax in query
:idinteger1, 2, 3
:binary_idbinary<<int, int, int, ...>>
:integerinteger1, 2, 3
:floatfloat1.0, 2.0, 3.0
:booleanbooleantrue, false
:stringUTF-8 encoded string“hello”
:binarybinary<<int, int, int, ...>>
{:array, inner_type}list[value, value, value, ...]
:decimalDecimal
:mapmap

Note: For the :array type, replace inner_type with one of the valid types, such as :string.

Custom types

Besides providing primitive types, Ecto allows custom types to be implemented by developers, allowing Ecto behaviour to be extended.

A custom type is a module that implements the Ecto.Type behaviour. By default, Ecto provides the following custom types:

Custom typeDatabase typeElixir type
Ecto.DateTime:datetime%Ecto.DateTime{}
Ecto.Date:date%Ecto.Date{}
Ecto.Time:time%Ecto.Time{}
Ecto.UUID:uuid“uuid-string”

Read the Ecto.Type documentation for more information on implementing your own types.

The map type

The map type allows developers to store an Elixir map directly in the database:

# In your migration
create table(:users) do
  add :data, :map
end

# In your model
field :data, :map

# Now in your code
%User{data: %{"foo" => "bar"}} |> Repo.insert!
%User{data: %{"foo" => value}} = Repo.one(User)
value #=> "bar"

Keep in mind that we advise the map keys to be strings or integers instead of atoms. Atoms may be accepted depending on how maps are serialized but the database will always return atom keys as strings due to security reasons.

In order to support maps, different databases may employ different techniques. For example, PostgreSQL will store those values in jsonb fields, allowing you to even query parts of it. MySQL and MSSQL, on the other hand, do not yet provide a JSON type, so the value will be stored in a text field.

For maps to work in such databases, Ecto will need a JSON library. By default Ecto will use Poison which needs to be added your deps in mix.exs:

{:poison, "~> 1.0"}

You can however tell Ecto to use any other library by configuring it:

config :ecto, :json_library, YourLibraryOfChoice

Casting

When directly manipulating the struct, it is the responsibility of the developer to ensure the field values have the proper type. For example, you can create a user struct with an invalid value for age:

iex> user = %User{age: "0"}
iex> user.age
"0"

However, if you attempt to persist the struct above, an error will be raised since Ecto validates the types when sending them to the adapter/database.

Therefore, when working and manipulating external data, it is recommended the usage of Ecto.Changeset’s that are able to filter and properly cast external data:

changeset = Ecto.Changeset.cast(%User{}, %{"age" => "0"}, [:age], [])
user = Repo.insert!(changeset)

In fact, Ecto.Changeset and custom types provide a powerful combination to extend Ecto types and queries.

Finally, models can also have virtual fields by passing the virtual: true option. These fields are not persisted to the database and can optionally not be type checked by declaring type :any.

Reflection

Any schema module will generate the __schema__ function that can be used for runtime introspection of the schema:

  • __schema__(:source) - Returns the source as given to schema/2;
  • __schema__(:primary_key) - Returns a list of primary key fields (empty if there is none);

  • __schema__(:fields) - Returns a list of all non-virtual field names;
  • __schema__(:type, field) - Returns the type of the given non-virtual field;
  • __schema__(:types) - Returns a keyword list of all non-virtual field names and their type;

  • __schema__(:associations) - Returns a list of all association field names;
  • __schema__(:association, assoc) - Returns the association reflection of the given assoc;

  • __schema__(:embeds) - Returns a list of all embedded field names;
  • __schema__(:embed, embed) - Returns the embedding reflection of the given embed;

  • __schema__(:read_after_writes) - Non-virtual fields that must be read back from the database after every write (insert or update);

  • __schema__(:autogenerate_id) - Primary key that is auto generated on insert;

Furthermore, both __struct__ and __changeset__ functions are defined so structs and changeset functionalities are available.

Summary

Macros

Indicates a one-to-one association with another model

Defines an embedded schema

Indicates an embedding of many models

Indicates an embedding of one model

Defines a field on the model schema with given name and type

Indicates a one-to-many association with another model

Indicates a one-to-one association with another model

Defines a schema with a source name and field definitions

Generates :inserted_at and :updated_at timestamp fields

Types

t :: %atom{}

Macros

belongs_to(name, queryable, opts \\ [])

Indicates a one-to-one association with another model.

The current model belongs to zero or one records of the other model. The other model often has a has_one or a has_many field with the reverse association.

You should use belongs_to in the table that contains the foreign key. Imagine a company <-> manager relationship. If the company contains the manager_id in the underlying database table, we say the company belongs to manager.

In fact, when you invoke this macro, a field with the name of foreign key is automatically defined in the schema for you.

Options

  • :foreign_key - Sets the foreign key field name, defaults to the name of the association suffixed by _id. For example, belongs_to :company will define foreign key of :company_id

  • :references - Sets the key on the other model to be used for the association, defaults to: :id

  • :define_field - When false, does not automatically define a :foreign_key field, implying the user is defining the field manually elsewhere

  • :type - Sets the type of automatically defined :foreign_key. Defaults to: :integer and be set per schema via @foreign_key_type

All other options are forwarded to the underlying foreign key definition and therefore accept the same options as field/3.

Examples

defmodule Comment do
  use Ecto.Schema

  schema "comments" do
    belongs_to :post, Post
  end
end

# The post can come preloaded on the comment record
[comment] = Repo.all(from(c in Comment, where: c.id == 42, preload: :post))
comment.post #=> %Post{...}

Polymorphic associations

One common use case for belongs to associations is to handle polymorphism. For example, imagine you have defined a Comment model and you wish to use it for commenting on both tasks and posts.

Some abstractions would force you to define some sort of polymorphic association with two fields in your database:

* commentable_type
* commentable_id

The problem with this approach is that it breaks references in the database. You can’t use foreign keys and it is very inneficient both in terms of query time and storage.

In Ecto, we have two ways to solve this issue. The simplest one is to define multiple fields in the Comment model, one for each association:

* task_id
* post_id

Unless you have dozens of columns, this is simpler for the developer, more DB friendly and more efficient on all aspects.

Alternatively, because Ecto does not tie a model to a given table, we can use separate tables for each association. Let’s start over and define a new Comment model:

defmodule Comment do
  use Ecto.Schema

  schema "abstract table: comments" do
    # This will be used by associations on each "concrete" table
    field :assoc_id, :integer
  end
end

Notice we have changed the table name to “abstract table: comments”. You can choose whatever name you want, the point here is that this particular table will never exist.

Now in your Post and Task models:

defmodule Post do
  use Ecto.Schema

  schema "posts" do
    has_many :comments, {"posts_comments", Comment}, foreign_key: :assoc_id
  end
end

defmodule Task do
  use Ecto.Schema

  schema "tasks" do
    has_many :comments, {"tasks_comments", Comment}, foreign_key: :assoc_id
  end
end

Now each association uses its own specific table, “posts_comments” and “tasks_comments”, which must be created on migrations. The advantage of this approach is that we never store unrelated data together, also ensuring we keep databases references fast and correct.

When using this technique, the only limitation is that you cannot build comments directly. For example, the command below

Repo.insert!(%Comment{})

will attempt to use the abstract table. Instead, one should

Repo.insert!(build_assoc(post, :comments))

where build/2 is defined in Ecto.Model. You can also use assoc/2 in both Ecto.Model and in the query syntax to easily retrieve associated comments to a given post or task:

# Fetch all comments associated to the given task
Repo.all(assoc(task, :comments))

Finally, if for some reason you wish to query one of comments table directly, you can also specify the tuple source in the query syntax:

Repo.all from(c in {"posts_comments", Comment}), ...)
embedded_schema(opts)

Defines an embedded schema.

This function is literally a shortcut for:

@primary_key {:id, :binary_id, autogenerate: true}
  schema "embedded Model" do
embeds_many(name, model, opts \\ [])

Indicates an embedding of many models.

The current model has zero or more records of the other model embedded inside of it, contained in a list. Embedded models have all the things regular models do - callbacks, structured fields, etc.

It is recommended to declare your embeds_many/3 field with type {:array, :map} and default value of [] at the database level. In fact, Ecto will automatically translate nil values from the database into empty lists for embeds many (this behaviour is specific to embeds_many/3 fields in order to mimic has_many/3).

Options

  • :on_cast - the default changeset function to call during casting, which can be overridden in Ecto.Changeset.cast/4. It’s an atom representing the function name in the embedded model’s module which will receive the module and the parameters for casting (default: :changeset).

  • :strategy - the strategy for storing models in the database. Ecto supports only the :replace strategy out of the box which is the default. Read strategy section below for more info.

  • :on_replace - The action taken on associations when the model is replaced when casting or manipulating parent changeset. May be :raise (default), :mark_as_invalid, or :delete. See Ecto.Changeset’s section on related models for more info.

Examples

defmodule Order do
  use Ecto.Schema

  schema "orders" do
    embeds_many :items, Item
  end
end

defmodule Item do
  use Ecto.Schema

  # embedded_schema is a shorcut for:
  #
  #   @primary_key {:id, :binary_id, autogenerate: true}
  #   schema "embedded Item" do
  #
  embedded_schema do
    field :name
  end
end

# The items are loaded with the order
order = Repo.get!(Order, 42)
order.items #=> [%Item{...}, ...]

Adding and removal of embeds can only be done via the Ecto.Changeset API so Ecto can properly track the embeded models life-cycle:

order = Repo.get!(Order, 42)

# Generate a changeset
changeset = Ecto.Changeset.change(order)

# Change, put a new one or remove all items
changeset = Ecto.Changeset.put_change(order, :items, [])

# Update the order
changeset = Repo.update!(changeset)

Strategy

A strategy configures how modules should be inserted, updated and deleted from the database. Changing the strategy may affect how items are stored in the database, although embeds_many will always have them as a list in the model.

Ecto supports only the :replace strategy out of the box which is the default. This means all embeds in the model always fully replace the entries in the database.

For example, if you have a collection with a 100 items, the 100 items will be sent whenever any of them change. The approach is useful when you need the parent and embeds to always be consistent.

Other databases may support different strategies, like one that only changes the embeds that have effectively changed, also reducing the amount of data send to the database. This is specially common in NoSQL databases.

Please check your adapter documentation in case it supports other strategies.

embeds_one(name, model, opts \\ [])

Indicates an embedding of one model.

The current model has zero or one records of the other model embedded inside of it. It uses a field similar to the :map type for storage, but allows embedded models to have all the things regular models can - callbacks, structured fields, etc. All typecasting operations are performed on an embedded model alongside the operations on the parent model.

You must declare your embeds_one/3 field with type :map at the database level.

Options

  • :on_cast - the default changeset function to call during casting, which can be overridden in Ecto.Changeset.cast/4. It’s an atom representing the function name in the embedded model’s module which will receive the module and the parameters for casting (default: :changeset).

  • :strategy - the strategy for storing models in the database. Ecto supports only the :replace strategy out of the box which is the default. Read the strategy in embeds_many/3 for more info.

  • :on_replace - The action taken on associations when the model is replaced when casting or manipulating parent changeset. May be :raise (default), :mark_as_invalid, or :delete. See Ecto.Changeset’s section on related models for more info.

Examples

defmodule Order do
  use Ecto.Schema

  schema "orders" do
    embeds_one :item, Item
  end
end

defmodule Item do
  use Ecto.Schema

  # A required field for all embedded documents
  @primary_key {:id, :binary_id, autogenerate: true}
  schema "" do
    field :name
  end
end

# The item is loaded with the order
order = Repo.get!(Order, 42)
order.item #=> %Item{...}

Adding and removal of embeds can only be done via the Ecto.Changeset API so Ecto can properly track the embeded model life-cycle:

order = Repo.get!(Order, 42)

# Generate a changeset
changeset = Ecto.Changeset.change(order)

# Change, put a new one or remove an item
changeset = Ecto.Changeset.put_change(order, :item, nil)

# Update the order
changeset = Repo.update!(changeset)
field(name, type \\ :string, opts \\ [])

Defines a field on the model schema with given name and type.

Options

  • :default - Sets the default value on the schema and the struct. The default value is calculated at compilation time, so don’t use expressions like Ecto.DateTime.utc or Ecto.UUID.generate as they would then be the same for all records

  • :autogenerate - Annotates the field to be autogenerated before insertion if not value is set.

  • :read_after_writes - When true, the field is always read back from the database after insert and updates.

    For relational databases, this means the RETURNING option of those statements are used. For this reason, MySQL does not support this option and will raise an error if a model is inserted/updated with read after writes fields.

  • :virtual - When true, the field is not persisted to the database. Notice virtual fields do not support :autogenerate nor :read_after_writes.
has_many(name, queryable, opts \\ [])

Indicates a one-to-many association with another model.

The current model has zero or more records of the other model. The other model often has a belongs_to field with the reverse association.

Options

  • :foreign_key - Sets the foreign key, this should map to a field on the other model, defaults to the underscored name of the current model suffixed by _id

  • :references - Sets the key on the current model to be used for the association, defaults to the primary key on the model

  • :through - If this association must be defined in terms of existing associations. Read below for more information

  • :on_delete - The action taken on associations when parent model is deleted. May be :nothing (default), :nilify_all and :delete_all. Notice :on_delete may also be set in migrations when creating a reference. If supported, relying on the database via migrations is prefered

  • :on_replace - The action taken on associations when the model is replaced when casting or manipulating parent changeset. May be :raise (default), :mark_as_invalid, :nilify, or :delete. See Ecto.Changeset’s section on related models for more info.

  • :on_cast - The default changeset function to call during casting of a nested association which can be overridden in Ecto.Changeset.cast/4. It’s an atom representing the function name in the associated model’s module which will receive the module and the parameters for casting (default: :changeset)

  • :defaults - Default values to use when building the association

Examples

defmodule Post do
  use Ecto.Schema
  schema "posts" do
    has_many :comments, Comment
  end
end

# Get all comments for a given post
post = Repo.get(Post, 42)
comments = Repo.all assoc(post, :comments)

# The comments can come preloaded on the post struct
[post] = Repo.all(from(p in Post, where: p.id == 42, preload: :comments))
post.comments #=> [%Comment{...}, ...]

has_many/has_one :through

Ecto also supports defining associations in terms of other associations via the :through option. Let’s see an example:

defmodule Post do
  use Ecto.Schema

  schema "posts" do
    has_many :comments, Comment
    has_one :permalink, Permalink

    # In the has_many :through example below, in the list
    # `[:comments, :author]` the `:comments` refers to the
    # `has_many :comments` in the Post model's own schema
    # and the `:author` refers to the `belongs_to :author`
    # of the Comment module's schema (the module below).
    # (see the description below for more details)
    has_many :comments_authors, through: [:comments, :author]

    # Specify the association with custom source
    has_many :tags, {"posts_tags", Tag}
  end
end

defmodule Comment do
  use Ecto.Schema

  schema "comments" do
    belongs_to :author, Author
    belongs_to :post, Post
    has_one :post_permalink, through: [:post, :permalink]
  end
end

In the example above, we have defined a has_many :through association named :comments_authors. A :through association always expect a list and the first element of the list must be a previously defined association in the current module. For example, :comments_authors first points to :comments in the same module (Post), which then points to :author in the next model Comment.

This :through associations will return all authors for all comments that belongs to that post:

# Get all comments for a given post
post = Repo.get(Post, 42)
authors = Repo.all assoc(post, :comments_authors)

:through associations are read-only as they are useful to avoid repetition allowing the developer to easily retrieve data that is often seem together but stored across different tables.

:through associations can also be preloaded. In such cases, not only the :through association is preloaded but all intermediate steps are preloaded too:

[post] = Repo.all(from(p in Post, where: p.id == 42, preload: :comments_authors))
post.comments_authors #=> [%Author{...}, ...]

# The comments for each post will be preloaded too
post.comments #=> [%Comment{...}, ...]

# And the author for each comment too
hd(post.comments).author #=> %Author{...}

Finally, :through can be used with multiple associations (not only 2) and with associations of any kind, including belongs_to and others :through associations. When the :through association is expected to return one or no item, has_one :through should be used instead, as in the example at the beginning of this section:

# How we defined the association above
has_one :post_permalink, through: [:post, :permalink]

# Get a preloaded comment
[comment] = Repo.all(Comment) |> Repo.preload(:post_permalink)
comment.post_permalink #=> %Permalink{...}
has_one(name, queryable, opts \\ [])

Indicates a one-to-one association with another model.

The current model has zero or one records of the other model. The other model often has a belongs_to field with the reverse association.

Options

  • :foreign_key - Sets the foreign key, this should map to a field on the other model, defaults to the underscored name of the current model suffixed by _id

  • :references - Sets the key on the current model to be used for the association, defaults to the primary key on the model

  • :through - If this association must be defined in terms of existing associations. Read the section in has_many/3 for more information

  • :on_delete - The action taken on associations when parent model is deleted. May be :nothing (default), :nilify_all and :delete_all. Notice :on_delete may also be set in migrations when creating a reference. If supported, relying on the database via migrations is prefered

  • :on_replace - The action taken on associations when the model is replaced when casting or manipulating parent changeset. May be :raise (default), :mark_as_invalid, :nilify, or :delete. See Ecto.Changeset’s section on related models for more info.

  • :on_cast - The default changeset function to call during casting of a nested association which can be overridden in Ecto.Changeset.cast/4. It’s an atom representing the function name in the associated model’s module which will receive the module and the parameters for casting (default: :changeset)

  • :defaults - Default values to use when building the association

Examples

defmodule Post do
  use Ecto.Schema

  schema "posts" do
    has_one :permalink, Permalink

    # Specify the association with custom source
    has_one :category, {"posts_categories", Category}
  end
end

# The permalink can come preloaded on the post struct
[post] = Repo.all(from(p in Post, where: p.id == 42, preload: :permalink))
post.permalink #=> %Permalink{...}
schema(source, list)

Defines a schema with a source name and field definitions.

timestamps(opts \\ [])

Generates :inserted_at and :updated_at timestamp fields.

When using Ecto.Model, the fields generated by this macro will automatically be set to the current time when inserting and updating values in a repository.

Options

  • :type - the timestamps type, defaults to Ecto.DateTime.
  • :usec - boolean, sets whether microseconds are used in timestamps. Microseconds will be 0 if false. Defaults to false.
  • :inserted_at - the name of the column for insertion times or false
  • :updated_at - the name of the column for update times or false

All options can be pre-configured by setting @timestamps_opts.