Ecto v2.0.0-rc.1 Ecto.Schema
Defines a schema.
An Ecto schema is used to map any data source into an Elixir struct. One of such use cases is to map data coming from a repository, usually a table, into Elixir structs.
Example
defmodule User do
use Ecto.Schema
schema "users" do
field :name, :string
field :age, :integer, default: 0
has_many :posts, Post
end
end
By default, a schema will generate a primary key, named id
and
of type :integer
, and belongs_to
associations in the schema will generate
foreign keys of type :integer
. These setting can be configured
below.
Schema attributes
Supported attributes, to be set beforehand, for configuring the defined schema.
These attributes are:
@primary_key
- configures the schema primary key. It expects a tuple{field_name, type, options}
with the primary key field name, type (typically:id
or:binary_id
, but can be any type) and options. Defaults to{:id, :id, autogenerate: true}
. When set tofalse
, does not define a primary key in the schema;@schema_prefix
- configures the schema prefix. Defaults tonil
, which generates structs and queries without prefix. When set, the prefix will be used by every built struct as well as queries where thefrom
is the current schema. In PostgreSQL, the prefix is called “SCHEMA” (typically set with schema search_path). In MySQL the prefix points to databases.@foreign_key_type
- configures the default foreign key type used bybelongs_to
associations. Defaults to:integer
;@timestamps_opts
- configures the default timestamps type used bytimestamps
. Defaults to[type: Ecto.DateTime, usec: false]
;@derive
- the same as@derive
available inKernel.defstruct/1
as the schema defines a struct behind the scenes;
The advantage of configuring the schema via those attributes is that they can be set with a macro to configure application wide defaults.
For example, if your database does not support autoincrementing
primary keys and requires something like UUID or a RecordID, you
can configure and use:binary_id
as your primary key type as follows:
# Define a module to be used as base
defmodule MyApp.Schema do
defmacro __using__(_) do
quote do
use Ecto.Schema
@primary_key {:id, :binary_id, autogenerate: true}
@foreign_key_type :binary_id
end
end
end
# Now use MyApp.Schema to define new schemas
defmodule MyApp.Comment do
use MyApp.Schema
schema "comments" do
belongs_to :post, MyApp.Post
end
end
Any schemas using MyApp.Schema
will get the :id
field with type
:binary_id
as the primary key. We explain what the :binary_id
type
entails in the next section.
The belongs_to
association on MyApp.Comment
will also define
a :post_id
field with :binary_id
type that references the :id
field of the MyApp.Post
schema.
Primary keys
Ecto supports two ID types, called :id
and :binary_id
, which are
often used as the type for primary keys and associations.
The :id
type is used when the primary key is an integer while the
:binary_id
is used for primary keys in particular binary formats,
which may be Ecto.UUID
for databases like PostgreSQL and MySQL,
or some specific ObjectID or RecordID often imposed by NoSQL databases.
In both cases, both types have their semantics specified by the
underlying adapter/database. If you use the :id
type with
:autogenerate
, it means the database will be responsible for
auto-generation of the id. This is often the case for primary keys
in relational databases which are auto-incremented.
Similarly, the :binary_id
type may be generated in the adapter
for cases like UUID but it may also be handled by the database if
required. In any case, both scenarios are handled transparently by
Ecto.
Besides :id
and :binary_id
, which are often used by primary
and foreign keys, Ecto provides a huge variety of types to be used
by any column.
Types and casting
When defining the schema, types need to be given. Types are split into two categories, primitive types and custom types.
Primitive types
The primitive types are:
Ecto type | Elixir type | Literal syntax in query |
---|---|---|
:id | integer | 1, 2, 3 |
:binary_id | binary | <<int, int, int, ...>> |
:integer | integer | 1, 2, 3 |
:float | float | 1.0, 2.0, 3.0 |
:boolean | boolean | true, false |
:string | UTF-8 encoded string | “hello” |
:binary | binary | <<int, int, int, ...>> |
{:array, inner_type} | list | [value, value, value, ...] |
:decimal | Decimal | |
:map | map |
Note: For the :array
type, replace inner_type
with one of
the valid types, such as :string
.
Custom types
Besides providing primitive types, Ecto allows custom types to be implemented by developers, allowing Ecto behaviour to be extended.
A custom type is a module that implements the Ecto.Type
behaviour.
By default, Ecto provides the following custom types:
Custom type | Database type | Elixir type |
---|---|---|
Ecto.DateTime | :datetime | %Ecto.DateTime{} |
Ecto.Date | :date | %Ecto.Date{} |
Ecto.Time | :time | %Ecto.Time{} |
Ecto.UUID | :uuid | uuid-string |
Read the Ecto.Type
documentation for more information on implementing
your own types.
Finally, schemas can also have virtual fields by passing the
virtual: true
option. These fields are not persisted to the database
and can optionally not be type checked by declaring type :any
.
The map type
The map type allows developers to store an Elixir map directly in the database:
# In your migration
create table(:users) do
add :data, :map
end
# In your schema
field :data, :map
# Now in your code
user = Repo.insert! %User{data: %{"foo" => "bar"}}
Keep in mind that we advise the map keys to be strings or integers instead of atoms. Atoms may be accepted depending on how maps are serialized but the database will always return atom keys as strings due to security reasons.
In order to support maps, different databases may employ different techniques. For example, PostgreSQL will store those values in jsonb fields, allowing you to just query parts of it. MySQL and MSSQL, on the other hand, do not yet provide a JSON type, so the value will be stored in a text field.
For maps to work in such databases, Ecto will need a JSON library.
By default Ecto will use Poison
which needs to be added your deps in mix.exs
:
{:poison, "~> 1.0"}
You can however tell Ecto to use any other library by configuring it:
config :ecto, :json_library, YourLibraryOfChoice
Casting
When directly manipulating the struct, it is the responsibility of
the developer to ensure the field values have the proper type. For
example, you can create a user struct with an invalid value
for age
:
iex> user = %User{age: "0"}
iex> user.age
"0"
However, if you attempt to persist the struct above, an error will be raised since Ecto validates the types when sending them to the adapter/database.
Therefore, when working with and manipulating external data, it is
recommended to use Ecto.Changeset
’s that are able to filter
and properly cast external data:
changeset = Ecto.Changeset.cast(%User{}, %{"age" => "0"}, [:age])
user = Repo.insert!(changeset)
You can use Ecto schemas and changesets to cast and valid any kind of data, regardless if the data will be persisted to an Ecto repository or not.
Reflection
Any schema module will generate the __schema__
function that can be
used for runtime introspection of the schema:
__schema__(:source)
- Returns the source as given toschema/2
;__schema__(:prefix)
- Returns optional prefix for source provided by@schema_prefix
schema attribute;__schema__(:primary_key)
- Returns a list of primary key fields (empty if there is none);__schema__(:fields)
- Returns a list of all non-virtual field names;__schema__(:type, field)
- Returns the type of the given non-virtual field;__schema__(:types)
- Returns a keyword list of all non-virtual field names and their type;__schema__(:associations)
- Returns a list of all association field names;__schema__(:association, assoc)
- Returns the association reflection of the given assoc;__schema__(:embeds)
- Returns a list of all embedded field names;__schema__(:embed, embed)
- Returns the embedding reflection of the given embed;__schema__(:read_after_writes)
- Non-virtual fields that must be read back from the database after every write (insert or update);__schema__(:autogenerate_id)
- Primary key that is auto generated on insert;
Furthermore, both __struct__
and __changeset__
functions are
defined so structs and changeset functionalities are available.
Summary
Macros
Indicates a one-to-one association with another schema
Defines an embedded schema
Indicates an embedding of many schemas
Indicates an embedding of a schema
Defines a field on the schema with given name and type
Indicates a one-to-many association with another schema
Indicates a one-to-one association with another schema
Indicates a many-to-many association with another schema
Defines a schema with a source name and field definitions
Generates :inserted_at
and :updated_at
timestamp fields
Types
t :: %atom{}
Macros
Indicates a one-to-one association with another schema.
The current schema belongs to zero or one records of the other schema. The other
schema often has a has_one
or a has_many
field with the reverse association.
You should use belongs_to
in the table that contains the foreign key. Imagine
a company <-> manager relationship. If the company contains the manager_id
in
the underlying database table, we say the company belongs to manager.
In fact, when you invoke this macro, a field with the name of foreign key is automatically defined in the schema for you.
Options
:foreign_key
- Sets the foreign key field name, defaults to the name of the association suffixed by_id
. For example,belongs_to :company
will define foreign key of:company_id
:references
- Sets the key on the other schema to be used for the association, defaults to::id
:define_field
- When false, does not automatically define a:foreign_key
field, implying the user is defining the field manually elsewhere:type
- Sets the type of automatically defined:foreign_key
. Defaults to::id
and can be set per schema via@foreign_key_type
:on_replace
- The action taken on associations when the record is replaced when casting or manipulating parent changeset. May be:raise
(default),:mark_as_invalid
,:nilify
, or:delete
. SeeEcto.Changeset
’s section on related data for more info.:defaults
- Default values to use when building the association
All other options are forwarded to the underlying foreign key definition
and therefore accept the same options as field/3
.
Examples
defmodule Comment do
use Ecto.Schema
schema "comments" do
belongs_to :post, Post
end
end
# The post can come preloaded on the comment record
[comment] = Repo.all(from(c in Comment, where: c.id == 42, preload: :post))
comment.post #=> %Post{...}
Polymorphic associations
One common use case for belongs to associations is to handle polymorphism. For example, imagine you have defined a Comment schema and you wish to use it for commenting on both tasks and posts.
Some abstractions would force you to define some sort of polymorphic association with two fields in your database:
* commentable_type
* commentable_id
The problem with this approach is that it breaks references in the database. You can’t use foreign keys and it is very inefficient, both in terms of query time and storage.
In Ecto, we have two ways to solve this issue. The simplest is to define multiple fields in the Comment schema, one for each association:
* task_id
* post_id
Unless you have dozens of columns, this is simpler for the developer, more DB friendly and more efficient in all aspects.
Alternatively, because Ecto does not tie a schema to a given table, we can use separate tables for each association. Let’s start over and define a new Comment schema:
defmodule Comment do
use Ecto.Schema
schema "abstract table: comments" do
# This will be used by associations on each "concrete" table
field :assoc_id, :integer
end
end
Notice we have changed the table name to “abstract table: comments”. You can choose whatever name you want, the point here is that this particular table will never exist.
Now in your Post and Task schemas:
defmodule Post do
use Ecto.Schema
schema "posts" do
has_many :comments, {"posts_comments", Comment}, foreign_key: :assoc_id
end
end
defmodule Task do
use Ecto.Schema
schema "tasks" do
has_many :comments, {"tasks_comments", Comment}, foreign_key: :assoc_id
end
end
Now each association uses its own specific table, “posts_comments” and “tasks_comments”, which must be created on migrations. The advantage of this approach is that we never store unrelated data together, also ensuring we keep database references fast and correct.
When using this technique, the only limitation is that you cannot build comments directly. For example, the command below
Repo.insert!(%Comment{})
will attempt to use the abstract table. Instead, one should use
Repo.insert!(build_assoc(post, :comments))
where build_assoc/3
is defined in Ecto
. You can also
use assoc/2
in both Ecto
and in the query syntax
to easily retrieve associated comments to a given post or
task:
# Fetch all comments associated to the given task
Repo.all(assoc(task, :comments))
Finally, if for some reason you wish to query one of comments tables directly, you can also specify the tuple source in the query syntax:
Repo.all from(c in {"posts_comments", Comment}), ...)
Defines an embedded schema.
An embedded schema does not require a source name and it does not include a metadata field.
Embedded schemas by default set the primary key type
to :binary_id
but such can be configured with the
@primary_key
attribute.
Indicates an embedding of many schemas.
The current schema has zero or more records of the other schema embedded inside of it. Embeds have all the things regular schema can.
It is recommended to declare your embeds_many/3
field with type
{:array, :map}
and default value of []
at the database level.
In fact, Ecto will automatically translate nil
values from the
database into empty lists for embeds many (this behaviour is specific
to embeds_many/3
fields in order to mimic has_many/3
).
For encoding and decoding of embeds, please read the docs for
embeds_one/3
.
Options
:on_replace
- The action taken on associations when the embed is replaced when casting or manipulating parent changeset. May be:raise
(default),:mark_as_invalid
, or:delete
. SeeEcto.Changeset
’s section on related data for more info.
Examples
defmodule Order do
use Ecto.Schema
schema "orders" do
embeds_many :items, Item
end
end
defmodule Item do
use Ecto.Schema
embedded_schema do
field :name
end
end
# The items are loaded with the order
order = Repo.get!(Order, 42)
order.items #=> [%Item{...}, ...]
Adding and removal of embeds can only be done via the Ecto.Changeset
API so Ecto can properly track the embed life-cycle:
order = Repo.get!(Order, 42)
# Generate a changeset
changeset = Ecto.Changeset.change(order)
# Change, put a new one or remove all items
changeset = Ecto.Changeset.put_change(order, :items, [])
# Update the order
changeset = Repo.update!(changeset)
Indicates an embedding of a schema.
The current schema has zero or one records of the other schema embedded
inside of it. It uses a field similar to the :map
type for storage,
but allows embeds to have all the things regular schema can.
You must declare your embeds_one/3
field with type :map
at the
database level.
Options
:on_replace
- The action taken on associations when the embed is replaced when casting or manipulating parent changeset. May be:raise
(default),:mark_as_invalid
, or:delete
. SeeEcto.Changeset
’s section on related data for more info.
Examples
defmodule Order do
use Ecto.Schema
schema "orders" do
embeds_one :item, Item
end
end
defmodule Item do
use Ecto.Schema
# A required field for all embedded documents
@primary_key {:id, :binary_id, autogenerate: true}
schema "" do
field :name
end
end
# The item is loaded with the order
order = Repo.get!(Order, 42)
order.item #=> %Item{...}
Adding and removal of embeds can only be done via the Ecto.Changeset
API so Ecto can properly track the embed life-cycle:
order = Repo.get!(Order, 42)
# Generate a changeset
changeset = Ecto.Changeset.change(order)
# Change, put a new one or remove an item
changeset = Ecto.Changeset.put_change(order, :item, nil)
# Update the order
changeset = Repo.update!(changeset)
Encoding and decoding
Because many databases do not support direct encoding and decoding of embeds, it is often emulated by Ecto by using specific encoding and decoding rules.
For example, PostgreSQL will store embeds on top of JSONB columns, which means types in embedded schemas won’t go through the usual dump->DB->load cycle but rather encode->DB->decode->cast. This means that, when using embedded schemas with databases like PG or MySQL, make sure all of your types can be JSON encoded/decoded correctly. Ecto provides this guarantee for all built-in types.
Defines a field on the schema with given name and type.
Options
:default
- Sets the default value on the schema and the struct. The default value is calculated at compilation time, so don’t use expressions likeEcto.DateTime.utc
orEcto.UUID.generate
as they would then be the same for all records:autogenerate
- Annotates the field to be autogenerated before insertion if value is not set.:read_after_writes
- When true, the field is always read back from the database after insert and updates.For relational databases, this means the RETURNING option of those statements is used. For this reason, MySQL does not support this option and will raise an error if a schema is inserted/updated with read after writes fields.
:virtual
- When true, the field is not persisted to the database. Notice virtual fields do not support:autogenerate
nor:read_after_writes
.
Indicates a one-to-many association with another schema.
The current schema has zero or more records of the other schema. The other
schema often has a belongs_to
field with the reverse association.
Options
:foreign_key
- Sets the foreign key, this should map to a field on the other schema, defaults to the underscored name of the current schema suffixed by_id
:references
- Sets the key on the current schema to be used for the association, defaults to the primary key on the schema:through
- If this association must be defined in terms of existing associations. Read below for more information:on_delete
- The action taken on associations when parent record is deleted. May be:nothing
(default),:nilify_all
and:delete_all
. Notice:on_delete
may also be set in migrations when creating a reference. If supported, relying on the database via migrations is preferred:on_replace
- The action taken on associations when the record is replaced when casting or manipulating parent changeset. May be:raise
(default),:mark_as_invalid
,:nilify
, or:delete
. SeeEcto.Changeset
’s section on related data for more info.:defaults
- Default values to use when building the association
Examples
defmodule Post do
use Ecto.Schema
schema "posts" do
has_many :comments, Comment
end
end
# Get all comments for a given post
post = Repo.get(Post, 42)
comments = Repo.all assoc(post, :comments)
# The comments can come preloaded on the post struct
[post] = Repo.all(from(p in Post, where: p.id == 42, preload: :comments))
post.comments #=> [%Comment{...}, ...]
has_many/has_one :through
Ecto also supports defining associations in terms of other associations
via the :through
option. Let’s see an example:
defmodule Post do
use Ecto.Schema
schema "posts" do
has_many :comments, Comment
has_one :permalink, Permalink
# In the has_many :through example below, the `:comments`
# in the list [:comments, :author] refers to the
# `has_many :comments` in the Post own schema and the
# `:author` refers to the `belongs_to :author` of the
# Comment's schema (the module below).
# (see the description below for more details)
has_many :comments_authors, through: [:comments, :author]
# Specify the association with custom source
has_many :tags, {"posts_tags", Tag}
end
end
defmodule Comment do
use Ecto.Schema
schema "comments" do
belongs_to :author, Author
belongs_to :post, Post
has_one :post_permalink, through: [:post, :permalink]
end
end
In the example above, we have defined a has_many :through
association
named :comments_authors
. A :through
association always expects a list
and the first element of the list must be a previously defined association
in the current module. For example, :comments_authors
first points to
:comments
in the same module (Post), which then points to :author
in
the next schema, Comment
.
This :through
association will return all authors for all comments
that belongs to that post:
# Get all comments for a given post
post = Repo.get(Post, 42)
authors = Repo.all assoc(post, :comments_authors)
:through
associations are read-only as they are useful for avoiding repetition,
allowing the developer to easily retrieve data that is often seen together
but stored across different tables.
:through
associations can also be preloaded. In such cases, not only
the :through
association is preloaded but all intermediate steps are
preloaded too:
[post] = Repo.all(from(p in Post, where: p.id == 42, preload: :comments_authors))
post.comments_authors #=> [%Author{...}, ...]
# The comments for each post will be preloaded too
post.comments #=> [%Comment{...}, ...]
# And the author for each comment too
hd(post.comments).author #=> %Author{...}
Finally, :through
can be used with multiple associations (not only 2)
and with associations of any kind, including belongs_to
and other
:through
associations. When the :through
association is expected to
return one or zero items, has_one :through
should be used instead, as in
the example at the beginning of this section:
# How we defined the association above
has_one :post_permalink, through: [:post, :permalink]
# Get a preloaded comment
[comment] = Repo.all(Comment) |> Repo.preload(:post_permalink)
comment.post_permalink #=> %Permalink{...}
Indicates a one-to-one association with another schema.
The current schema has zero or one records of the other schema. The other
schema often has a belongs_to
field with the reverse association.
Options
:foreign_key
- Sets the foreign key, this should map to a field on the other schema, defaults to the underscored name of the current schema suffixed by_id
:references
- Sets the key on the current schema to be used for the association, defaults to the primary key on the schema:through
- If this association must be defined in terms of existing associations. Read the section inhas_many/3
for more information:on_delete
- The action taken on associations when parent record is deleted. May be:nothing
(default),:nilify_all
and:delete_all
. Notice:on_delete
may also be set in migrations when creating a reference. If supported, relying on the database via migrations is preferred:on_replace
- The action taken on associations when the record is replaced when casting or manipulating parent changeset. May be:raise
(default),:mark_as_invalid
,:nilify
, or:delete
. SeeEcto.Changeset
’s section on related data for more info.:defaults
- Default values to use when building the association
Examples
defmodule Post do
use Ecto.Schema
schema "posts" do
has_one :permalink, Permalink
# Specify the association with custom source
has_one :category, {"posts_categories", Category}
end
end
# The permalink can come preloaded on the post struct
[post] = Repo.all(from(p in Post, where: p.id == 42, preload: :permalink))
post.permalink #=> %Permalink{...}
Indicates a many-to-many association with another schema.
The association happens through a join schema or source, containing foreign keys to the associated schemas. For example, the association below:
# from MyApp.Post
many_to_many :tags, MyApp.Tag, join_through: "posts_tags"
is backed by relational databases through a join table as follows:
[Post] <-> [posts_tags] <-> [Tag]
id <-- post_id
tag_id --> id
More information on the migration for creating such a schema is shown below.
Options
:join_through
- specifies the source of the associated data. It may be a string, like “posts_tags”, representing the underlying storage table or an atom, likeMyApp.PostTag
, representing a schema. This option is required.:join_keys
- specifies how the schemas are associated. It expects a keyword list with two entries, the first being how the join table should reach the current schema and the second how the join table should reach the associated schema. In the example above, it defaults to:[post_id: :id, tag_id: :id]
. The keys are inflected from the schema names.:on_delete
- The action taken on associations when the parent record is deleted. May be:nothing
(default) or:delete_all
.:delete_all
will only remove data from the join source, never the associated records. Notice:on_delete
may also be set in migrations when creating a reference. If supported, relying on the database via migrations is preferred:on_replace
- The action taken on associations when the record is replaced when casting or manipulating parent changeset. May be:raise
(default),:mark_as_invalid
, or:delete
.:delete
will only remove data from the join source, never the associated records. SeeEcto.Changeset
’s section on related data for more info.:defaults
- Default values to use when building the association
Removing data
If you attempt to remove associated many_to_many
data, be it by
setting :on_replace
to :delete
, :on_delete
to :delete_all
or by using Ecto.Changeset.put_assoc/3
and Ecto.Changeset.cast_assoc/3
,
Ecto will always remove data from the join schema and never from
the target associations. For example, if a Post
has a many to many
relationship with Tag
, setting :on_delete
to :delete_all
will
only delete entries from the “posts_tags” table in case Post
is
deleted.
Migration
How your migration should be structured depends on the value you pass
in :join_through
. If :join_through
is simply a string, representing
a table, you may define a table without primary keys and you must not
include any further columns, as those values won’t be set by Ecto:
create table(:posts_tags, primary_key: false) do
add :post_id, references(:posts)
add :tag_id, references(:tags)
end
However, if your :join_through
is a schema, like MyApp.PostTag
, your
join table may be structured as any other table in your codebase,
including timestamps:
create table(:posts_tags) do
add :post_id, references(:posts)
add :tag_id, references(:tags)
timestamps
end
Because :join_through
contains a schema, in such cases, autogenerated
values and primary keys will be automatically handled by Ecto.
Examples
defmodule Post do
use Ecto.Schema
schema "posts" do
many_to_many :tags, Tag, join_through: "posts_tags"
end
end
# Get all comments for a given post
post = Repo.get(Post, 42)
tags = Repo.all assoc(post, :tags)
# The comments can come preloaded on the post struct
[post] = Repo.all(from(p in Post, where: p.id == 42, preload: :tags))
post.tags #=> [%Tag{...}, ...]
Generates :inserted_at
and :updated_at
timestamp fields.
The fields generated by this macro will automatically be set to the current time when inserting and updating values in a repository.
Options
:type
- the timestamps type, defaults toEcto.DateTime
.:usec
- boolean, sets whether microseconds are used in timestamps. Microseconds will be 0 if false. Defaults to false.:inserted_at
- the name of the column for insertion times orfalse
:updated_at
- the name of the column for update times orfalse
All options can be pre-configured by setting @timestamps_opts
.