ecto.ex (23064B)
1 defmodule Ecto do 2 @moduledoc ~S""" 3 Ecto is split into 4 main components: 4 5 * `Ecto.Repo` - repositories are wrappers around the data store. 6 Via the repository, we can create, update, destroy and query 7 existing entries. A repository needs an adapter and credentials 8 to communicate to the database 9 10 * `Ecto.Schema` - schemas are used to map external data into Elixir 11 structs. We often use them to map database tables to Elixir data but 12 they have many other use cases 13 14 * `Ecto.Query` - written in Elixir syntax, queries are used to retrieve 15 information from a given repository. Ecto queries are secure and composable 16 17 * `Ecto.Changeset` - changesets provide a way for track and validate changes 18 before they are applied to the data 19 20 In summary: 21 22 * `Ecto.Repo` - **where** the data is 23 * `Ecto.Schema` - **what** the data is 24 * `Ecto.Query` - **how to read** the data 25 * `Ecto.Changeset` - **how to change** the data 26 27 Besides the four components above, most developers use Ecto to interact 28 with SQL databases, such as PostgreSQL and MySQL via the 29 [`ecto_sql`](https://hexdocs.pm/ecto_sql) project. `ecto_sql` provides many 30 conveniences for working with SQL databases as well as the ability to version 31 how your database changes through time via 32 [database migrations](https://hexdocs.pm/ecto_sql/Ecto.Adapters.SQL.html#module-migrations). 33 34 If you want to quickly check a sample application using Ecto, please check 35 the [getting started guide](https://hexdocs.pm/ecto/getting-started.html) and 36 the accompanying sample application. [Ecto's README](https://github.com/elixir-ecto/ecto) 37 also links to other resources. 38 39 In the following sections, we will provide an overview of those components and 40 how they interact with each other. Feel free to access their respective module 41 documentation for more specific examples, options and configuration. 42 43 ## Repositories 44 45 `Ecto.Repo` is a wrapper around the database. We can define a 46 repository as follows: 47 48 defmodule Repo do 49 use Ecto.Repo, 50 otp_app: :my_app, 51 adapter: Ecto.Adapters.Postgres 52 end 53 54 Where the configuration for the Repo must be in your application 55 environment, usually defined in your `config/config.exs`: 56 57 config :my_app, Repo, 58 database: "ecto_simple", 59 username: "postgres", 60 password: "postgres", 61 hostname: "localhost", 62 # OR use a URL to connect instead 63 url: "postgres://postgres:postgres@localhost/ecto_simple" 64 65 Each repository in Ecto defines a `start_link/0` function that needs to be invoked 66 before using the repository. In general, this function is not called directly, 67 but used as part of your application supervision tree. 68 69 If your application was generated with a supervisor (by passing `--sup` to `mix new`) 70 you will have a `lib/my_app/application.ex` file containing the application start 71 callback that defines and starts your supervisor. You just need to edit the `start/2` 72 function to start the repo as a supervisor on your application's supervisor: 73 74 def start(_type, _args) do 75 children = [ 76 MyApp.Repo, 77 ] 78 79 opts = [strategy: :one_for_one, name: MyApp.Supervisor] 80 Supervisor.start_link(children, opts) 81 end 82 83 ## Schema 84 85 Schemas allow developers to define the shape of their data. 86 Let's see an example: 87 88 defmodule Weather do 89 use Ecto.Schema 90 91 # weather is the DB table 92 schema "weather" do 93 field :city, :string 94 field :temp_lo, :integer 95 field :temp_hi, :integer 96 field :prcp, :float, default: 0.0 97 end 98 end 99 100 By defining a schema, Ecto automatically defines a struct with 101 the schema fields: 102 103 iex> weather = %Weather{temp_lo: 30} 104 iex> weather.temp_lo 105 30 106 107 The schema also allows us to interact with a repository: 108 109 iex> weather = %Weather{temp_lo: 0, temp_hi: 23} 110 iex> Repo.insert!(weather) 111 %Weather{...} 112 113 After persisting `weather` to the database, it will return a new copy of 114 `%Weather{}` with the primary key (the `id`) set. We can use this value 115 to read a struct back from the repository: 116 117 # Get the struct back 118 iex> weather = Repo.get Weather, 1 119 %Weather{id: 1, ...} 120 121 # Delete it 122 iex> Repo.delete!(weather) 123 %Weather{...} 124 125 > NOTE: by using `Ecto.Schema`, an `:id` field with type `:id` (:id means :integer) is 126 > generated by default, which is the primary key of the Schema. If you want 127 > to use a different primary key, you can declare custom `@primary_key` 128 > before the `schema/2` call. Consult the `Ecto.Schema` documentation 129 > for more information. 130 131 Notice how the storage (repository) and the data are decoupled. This provides 132 two main benefits: 133 134 * By having structs as data, we guarantee they are light-weight, 135 serializable structures. In many languages, the data is often represented 136 by large, complex objects, with entwined state transactions, which makes 137 serialization, maintenance and understanding hard; 138 139 * You do not need to define schemas in order to interact with repositories, 140 operations like `all`, `insert_all` and so on allow developers to directly 141 access and modify the data, keeping the database at your fingertips when 142 necessary; 143 144 ## Changesets 145 146 Although in the example above we have directly inserted and deleted the 147 struct in the repository, operations on top of schemas are done through 148 changesets so Ecto can efficiently track changes. 149 150 Changesets allow developers to filter, cast, and validate changes before 151 we apply them to the data. Imagine the given schema: 152 153 defmodule User do 154 use Ecto.Schema 155 156 import Ecto.Changeset 157 158 schema "users" do 159 field :name 160 field :email 161 field :age, :integer 162 end 163 164 def changeset(user, params \\ %{}) do 165 user 166 |> cast(params, [:name, :email, :age]) 167 |> validate_required([:name, :email]) 168 |> validate_format(:email, ~r/@/) 169 |> validate_inclusion(:age, 18..100) 170 end 171 end 172 173 The `changeset/2` function first invokes `Ecto.Changeset.cast/4` with 174 the struct, the parameters and a list of allowed fields; this returns a changeset. 175 The parameters is a map with binary keys and values that will be cast based 176 on the type defined on the schema. 177 178 Any parameter that was not explicitly listed in the fields list will be ignored. 179 180 After casting, the changeset is given to many `Ecto.Changeset.validate_*` 181 functions that validate only the **changed fields**. In other words: 182 if a field was not given as a parameter, it won't be validated at all. 183 For example, if the params map contain only the "name" and "email" keys, 184 the "age" validation won't run. 185 186 Once a changeset is built, it can be given to functions like `insert` and 187 `update` in the repository that will return an `:ok` or `:error` tuple: 188 189 case Repo.update(changeset) do 190 {:ok, user} -> 191 # user updated 192 {:error, changeset} -> 193 # an error occurred 194 end 195 196 The benefit of having explicit changesets is that we can easily provide 197 different changesets for different use cases. For example, one 198 could easily provide specific changesets for registering and updating 199 users: 200 201 def registration_changeset(user, params) do 202 # Changeset on create 203 end 204 205 def update_changeset(user, params) do 206 # Changeset on update 207 end 208 209 Changesets are also capable of transforming database constraints, 210 like unique indexes and foreign key checks, into errors. Allowing 211 developers to keep their database consistent while still providing 212 proper feedback to end users. Check `Ecto.Changeset.unique_constraint/3` 213 for some examples as well as the other `_constraint` functions. 214 215 ## Query 216 217 Last but not least, Ecto allows you to write queries in Elixir and send 218 them to the repository, which translates them to the underlying database. 219 Let's see an example: 220 221 import Ecto.Query, only: [from: 2] 222 223 query = from u in User, 224 where: u.age > 18 or is_nil(u.email), 225 select: u 226 227 # Returns %User{} structs matching the query 228 Repo.all(query) 229 230 In the example above we relied on our schema but queries can also be 231 made directly against a table by giving the table name as a string. In 232 such cases, the data to be fetched must be explicitly outlined: 233 234 query = from u in "users", 235 where: u.age > 18 or is_nil(u.email), 236 select: %{name: u.name, age: u.age} 237 238 # Returns maps as defined in select 239 Repo.all(query) 240 241 Queries are defined and extended with the `from` macro. The supported 242 keywords are: 243 244 * `:distinct` 245 * `:where` 246 * `:order_by` 247 * `:offset` 248 * `:limit` 249 * `:lock` 250 * `:group_by` 251 * `:having` 252 * `:join` 253 * `:select` 254 * `:preload` 255 256 Examples and detailed documentation for each of those are available 257 in the `Ecto.Query` module. Functions supported in queries are listed 258 in `Ecto.Query.API`. 259 260 When writing a query, you are inside Ecto's query syntax. In order to 261 access params values or invoke Elixir functions, you need to use the `^` 262 operator, which is overloaded by Ecto: 263 264 def min_age(min) do 265 from u in User, where: u.age > ^min 266 end 267 268 Besides `Repo.all/1` which returns all entries, repositories also 269 provide `Repo.one/1` which returns one entry or nil, `Repo.one!/1` 270 which returns one entry or raises, `Repo.get/2` which fetches 271 entries for a particular ID and more. 272 273 Finally, if you need an escape hatch, Ecto provides fragments 274 (see `Ecto.Query.API.fragment/1`) to inject SQL (and non-SQL) 275 fragments into queries. Also, most adapters provide direct 276 APIs for queries, like `Ecto.Adapters.SQL.query/4`, allowing 277 developers to completely bypass Ecto queries. 278 279 ## Other topics 280 281 ### Associations 282 283 Ecto supports defining associations on schemas: 284 285 defmodule Post do 286 use Ecto.Schema 287 288 schema "posts" do 289 has_many :comments, Comment 290 end 291 end 292 293 defmodule Comment do 294 use Ecto.Schema 295 296 schema "comments" do 297 field :title, :string 298 belongs_to :post, Post 299 end 300 end 301 302 When an association is defined, Ecto also defines a field in the schema 303 with the association name. By default, associations are not loaded into 304 this field: 305 306 iex> post = Repo.get(Post, 42) 307 iex> post.comments 308 #Ecto.Association.NotLoaded<...> 309 310 However, developers can use the preload functionality in queries to 311 automatically pre-populate the field: 312 313 Repo.all from p in Post, preload: [:comments] 314 315 Preloading can also be done with a pre-defined join value: 316 317 Repo.all from p in Post, 318 join: c in assoc(p, :comments), 319 where: c.votes > p.votes, 320 preload: [comments: c] 321 322 Finally, for the simple cases, preloading can also be done after 323 a collection was fetched: 324 325 posts = Repo.all(Post) |> Repo.preload(:comments) 326 327 The `Ecto` module also provides conveniences for working 328 with associations. For example, `Ecto.assoc/2` returns a query 329 with all associated data to a given struct: 330 331 import Ecto 332 333 # Get all comments for the given post 334 Repo.all assoc(post, :comments) 335 336 # Or build a query on top of the associated comments 337 query = from c in assoc(post, :comments), where: not is_nil(c.title) 338 Repo.all(query) 339 340 Another function in `Ecto` is `build_assoc/3`, which allows 341 someone to build an associated struct with the proper fields: 342 343 Repo.transaction fn -> 344 post = Repo.insert!(%Post{title: "Hello", body: "world"}) 345 346 # Build a comment from post 347 comment = Ecto.build_assoc(post, :comments, body: "Excellent!") 348 349 Repo.insert!(comment) 350 end 351 352 In the example above, `Ecto.build_assoc/3` is equivalent to: 353 354 %Comment{post_id: post.id, body: "Excellent!"} 355 356 You can find more information about defining associations and each 357 respective association module in `Ecto.Schema` docs. 358 359 > NOTE: Ecto does not lazy load associations. While lazily loading 360 > associations may sound convenient at first, in the long run it 361 > becomes a source of confusion and performance issues. 362 363 ### Embeds 364 365 Ecto also supports embeds. While associations keep parent and child 366 entries in different tables, embeds stores the child along side the 367 parent. 368 369 Databases like MongoDB have native support for embeds. Databases 370 like PostgreSQL uses a mixture of JSONB (`embeds_one/3`) and ARRAY 371 columns to provide this functionality. 372 373 Check `Ecto.Schema.embeds_one/3` and `Ecto.Schema.embeds_many/3` 374 for more information. 375 376 ### Mix tasks and generators 377 378 Ecto provides many tasks to help your workflow as well as code generators. 379 You can find all available tasks by typing `mix help` inside a project 380 with Ecto listed as a dependency. 381 382 Ecto generators will automatically open the generated files if you have 383 `ECTO_EDITOR` set in your environment variable. 384 385 #### Repo resolution 386 387 Ecto requires developers to specify the key `:ecto_repos` in their 388 application configuration before using tasks like `ecto.create` and 389 `ecto.migrate`. For example: 390 391 config :my_app, :ecto_repos, [MyApp.Repo] 392 393 config :my_app, MyApp.Repo, 394 database: "ecto_simple", 395 username: "postgres", 396 password: "postgres", 397 hostname: "localhost" 398 399 """ 400 401 @doc """ 402 Returns the schema primary keys as a keyword list. 403 """ 404 @spec primary_key(Ecto.Schema.t) :: Keyword.t 405 def primary_key(%{__struct__: schema} = struct) do 406 Enum.map schema.__schema__(:primary_key), fn(field) -> 407 {field, Map.fetch!(struct, field)} 408 end 409 end 410 411 @doc """ 412 Returns the schema primary keys as a keyword list. 413 414 Raises `Ecto.NoPrimaryKeyFieldError` if the schema has no 415 primary key field. 416 """ 417 @spec primary_key!(Ecto.Schema.t) :: Keyword.t 418 def primary_key!(%{__struct__: schema} = struct) do 419 case primary_key(struct) do 420 [] -> raise Ecto.NoPrimaryKeyFieldError, schema: schema 421 pk -> pk 422 end 423 end 424 425 @doc """ 426 Builds a struct from the given `assoc` in `struct`. 427 428 ## Examples 429 430 If the relationship is a `has_one` or `has_many` and 431 the primary key is set in the parent struct, the key will 432 automatically be set in the built association: 433 434 iex> post = Repo.get(Post, 13) 435 %Post{id: 13} 436 iex> build_assoc(post, :comments) 437 %Comment{id: nil, post_id: 13} 438 439 Note though it doesn't happen with `belongs_to` cases, as the 440 key is often the primary key and such is usually generated 441 dynamically: 442 443 iex> comment = Repo.get(Comment, 13) 444 %Comment{id: 13, post_id: 25} 445 iex> build_assoc(comment, :post) 446 %Post{id: nil} 447 448 You can also pass the attributes, which can be a map or 449 a keyword list, to set the struct's fields except the 450 association key. 451 452 iex> build_assoc(post, :comments, text: "cool") 453 %Comment{id: nil, post_id: 13, text: "cool"} 454 455 iex> build_assoc(post, :comments, %{text: "cool"}) 456 %Comment{id: nil, post_id: 13, text: "cool"} 457 458 iex> build_assoc(post, :comments, post_id: 1) 459 %Comment{id: nil, post_id: 13} 460 461 The given attributes are expected to be structured data. 462 If you want to build an association with external data, 463 such as a request parameters, you can use `Ecto.Changeset.cast/3` 464 after `build_assoc/3`: 465 466 parent 467 |> Ecto.build_assoc(:child) 468 |> Ecto.Changeset.cast(params, [:field1, :field2]) 469 470 """ 471 def build_assoc(%{__struct__: schema} = struct, assoc, attributes \\ %{}) do 472 assoc = Ecto.Association.association_from_schema!(schema, assoc) 473 assoc.__struct__.build(assoc, struct, drop_meta(attributes)) 474 end 475 476 defp drop_meta(%{} = attrs), do: Map.drop(attrs, [:__struct__, :__meta__]) 477 defp drop_meta([_|_] = attrs), do: Keyword.drop(attrs, [:__struct__, :__meta__]) 478 479 @doc """ 480 Builds a query for the association in the given struct or structs. 481 482 ## Examples 483 484 In the example below, we get all comments associated to the given 485 post: 486 487 post = Repo.get Post, 1 488 Repo.all Ecto.assoc(post, :comments) 489 490 `assoc/2` can also receive a list of posts, as long as the posts are 491 not empty: 492 493 posts = Repo.all from p in Post, where: is_nil(p.published_at) 494 Repo.all Ecto.assoc(posts, :comments) 495 496 This function can also be used to dynamically load through associations 497 by giving it a list. For example, to get all authors for all comments for 498 the given posts, do: 499 500 posts = Repo.all from p in Post, where: is_nil(p.published_at) 501 Repo.all Ecto.assoc(posts, [:comments, :author]) 502 503 ## Options 504 505 * `:prefix` - the prefix to fetch assocs from. By default, queries 506 will use the same prefix as the first struct in the given collection. 507 This option allows the prefix to be changed. 508 509 """ 510 def assoc(struct_or_structs, assocs, opts \\ []) do 511 [assoc | assocs] = List.wrap(assocs) 512 513 structs = 514 case struct_or_structs do 515 nil -> raise ArgumentError, "cannot retrieve association #{inspect(assoc)} for nil" 516 [] -> raise ArgumentError, "cannot retrieve association #{inspect(assoc)} for empty list" 517 struct_or_structs -> List.wrap(struct_or_structs) 518 end 519 520 sample = hd(structs) 521 prefix = assoc_prefix(sample, opts) 522 schema = sample.__struct__ 523 refl = %{owner_key: owner_key} = Ecto.Association.association_from_schema!(schema, assoc) 524 525 values = 526 Enum.uniq for(struct <- structs, 527 assert_struct!(schema, struct), 528 key = Map.fetch!(struct, owner_key), 529 do: key) 530 531 case assocs do 532 [] -> 533 %module{} = refl 534 %{module.assoc_query(refl, nil, values) | prefix: prefix} 535 536 assocs -> 537 %{Ecto.Association.filter_through_chain(schema, [assoc | assocs], values) | prefix: prefix} 538 end 539 end 540 541 defp assoc_prefix(sample, opts) do 542 case Keyword.fetch(opts, :prefix) do 543 {:ok, prefix} -> 544 prefix 545 546 :error -> 547 case sample do 548 %{__meta__: %{prefix: prefix}} -> prefix 549 # Must be an embedded schema 550 _ -> nil 551 end 552 end 553 end 554 555 @doc """ 556 Checks if an association is loaded. 557 558 ## Examples 559 560 iex> post = Repo.get(Post, 1) 561 iex> Ecto.assoc_loaded?(post.comments) 562 false 563 iex> post = post |> Repo.preload(:comments) 564 iex> Ecto.assoc_loaded?(post.comments) 565 true 566 567 """ 568 def assoc_loaded?(%Ecto.Association.NotLoaded{}), do: false 569 def assoc_loaded?(list) when is_list(list), do: true 570 def assoc_loaded?(%_{}), do: true 571 def assoc_loaded?(nil), do: true 572 573 @doc """ 574 Gets the metadata from the given struct. 575 """ 576 def get_meta(struct, :context), 577 do: struct.__meta__.context 578 def get_meta(struct, :state), 579 do: struct.__meta__.state 580 def get_meta(struct, :source), 581 do: struct.__meta__.source 582 def get_meta(struct, :prefix), 583 do: struct.__meta__.prefix 584 585 @doc """ 586 Returns a new struct with updated metadata. 587 588 It is possible to set: 589 590 * `:source` - changes the struct query source 591 * `:prefix` - changes the struct query prefix 592 * `:context` - changes the struct meta context 593 * `:state` - changes the struct state 594 595 Please refer to the `Ecto.Schema.Metadata` module for more information. 596 """ 597 @spec put_meta(Ecto.Schema.schema, meta) :: Ecto.Schema.schema 598 when meta: [source: Ecto.Schema.source, prefix: Ecto.Schema.prefix, 599 context: Ecto.Schema.Metadata.context, state: Ecto.Schema.Metadata.state] 600 def put_meta(%{__meta__: meta} = struct, opts) do 601 case put_or_noop_meta(opts, meta, false) do 602 :noop -> struct 603 meta -> %{struct | __meta__: meta} 604 end 605 end 606 607 defp put_or_noop_meta([{key, value}|t], meta, updated?) do 608 case meta do 609 %{^key => ^value} -> put_or_noop_meta(t, meta, updated?) 610 _ -> put_or_noop_meta(t, put_meta(meta, key, value), true) 611 end 612 end 613 614 defp put_or_noop_meta([], meta, true), do: meta 615 defp put_or_noop_meta([], _meta, false), do: :noop 616 617 defp put_meta(meta, :state, state) do 618 if state in [:built, :loaded, :deleted] do 619 %{meta | state: state} 620 else 621 raise ArgumentError, "invalid state #{inspect state}" 622 end 623 end 624 625 defp put_meta(meta, :source, source) do 626 %{meta | source: source} 627 end 628 629 defp put_meta(meta, :prefix, prefix) do 630 %{meta | prefix: prefix} 631 end 632 633 defp put_meta(meta, :context, context) do 634 %{meta | context: context} 635 end 636 637 defp put_meta(_meta, key, _value) do 638 raise ArgumentError, "unknown meta key #{inspect key}" 639 end 640 641 defp assert_struct!(module, %{__struct__: struct}) do 642 if struct != module do 643 raise ArgumentError, "expected a homogeneous list containing the same struct, " <> 644 "got: #{inspect module} and #{inspect struct}" 645 else 646 true 647 end 648 end 649 650 @doc """ 651 Loads previously dumped `data` in the given `format` into a schema. 652 653 The first argument can be a an embedded schema module, or a map (of types) and 654 determines the return value: a struct or a map, respectively. 655 656 The second argument `data` specifies fields and values that are to be loaded. 657 It can be a map, a keyword list, or a `{fields, values}` tuple. Fields can be 658 atoms or strings. 659 660 The third argument `format` is the format the data has been dumped as. For 661 example, databases may dump embedded to `:json`, this function allows such 662 dumped data to be put back into the schemas. 663 664 Fields that are not present in the schema (or `types` map) are ignored. 665 If any of the values has invalid type, an error is raised. 666 667 Note that if you want to load data into a non-embedded schema that was 668 directly persisted into a given repository, then use `c:Ecto.Repo.load/2`. 669 670 ## Examples 671 672 iex> result = Ecto.Adapters.SQL.query!(MyRepo, "SELECT users.settings FROM users", []) 673 iex> Enum.map(result.rows, fn [settings] -> Ecto.embedded_load(Setting, Jason.decode!(settings), :json) end) 674 [%Setting{...}, ...] 675 """ 676 @spec embedded_load( 677 module_or_map :: module | map(), 678 data :: map(), 679 format :: atom() 680 ) :: Ecto.Schema.t() | map() 681 def embedded_load(schema_or_types, data, format) do 682 Ecto.Schema.Loader.unsafe_load(schema_or_types, data, &Ecto.Type.embedded_load(&1, &2, format)) 683 end 684 685 @doc """ 686 Dumps the given struct defined by an embedded schema. 687 688 This converts the given embedded schema to a map to be serialized 689 with the given format. For example: 690 691 iex> Ecto.embedded_dump(%Post{}, :json) 692 %{title: "hello"} 693 694 """ 695 @spec embedded_dump(Ecto.Schema.t(), format :: atom()) :: map() 696 def embedded_dump(%schema{} = data, format) do 697 Ecto.Schema.Loader.safe_dump(data, schema.__schema__(:dump), &Ecto.Type.embedded_dump(&1, &2, format)) 698 end 699 end