query.ex (83782B)
1 defmodule Ecto.SubQuery do 2 @moduledoc """ 3 A struct representing subqueries. 4 5 See `Ecto.Query.subquery/2` for more information. 6 """ 7 defstruct [:query, :params, :select, :cache] 8 9 @type t :: %__MODULE__{} 10 end 11 12 defmodule Ecto.Query do 13 @moduledoc ~S""" 14 Provides the Query DSL. 15 16 Queries are used to retrieve and manipulate data from a repository 17 (see `Ecto.Repo`). Ecto queries come in two flavors: keyword-based 18 and macro-based. Most examples will use the keyword-based syntax, 19 the macro one will be explored in later sections. 20 21 Let's see a sample query: 22 23 # Imports only from/2 of Ecto.Query 24 import Ecto.Query, only: [from: 2] 25 26 # Create a query 27 query = from u in "users", 28 where: u.age > 18, 29 select: u.name 30 31 # Send the query to the repository 32 Repo.all(query) 33 34 In the example above, we are directly querying the "users" table 35 from the database. 36 37 ## Query expressions 38 39 Ecto allows a limited set of expressions inside queries. In the 40 query below, for example, we use `u.age` to access a field, the 41 `>` comparison operator and the literal `0`: 42 43 query = from u in "users", where: u.age > 0, select: u.name 44 45 You can find the full list of operations in `Ecto.Query.API`. 46 Besides the operations listed there, the following literals are 47 supported in queries: 48 49 * Integers: `1`, `2`, `3` 50 * Floats: `1.0`, `2.0`, `3.0` 51 * Booleans: `true`, `false` 52 * Binaries: `<<1, 2, 3>>` 53 * Strings: `"foo bar"`, `~s(this is a string)` 54 * Atoms (other than booleans and `nil`): `:foo`, `:bar` 55 * Arrays: `[1, 2, 3]`, `~w(interpolate words)` 56 57 All other types and dynamic values must be passed as a parameter using 58 interpolation as explained below. 59 60 ## Interpolation and casting 61 62 External values and Elixir expressions can be injected into a query 63 expression with `^`: 64 65 def with_minimum(age, height_ft) do 66 from u in "users", 67 where: u.age > ^age and u.height > ^(height_ft * 3.28), 68 select: u.name 69 end 70 71 with_minimum(18, 5.0) 72 73 When interpolating values, you may want to explicitly tell Ecto 74 what is the expected type of the value being interpolated: 75 76 age = "18" 77 Repo.all(from u in "users", 78 where: u.age > type(^age, :integer), 79 select: u.name) 80 81 In the example above, Ecto will cast the age to type integer. When 82 a value cannot be cast, `Ecto.Query.CastError` is raised. 83 84 To avoid the repetition of always specifying the types, you may define 85 an `Ecto.Schema`. In such cases, Ecto will analyze your queries and 86 automatically cast the interpolated "age" when compared to the `u.age` 87 field, as long as the age field is defined with type `:integer` in 88 your schema: 89 90 age = "18" 91 Repo.all(from u in User, where: u.age > ^age, select: u.name) 92 93 Another advantage of using schemas is that we no longer need to specify 94 the select option in queries, as by default Ecto will retrieve all 95 fields specified in the schema: 96 97 age = "18" 98 Repo.all(from u in User, where: u.age > ^age) 99 100 For this reason, we will use schemas on the remaining examples but 101 remember Ecto does not require them in order to write queries. 102 103 ## `nil` comparison 104 105 `nil` comparison in filters, such as where and having, is forbidden 106 and it will raise an error: 107 108 # Raises if age is nil 109 from u in User, where: u.age == ^age 110 111 This is done as a security measure to avoid attacks that attempt 112 to traverse entries with nil columns. To check that value is `nil`, 113 use `is_nil/1` instead: 114 115 from u in User, where: is_nil(u.age) 116 117 ## Composition 118 119 Ecto queries are composable. For example, the query above can 120 actually be defined in two parts: 121 122 # Create a query 123 query = from u in User, where: u.age > 18 124 125 # Extend the query 126 query = from u in query, select: u.name 127 128 Composing queries uses the same syntax as creating a query. 129 The difference is that, instead of passing a schema like `User` 130 on the right-hand side of `in`, we passed the query itself. 131 132 Any value can be used on the right-hand side of `in` as long as it implements 133 the `Ecto.Queryable` protocol. For now, we know the protocol is 134 implemented for both atoms (like `User`) and strings (like "users"). 135 136 In any case, regardless if a schema has been given or not, Ecto 137 queries are always composable thanks to its binding system. 138 139 ### Positional bindings 140 141 On the left-hand side of `in` we specify the query bindings. This is 142 done inside `from` and `join` clauses. In the query below `u` is a 143 binding and `u.age` is a field access using this binding. 144 145 query = from u in User, where: u.age > 18 146 147 Bindings are not exposed from the query. When composing queries, you 148 must specify bindings again for each refinement query. For example, 149 to further narrow down the above query, we again need to tell Ecto what 150 bindings to expect: 151 152 query = from u in query, select: u.city 153 154 Bindings in Ecto are positional, and the names do not have to be 155 consistent between input and refinement queries. For example, the 156 query above could also be written as: 157 158 query = from q in query, select: q.city 159 160 It would make no difference to Ecto. This is important because 161 it allows developers to compose queries without caring about 162 the bindings used in the initial query. 163 164 When using joins, the bindings should be matched in the order they 165 are specified: 166 167 # Create a query 168 query = from p in Post, 169 join: c in Comment, on: c.post_id == p.id 170 171 # Extend the query 172 query = from [p, c] in query, 173 select: {p.title, c.body} 174 175 You are not required to specify all bindings when composing. 176 For example, if we would like to order the results above by 177 post insertion date, we could further extend it as: 178 179 query = from q in query, order_by: q.inserted_at 180 181 The example above will work if the input query has 1 or 10 182 bindings. As long as the number of bindings is less than the 183 number of `from`s + `join`s, Ecto will match only what you have 184 specified. The first binding always matches the source given 185 in `from`. 186 187 Similarly, if you are interested only in the last binding 188 (or the last bindings) in a query, you can use `...` to 189 specify "all bindings before" and match on the last one. 190 191 For instance, imagine you wrote: 192 193 posts_with_comments = 194 from p in query, join: c in Comment, on: c.post_id == p.id 195 196 And now we want to make sure to return both the post title 197 and the comment body. Although we may not know how many 198 bindings there are in the query, we are sure posts is the 199 first binding and comments are the last one, so we can write: 200 201 from [p, ..., c] in posts_with_comments, select: {p.title, c.body} 202 203 In other words, `...` will include all the bindings between the 204 first and the last, which may be one, many or no bindings at all. 205 206 ### Named bindings 207 208 Another option for flexibly building queries with joins are named 209 bindings. Coming back to the previous example, we can use the 210 `as: :comment` option to bind the comments join to a concrete name: 211 212 posts_with_comments = 213 from p in Post, 214 join: c in Comment, as: :comment, on: c.post_id == p.id 215 216 Now we can refer to it using the following form of a bindings list: 217 218 from [p, comment: c] in posts_with_comments, select: {p.title, c.body} 219 220 This approach lets us not worry about keeping track of the position 221 of the bindings when composing the query. The `:as` option can be 222 given both on joins and on `from`: 223 224 from p in Post, as: :post 225 226 Only atoms are accepted for binding names. Named binding references 227 must always be placed at the end of the bindings list: 228 229 [positional_binding_1, positional_binding_2, named_1: binding, named_2: binding] 230 231 Named bindings can also be used for late binding with the `as/1` 232 construct, allowing you to refer to a binding that has not been 233 defined yet: 234 235 from c in Comment, where: as(:posts).id == c.post_id 236 237 This is especially useful when working with subqueries, where you 238 may need to refer to a parent binding with `parent_as`, which is 239 not known when writing the subquery: 240 241 child_query = from c in Comment, where: parent_as(:posts).id == c.post_id 242 from p in Post, as: :posts, inner_lateral_join: c in subquery(child_query) 243 244 You can also match on a specific binding when building queries. For 245 example, let's suppose you want to create a generic sort function 246 that will order by a given `field` with a given `as` in `query`: 247 248 # Knowing the name of the binding 249 def sort(query, as, field) do 250 from [{^as, x}] in query, order_by: field(x, ^field) 251 end 252 253 ### Bindingless operations 254 255 Although bindings are extremely useful when working with joins, 256 they are not necessary when the query has only the `from` clause. 257 For such cases, Ecto supports a way for building queries 258 without specifying the binding: 259 260 from Post, 261 where: [category: "fresh and new"], 262 order_by: [desc: :published_at], 263 select: [:id, :title, :body] 264 265 The query above will select all posts with category "fresh and new", 266 order by the most recently published, and return Post structs with 267 only the id, title and body fields set. It is equivalent to: 268 269 from p in Post, 270 where: p.category == "fresh and new", 271 order_by: [desc: p.published_at], 272 select: struct(p, [:id, :title, :body]) 273 274 One advantage of bindingless queries is that they are data-driven 275 and therefore useful for dynamically building queries. For example, 276 the query above could also be written as: 277 278 where = [category: "fresh and new"] 279 order_by = [desc: :published_at] 280 select = [:id, :title, :body] 281 from Post, where: ^where, order_by: ^order_by, select: ^select 282 283 This feature is very useful when queries need to be built based 284 on some user input, like web search forms, CLIs and so on. 285 286 ## Fragments 287 288 If you need an escape hatch, Ecto provides fragments 289 (see `Ecto.Query.API.fragment/1`) to inject SQL (and non-SQL) 290 fragments into queries. 291 292 For example, to get all posts while running the "lower(?)" 293 function in the database where `p.title` is interpolated 294 in place of `?`, one can write: 295 296 from p in Post, 297 where: is_nil(p.published_at) and 298 fragment("lower(?)", p.title) == ^title 299 300 Also, most adapters provide direct APIs for queries, like 301 `Ecto.Adapters.SQL.query/4`, allowing developers to 302 completely bypass Ecto queries. 303 304 ## Macro API 305 306 In all examples so far we have used the **keywords query syntax** to 307 create a query: 308 309 import Ecto.Query 310 from u in "users", where: u.age > 18, select: u.name 311 312 Due to the prevalence of the pipe operator in Elixir, Ecto also supports 313 a pipe-based syntax: 314 315 "users" 316 |> where([u], u.age > 18) 317 |> select([u], u.name) 318 319 The keyword-based and pipe-based examples are equivalent. The downside 320 of using macros is that the binding must be specified for every operation. 321 However, since keyword-based and pipe-based examples are equivalent, the 322 bindingless syntax also works for macros: 323 324 "users" 325 |> where([u], u.age > 18) 326 |> select([:name]) 327 328 Such a syntax allows developers to write queries using bindings only in more 329 complex query expressions. 330 331 This module documents each of those macros, providing examples in 332 both the keywords query and pipe expression formats. 333 334 ## Query prefix 335 336 It is possible to set a prefix for the queries. For Postgres users, 337 this will specify the schema where the table is located, while for 338 MySQL users this will specify the database where the table is 339 located. When no prefix is set, Postgres queries are assumed to be 340 in the public schema, while MySQL queries are assumed to be in the 341 database set in the config for the repo. 342 343 The query prefix may be set either for the whole query or on each 344 individual `from` and `join` expression. If a `prefix` is not given 345 to a `from` or a `join`, the prefix of the schema given to the `from` 346 or `join` is used. The query prefix is used only if none of the above 347 are declared. 348 349 Let's see some examples. To see the query prefix globally, the simplest 350 mechanism is to pass an option to the repository operation: 351 352 results = Repo.all(query, prefix: "accounts") 353 354 You may also set the prefix for the whole query by setting the prefix field: 355 356 results = 357 query # May be User or an Ecto.Query itself 358 |> Ecto.Query.put_query_prefix("accounts") 359 |> Repo.all() 360 361 Setting the prefix in the query changes the default prefix of all `from` 362 and `join` expressions. You can override the query prefix by either setting 363 the `@schema_prefix` in your schema definitions or by passing the prefix 364 option: 365 366 from u in User, 367 prefix: "accounts", 368 join: p in assoc(u, :posts), 369 prefix: "public" 370 371 Overall, here is the prefix lookup precedence: 372 373 1. The `:prefix` option given to `from`/`join` has the highest precedence 374 2. Then it falls back to the `@schema_prefix` attribute declared in the schema 375 given to `from`/`join` 376 3. Then it falls back to the query prefix 377 378 The prefixes set in the query will be preserved when loading data. 379 """ 380 381 defstruct [prefix: nil, sources: nil, from: nil, joins: [], aliases: %{}, wheres: [], select: nil, 382 order_bys: [], limit: nil, offset: nil, group_bys: [], combinations: [], updates: [], 383 havings: [], preloads: [], assocs: [], distinct: nil, lock: nil, windows: [], 384 with_ctes: nil] 385 386 defmodule FromExpr do 387 @moduledoc false 388 defstruct [:source, :file, :line, :as, :prefix, params: [], hints: []] 389 end 390 391 defmodule DynamicExpr do 392 @moduledoc false 393 defstruct [:fun, :binding, :file, :line] 394 end 395 396 defmodule QueryExpr do 397 @moduledoc false 398 defstruct [:expr, :file, :line, params: []] 399 end 400 401 defmodule BooleanExpr do 402 @moduledoc false 403 defstruct [:op, :expr, :file, :line, params: [], subqueries: []] 404 end 405 406 defmodule SelectExpr do 407 @moduledoc false 408 defstruct [:expr, :file, :line, :fields, params: [], take: %{}, subqueries: [], aliases: %{}] 409 end 410 411 defmodule JoinExpr do 412 @moduledoc false 413 defstruct [:qual, :source, :on, :file, :line, :assoc, :as, :ix, :prefix, params: [], hints: []] 414 end 415 416 defmodule WithExpr do 417 @moduledoc false 418 defstruct [recursive: false, queries: []] 419 end 420 421 defmodule Tagged do 422 @moduledoc false 423 # * value is the tagged value 424 # * tag is the directly tagged value, like Ecto.UUID 425 # * type is the underlying tag type, like :string 426 defstruct [:tag, :type, :value] 427 end 428 429 @type t :: %__MODULE__{} 430 @opaque dynamic :: %DynamicExpr{} 431 432 alias Ecto.Query.Builder 433 434 @doc """ 435 Builds a dynamic query expression. 436 437 Dynamic query expressions allow developers to compose query 438 expressions bit by bit, so that they can be interpolated into 439 parts of a query or another dynamic expression later on. 440 441 ## Examples 442 443 Imagine you have a set of conditions you want to build your query on: 444 445 conditions = false 446 447 conditions = 448 if params["is_public"] do 449 dynamic([p], p.is_public or ^conditions) 450 else 451 conditions 452 end 453 454 conditions = 455 if params["allow_reviewers"] do 456 dynamic([p, a], a.reviewer == true or ^conditions) 457 else 458 conditions 459 end 460 461 from query, where: ^conditions 462 463 In the example above, we were able to build the query expressions 464 bit by bit, using different bindings, and later interpolate it all 465 at once into the actual query. 466 467 A dynamic expression can always be interpolated inside another dynamic 468 expression and into the constructs described below. 469 470 ## `where`, `having` and a `join`'s `on` 471 472 The `dynamic` macro can be interpolated at the root of a `where`, 473 `having` or a `join`'s `on`. 474 475 For example, assuming the `conditions` variable defined in the 476 previous section, the following is forbidden because it is not 477 at the root of a `where`: 478 479 from q in query, where: q.some_condition and ^conditions 480 481 Fortunately that's easily solved by simply rewriting it to: 482 483 conditions = dynamic([q], q.some_condition and ^conditions) 484 from query, where: ^conditions 485 486 ## `order_by` 487 488 Dynamics can be interpolated inside keyword lists at the root of 489 `order_by`. For example, you can write: 490 491 order_by = [ 492 asc: :some_field, 493 desc: dynamic([p], fragment("?->>?", p.another_field, "json_key")) 494 ] 495 496 from query, order_by: ^order_by 497 498 Dynamics are also supported in `order_by/2` clauses inside `windows/2`. 499 500 As with `where` and friends, it is not possible to pass dynamics 501 outside of a root. For example, this won't work: 502 503 from query, order_by: [asc: ^dynamic(...)] 504 505 But this will: 506 507 from query, order_by: ^[asc: dynamic(...)] 508 509 ## `group_by` 510 511 Dynamics can be interpolated inside keyword lists at the root of 512 `group_by`. For example, you can write: 513 514 group_by = [ 515 :some_field, 516 dynamic([p], fragment("?->>?", p.another_field, "json_key")) 517 ] 518 519 from query, group_by: ^group_by 520 521 Dynamics are also supported in `partition_by/2` clauses inside `windows/2`. 522 523 As with `where` and friends, it is not possible to pass dynamics 524 outside of a root. For example, this won't work: 525 526 from query, group_by: [:some_field, ^dynamic(...)] 527 528 But this will: 529 530 from query, group_by: ^[:some_field, dynamic(...)] 531 532 ## `select` and `select_merge` 533 534 Dynamics can be inside maps interpolated at the root of a 535 `select` or `select_merge`. For example, you can write: 536 537 fields = %{ 538 period: dynamic([p], p.month), 539 metric: dynamic([p], p.distance) 540 } 541 542 from query, select: ^fields 543 544 As with `where` and friends, it is not possible to pass dynamics 545 outside of a root. For example, this won't work: 546 547 from query, select: %{field: ^dynamic(...)} 548 549 But this will: 550 551 from query, select: ^%{field: dynamic(...)} 552 553 Maps with dynamics can also be merged into existing `select` structures, 554 enabling a variety of possibilities for partially dynamic selects: 555 556 metric = dynamic([p], p.distance) 557 558 from query, select: [:period, :metric], select_merge: ^%{metric: metric} 559 560 Aliasing fields with `selected_as/2` and referencing them with `selected_as/1` 561 is also allowed: 562 563 fields = %{ 564 period: dynamic([p], selected_as(p.month, :month)), 565 metric: dynamic([p], p.distance) 566 } 567 568 order = dynamic(selected_as(:month)) 569 570 from query, select: ^fields, order_by: ^order 571 572 ## Updates 573 574 A `dynamic` is also supported inside updates, for example: 575 576 updates = [ 577 set: [average: dynamic([p], p.sum / p.count)] 578 ] 579 580 from query, update: ^updates 581 """ 582 defmacro dynamic(binding \\ [], expr) do 583 Builder.Dynamic.build(binding, expr, __CALLER__) 584 end 585 586 @doc """ 587 Defines windows which can be used with `Ecto.Query.WindowAPI`. 588 589 Receives a keyword list where keys are names of the windows 590 and values are a keyword list with window expressions. 591 592 ## Examples 593 594 # Compare each employee's salary with the average salary in his or her department 595 from e in Employee, 596 select: {e.depname, e.empno, e.salary, over(avg(e.salary), :department)}, 597 windows: [department: [partition_by: e.depname]] 598 599 In the example above, we get the average salary per department. 600 `:department` is the window name, partitioned by `e.depname` 601 and `avg/1` is the window function. For more information 602 on windows functions, see `Ecto.Query.WindowAPI`. 603 604 ## Window expressions 605 606 The following keys are allowed when specifying a window. 607 608 ### :partition_by 609 610 A list of fields to partition the window by, for example: 611 612 windows: [department: [partition_by: e.depname]] 613 614 A list of atoms can also be interpolated for dynamic partitioning: 615 616 fields = [:depname, :year] 617 windows: [dynamic_window: [partition_by: ^fields]] 618 619 ### :order_by 620 621 A list of fields to order the window by, for example: 622 623 windows: [ordered_names: [order_by: e.name]] 624 625 It works exactly as the keyword query version of `order_by/3`. 626 627 ### :frame 628 629 A fragment which defines the frame for window functions. 630 631 ## Examples 632 633 # Compare each employee's salary for each month with his average salary for previous 3 months 634 from p in Payroll, 635 select: {p.empno, p.date, p.salary, over(avg(p.salary), :prev_months)}, 636 windows: [prev_months: [partition_by: p.empno, order_by: p.date, frame: fragment("ROWS 3 PRECEDING EXCLUDE CURRENT ROW")]] 637 638 """ 639 defmacro windows(query, binding \\ [], expr) do 640 Builder.Windows.build(query, binding, expr, __CALLER__) 641 end 642 643 @doc """ 644 Converts a query into a subquery. 645 646 If a subquery is given, returns the subquery itself. 647 If any other value is given, it is converted to a query via 648 `Ecto.Queryable` and wrapped in the `Ecto.SubQuery` struct. 649 650 `subquery` is supported in `from`, `join`, and `where`, in the 651 form `p.x in subquery(q)`. 652 653 ## Examples 654 655 # Get the average salary of the top 10 highest salaries 656 query = from Employee, order_by: [desc: :salary], limit: 10 657 from e in subquery(query), select: avg(e.salary) 658 659 A prefix can be specified for a subquery, similar to standard repo operations: 660 661 query = from Employee, order_by: [desc: :salary], limit: 10 662 from e in subquery(query, prefix: "my_prefix"), select: avg(e.salary) 663 664 665 Subquery can also be used in a `join` expression. 666 667 UPDATE posts 668 SET sync_started_at = $1 669 WHERE id IN ( 670 SELECT id FROM posts 671 WHERE synced = false AND (sync_started_at IS NULL OR sync_started_at < $1) 672 LIMIT $2 673 ) 674 675 We can write it as a join expression: 676 677 subset = from(p in Post, 678 where: p.synced == false and 679 (is_nil(p.sync_started_at) or p.sync_started_at < ^min_sync_started_at), 680 limit: ^batch_size 681 ) 682 683 Repo.update_all( 684 from(p in Post, join: s in subquery(subset), on: s.id == p.id), 685 set: [sync_started_at: NaiveDateTime.utc_now()] 686 ) 687 688 Or as a `where` condition: 689 690 subset_ids = from(p in subset, select: p.id) 691 Repo.update_all( 692 from(p in Post, where: p.id in subquery(subset_ids)), 693 set: [sync_started_at: NaiveDateTime.utc_now()] 694 ) 695 696 If you need to refer to a parent binding which is not known when writing the subquery, 697 you can use `parent_as` as shown in the examples under "Named bindings" in this module doc. 698 """ 699 def subquery(query, opts \\ []) do 700 subquery = wrap_in_subquery(query) 701 case Keyword.fetch(opts, :prefix) do 702 {:ok, prefix} when is_binary(prefix) or is_nil(prefix) -> put_in(subquery.query.prefix, prefix) 703 :error -> subquery 704 end 705 end 706 707 defp wrap_in_subquery(%Ecto.SubQuery{} = subquery), do: subquery 708 defp wrap_in_subquery(%Ecto.Query{} = query), do: %Ecto.SubQuery{query: query} 709 defp wrap_in_subquery(queryable), do: %Ecto.SubQuery{query: Ecto.Queryable.to_query(queryable)} 710 711 @joins [:join, :inner_join, :cross_join, :left_join, :right_join, :full_join, 712 :inner_lateral_join, :left_lateral_join] 713 714 @doc """ 715 Puts the given prefix in a query. 716 """ 717 def put_query_prefix(%Ecto.Query{} = query, prefix) when is_binary(prefix) do 718 %{query | prefix: prefix} 719 end 720 721 def put_query_prefix(other, prefix) when is_binary(prefix) do 722 other |> Ecto.Queryable.to_query() |> put_query_prefix(prefix) 723 end 724 725 @doc """ 726 Resets a previously set field on a query. 727 728 It can reset many fields except the query source (`from`). When excluding 729 a `:join`, it will remove *all* types of joins. If you prefer to remove a 730 single type of join, please see paragraph below. 731 732 ## Examples 733 734 Ecto.Query.exclude(query, :join) 735 Ecto.Query.exclude(query, :where) 736 Ecto.Query.exclude(query, :order_by) 737 Ecto.Query.exclude(query, :group_by) 738 Ecto.Query.exclude(query, :having) 739 Ecto.Query.exclude(query, :distinct) 740 Ecto.Query.exclude(query, :select) 741 Ecto.Query.exclude(query, :combinations) 742 Ecto.Query.exclude(query, :with_ctes) 743 Ecto.Query.exclude(query, :limit) 744 Ecto.Query.exclude(query, :offset) 745 Ecto.Query.exclude(query, :lock) 746 Ecto.Query.exclude(query, :preload) 747 748 You can also remove specific joins as well such as `left_join` and 749 `inner_join`: 750 751 Ecto.Query.exclude(query, :inner_join) 752 Ecto.Query.exclude(query, :cross_join) 753 Ecto.Query.exclude(query, :left_join) 754 Ecto.Query.exclude(query, :right_join) 755 Ecto.Query.exclude(query, :full_join) 756 Ecto.Query.exclude(query, :inner_lateral_join) 757 Ecto.Query.exclude(query, :left_lateral_join) 758 759 However, keep in mind that if a join is removed and its bindings 760 were referenced elsewhere, the bindings won't be removed, leading 761 to a query that won't compile. 762 """ 763 def exclude(%Ecto.Query{} = query, field), do: do_exclude(query, field) 764 def exclude(query, field), do: do_exclude(Ecto.Queryable.to_query(query), field) 765 766 defp do_exclude(%Ecto.Query{} = query, :join) do 767 %{query | joins: [], aliases: Map.take(query.aliases, [query.from.as])} 768 end 769 defp do_exclude(%Ecto.Query{} = query, join_keyword) when join_keyword in @joins do 770 qual = join_qual(join_keyword) 771 {excluded, remaining} = Enum.split_with(query.joins, &(&1.qual == qual)) 772 aliases = Map.drop(query.aliases, Enum.map(excluded, & &1.as)) 773 %{query | joins: remaining, aliases: aliases} 774 end 775 defp do_exclude(%Ecto.Query{} = query, :where), do: %{query | wheres: []} 776 defp do_exclude(%Ecto.Query{} = query, :order_by), do: %{query | order_bys: []} 777 defp do_exclude(%Ecto.Query{} = query, :group_by), do: %{query | group_bys: []} 778 defp do_exclude(%Ecto.Query{} = query, :combinations), do: %{query | combinations: []} 779 defp do_exclude(%Ecto.Query{} = query, :with_ctes), do: %{query | with_ctes: nil} 780 defp do_exclude(%Ecto.Query{} = query, :having), do: %{query | havings: []} 781 defp do_exclude(%Ecto.Query{} = query, :distinct), do: %{query | distinct: nil} 782 defp do_exclude(%Ecto.Query{} = query, :select), do: %{query | select: nil} 783 defp do_exclude(%Ecto.Query{} = query, :limit), do: %{query | limit: nil} 784 defp do_exclude(%Ecto.Query{} = query, :offset), do: %{query | offset: nil} 785 defp do_exclude(%Ecto.Query{} = query, :lock), do: %{query | lock: nil} 786 defp do_exclude(%Ecto.Query{} = query, :preload), do: %{query | preloads: [], assocs: []} 787 788 @doc """ 789 Creates a query. 790 791 It can either be a keyword query or a query expression. 792 793 If it is a keyword query the first argument must be 794 either an `in` expression, a value that implements 795 the `Ecto.Queryable` protocol, or an `Ecto.Query.API.fragment/1`. If the query needs a 796 reference to the data source in any other part of the 797 expression, then an `in` must be used to create a reference 798 variable. The second argument should be a keyword query 799 where the keys are expression types and the values are 800 expressions. 801 802 If it is a query expression the first argument must be 803 a value that implements the `Ecto.Queryable` protocol 804 and the second argument the expression. 805 806 ## Keywords examples 807 808 # `in` expression 809 from(c in City, select: c) 810 811 # Ecto.Queryable 812 from(City, limit: 1) 813 814 # Fragment 815 from(f in fragment("generate_series(?, ?) as x", ^0, ^100000), select f.x) 816 817 ## Expressions examples 818 819 # Schema 820 City |> select([c], c) 821 822 # Source 823 "cities" |> select([c], c) 824 825 # Source with schema 826 {"cities", Source} |> select([c], c) 827 828 # Ecto.Query 829 from(c in Cities) |> select([c], c) 830 831 ## Examples 832 833 def paginate(query, page, size) do 834 from query, 835 limit: ^size, 836 offset: ^((page-1) * size) 837 end 838 839 The example above does not use `in` because `limit` and `offset` 840 do not require a reference to the data source. However, extending 841 the query with a where expression would require the use of `in`: 842 843 def published(query) do 844 from p in query, where: not(is_nil(p.published_at)) 845 end 846 847 Notice we have created a `p` variable to reference the query's 848 original data source. This assumes that the original query 849 only had one source. When the given query has more than one source, 850 positional or named bindings may be used to access the additional sources. 851 852 def published_multi(query) do 853 from [p,o] in query, 854 where: not(is_nil(p.published_at)) and not(is_nil(o.published_at)) 855 end 856 857 Note that the variables `p` and `o` can be named whatever you like 858 as they have no importance in the query sent to the database. 859 """ 860 defmacro from(expr, kw \\ []) do 861 unless Keyword.keyword?(kw) do 862 raise ArgumentError, "second argument to `from` must be a compile time keyword list" 863 end 864 865 {kw, as, prefix, hints} = collect_as_and_prefix_and_hints(kw, nil, nil, nil) 866 {quoted, binds, count_bind} = Builder.From.build(expr, __CALLER__, as, prefix, hints) 867 from(kw, __CALLER__, count_bind, quoted, to_query_binds(binds)) 868 end 869 870 @from_join_opts [:as, :prefix, :hints] 871 @no_binds [:union, :union_all, :except, :except_all, :intersect, :intersect_all] 872 @binds [:lock, :where, :or_where, :select, :distinct, :order_by, :group_by, :windows] ++ 873 [:having, :or_having, :limit, :offset, :preload, :update, :select_merge, :with_ctes] 874 875 defp from([{type, expr}|t], env, count_bind, quoted, binds) when type in @binds do 876 # If all bindings are integer indexes keep AST Macro expandable to %Query{}, 877 # otherwise ensure that quoted code is evaluated before macro call 878 quoted = 879 if Enum.all?(binds, fn {_, value} -> is_integer(value) end) do 880 quote do 881 Ecto.Query.unquote(type)(unquote(quoted), unquote(binds), unquote(expr)) 882 end 883 else 884 quote do 885 query = unquote(quoted) 886 Ecto.Query.unquote(type)(query, unquote(binds), unquote(expr)) 887 end 888 end 889 890 from(t, env, count_bind, quoted, binds) 891 end 892 893 defp from([{type, expr}|t], env, count_bind, quoted, binds) when type in @no_binds do 894 quoted = 895 quote do 896 Ecto.Query.unquote(type)(unquote(quoted), unquote(expr)) 897 end 898 899 from(t, env, count_bind, quoted, binds) 900 end 901 902 defp from([{join, expr}|t], env, count_bind, quoted, binds) when join in @joins do 903 qual = join_qual(join) 904 {t, on, as, prefix, hints} = collect_on(t, nil, nil, nil, nil) 905 906 {quoted, binds, count_bind} = 907 Builder.Join.build(quoted, qual, binds, expr, count_bind, on, as, prefix, hints, env) 908 909 from(t, env, count_bind, quoted, to_query_binds(binds)) 910 end 911 912 defp from([{:on, _value}|_], _env, _count_bind, _quoted, _binds) do 913 Builder.error! "`on` keyword must immediately follow a join" 914 end 915 916 defp from([{key, _value}|_], _env, _count_bind, _quoted, _binds) when key in @from_join_opts do 917 Builder.error! "`#{key}` keyword must immediately follow a from/join" 918 end 919 920 defp from([{key, _value}|_], _env, _count_bind, _quoted, _binds) do 921 Builder.error! "unsupported #{inspect key} in keyword query expression" 922 end 923 924 defp from([], _env, _count_bind, quoted, _binds) do 925 quoted 926 end 927 928 defp to_query_binds(binds) do 929 for {k, v} <- binds, do: {{k, [], nil}, v} 930 end 931 932 defp join_qual(:join), do: :inner 933 defp join_qual(:full_join), do: :full 934 defp join_qual(:left_join), do: :left 935 defp join_qual(:right_join), do: :right 936 defp join_qual(:inner_join), do: :inner 937 defp join_qual(:cross_join), do: :cross 938 defp join_qual(:left_lateral_join), do: :left_lateral 939 defp join_qual(:inner_lateral_join), do: :inner_lateral 940 941 defp collect_on([{key, _} | _] = t, on, as, prefix, hints) when key in @from_join_opts do 942 {t, as, prefix, hints} = collect_as_and_prefix_and_hints(t, as, prefix, hints) 943 collect_on(t, on, as, prefix, hints) 944 end 945 946 defp collect_on([{:on, on} | t], nil, as, prefix, hints), 947 do: collect_on(t, on, as, prefix, hints) 948 defp collect_on([{:on, expr} | t], on, as, prefix, hints), 949 do: collect_on(t, {:and, [], [on, expr]}, as, prefix, hints) 950 defp collect_on(t, on, as, prefix, hints), 951 do: {t, on, as, prefix, hints} 952 953 defp collect_as_and_prefix_and_hints([{:as, as} | t], nil, prefix, hints), 954 do: collect_as_and_prefix_and_hints(t, as, prefix, hints) 955 defp collect_as_and_prefix_and_hints([{:as, _} | _], _, _, _), 956 do: Builder.error! "`as` keyword was given more than once to the same from/join" 957 defp collect_as_and_prefix_and_hints([{:prefix, prefix} | t], as, nil, hints), 958 do: collect_as_and_prefix_and_hints(t, as, {:ok, prefix}, hints) 959 defp collect_as_and_prefix_and_hints([{:prefix, _} | _], _, _, _), 960 do: Builder.error! "`prefix` keyword was given more than once to the same from/join" 961 defp collect_as_and_prefix_and_hints([{:hints, hints} | t], as, prefix, nil), 962 do: collect_as_and_prefix_and_hints(t, as, prefix, hints) 963 defp collect_as_and_prefix_and_hints([{:hints, _} | _], _, _, _), 964 do: Builder.error! "`hints` keyword was given more than once to the same from/join" 965 defp collect_as_and_prefix_and_hints(t, as, prefix, hints), 966 do: {t, as, prefix, hints} 967 968 @doc """ 969 A join query expression. 970 971 Receives a source that is to be joined to the query and a condition for 972 the join. The join condition can be any expression that evaluates 973 to a boolean value. The qualifier must be one of `:inner`, `:left`, 974 `:right`, `:cross`, `:full`, `:inner_lateral` or `:left_lateral`. 975 976 For a keyword query the `:join` keyword can be changed to `:inner_join`, 977 `:left_join`, `:right_join`, `:cross_join`, `:full_join`, `:inner_lateral_join` 978 or `:left_lateral_join`. `:join` is equivalent to `:inner_join`. 979 980 Currently it is possible to join on: 981 982 * an `Ecto.Schema`, such as `p in Post` 983 * an interpolated Ecto query with zero or more `where` clauses, 984 such as `c in ^(from "posts", where: [public: true])` 985 * an association, such as `c in assoc(post, :comments)` 986 * a subquery, such as `c in subquery(another_query)` 987 * a query fragment, such as `c in fragment("SOME COMPLEX QUERY")`, 988 see "Joining with fragments" below. 989 990 ## Options 991 992 Each join accepts the following options: 993 994 * `:on` - a query expression or keyword list to filter the join 995 * `:as` - a named binding for the join 996 * `:prefix` - the prefix to be used for the join when issuing a database query 997 * `:hints` - a string or a list of strings to be used as database hints 998 999 In the keyword query syntax, those options must be given immediately 1000 after the join. In the expression syntax, the options are given as 1001 the fifth argument. 1002 1003 ## Keywords examples 1004 1005 from c in Comment, 1006 join: p in Post, 1007 on: p.id == c.post_id, 1008 select: {p.title, c.text} 1009 1010 from p in Post, 1011 left_join: c in assoc(p, :comments), 1012 select: {p, c} 1013 1014 Keywords can also be given or interpolated as part of `on`: 1015 1016 from c in Comment, 1017 join: p in Post, 1018 on: [id: c.post_id], 1019 select: {p.title, c.text} 1020 1021 Any key in `on` will apply to the currently joined expression. 1022 1023 It is also possible to interpolate an Ecto query on the right-hand side 1024 of `in`. For example, the query above can also be written as: 1025 1026 posts = Post 1027 from c in Comment, 1028 join: p in ^posts, 1029 on: [id: c.post_id], 1030 select: {p.title, c.text} 1031 1032 The above is specially useful to dynamically join on existing 1033 queries, for example, to dynamically choose a source, or by 1034 choosing between public posts or posts that have been recently 1035 published: 1036 1037 posts = 1038 if params["drafts"] do 1039 from p in Post, where: [drafts: true] 1040 else 1041 from p in Post, where: [public: true] 1042 end 1043 1044 from c in Comment, 1045 join: p in ^posts, on: [id: c.post_id], 1046 select: {p.title, c.text} 1047 1048 Only simple queries with `where` expressions can be interpolated 1049 in a join. 1050 1051 ## Expressions examples 1052 1053 Comment 1054 |> join(:inner, [c], p in Post, on: c.post_id == p.id) 1055 |> select([c, p], {p.title, c.text}) 1056 1057 Post 1058 |> join(:left, [p], c in assoc(p, :comments)) 1059 |> select([p, c], {p, c}) 1060 1061 Post 1062 |> join(:left, [p], c in Comment, on: c.post_id == p.id and c.is_visible == true) 1063 |> select([p, c], {p, c}) 1064 1065 ## Joining with fragments 1066 1067 When you need to join on a complex query, Ecto supports fragments in joins: 1068 1069 Comment 1070 |> join(:inner, [c], p in fragment("SOME COMPLEX QUERY", c.id, ^some_param)) 1071 1072 Although using fragments in joins is discouraged in favor of Ecto 1073 Query syntax, they are necessary when writing lateral joins as 1074 lateral joins require a subquery that refer to previous bindings: 1075 1076 Game 1077 |> join(:inner_lateral, [g], gs in fragment("SELECT * FROM games_sold AS gs WHERE gs.game_id = ? ORDER BY gs.sold_on LIMIT 2", g.id)) 1078 |> select([g, gs], {g.name, gs.sold_on}) 1079 1080 Note that the `join` does not automatically wrap the fragment in 1081 parentheses, since some expressions require parens and others 1082 require no parens. Therefore, in cases such as common table 1083 expressions, you will have to explicitly wrap the fragment content 1084 in parens. 1085 1086 ## Hints 1087 1088 `from` and `join` also support index hints, as found in databases such as 1089 [MySQL](https://dev.mysql.com/doc/refman/8.0/en/index-hints.html), 1090 [MSSQL](https://docs.microsoft.com/en-us/sql/t-sql/queries/hints-transact-sql-table?view=sql-server-2017) and 1091 [Clickhouse](https://clickhouse.tech/docs/en/sql-reference/statements/select/sample/). 1092 1093 For example, a developer using MySQL may write: 1094 1095 from p in Post, 1096 join: c in Comment, 1097 hints: ["USE INDEX FOO", "USE INDEX BAR"], 1098 where: p.id == c.post_id, 1099 select: c 1100 1101 Keep in mind you want to use hints rarely, so don't forget to read the database 1102 disclaimers about such functionality. 1103 1104 Hints must be static compile-time strings when they are specified as (list of) strings. 1105 Certain Ecto adapters may also accept dynamic hints using the tuple form: 1106 1107 from e in Event, 1108 hints: [sample: sample_threshold()], 1109 select: e 1110 1111 """ 1112 @join_opts [:on | @from_join_opts] 1113 1114 defmacro join(query, qual, binding \\ [], expr, opts \\ []) 1115 defmacro join(query, qual, binding, expr, opts) 1116 when is_list(binding) and is_list(opts) do 1117 {t, on, as, prefix, hints} = collect_on(opts, nil, nil, nil, nil) 1118 1119 with [{key, _} | _] <- t do 1120 raise ArgumentError, "invalid option `#{key}` passed to Ecto.Query.join/5, " <> 1121 "valid options are: #{inspect(@join_opts)}" 1122 end 1123 1124 query 1125 |> Builder.Join.build(qual, binding, expr, nil, on, as, prefix, hints, __CALLER__) 1126 |> elem(0) 1127 end 1128 1129 defmacro join(_query, _qual, binding, _expr, opts) when is_list(opts) do 1130 raise ArgumentError, "invalid binding passed to Ecto.Query.join/5, should be " <> 1131 "list of variables, got: #{Macro.to_string(binding)}" 1132 end 1133 1134 defmacro join(_query, _qual, _binding, _expr, opts) do 1135 raise ArgumentError, "invalid opts passed to Ecto.Query.join/5, should be " <> 1136 "list, got: #{Macro.to_string(opts)}" 1137 end 1138 1139 @doc ~S''' 1140 A common table expression (CTE) also known as WITH expression. 1141 1142 `name` must be a compile-time literal string that is being used 1143 as the table name to join the CTE in the main query or in the 1144 recursive CTE. 1145 1146 **IMPORTANT!** Beware of using CTEs. In raw SQL, CTEs can be 1147 used as a mechanism to organize queries, but said mechanism 1148 has no purpose in Ecto since Ecto queries are composable by 1149 definition. In other words, if you need to break a large query 1150 into parts, use all of the functionality in Elixir and in this 1151 module to structure your code. Furthermore, breaking a query 1152 into CTEs can negatively impact performance, as the database 1153 may not optimize efficiently across CTEs. The main use case 1154 for CTEs in Ecto is to provide recursive definitions, which 1155 we outline in the following section. Non-recursive CTEs can 1156 often be written as joins or subqueries, which provide better 1157 performance. 1158 1159 ## Options 1160 1161 * `:as` - the CTE query itself or a fragment 1162 1163 ## Recursive CTEs 1164 1165 Use `recursive_ctes/2` to enable recursive mode for CTEs. 1166 1167 In the CTE query itself use the same table name to leverage 1168 recursion that has been passed to the `name` argument. Make sure 1169 to write a stop condition to avoid an infinite recursion loop. 1170 Generally speaking, you should only use CTEs in Ecto for 1171 writing recursive queries. 1172 1173 ## Expression examples 1174 1175 Products and their category names for breadcrumbs: 1176 1177 category_tree_initial_query = 1178 Category 1179 |> where([c], is_nil(c.parent_id)) 1180 1181 category_tree_recursion_query = 1182 Category 1183 |> join(:inner, [c], ct in "category_tree", on: c.parent_id == ct.id) 1184 1185 category_tree_query = 1186 category_tree_initial_query 1187 |> union_all(^category_tree_recursion_query) 1188 1189 Product 1190 |> recursive_ctes(true) 1191 |> with_cte("category_tree", as: ^category_tree_query) 1192 |> join(:left, [p], c in "category_tree", on: c.id == p.category_id) 1193 |> group_by([p], p.id) 1194 |> select([p, c], %{p | category_names: fragment("ARRAY_AGG(?)", c.name)}) 1195 1196 It's also possible to pass a raw SQL fragment: 1197 1198 @raw_sql_category_tree """ 1199 SELECT * FROM categories WHERE c.parent_id IS NULL 1200 UNION ALL 1201 SELECT * FROM categories AS c, category_tree AS ct WHERE ct.id = c.parent_id 1202 """ 1203 1204 Product 1205 |> recursive_ctes(true) 1206 |> with_cte("category_tree", as: fragment(@raw_sql_category_tree)) 1207 |> join(:inner, [p], c in "category_tree", on: c.id == p.category_id) 1208 1209 If you don't have any Ecto schema pointing to the CTE table, you can pass a 1210 tuple with the CTE table name as the first element and an Ecto schema as the second 1211 element. This will cast the result rows to Ecto structs as long as the Ecto 1212 schema maps to the same fields in the CTE table: 1213 1214 {"category_tree", Category} 1215 |> recursive_ctes(true) 1216 |> with_cte("category_tree", as: ^category_tree_query) 1217 |> join(:left, [c], p in assoc(c, :products)) 1218 |> group_by([c], c.id) 1219 |> select([c, p], %{c | products_count: count(p.id)}) 1220 1221 Keyword syntax is not supported for this feature. 1222 1223 ## Limitation: CTEs on schemas with source fields 1224 1225 Ecto allows developers to say that a table in their Ecto schema 1226 maps to a different column in their database: 1227 1228 field :group_id, :integer, source: :iGroupId 1229 1230 At the moment, using a schema with source fields in CTE may emit 1231 invalid queries. If you are running into such scenarios, your best 1232 option is to use a fragment as your CTE. 1233 ''' 1234 defmacro with_cte(query, name, as: with_query) do 1235 Builder.CTE.build(query, name, with_query, __CALLER__) 1236 end 1237 1238 @doc """ 1239 Enables or disables recursive mode for CTEs. 1240 1241 According to the SQL standard it affects all CTEs in the query, not individual ones. 1242 1243 See `with_cte/3` on example of how to build a query with a recursive CTE. 1244 """ 1245 def recursive_ctes(%__MODULE__{with_ctes: with_expr} = query, value) when is_boolean(value) do 1246 with_expr = with_expr || %WithExpr{} 1247 with_expr = %{with_expr | recursive: value} 1248 %{query | with_ctes: with_expr} 1249 end 1250 1251 def recursive_ctes(queryable, value) do 1252 recursive_ctes(Ecto.Queryable.to_query(queryable), value) 1253 end 1254 1255 @doc """ 1256 A select query expression. 1257 1258 Selects which fields will be selected from the schema and any transformations 1259 that should be performed on the fields. Any expression that is accepted in a 1260 query can be a select field. 1261 1262 Select also allows each expression to be wrapped in lists, tuples or maps as 1263 shown in the examples below. A full schema can also be selected. 1264 1265 There can only be one select expression in a query, if the select expression 1266 is omitted, the query will by default select the full schema. If `select` is 1267 given more than once, an error is raised. Use `exclude/2` if you would like 1268 to remove a previous select for overriding or see `select_merge/3` for a 1269 limited version of `select` that is composable and can be called multiple 1270 times. 1271 1272 `select` also accepts a list of atoms where each atom refers to a field in 1273 the source to be selected. 1274 1275 ## Keywords examples 1276 1277 from(c in City, select: c) # returns the schema as a struct 1278 from(c in City, select: {c.name, c.population}) 1279 from(c in City, select: [c.name, c.county]) 1280 from(c in City, select: %{n: c.name, answer: 42}) 1281 from(c in City, select: %{c | alternative_name: c.name}) 1282 from(c in City, select: %Data{name: c.name}) 1283 1284 It is also possible to select a struct and limit the returned 1285 fields at the same time: 1286 1287 from(City, select: [:name]) 1288 1289 The syntax above is equivalent to: 1290 1291 from(city in City, select: struct(city, [:name])) 1292 1293 You can also write: 1294 1295 from(city in City, select: map(city, [:name])) 1296 1297 If you want a map with only the selected fields to be returned. 1298 1299 To select a struct but omit only given fields, you can 1300 override them with `nil` or another default value: 1301 1302 from(city in City, select: %{city | geojson: nil, text: "<redacted>"}) 1303 1304 For more information, read the docs for `Ecto.Query.API.struct/2` 1305 and `Ecto.Query.API.map/2`. 1306 1307 ## Expressions examples 1308 1309 City |> select([c], c) 1310 City |> select([c], {c.name, c.country}) 1311 City |> select([c], %{"name" => c.name}) 1312 City |> select([:name]) 1313 City |> select([c], struct(c, [:name])) 1314 City |> select([c], map(c, [:name])) 1315 City |> select([c], %{c | geojson: nil, text: "<redacted>"}) 1316 1317 ## Dynamic parts 1318 1319 Dynamics can be part of a `select` as values in a map that must be interpolated 1320 at the root level: 1321 1322 period = if monthly?, do: dynamic([p], p.month), else: dynamic([p], p.date) 1323 metric = if distance?, do: dynamic([p], p.distance), else: dynamic([p], p.time) 1324 1325 from(c in City, select: ^%{period: period, metric: metric}) 1326 """ 1327 defmacro select(query, binding \\ [], expr) do 1328 Builder.Select.build(:select, query, binding, expr, __CALLER__) 1329 end 1330 1331 @doc """ 1332 Mergeable select query expression. 1333 1334 This macro is similar to `select/3` except it may be specified 1335 multiple times as long as every entry is a map. This is useful 1336 for merging and composing selects. For example: 1337 1338 query = from p in Post, select: %{} 1339 1340 query = 1341 if include_title? do 1342 from p in query, select_merge: %{title: p.title} 1343 else 1344 query 1345 end 1346 1347 query = 1348 if include_visits? do 1349 from p in query, select_merge: %{visits: p.visits} 1350 else 1351 query 1352 end 1353 1354 In the example above, the query is built little by little by merging 1355 into a final map. If both conditions above are true, the final query 1356 would be equivalent to: 1357 1358 from p in Post, select: %{title: p.title, visits: p.visits} 1359 1360 If `:select_merge` is called and there is no value selected previously, 1361 it will default to the source, `p` in the example above. 1362 1363 The argument given to `:select_merge` must always be a map. The value 1364 being merged on must be a struct or a map. If it is a struct, the fields 1365 merged later on must be part of the struct, otherwise an error is raised. 1366 1367 If the argument to `:select_merge` is a constructed struct 1368 (`Ecto.Query.API.struct/2`) or map (`Ecto.Query.API.map/2`) where the source 1369 to struct or map may be a `nil` value (as in an outer join), the source will 1370 be returned unmodified. 1371 1372 query = 1373 Post 1374 |> join(:left, [p], t in Post.Translation, 1375 on: t.post_id == p.id and t.locale == ^"en" 1376 ) 1377 |> select_merge([_p, t], map(t, ^~w(title summary)a)) 1378 1379 If there is no English translation for the post, the untranslated post 1380 `title` will be returned and `summary` will be `nil`. If there is, both 1381 `title` and `summary` will be the value from `Post.Translation`. 1382 1383 `select_merge` cannot be used to set fields in associations, as 1384 associations are always loaded later, overriding any previous value. 1385 1386 Dynamics can be part of a `select_merge` as values in a map that must be 1387 interpolated at the root level. The rules for merging detailed above apply. 1388 This allows merging dynamic values into previsouly selected maps and structs. 1389 """ 1390 defmacro select_merge(query, binding \\ [], expr) do 1391 Builder.Select.build(:merge, query, binding, expr, __CALLER__) 1392 end 1393 1394 @doc """ 1395 A distinct query expression. 1396 1397 When true, only keeps distinct values from the resulting 1398 select expression. 1399 1400 If supported by your database, you can also pass query expressions 1401 to distinct and it will generate a query with DISTINCT ON. In such 1402 cases, `distinct` accepts exactly the same expressions as `order_by` 1403 and any `distinct` expression will be automatically prepended to the 1404 `order_by` expressions in case there is any `order_by` expression. 1405 1406 ## Keywords examples 1407 1408 # Returns the list of different categories in the Post schema 1409 from(p in Post, distinct: true, select: p.category) 1410 1411 # If your database supports DISTINCT ON(), 1412 # you can pass expressions to distinct too 1413 from(p in Post, 1414 distinct: p.category, 1415 order_by: [p.date]) 1416 1417 # The DISTINCT ON() also supports ordering similar to ORDER BY. 1418 from(p in Post, 1419 distinct: [desc: p.category], 1420 order_by: [p.date]) 1421 1422 # Using atoms 1423 from(p in Post, distinct: :category, order_by: :date) 1424 1425 ## Expressions example 1426 1427 Post 1428 |> distinct(true) 1429 |> order_by([p], [p.category, p.author]) 1430 1431 """ 1432 defmacro distinct(query, binding \\ [], expr) do 1433 Builder.Distinct.build(query, binding, expr, __CALLER__) 1434 end 1435 1436 @doc """ 1437 An AND where query expression. 1438 1439 `where` expressions are used to filter the result set. If there is more 1440 than one where expression, they are combined with an `and` operator. All 1441 where expressions have to evaluate to a boolean value. 1442 1443 `where` also accepts a keyword list where the field given as key is going to 1444 be compared with the given value. The fields will always refer to the source 1445 given in `from`. 1446 1447 ## Keywords example 1448 1449 from(c in City, where: c.country == "Sweden") 1450 from(c in City, where: [country: "Sweden"]) 1451 1452 It is also possible to interpolate the whole keyword list, allowing you to 1453 dynamically filter the source: 1454 1455 filters = [country: "Sweden"] 1456 from(c in City, where: ^filters) 1457 1458 ## Expressions examples 1459 1460 City |> where([c], c.country == "Sweden") 1461 City |> where(country: "Sweden") 1462 1463 """ 1464 defmacro where(query, binding \\ [], expr) do 1465 Builder.Filter.build(:where, :and, query, binding, expr, __CALLER__) 1466 end 1467 1468 @doc """ 1469 An OR where query expression. 1470 1471 Behaves exactly the same as `where` except it combines with any previous 1472 expression by using an `OR`. All expressions have to evaluate to a boolean 1473 value. 1474 1475 `or_where` also accepts a keyword list where each key is a field to be 1476 compared with the given value. Each key-value pair will be combined 1477 using `AND`, exactly as in `where`. 1478 1479 ## Keywords example 1480 1481 from(c in City, where: [country: "Sweden"], or_where: [country: "Brazil"]) 1482 1483 If interpolating keyword lists, the keyword list entries are combined 1484 using ANDs and joined to any existing expression with an OR: 1485 1486 filters = [country: "USA", name: "New York"] 1487 from(c in City, where: [country: "Sweden"], or_where: ^filters) 1488 1489 is equivalent to: 1490 1491 from c in City, where: (c.country == "Sweden") or 1492 (c.country == "USA" and c.name == "New York") 1493 1494 The behaviour above is by design to keep the changes between `where` 1495 and `or_where` minimal. Plus, if you have a keyword list and you 1496 would like each pair to be combined using `or`, it can be easily done 1497 with `Enum.reduce/3`: 1498 1499 filters = [country: "USA", is_tax_exempt: true] 1500 Enum.reduce(filters, City, fn {key, value}, query -> 1501 from q in query, or_where: field(q, ^key) == ^value 1502 end) 1503 1504 which will be equivalent to: 1505 1506 from c in City, or_where: (c.country == "USA"), or_where: c.is_tax_exempt == true 1507 1508 ## Expressions example 1509 1510 City |> where([c], c.country == "Sweden") |> or_where([c], c.country == "Brazil") 1511 1512 """ 1513 defmacro or_where(query, binding \\ [], expr) do 1514 Builder.Filter.build(:where, :or, query, binding, expr, __CALLER__) 1515 end 1516 1517 @doc """ 1518 An order by query expression. 1519 1520 Orders the fields based on one or more fields. It accepts a single field 1521 or a list of fields. The default direction is ascending (`:asc`) and can be 1522 customized in a keyword list as one of the following: 1523 1524 * `:asc` 1525 * `:asc_nulls_last` 1526 * `:asc_nulls_first` 1527 * `:desc` 1528 * `:desc_nulls_last` 1529 * `:desc_nulls_first` 1530 1531 The `*_nulls_first` and `*_nulls_last` variants are not supported by all 1532 databases. While all databases default to ascending order, the choice of 1533 "nulls first" or "nulls last" is specific to each database implementation. 1534 1535 `order_by` may be invoked or listed in a query many times. New expressions 1536 are always appended to the previous ones. 1537 1538 `order_by` also accepts a list of atoms where each atom refers to a field in 1539 source or a keyword list where the direction is given as key and the field 1540 to order as value. 1541 1542 ## Keywords examples 1543 1544 from(c in City, order_by: c.name, order_by: c.population) 1545 from(c in City, order_by: [c.name, c.population]) 1546 from(c in City, order_by: [asc: c.name, desc: c.population]) 1547 1548 from(c in City, order_by: [:name, :population]) 1549 from(c in City, order_by: [asc: :name, desc_nulls_first: :population]) 1550 1551 A keyword list can also be interpolated: 1552 1553 values = [asc: :name, desc_nulls_first: :population] 1554 from(c in City, order_by: ^values) 1555 1556 A fragment can also be used: 1557 1558 from c in City, order_by: [ 1559 # A deterministic shuffled order 1560 fragment("? % ? DESC", c.id, ^modulus), 1561 desc: c.id, 1562 ] 1563 1564 It's also possible to order by an aliased or calculated column: 1565 1566 from(c in City, 1567 select: %{ 1568 name: c.name, 1569 total_population: 1570 fragment( 1571 "COALESCE(?, ?) + ? AS total_population", 1572 c.animal_population, 1573 0, 1574 c.human_population 1575 ) 1576 }, 1577 order_by: [ 1578 # based on `AS total_population` in the previous fragment 1579 {:desc, fragment("total_population")} 1580 ] 1581 ) 1582 1583 ## Expressions examples 1584 1585 City |> order_by([c], asc: c.name, desc: c.population) 1586 City |> order_by(asc: :name) # Sorts by the cities name 1587 1588 """ 1589 defmacro order_by(query, binding \\ [], expr) do 1590 Builder.OrderBy.build(query, binding, expr, __CALLER__) 1591 end 1592 1593 @doc """ 1594 A union query expression. 1595 1596 Combines result sets of multiple queries. The `select` of each query 1597 must be exactly the same, with the same types in the same order. 1598 1599 Union expression returns only unique rows as if each query returned 1600 distinct results. This may cause a performance penalty. If you need 1601 to combine multiple result sets without removing duplicate rows 1602 consider using `union_all/2`. 1603 1604 Note that the operations `order_by`, `limit` and `offset` of the 1605 current `query` apply to the result of the union. `order_by` must 1606 be specified in one of the following ways, since the union of two 1607 or more queries is not automatically aliased: 1608 1609 - Use `Ecto.Query.API.fragment/1` to pass an `order_by` statement that directly access the union fields. 1610 - Wrap the union in a subquery and refer to the binding of the subquery. 1611 1612 ## Keywords examples 1613 1614 # Unordered result 1615 supplier_query = from s in Supplier, select: s.city 1616 from c in Customer, select: c.city, union: ^supplier_query 1617 1618 # Ordered result 1619 supplier_query = from s in Supplier, select: s.city 1620 union_query = from c in Customer, select: c.city, union: ^supplier_query 1621 from s in subquery(union_query), order_by: s.city 1622 1623 ## Expressions examples 1624 1625 # Unordered result 1626 supplier_query = Supplier |> select([s], s.city) 1627 Customer |> select([c], c.city) |> union(^supplier_query) 1628 1629 # Ordered result 1630 customer_query = Customer |> select([c], c.city) |> order_by(fragment("city")) 1631 supplier_query = Supplier |> select([s], s.city) 1632 union(customer_query, ^supplier_query) 1633 1634 """ 1635 defmacro union(query, other_query) do 1636 Builder.Combination.build(:union, query, other_query, __CALLER__) 1637 end 1638 1639 @doc """ 1640 A union all query expression. 1641 1642 Combines result sets of multiple queries. The `select` of each query 1643 must be exactly the same, with the same types in the same order. 1644 1645 Note that the operations `order_by`, `limit` and `offset` of the 1646 current `query` apply to the result of the union. `order_by` must 1647 be specified in one of the following ways, since the union of two 1648 or more queries is not automatically aliased: 1649 1650 - Use `Ecto.Query.API.fragment/1` to pass an `order_by` statement that directly access the union fields. 1651 - Wrap the union in a subquery and refer to the binding of the subquery. 1652 1653 ## Keywords examples 1654 1655 # Unordered result 1656 supplier_query = from s in Supplier, select: s.city 1657 from c in Customer, select: c.city, union_all: ^supplier_query 1658 1659 # Ordered result 1660 supplier_query = from s in Supplier, select: s.city 1661 union_all_query = from c in Customer, select: c.city, union_all: ^supplier_query 1662 from s in subquery(union_all_query), order_by: s.city 1663 1664 ## Expressions examples 1665 1666 # Unordered result 1667 supplier_query = Supplier |> select([s], s.city) 1668 Customer |> select([c], c.city) |> union_all(^supplier_query) 1669 1670 # Ordered result 1671 customer_query = Customer |> select([c], c.city) |> order_by(fragment("city")) 1672 supplier_query = Supplier |> select([s], s.city) 1673 union_all(customer_query, ^supplier_query) 1674 """ 1675 defmacro union_all(query, other_query) do 1676 Builder.Combination.build(:union_all, query, other_query, __CALLER__) 1677 end 1678 1679 @doc """ 1680 An except (set difference) query expression. 1681 1682 Takes the difference of the result sets of multiple queries. The 1683 `select` of each query must be exactly the same, with the same 1684 types in the same order. 1685 1686 Except expression returns only unique rows as if each query returned 1687 distinct results. This may cause a performance penalty. If you need 1688 to take the difference of multiple result sets without 1689 removing duplicate rows consider using `except_all/2`. 1690 1691 Note that the operations `order_by`, `limit` and `offset` of the 1692 current `query` apply to the result of the set difference. `order_by` 1693 must be specified in one of the following ways, since the set difference 1694 of two or more queries is not automatically aliased: 1695 1696 - Use `Ecto.Query.API.fragment/1` to pass an `order_by` statement that directly access the set difference fields. 1697 - Wrap the set difference in a subquery and refer to the binding of the subquery. 1698 1699 ## Keywords examples 1700 1701 # Unordered result 1702 supplier_query = from s in Supplier, select: s.city 1703 from c in Customer, select: c.city, except: ^supplier_query 1704 1705 # Ordered result 1706 supplier_query = from s in Supplier, select: s.city 1707 except_query = from c in Customer, select: c.city, except: ^supplier_query 1708 from s in subquery(except_query), order_by: s.city 1709 1710 ## Expressions examples 1711 1712 # Unordered result 1713 supplier_query = Supplier |> select([s], s.city) 1714 Customer |> select([c], c.city) |> except(^supplier_query) 1715 1716 # Ordered result 1717 customer_query = Customer |> select([c], c.city) |> order_by(fragment("city")) 1718 supplier_query = Supplier |> select([s], s.city) 1719 except(customer_query, ^supplier_query) 1720 """ 1721 defmacro except(query, other_query) do 1722 Builder.Combination.build(:except, query, other_query, __CALLER__) 1723 end 1724 1725 @doc """ 1726 An except (set difference) query expression. 1727 1728 Takes the difference of the result sets of multiple queries. The 1729 `select` of each query must be exactly the same, with the same 1730 types in the same order. 1731 1732 Note that the operations `order_by`, `limit` and `offset` of the 1733 current `query` apply to the result of the set difference. `order_by` 1734 must be specified in one of the following ways, since the set difference 1735 of two or more queries is not automatically aliased: 1736 1737 - Use `Ecto.Query.API.fragment/1` to pass an `order_by` statement that directly access the set difference fields. 1738 - Wrap the set difference in a subquery and refer to the binding of the subquery. 1739 1740 ## Keywords examples 1741 1742 # Unordered result 1743 supplier_query = from s in Supplier, select: s.city 1744 from c in Customer, select: c.city, except_all: ^supplier_query 1745 1746 # Ordered result 1747 supplier_query = from s in Supplier, select: s.city 1748 except_all_query = from c in Customer, select: c.city, except_all: ^supplier_query 1749 from s in subquery(except_all_query), order_by: s.city 1750 1751 ## Expressions examples 1752 1753 # Unordered result 1754 supplier_query = Supplier |> select([s], s.city) 1755 Customer |> select([c], c.city) |> except_all(^supplier_query) 1756 1757 # Ordered result 1758 customer_query = Customer |> select([c], c.city) |> order_by(fragment("city")) 1759 supplier_query = Supplier |> select([s], s.city) 1760 except_all(customer_query, ^supplier_query) 1761 """ 1762 defmacro except_all(query, other_query) do 1763 Builder.Combination.build(:except_all, query, other_query, __CALLER__) 1764 end 1765 1766 @doc """ 1767 An intersect query expression. 1768 1769 Takes the overlap of the result sets of multiple queries. The 1770 `select` of each query must be exactly the same, with the same 1771 types in the same order. 1772 1773 Intersect expression returns only unique rows as if each query returned 1774 distinct results. This may cause a performance penalty. If you need 1775 to take the intersection of multiple result sets without 1776 removing duplicate rows consider using `intersect_all/2`. 1777 1778 Note that the operations `order_by`, `limit` and `offset` of the 1779 current `query` apply to the result of the set difference. `order_by` 1780 must be specified in one of the following ways, since the intersection 1781 of two or more queries is not automatically aliased: 1782 1783 - Use `Ecto.Query.API.fragment/1` to pass an `order_by` statement that directly access the intersection fields. 1784 - Wrap the intersection in a subquery and refer to the binding of the subquery. 1785 1786 ## Keywords examples 1787 1788 # Unordered result 1789 supplier_query = from s in Supplier, select: s.city 1790 from c in Customer, select: c.city, intersect: ^supplier_query 1791 1792 # Ordered result 1793 supplier_query = from s in Supplier, select: s.city 1794 intersect_query = from c in Customer, select: c.city, intersect: ^supplier_query 1795 from s in subquery(intersect_query), order_by: s.city 1796 1797 ## Expressions examples 1798 1799 # Unordered result 1800 supplier_query = Supplier |> select([s], s.city) 1801 Customer |> select([c], c.city) |> intersect(^supplier_query) 1802 1803 # Ordered result 1804 customer_query = Customer |> select([c], c.city) |> order_by(fragment("city")) 1805 supplier_query = Supplier |> select([s], s.city) 1806 intersect(customer_query, ^supplier_query) 1807 """ 1808 defmacro intersect(query, other_query) do 1809 Builder.Combination.build(:intersect, query, other_query, __CALLER__) 1810 end 1811 1812 @doc """ 1813 An intersect query expression. 1814 1815 Takes the overlap of the result sets of multiple queries. The 1816 `select` of each query must be exactly the same, with the same 1817 types in the same order. 1818 1819 Note that the operations `order_by`, `limit` and `offset` of the 1820 current `query` apply to the result of the set difference. `order_by` 1821 must be specified in one of the following ways, since the intersection 1822 of two or more queries is not automatically aliased: 1823 1824 - Use `Ecto.Query.API.fragment/1` to pass an `order_by` statement that directly access the intersection fields. 1825 - Wrap the intersection in a subquery and refer to the binding of the subquery. 1826 1827 ## Keywords examples 1828 1829 # Unordered result 1830 supplier_query = from s in Supplier, select: s.city 1831 from c in Customer, select: c.city, intersect_all: ^supplier_query 1832 1833 # Ordered result 1834 supplier_query = from s in Supplier, select: s.city 1835 intersect_all_query = from c in Customer, select: c.city, intersect_all: ^supplier_query 1836 from s in subquery(intersect_all_query), order_by: s.city 1837 1838 ## Expressions examples 1839 1840 # Unordered result 1841 supplier_query = Supplier |> select([s], s.city) 1842 Customer |> select([c], c.city) |> intersect_all(^supplier_query) 1843 1844 # Ordered result 1845 customer_query = Customer |> select([c], c.city) |> order_by(fragment("city")) 1846 supplier_query = Supplier |> select([s], s.city) 1847 intersect_all(customer_query, ^supplier_query) 1848 """ 1849 defmacro intersect_all(query, other_query) do 1850 Builder.Combination.build(:intersect_all, query, other_query, __CALLER__) 1851 end 1852 1853 @doc """ 1854 A limit query expression. 1855 1856 Limits the number of rows returned from the result. Can be any expression but 1857 has to evaluate to an integer value and it can't include any field. 1858 1859 If `limit` is given twice, it overrides the previous value. 1860 1861 ## Keywords example 1862 1863 from(u in User, where: u.id == ^current_user, limit: 1) 1864 1865 ## Expressions example 1866 1867 User |> where([u], u.id == ^current_user) |> limit(1) 1868 1869 """ 1870 defmacro limit(query, binding \\ [], expr) do 1871 Builder.LimitOffset.build(:limit, query, binding, expr, __CALLER__) 1872 end 1873 1874 @doc """ 1875 An offset query expression. 1876 1877 Offsets the number of rows selected from the result. Can be any expression 1878 but it must evaluate to an integer value and it can't include any field. 1879 1880 If `offset` is given twice, it overrides the previous value. 1881 1882 ## Keywords example 1883 1884 # Get all posts on page 4 1885 from(p in Post, limit: 10, offset: 30) 1886 1887 ## Expressions example 1888 1889 Post |> limit(10) |> offset(30) 1890 1891 """ 1892 defmacro offset(query, binding \\ [], expr) do 1893 Builder.LimitOffset.build(:offset, query, binding, expr, __CALLER__) 1894 end 1895 1896 @doc ~S""" 1897 A lock query expression. 1898 1899 Provides support for row-level pessimistic locking using 1900 `SELECT ... FOR UPDATE` or other, database-specific, locking clauses. 1901 `expr` can be any expression but has to evaluate to a boolean value or to a 1902 string and it can't include any fields. 1903 1904 If `lock` is used more than once, the last one used takes precedence. 1905 1906 Ecto also supports [optimistic 1907 locking](https://en.wikipedia.org/wiki/Optimistic_concurrency_control) but not 1908 through queries. For more information on optimistic locking, have a look at 1909 the `Ecto.Changeset.optimistic_lock/3` function. 1910 1911 ## Keywords example 1912 1913 from(u in User, where: u.id == ^current_user, lock: "FOR SHARE NOWAIT") 1914 1915 ## Expressions example 1916 1917 User |> where([u], u.id == ^current_user) |> lock("FOR SHARE NOWAIT") 1918 1919 """ 1920 defmacro lock(query, binding \\ [], expr) do 1921 Builder.Lock.build(query, binding, expr, __CALLER__) 1922 end 1923 1924 @doc ~S""" 1925 An update query expression. 1926 1927 Updates are used to update the filtered entries. In order for 1928 updates to be applied, `c:Ecto.Repo.update_all/3` must be invoked. 1929 1930 ## Keywords example 1931 1932 from(u in User, update: [set: [name: "new name"]]) 1933 1934 ## Expressions examples 1935 1936 User |> update([u], set: [name: "new name"]) 1937 User |> update(set: [name: "new name"]) 1938 1939 ## Interpolation 1940 1941 new_name = "new name" 1942 from(u in User, update: [set: [name: ^new_name]]) 1943 1944 new_name = "new name" 1945 from(u in User, update: [set: [name: fragment("upper(?)", ^new_name)]]) 1946 1947 ## Operators 1948 1949 The update expression in Ecto supports the following operators: 1950 1951 * `set` - sets the given field in the table to the given value 1952 1953 from(u in User, update: [set: [name: "new name"]]) 1954 1955 * `inc` - increments (or decrements if the value is negative) the given field in the table by the given value 1956 1957 from(u in User, update: [inc: [accesses: 1]]) 1958 1959 * `push` - pushes (appends) the given value to the end of the array field 1960 1961 from(u in User, update: [push: [tags: "cool"]]) 1962 1963 * `pull` - pulls (removes) the given value from the array field 1964 1965 from(u in User, update: [pull: [tags: "not cool"]]) 1966 1967 """ 1968 defmacro update(query, binding \\ [], expr) do 1969 Builder.Update.build(query, binding, expr, __CALLER__) 1970 end 1971 1972 @doc """ 1973 A group by query expression. 1974 1975 Groups together rows from the schema that have the same values in the given 1976 fields. Using `group_by` "groups" the query giving it different semantics 1977 in the `select` expression. If a query is grouped, only fields that were 1978 referenced in the `group_by` can be used in the `select` or if the field 1979 is given as an argument to an aggregate function. 1980 1981 `group_by` also accepts a list of atoms where each atom refers to 1982 a field in source. For more complicated queries you can access fields 1983 directly instead of atoms. 1984 1985 ## Keywords examples 1986 1987 # Returns the number of posts in each category 1988 from(p in Post, 1989 group_by: p.category, 1990 select: {p.category, count(p.id)}) 1991 1992 # Using atoms 1993 from(p in Post, group_by: :category, select: {p.category, count(p.id)}) 1994 1995 # Using direct fields access 1996 from(p in Post, 1997 join: c in assoc(p, :category), 1998 group_by: [p.id, c.name]) 1999 2000 ## Expressions example 2001 2002 Post |> group_by([p], p.category) |> select([p], count(p.id)) 2003 2004 """ 2005 defmacro group_by(query, binding \\ [], expr) do 2006 Builder.GroupBy.build(query, binding, expr, __CALLER__) 2007 end 2008 2009 @doc """ 2010 An AND having query expression. 2011 2012 Like `where`, `having` filters rows from the schema, but after the grouping is 2013 performed giving it the same semantics as `select` for a grouped query 2014 (see `group_by/3`). `having` groups the query even if the query has no 2015 `group_by` expression. 2016 2017 ## Keywords example 2018 2019 # Returns the number of posts in each category where the 2020 # average number of comments is above ten 2021 from(p in Post, 2022 group_by: p.category, 2023 having: avg(p.num_comments) > 10, 2024 select: {p.category, count(p.id)}) 2025 2026 ## Expressions example 2027 2028 Post 2029 |> group_by([p], p.category) 2030 |> having([p], avg(p.num_comments) > 10) 2031 |> select([p], count(p.id)) 2032 """ 2033 defmacro having(query, binding \\ [], expr) do 2034 Builder.Filter.build(:having, :and, query, binding, expr, __CALLER__) 2035 end 2036 2037 @doc """ 2038 An OR having query expression. 2039 2040 Like `having` but combines with the previous expression by using 2041 `OR`. `or_having` behaves for `having` the same way `or_where` 2042 behaves for `where`. 2043 2044 ## Keywords example 2045 2046 # Augment a previous group_by with a having condition. 2047 from(p in query, or_having: avg(p.num_comments) > 10) 2048 2049 ## Expressions example 2050 2051 # Augment a previous group_by with a having condition. 2052 Post |> or_having([p], avg(p.num_comments) > 10) 2053 2054 """ 2055 defmacro or_having(query, binding \\ [], expr) do 2056 Builder.Filter.build(:having, :or, query, binding, expr, __CALLER__) 2057 end 2058 2059 @doc """ 2060 Preloads the associations into the result set. 2061 2062 Imagine you have a schema `Post` with a `has_many :comments` 2063 association and you execute the following query: 2064 2065 Repo.all from p in Post, preload: [:comments] 2066 2067 The example above will fetch all posts from the database and then do 2068 a separate query returning all comments associated with the given posts. 2069 The comments are then processed and associated to each returned `post` 2070 under the `comments` field. 2071 2072 Often times, you may want posts and comments to be selected and 2073 filtered in the same query. For such cases, you can explicitly tell 2074 an existing join to be preloaded into the result set: 2075 2076 Repo.all from p in Post, 2077 join: c in assoc(p, :comments), 2078 where: c.published_at > p.updated_at, 2079 preload: [comments: c] 2080 2081 In the example above, instead of issuing a separate query to fetch 2082 comments, Ecto will fetch posts and comments in a single query and 2083 then do a separate pass associating each comment to its parent post. 2084 Therefore, instead of returning `number_of_posts * number_of_comments` 2085 results, like a `join` would, it returns only posts with the `comments` 2086 fields properly filled in. 2087 2088 Nested associations can also be preloaded in both formats: 2089 2090 Repo.all from p in Post, 2091 preload: [comments: :likes] 2092 2093 Repo.all from p in Post, 2094 join: c in assoc(p, :comments), 2095 join: l in assoc(c, :likes), 2096 where: l.inserted_at > c.updated_at, 2097 preload: [comments: {c, likes: l}] 2098 2099 Applying a limit to the association can be achieved with `inner_lateral_join`: 2100 2101 Repo.all from p in Post, as: :post, 2102 join: c in assoc(p, :comments), 2103 inner_lateral_join: top_five in subquery( 2104 from Comment, 2105 where: [post_id: parent_as(:post).id], 2106 order_by: :popularity, 2107 limit: 5, 2108 select: [:id] 2109 ), on: top_five.id == c.id, 2110 preload: [comments: c] 2111 2112 ## Preload queries 2113 2114 Preload also allows queries to be given, allowing you to filter or 2115 customize how the preloads are fetched: 2116 2117 comments_query = from c in Comment, order_by: c.published_at 2118 Repo.all from p in Post, preload: [comments: ^comments_query] 2119 2120 The example above will issue two queries, one for loading posts and 2121 then another for loading the comments associated with the posts. 2122 Comments will be ordered by `published_at`. 2123 2124 When specifying a preload query, you can still preload the associations of 2125 those records. For instance, you could preload an author's published posts and 2126 the comments on those posts: 2127 2128 posts_query = from p in Post, where: p.state == :published 2129 Repo.all from a in Author, preload: [posts: ^{posts_query, [:comments]}] 2130 2131 Note: keep in mind operations like limit and offset in the preload 2132 query will affect the whole result set and not each association. For 2133 example, the query below: 2134 2135 comments_query = from c in Comment, order_by: c.popularity, limit: 5 2136 Repo.all from p in Post, preload: [comments: ^comments_query] 2137 2138 won't bring the top of comments per post. Rather, it will only bring 2139 the 5 top comments across all posts. Instead, use a window: 2140 2141 ranking_query = 2142 from c in Comment, 2143 select: %{id: c.id, row_number: over(row_number(), :posts_partition)}, 2144 windows: [posts_partition: [partition_by: :post_id, order_by: :popularity]] 2145 2146 comments_query = 2147 from c in Comment, 2148 join: r in subquery(ranking_query), 2149 on: c.id == r.id and r.row_number <= 5 2150 2151 Repo.all from p in Post, preload: [comments: ^comments_query] 2152 2153 ## Preload functions 2154 2155 Preload also allows functions to be given. In such cases, the function 2156 receives the IDs of the parent association and it must return the associated 2157 data. Ecto then will map this data and sort it by the relationship key: 2158 2159 comment_preloader = fn post_ids -> fetch_comments_by_post_ids(post_ids) end 2160 Repo.all from p in Post, preload: [comments: ^comment_preloader] 2161 2162 This is useful when the whole dataset was already loaded or must be 2163 explicitly fetched from elsewhere. The IDs received by the preloading 2164 function and the result returned depends on the association type: 2165 2166 * For `has_many` and `belongs_to` - the function receives the IDs of 2167 the parent association and it must return a list of maps or structs 2168 with the associated entries. The associated map/struct must contain 2169 the "foreign_key" field. For example, if a post has many comments, 2170 when preloading the comments with a custom function, the function 2171 will receive a list of "post_ids" as the argument and it must return 2172 maps or structs representing the comments. The maps/structs must 2173 include the `:post_id` field 2174 2175 * For `has_many :through` - it behaves similarly to a regular `has_many` 2176 but note that the IDs received are of the last association. Imagine, 2177 for example, a post has many comments and each comment has an author. 2178 Therefore, a post may have many comments_authors, written as 2179 `has_many :comments_authors, through: [:comments, :author]`. When 2180 preloading authors with a custom function via `:comments_authors`, 2181 the function will receive the IDs of the authors as the last step 2182 2183 * For `many_to_many` - the function receives the IDs of the parent 2184 association and it must return a tuple with the parent id as the first 2185 element and the association map or struct as the second. For example, 2186 if a post has many tags, when preloading the tags with a custom 2187 function, the function will receive a list of "post_ids" as the argument 2188 and it must return a tuple in the format of `{post_id, tag}` 2189 2190 ## Keywords example 2191 2192 # Returns all posts, their associated comments, and the associated 2193 # likes for those comments. 2194 from(p in Post, 2195 preload: [comments: :likes], 2196 select: p 2197 ) 2198 2199 ## Expressions examples 2200 2201 Post |> preload(:comments) |> select([p], p) 2202 2203 Post 2204 |> join(:left, [p], c in assoc(p, :comments)) 2205 |> preload([p, c], [:user, comments: c]) 2206 |> select([p], p) 2207 2208 """ 2209 defmacro preload(query, bindings \\ [], expr) do 2210 Builder.Preload.build(query, bindings, expr, __CALLER__) 2211 end 2212 2213 @doc """ 2214 Restricts the query to return the first result ordered by primary key. 2215 2216 The query will be automatically ordered by the primary key 2217 unless `order_by` is given or `order_by` is set in the query. 2218 Limit is always set to 1. 2219 2220 ## Examples 2221 2222 Post |> first |> Repo.one 2223 query |> first(:inserted_at) |> Repo.one 2224 """ 2225 def first(queryable, order_by \\ nil) 2226 2227 def first(%Ecto.Query{} = query, nil) do 2228 query = %{query | limit: limit()} 2229 case query do 2230 %{order_bys: []} -> 2231 %{query | order_bys: [order_by_pk(query, :asc)]} 2232 %{} -> 2233 query 2234 end 2235 end 2236 def first(queryable, nil), do: first(Ecto.Queryable.to_query(queryable), nil) 2237 def first(queryable, key), do: first(order_by(queryable, ^key), nil) 2238 2239 @doc """ 2240 Restricts the query to return the last result ordered by primary key. 2241 2242 The query ordering will be automatically reversed, with ASC 2243 columns becoming DESC columns (and vice-versa) and limit is set 2244 to 1. If there is no ordering, the query will be automatically 2245 ordered decreasingly by primary key. 2246 2247 ## Examples 2248 2249 Post |> last |> Repo.one 2250 query |> last(:inserted_at) |> Repo.one 2251 """ 2252 def last(queryable, order_by \\ nil) 2253 def last(queryable, nil), do: %{reverse_order(queryable) | limit: limit()} 2254 def last(queryable, key), do: last(order_by(queryable, ^key), nil) 2255 2256 defp limit do 2257 %QueryExpr{expr: 1, params: [], file: __ENV__.file, line: __ENV__.line} 2258 end 2259 2260 defp field(ix, field) when is_integer(ix) and is_atom(field) do 2261 {{:., [], [{:&, [], [ix]}, field]}, [], []} 2262 end 2263 2264 defp order_by_pk(query, dir) do 2265 schema = assert_schema!(query) 2266 pks = schema.__schema__(:primary_key) 2267 expr = for pk <- pks, do: {dir, field(0, pk)} 2268 %QueryExpr{expr: expr, file: __ENV__.file, line: __ENV__.line} 2269 end 2270 2271 defp assert_schema!(%{from: %Ecto.Query.FromExpr{source: {_source, schema}}}) when schema != nil, do: schema 2272 defp assert_schema!(query) do 2273 raise Ecto.QueryError, query: query, message: "expected a from expression with a schema" 2274 end 2275 2276 @doc """ 2277 Returns `true` if the query has a binding with the given name, otherwise `false`. 2278 2279 For more information on named bindings see "Named bindings" in this module doc. 2280 """ 2281 def has_named_binding?(%Ecto.Query{aliases: aliases}, key) do 2282 Map.has_key?(aliases, key) 2283 end 2284 2285 def has_named_binding?(queryable, _key) 2286 when is_atom(queryable) or is_binary(queryable) or is_tuple(queryable) do 2287 false 2288 end 2289 2290 def has_named_binding?(queryable, key) do 2291 has_named_binding?(Ecto.Queryable.to_query(queryable), key) 2292 end 2293 2294 @doc """ 2295 Applies a callback function to a query if it doesn't contain the given named binding. 2296 Otherwise, returns the original query. 2297 2298 The callback function must accept a queryable and return an `Ecto.Query` struct 2299 that contains the provided named binding, otherwise an error is raised. It can also 2300 accept second argument which is the atom representing the name of a binding. 2301 2302 For example, one might use this function as a convenience to conditionally add a new 2303 named join to a query: 2304 2305 if has_named_binding?(query, :comments) do 2306 query 2307 else 2308 join(query, :left, c in assoc(p, :comments), as: :comments) 2309 end 2310 2311 With this function it can be simplified to: 2312 2313 with_named_binding(query, :comments, fn query, binding -> 2314 join(query, :left, a in assoc(p, ^binding), as: ^binding) 2315 end) 2316 2317 For more information on named bindings see "Named bindings" in this module doc or `has_named_binding/2`. 2318 """ 2319 def with_named_binding(%Ecto.Query{} = query, key, fun) do 2320 if has_named_binding?(query, key) do 2321 query 2322 else 2323 query 2324 |> apply_binding_callback(fun, key) 2325 |> raise_on_invalid_callback_return(key) 2326 end 2327 end 2328 2329 def with_named_binding(queryable, key, fun) do 2330 queryable 2331 |> Ecto.Queryable.to_query() 2332 |> with_named_binding(key, fun) 2333 end 2334 2335 defp apply_binding_callback(query, fun, _key) when is_function(fun, 1), do: query |> fun.() 2336 defp apply_binding_callback(query, fun, key) when is_function(fun, 2), do: query |> fun.(key) 2337 defp apply_binding_callback(_query, fun, _key) do 2338 raise ArgumentError, "callback function for with_named_binding/3 should accept one or two arguments, got: #{inspect(fun)}" 2339 end 2340 2341 defp raise_on_invalid_callback_return(%Ecto.Query{} = query, key) do 2342 if has_named_binding?(query, key) do 2343 query 2344 else 2345 raise RuntimeError, "callback function for with_named_binding/3 should create a named binding for key #{inspect(key)}" 2346 end 2347 end 2348 2349 defp raise_on_invalid_callback_return(other, _key) do 2350 raise RuntimeError, "callback function for with_named_binding/3 should return an Ecto.Query struct, got: #{inspect(other)}" 2351 end 2352 2353 @doc """ 2354 Reverses the ordering of the query. 2355 2356 ASC columns become DESC columns (and vice-versa). If the query 2357 has no `order_by`s, it orders by the inverse of the primary key. 2358 2359 ## Examples 2360 2361 query |> reverse_order() |> Repo.one() 2362 Post |> order(asc: :id) |> reverse_order() == Post |> order(desc: :id) 2363 """ 2364 def reverse_order(%Ecto.Query{} = query) do 2365 update_in(query.order_bys, fn 2366 [] -> [order_by_pk(query, :desc)] 2367 order_bys -> Enum.map(order_bys, &reverse_order_by/1) 2368 end) 2369 end 2370 2371 def reverse_order(queryable) do 2372 reverse_order(Ecto.Queryable.to_query(queryable)) 2373 end 2374 2375 defp reverse_order_by(%{expr: expr} = order_by) do 2376 %{ 2377 order_by 2378 | expr: 2379 Enum.map(expr, fn 2380 {:desc, ast} -> {:asc, ast} 2381 {:desc_nulls_last, ast} -> {:asc_nulls_first, ast} 2382 {:desc_nulls_first, ast} -> {:asc_nulls_last, ast} 2383 {:asc, ast} -> {:desc, ast} 2384 {:asc_nulls_last, ast} -> {:desc_nulls_first, ast} 2385 {:asc_nulls_first, ast} -> {:desc_nulls_last, ast} 2386 end) 2387 } 2388 end 2389 end