Skip to content

Introduce an AST for preprocessing + postprocessing #2155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 4, 2017

Conversation

josevalim
Copy link
Member

This reduces the complexity in the adapters, reduces
the caching cost per entry and speeds up deserialization.

@josevalim
Copy link
Member Author

josevalim commented Aug 4, 2017

Hi @shdblowers, @jbachhardie, @AndrewDryga!

If this pull request is merged, it will require changes in adapters in order to support v2.2. The good news are: those changes can be done in a way to remain compatible with earlier codes and they all simplify the adapter code!

I will leave comments in the pull request for you to tag along.

"Please specify a schema or specify exactly which fields you want to select")
end
intersperse_map(fields, ", ", &[name, ?. | quote_name(&1)])
defp expr({:&, _, [idx]}, sources, query) do
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ecto 2.2 will no longer pass those weird and invalid {:&, _, list_with_three_elements}. It will pass {:&, [], [counter]} which should behave as the old {:&, _, [counter, nil, nil]}. If you want to remain backwards compatible, you just need to add this new clause and keep the old one around.

@@ -377,8 +374,8 @@ if Code.ensure_loaded?(Mariaex) do
["NOT (", expr(expr, sources, query), ?)]
end

defp expr(%Ecto.SubQuery{query: query, fields: fields}, _sources, _query) do
query.select.fields |> put_in(fields) |> all()
defp expr(%Ecto.SubQuery{query: query}, _sources, _query) do
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to remain backwards compatible, use this implementation for subqueries:

defp expr(%Ecto.SubQuery{query: query} = subquery, _sources, _query) do
  if Map.has_key?(subquery, :fields) do
    query.select.fields |> put_in(subquery.fields) |> all()
  else
    all(query)
  end
end

@@ -570,33 +538,6 @@ defmodule Ecto.Adapters.SQL do
end
end

defp process_row(row, process, fields, sources) do
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The difference in this code and the previous code is that adapters are no longer responsible for the whole logic below. To handle this in a backwards compatible fashion, you can change this code in v2.1:

mapper = &process_row(&1, process, fields, sources)

to something like this:

mapper = processor(process, fields, sources)

where

defp processor(process, fields, source) when is_function(process, 3) do
  &process_row(&1, process, fields, sources)
end
defp processor(process, _fields, _source) when is_function(process, 1) do
  process
end
defp processor(nil, _fields, _source) do
  nil
end

type =
# TODO: Support the :number type

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO found

@@ -1405,6 +1405,9 @@ defmodule Ecto.Schema do
defp type_to_module(:utc_datetime), do: DateTime
defp type_to_module(other), do: other

# TODO: Remove one more variations of this.
# TODO: Consider making :types a list once all is done.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO found

@@ -1405,6 +1405,9 @@ defmodule Ecto.Schema do
defp type_to_module(:utc_datetime), do: DateTime
defp type_to_module(other), do: other

# TODO: Remove one more variations of this.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO found

end

defp take!(source, query, fetched, field) do
defp take!(source, query, fetched, ix, field) do

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function is too complex (CC is 13, max is 9).

{{:value, :map}, [{:&, [], [ix]}]}

{:error, {_, schema}} ->
types = schema.__schema__(:types) |> Enum.to_list()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a function call when a pipeline is only one function long

@sourcelevel-bot
Copy link

Ebert has finished reviewing this Pull Request and has found:

  • 4 possible new issues (including those that may have been commented here).
  • 5 fixed issues! 🎉

You can see more details about this review at https://ebertapp.io/github/elixir-ecto/ecto/pulls/2155.

This reduces the complexity in the adapters, reduces
the caching cost per entry and speeds up deserialization.
@josevalim josevalim merged commit 0951213 into master Aug 4, 2017
@josevalim josevalim deleted the jv-expand-select branch August 4, 2017 21:25
@josevalim
Copy link
Member Author

❤️ 💚 💙 💛 💜

@josevalim
Copy link
Member Author

Hi @shdblowers, @jbachhardie, @AndrewDryga, @scouten!

Just a heads up that Ecto 2.2 will be released tomorrow (this monday). This issue should cover the new formats added to the query AST.

I have also invited all of you to the @elixir-ecto/adapters team so it is easier to discuss such changes in the future.

@AndrewDryga
Copy link
Contributor

Hello, @josevalim.

I will try to find some time over the next week or two to update Mensia adapter and make it compatible with Ecto 2.2. Thanks for this changes.

I really feel that there is a huge space for improvements in Adapter behavior to make it easy to build new ones.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants