I am trying to get a better understanding of Ecto adapters in elixir. I have begun trying to build my own adapter using Ecto.Adapters.Postgres as a base. This seemed like a good choice to start with as it is the default adapter used with Phoenix.
I can use my adapter in my own projects now by updating the the following line in my project's repo file...
defmodule UsingTestAdapter.Repo do
use Ecto.Repo,
otp_app: :using_test_adapter,
adapter: TestAdapter # <------ this line
end
At the moment it has the same functionality as the postgres adapter. I have been trying to edit some of the functions found in the Ecto.Adapters.Postgres.Connection and I have realised that they do not work quite how I expected.
The insert function for example does not actually use the params passed into Repo.insert.
To make this a little more clear imagine we have the following table, Comments...
| id | comment |
| -- | ------- |
Now Repo.insert(%Comments{comment: "hi"}) is called.
I want to modify the adapter, so that it ignores the "hi" value that is passed in and instead inserts a comment of "I am the adapter, and I control this database. Hahaha (evil laugh)"...
| id | comment |
| -- | ------------------------------------------------------------------ |
| 1 | I am the adapter and I control this database. Hahaha (evil laugh)" |
However, the insert function does not appear to actually take the data to be stored as an argument.
My initial thought of what happened with ecto adapters was that when a user calls one of the repo functions it called the corresponding function in the Ecto.Adapters.Postgres.Connection module. This does appear to happen but other steps seem to be happening before this.
If anyone has a better understanding of the chain of functions that are called when Repo.insert (and any other Repo function) is called, please explain below.
I have had the time to look into this more deeply and feel that I now have a better understanding.
I'm going to list the steps, in order, that happen when a user calls Repo.insert in an elixir app.
AppName.Repo.insert(%AppName.Comments{comment: "hi"})
defmodule AppName.Repo do
use Ecto.Repo, otp_app: :app_name, adapter: adapter_name
end
(This is the default set up for a phoenix application)
The
use Ecto.Repoallows for all the functions defined in that module to be used in the module that calls it. This means that when we callAppName.Repo.insert, it goes to our module, sees there is no function defined as insert, sees theusemarco, checks that module, sees a function calledinsertand calls that function (this is not exactly how it works but I feel it explains it well enough).
def insert(struct, opts \\ []) do
Ecto.Repo.Schema.insert(__MODULE__, struct, opts)
end
# if a changeset was passed in
def insert(name, %Changeset{} = changeset, opts) when is_list(opts) do
do_insert(name, changeset, opts)
end
# if a struct was passed in
# This will be called in this example
def insert(name, %{__struct__: _} = struct, opts) when is_list(opts) do
do_insert(name, Ecto.Changeset.change(struct), opts)
end
This step ensures that the data that is passed to do_insert in the the form of a changeset.
do_insert(name, Ecto.Changeset.change(struct), opts)
Not pasting whole function as it is very long. Where function is defined
This function does a fair amount of data manipulation and checks for errors. If all goes well it ends up calling the apply function
defp apply(changeset, adapter, action, args) do
case apply(adapter, action, args) do # <---- Kernel.apply/3
{:ok, values} ->
{:ok, values}
{:invalid, _} = constraints ->
constraints
{:error, :stale} ->
opts = List.last(args)
case Keyword.fetch(opts, :stale_error_field) do
{:ok, stale_error_field} when is_atom(stale_error_field) ->
stale_message = Keyword.get(opts, :stale_error_message, "is stale")
changeset = Changeset.add_error(changeset, stale_error_field, stale_message, [stale: true])
{:error, changeset}
_other ->
raise Ecto.StaleEntryError, struct: changeset.data, action: action
end
end
end
This apply/4 function calls the Kernel.apply/3 function with the module, function name and arguments. In our case the module is AdapterName and the function is :insert.
This is where our adapter comes into play :D (finally).
The apply/3 function call above takes us to our created adapter.
defmodule AdapterName do
# Inherit all behaviour from Ecto.Adapters.SQL
use Ecto.Adapters.SQL, driver: :postgrex, migration_lock: "FOR UPDATE"
end
There is no insert function defined in this module but as it is 'using' Ecto.Adapters.SQL let's look at this module next.
defmodule Ecto.Adapters.SQL do
...
@conn __MODULE__.Connection
...
@impl true
def insert(adapter_meta, %{source: source, prefix: prefix}, params,
{kind, conflict_params, _} = on_conflict, returning, opts) do
{fields, values} = :lists.unzip(params)
sql = @conn.insert(prefix, source, fields, [fields], on_conflict, returning)
Ecto.Adapters.SQL.struct(adapter_meta, @conn, sql, :insert, source, [], values ++ conflict_params, kind, returning, opts)
end
...
end
@conn is defined as a module attribute and is just the current calling module (MODULE) + .Connection.
The calling module, as discussed in point 5 is AdapterName
That means in the insert function, the following line...
@conn.insert(prefix, source, fields, [fields], on_conflict, returning)
is the same as
AdapterName.Connection.insert(prefix, source, fields, [fields], on_conflict, returning)
As our adapter is just the same as the postgres adapter, it takes us to this next function.
def insert(prefix, table, header, rows, on_conflict, returning) do
values =
if header == [] do
[" VALUES " | intersperse_map(rows, ?,, fn _ -> "(DEFAULT)" end)]
else
[?\s, ?(, intersperse_map(header, ?,, "e_name/1), ") VALUES " | insert_all(rows, 1)]
end
["INSERT INTO ", quote_table(prefix, table), insert_as(on_conflict),
values, on_conflict(on_conflict, header) | returning(returning)]
end
To save some text in an answer that is already too long, I won't go into too much detail. This function doesn't actually take the params we passed into Repo.insert (way back in set one).
If we want to edit the params we need to do so in the AdapterName module. We need to define our own insert function so that it no longer calls the insert function defined in step 6.
For the sake of simplicity, we are going to just copy the insert defined in step 6 into our AdapterName module. Then we can modify that function to update params as we see fit.
If we do this we end up with a function like...
def insert(adapter_meta, %{source: source, prefix: prefix}, params, on_conflict, returning, opts) do
Keyword.replace!(params, :comment, "I am the adapter and I control this database. Hahaha (evil laugh)") # <---- changing the comment like we wanted :D
{kind, conflict_params, _} = on_conflict
{fields, values} = :lists.unzip(params)
sql = @conn.insert(prefix, source, fields, [fields], on_conflict, returning)
Ecto.Adapters.SQL.struct(adapter_meta, @conn, sql, :insert, source, [], values ++ conflict_params, kind, returning, opts)
end
This now inserts a different value as we originally wanted.
Hopefully someone finds this helpful.