From the documentation of airflow:
Says this about autocommit
:
autocommit (bool) – if True, each command is automatically committed. (default value: False)
So if I have two commands(statements), as such:
my_operator = PostgresOperator(
task_id="mytask",
postgres_conn_id="myconn",
autocommit=True,
sql="""
SELECT * FROM FILMS;
DELETE FROM films USING producers
WHERE producer_id = producers.id AND producers.name = 'foo'""";
Does that mean this will be treated as two transactions with two commits ? Or one transation with two commits ? (guess this is not even possible). the point is if one of the statements fail will everything rollback in an atomic way ?
Thanks.
The postgres
provider uses psycopg2
as you can see in the source code:
from psycopg2.sql import SQL, Identifier
And if we refer to the autocommit
documentation:
It is possible to set the connection in autocommit mode: this way all the commands executed will be immediately committed and no rollback is possible...
So if you put a SELECT 1/0
in between your first and second action, I believe the first action will still be applied.
But the best way to make sure is to test it, replace the first action with an INSERT
and the modify the delete to cancel it out. Then put a failing operation in between, run it and go see the state of your DB.