Search code examples
postgresqlpartitioningpostgresql-13

Postgresql - CHECK constraint not preventing ACCESS EXCLUSIVE lock and table scan when attaching new partition


I'm running postgresql 13.

The below section of the postgres doc doc says I should be able to avoid a scan and ACCESS EXCLUSIVE lock to validate the partition constraint.

Before running the ATTACH PARTITION command, it is recommended to create a CHECK constraint on the table to be attached that matches the expected partition constraint, as illustrated above. That way, the system will be able to skip the scan which is otherwise needed to validate the implicit partition constraint. Without the CHECK constraint, the table will be scanned to validate the partition constraint while holding an ACCESS EXCLUSIVE lock on that partition.

But, when I create a new partition with a check constraint, insert data into it, and then attach it, an ACCESS EXCLUSIVE lock is held while the table is scanned.

The partitioned table:

CREATE TABLE IF NOT EXISTS tasks
(
    task_time timestamp(6) with time zone not null,
    task_sp_time timestamp(6) with time zone,
    task_org_id text not null,
    build_id text,
    unit_id  text,
    unit_req numeric(12,2),
    ... 30 columns truncated ...,
    constraint tasks_pkey1
        primary key (task_org_id, task_time)
)
partition by RANGE(task_time);

task_time is not null and of type timestamp (6) with timezone.

-- create new empty partition table
CREATE TABLE tasks_partitions.tasks_20230111
(LIKE tasks INCLUDING DEFAULTS INCLUDING CONSTRAINTS);

-- add CHECK constraint on new partition 
ALTER TABLE tasks_partitions.tasks_20230111 ADD CONSTRAINT tmp_20230111
CHECK (task_time >= '2023-01-11 00:00:00+00' AND task_time <= '2023-01-11 23:59:59.999999+00');

-- select around 100 million rows into the new partition from an old default partition that has been detached.
INSERT INTO tasks_partitions.tasks_20230111
SELECT * FROM tasks_partitions.tasks_default_old where (task_time >= '2023-01-11 00:00:00+00' AND task_time <= '2023-01-11 23:59:59.999999+00');

-- attach partition
ALTER TABLE tasks ATTACH PARTITION tasks_partitions_tasks_20230111
FOR VALUES FROM ('2023-01-11 00:00:00+00') TO ('2023-01-11 23:59:59.999999+00')

Attaching the partition still holds the ACCESS EXLUSIVE lock and the entire table is scanned.

The tasks table did have a default partition at one point, but I detached it and renamed it in order to resolve another issue. I currently do not have a default partition attached to tasks.

When I attach the partition from the example above, I see an ACCESS EXCLUSIVE lock on the new partition and a seemingly random relation, 468140. I cannot insert any records into the tasks table while the partition is being attached and the locks are in place.

If it helps, the query I run to see locks is:

SELECT a.datname,
       l.relation::regclass,
       l.transactionid,
       l.mode,
       l.GRANTED,
       l.usename,
       a.query,
       a.query_start,
       age(now(), a.query_start) AS "age",
       a.pid
FROM pg_stat_activity a
JOIN pg_locks l ON l.pid = a.pid
ORDER BY a.query_start;

Solution

  • The check constraint you are creating does not match the partition boundaries. You missed this statement from the documentation:

    When creating a range partition, the lower bound specified with FROM is an inclusive bound, whereas the upper bound specified with TO is an exclusive bound.

    So you should define the constraint as

    ALTER TABLE tasks_partitions.tasks_20230111 ADD
    CHECK (task_time >= '2023-01-11 00:00:00+00' AND
           task_time <  '2023-01-12 00:00:00+00');
    

    and attach the partition with

    ALTER TABLE tasks ATTACH PARTITION tasks_partitions_tasks_20230111
    FOR VALUES FROM ('2023-01-11 00:00:00+00')
                 TO ('2023-01-12 00:00:00+00');