You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Investigate how we end-up with large number of locks in non-transactional mode.
Make non-transactional metadata-sync truly idempotent:
e.g., the commands that we send to detach partitions is not truly idempotent even if we do ALTER TABLE IF EXISTS DETACH PARTITION.
This command guards us against the cases where the parent relation does not exist. However, it doesn't save us in the cases where:
the child relation does not exist
the child relation is not a partition of parent relation
The text was updated successfully, but these errors were encountered:
Those locks that are accumulating on the coordinator are normal. They are not that much even for a huge cluster, so it does not cause any failure. We verified non-transactional sync regularly clears accumulated locks at all workers via many small transactions.
This sounds true to me. We actually need to make sure that anywhere where we propagate "detach partition" is idempotent. It is extra important for non-transactional metadata-sync as it would leave the sync in an intermediate state that should be completed.
I found out one place where we accumulate locks on the coordinator. extern_IsColumnarTableAmTable points to IsColumnarTableAmTable which acquires lock on the relation without releasing it. See below:
if (accessMethod == NULL && extern_IsColumnarTableAmTable(relationId))
We can solve it with a variant of IsColumnarTableAmTable which takes no lock, which is fine to replace above snippet where we already take lock on metadata tables.
e.g., the commands that we send to detach partitions is not truly idempotent even if we do
ALTER TABLE IF EXISTS DETACH PARTITION
.This command guards us against the cases where the parent relation does not exist. However, it doesn't save us in the cases where:
The text was updated successfully, but these errors were encountered: