6

We are planning to partition a dataset of ~500million rows using list/range partitioning in PostgreSQL 13.4 on RDS. Our read access patterns should only require a very small number of partitions to be accessed for each query. We would like to prevent read queries that require accessing every partition, so that this strategy can serve well as the volume of data grows. Is it possible to enforce any checks or guards at the database layer to reject/block/fail on queries that require a query plan that includes every partition?

Essentially, I would expect this to manifiest as ensuring there is sufficient filtering in the WHERE clause for the query planner to know it does not need to route to every partition. We can examine the query plans for a sample of the queries coming from the business logic, but it would be nice to be able to enforce it automatically. Ideally we can enforce in production if the overhead is not prohibitive, but just doing it in lower environments would be a positive step. Are there any options or suggestions on how that might be possible? Many thanks in advance.

asked Jan 19, 2022 at 11:03
2
  • 1
    Hi, and welcome to dba.se! The most important thing is to ensure that your queres use an Index Scan rather than a Seq Scan - doing so improved a query's performance from 25mins to 5 seconds - see here. You can try setting enable_seqscan to off - for the transaction! This will not guarantee that they are not done but may help? YMMV! Commented Jan 19, 2022 at 12:17
  • 3
    "ensuring there is sufficient filtering in the WHERE clause" - there is nothing built into Postgres to do that. Commented Jan 19, 2022 at 12:54

1 Answer 1

3

There is no way to do that.

The best I can think of is to set the parameter statement_timeout in your database session so that statements that run too long fail with an error.

answered Jan 19, 2022 at 15:59
1
  • Thank you @larenz-albe for the suggestion, that could be a good compromise. Commented Jan 24, 2022 at 16:22

Your Answer

Draft saved
Draft discarded

Sign up or log in

Sign up using Google
Sign up using Email and Password

Post as a guest

Required, but never shown

Post as a guest

Required, but never shown

By clicking "Post Your Answer", you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.