git.postgresql.org Git - postgresql.git/commitdiff

git projects / postgresql.git / commitdiff

summary | shortlog | log | commit | commitdiff | tree
raw | patch | inline | side by side (parent: 3f1e529)

For bulk write operations (eg COPY IN), use a ring buffer of 16MB instead

author Tom Lane <tgl@sss.pgh.pa.us>

2009年6月22日 20:04:28 +0000 (20:04 +0000)

committer Tom Lane <tgl@sss.pgh.pa.us>

2009年6月22日 20:04:28 +0000 (20:04 +0000)

of the 256KB limit originally enforced by a patch committed 2008年11月06日.
Per recent test results, the smaller size resulted in an undesirable decrease
in bulk data loading speed, due to COPY processing frequently getting blocked
for WAL flushing. This area might need more tweaking later, but this setting
seems to be good enough for 8.4.

src/backend/storage/buffer/README patch | blob | blame | history

src/backend/storage/buffer/freelist.c patch | blob | blame | history

diff --git a/src/backend/storage/buffer/README b/src/backend/storage/buffer/README

index 62b2eccec7934c27486d684413b9765ad6c9510b..a5470b3753c6a73626e38219d3c5466ee2bf9fc3 100644 (file)

--- a/src/backend/storage/buffer/README

+++ b/src/backend/storage/buffer/README

@@ -1,4 +1,4 @@

-$PostgreSQL: pgsql/src/backend/storage/buffer/README,v 1.16 2009年02月18日 15:58:41 heikki Exp $

+$PostgreSQL: pgsql/src/backend/storage/buffer/README,v 1.17 2009年06月22日 20:04:28 tgl Exp $

Notes About Shared Buffer Access Rules

======================================

@@ -237,7 +237,12 @@ buffer, resulting in excessive WAL flushing. Allowing VACUUM to update

Bulk writes work similarly to VACUUM. Currently this applies only to

COPY IN and CREATE TABLE AS SELECT. (Might it be interesting to make

-seqscan UPDATE and DELETE use the bulkwrite strategy?)

+seqscan UPDATE and DELETE use the bulkwrite strategy?) For bulk writes

+we use a ring size of 16MB (but not more than 1/8th of shared_buffers).

+Smaller sizes have been shown to result in the COPY blocking too often

+for WAL flushes. While it's okay for a background vacuum to be slowed by

+doing its own WAL flushing, we'd prefer that COPY not be subject to that,

+so we let it use up a bit more of the buffer arena.

Background Writer's Processing

diff --git a/src/backend/storage/buffer/freelist.c b/src/backend/storage/buffer/freelist.c

index a42cbcf72644f94ff491a4c6ec4cfd645c3e8b43..e4d32d9bbe7c06fb29761f225576696412b2c77e 100644 (file)

--- a/src/backend/storage/buffer/freelist.c

+++ b/src/backend/storage/buffer/freelist.c

@@ -9,7 +9,7 @@

*

*

* IDENTIFICATION

- * $PostgreSQL: pgsql/src/backend/storage/buffer/freelist.c,v 1.66 2009年01月01日 17:23:47 momjian Exp $

+ * $PostgreSQL: pgsql/src/backend/storage/buffer/freelist.c,v 1.67 2009年06月22日 20:04:28 tgl Exp $

*

*-------------------------------------------------------------------------

*/

@@ -368,9 +368,7 @@ GetAccessStrategy(BufferAccessStrategyType btype)

int ring_size;

/*

- * Select ring size to use. See buffer/README for rationales. (Currently

- * all cases are the same size, but keep this code structure for

- * flexibility.)

+ * Select ring size to use. See buffer/README for rationales.

*

* Note: if you change the ring size for BAS_BULKREAD, see also

* SYNC_SCAN_REPORT_INTERVAL in access/heap/syncscan.c.

@@ -385,7 +383,7 @@ GetAccessStrategy(BufferAccessStrategyType btype)

ring_size = 256 * 1024 / BLCKSZ;

break;

case BAS_BULKWRITE:

- ring_size = 256 * 1024 / BLCKSZ;

+ ring_size = 16 * 1024 * 1024 / BLCKSZ;

break;

case BAS_VACUUM:

ring_size = 256 * 1024 / BLCKSZ;

This is the main PostgreSQL git repository.