Postgres replication: Do I need to rsync pg_xlog files if I have archiving setup in master?

Question

I am trying to replicate a master to slave, without shutting down my master.

In the tutorial here: http://wiki.postgresql.org/wiki/Binary_Replication_Tutorial, they recommend these steps:

1. pg_start_backup()
2. rsync -av --exclude pg_xlog --exclude postgresql.conf --exclude postgresql.pid \ data/* :/var/lib/postgresql/data/
3. pg_stop_backup()
4. rsync -av data/pg_xlog :/var/lib/postgresql/data/

Steps 1-3 are straightforward. However, If I setup archiving in the master and archiving in my slave, do I need to execute step 4? Will it mess things up? Should I NOT setup archiving until everything is rsync'd and my slave is running?

My master's config file looks like this:

wal_level = hot_standby
archive_mode = on
archive_command = 'rsync -a %p <INSERT SLAVE IP>:/var/lib/pgsql/data/archive/%f'

Likewise, my slave has a recovery.conf with a restore_command.

Answer 1

That tutorial appears to be old, and also says "work in progress" at the top. I'll amend it if I have time. Please refer to the main documentation.

If you have continuous archiving set up you don't need to manually rsync the WAL, because your archive_command will do that for you.

You may also wish to consider using pg_basebackup instead of manually rsync'ing the datadir. It'll take care of everything for you over the PostgreSQL replication protocol. If you weren't using WAL archiving you'd use -X stream mode to get it to copy the WAL for you too, but you don't need that.

• Henley Wing Chiu: If you have archiving setup, is it technically possible for the rsync for the initial data files to take a very very long time, and still have replication succeed if you have a very busy database?
• Craig Ringer: @HenleyChiu Yes, because the replica will catch up to the master using archives and begin streaming once it's caught up. As always, test.