Using rsync to Copy GoldenGate Trails from DBFS to ACFS

Posted in: Technical Track

I am currently working on project to migrate GoldenGate trail from Oracle Database File System (DBFS) to Oracle Automatic Storage Management Cluster File System (ACFS). The migration is performed in two phases.

This is the first phase, in which I create ACFS and synchronize DBFS to ACFS.

The rationale is to test if ACFS can sustain IO before actual migration and to ensure there are no issues with ACFS configuration.

Environment:

GoldenGate trail from DBFS resides at /ggdata01/FS1 (source)
GoldenGate trail from ACFS resides at /ggdata02/FS1 (target)

Currently, rsync is being used with the following parameters:

-v: verbose
-r: recursive
-p: preserve permissions
-o: preserve owner
-g: preserve group
-t: preserve modifications time
--delete-after

Syntax to copy all directories and files from /ggdata01/FS1 to /ggdata02/FS1. Any files deleted from /ggdata01/FS1 will be deleted at /ggdata02/FS1 after rsync has completed.

+ /bin/rsync -vrpogt --delete-after /ggdata01/FS1/ /ggdata02/FS1
dirdat/
dirdat/aa000302676
.....
dirdat/aa000302713

deleting dirdat/aa000301444
.....
deleting dirdat/aa000301413

sent 7,603,242,915 bytes  received 908 bytes  134,570,687.13 bytes/sec
total size is 259,700,906,128  speedup is 34.16

real	0m57.447s
user	0m39.999s
sys	0m22.851s

Check for differences between two directories. File cpe is expected to be different as this is continuously changed at source while GoldenGate processes are running.

+ /bin/diff -rq /ggdata01/FS1 /ggdata02/FS1
Files /ggdata01/FS1/dirchk/E_DB.cpe and /ggdata02/FS1/dirchk/E_DB.cpe differ
.....
Files /ggdata01/FS1/dirchk/P_TARGET.cpe and /ggdata02/FS1/dirchk/P_TARGET.cpe differ

real	86m24.642s
user	2m28.046s
sys	21m42.619s

Why did diff take so long and what was causing the delay? As it turns out, there are 1277 files from dirdat directory to compare between source and target.

+ ls /ggdata01/FS1/dirdat/
+ wc -l
1277

Why are there so many files? Currently, the trail size is 200M which is too small; hence, many files are created.
If the trail size is increased to 500M then there will be only 511 trail files (1277*200/500).

Also, there is a gap from source ih000302802 vs target ih000302713 as more trails were generated while diff was running for 86m.

+ ls -alrt /ggdata01/FS1/dirdat/
+ tail
-rw-r-----  1 ggsuser oinstall 199999648 Jul 18 11:19 aa000302794
.....
-rw-r-----  1 ggsuser oinstall 199998923 Jul 18 11:26 aa000302801
-rw-r-----  1 ggsuser oinstall  97219382 Jul 18 11:27 aa000302802

+ ls -alrt /ggdata02/FS1/dirdat/
+ tail
-rw-r-----  1 ggsuser oinstall 199999775 Jul 18 09:51 aa000302705
.....
-rw-r-----  1 ggsuser oinstall 199999891 Jul 18 09:58 aa000302712
-rw-r-----  1 ggsuser oinstall 199998069 Jul 18 10:00 aa000302713

Due to the duration for rsync to complete, it is scheduled to run every two hours at even hours, i.e. 00:00 2:00, 04:00, etc…

# script to sync dbfs to acfs for goldengate - dinh - July 17, 2019
0 */2 * * * /home/oracle/working/dinh/acfs_ggdata02_rsync.sh > /tmp/rsync_acfs_ggdata01_to_ggdata02.log 2>&1

Email log as attachment:

+ mailx -s 'SUCCESS: rsync_acfs_ggdata01_to_ggdata02.log' -a /tmp/rsync_acfs_ggdata01_to_ggdata02.log me@mail.com

In conclusion, rsync is a very good feature to use for synchronization and it was used for DBFS to ACFS migration.

email

Interested in working with Michael? Schedule a tech call.

No comments

Leave a Reply

Your email address will not be published. Required fields are marked *