Using rsync to Copy GoldenGate Trails from DBFS to ACFS

Posted in: Technical Track

I am currently working on project to migrate GoldenGate trail from Oracle Database File System (DBFS) to Oracle Automatic Storage Management Cluster File System (ACFS). The migration is performed in two phases.

This is the first phase, in which I create ACFS and synchronize DBFS to ACFS.

The rationale is to test if ACFS can sustain IO before actual migration and to ensure there are no issues with ACFS configuration.


GoldenGate trail from DBFS resides at /ggdata01/FS1 (source)
GoldenGate trail from ACFS resides at /ggdata02/FS1 (target)

Currently, rsync is being used with the following parameters:

-v: verbose
-r: recursive
-p: preserve permissions
-o: preserve owner
-g: preserve group
-t: preserve modifications time

Syntax to copy all directories and files from /ggdata01/FS1 to /ggdata02/FS1. Any files deleted from /ggdata01/FS1 will be deleted at /ggdata02/FS1 after rsync has completed.

+ /bin/rsync -vrpogt --delete-after /ggdata01/FS1/ /ggdata02/FS1
deleting dirdat/aa000301444
deleting dirdat/aa000301413
sent 7,603,242,915 bytes  received 908 bytes  134,570,687.13 bytes/sec
total size is 259,700,906,128  speedup is 34.16
real	0m57.447s
user	0m39.999s
sys	0m22.851s

Check for differences between two directories. File cpe is expected to be different as this is continuously changed at source while GoldenGate processes are running.

+ /bin/diff -rq /ggdata01/FS1 /ggdata02/FS1
Files /ggdata01/FS1/dirchk/E_DB.cpe and /ggdata02/FS1/dirchk/E_DB.cpe differ
Files /ggdata01/FS1/dirchk/P_TARGET.cpe and /ggdata02/FS1/dirchk/P_TARGET.cpe differ
real	86m24.642s
user	2m28.046s
sys	21m42.619s

Why did diff take so long and what was causing the delay? As it turns out, there are 1277 files from dirdat directory to compare between source and target.

+ ls /ggdata01/FS1/dirdat/
+ wc -l

Why are there so many files? Currently, the trail size is 200M which is too small; hence, many files are created.
If the trail size is increased to 500M then there will be only 511 trail files (1277*200/500).

Also, there is a gap from source ih000302802 vs target ih000302713 as more trails were generated while diff was running for 86m.

+ ls -alrt /ggdata01/FS1/dirdat/
+ tail
-rw-r-----  1 ggsuser oinstall 199999648 Jul 18 11:19 aa000302794
-rw-r-----  1 ggsuser oinstall 199998923 Jul 18 11:26 aa000302801
-rw-r-----  1 ggsuser oinstall  97219382 Jul 18 11:27 aa000302802
+ ls -alrt /ggdata02/FS1/dirdat/
+ tail
-rw-r-----  1 ggsuser oinstall 199999775 Jul 18 09:51 aa000302705
-rw-r-----  1 ggsuser oinstall 199999891 Jul 18 09:58 aa000302712
-rw-r-----  1 ggsuser oinstall 199998069 Jul 18 10:00 aa000302713

Due to the duration for rsync to complete, it is scheduled to run every two hours at even hours, i.e. 00:00 2:00, 04:00, etc…

# script to sync dbfs to acfs for goldengate - dinh - July 17, 2019
0 */2 * * * /home/oracle/working/dinh/ > /tmp/rsync_acfs_ggdata01_to_ggdata02.log 2>&1

Email log as attachment:

+ mailx -s 'SUCCESS: rsync_acfs_ggdata01_to_ggdata02.log' -a /tmp/rsync_acfs_ggdata01_to_ggdata02.log [email protected]

In conclusion, rsync is a very good feature to use for synchronization and it was used for DBFS to ACFS migration.

Want to talk with an expert? Schedule a call with our team to get the conversation started.

No comments

Leave a Reply

Your email address will not be published. Required fields are marked *