Syncing large datasets between 3 locations, long check times

Joshua · Joined: 26 Dec 2019 Posts: 3

Hello. I am currently using ViceVersa 2.5 to sync a very large dataset between 3 sites.
I have ViceVersa installed at Site A, and Site C. Site A has a synchronize job running between Site A and Site B, and Site C has a synchronize job running between Site C and Site B. This effectively allows this dataset to synchronize between all 3 sites. Changes to the dataset are made at all 3 sites.

The problem is this dataset is very large (around 900GB), and continually growing, and is taking an excessive amount of time to perform the target check (around 2 - 4 hours). The actual data transfer does not take very long in comparison to the target check. The internet speed at all 3 locations is good.

I've looked into the no Rescan options, but this does not some appropriate for my setup.

Is there anything I can do to make this setup run better? Does TGMRN offer any other products that would be able to do this type of synchronization any faster?

TGRMN Software · Site Admin Joined: 10 Jan 2005 Posts: 8762

Hi, did you try setting up the VVEngine add-on to work in real-time? Please see:

https://www.tgrmn.com/web/vvengine/vvengine.htm
https://www.tgrmn.com/web/kb/item108.htm

thanks
_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com

Joshua · Joined: 26 Dec 2019 Posts: 3

TGRMN Software · Site Admin Joined: 10 Jan 2005 Posts: 8762

Hi, yes, you would need to install VVEngine and ViceVersa on all locations to detect local changes and send the changes to the other sites in real-time.

Same setup as in this document:
https://www.tgrmn.com/web/docs/realtime.pdf

Part II - Method 2
_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com

Joshua · Joined: 26 Dec 2019 Posts: 3

I just wanted to update this thread that setting up VVEngine (and a third instance of ViceVersa at Site B) seems to be working pretty well.

I have considered setting up another set of jobs running one way real time syncs directly from Site A to Site C (and Site C to Site A), but I'm not sure how much this would rock the boat.