Replicate (Update Target) takes loooooooooooooooooooong time

[jcm4101]

Hi
Is there a clever little way of decreasing the time my replication takes?
Scenario is this.....

Server 1 contains 100,000 or so files I replicate (Update Target) nightly to Server 2
Server 1 and Server 2 are geographically distant, so replication happens over the wan (1Mb link)
There are very few new files that actually need to be copied over to Server 2 each night, so this part of the sync doesn't take that long.
The actual comparison takes 4 hours though.
Using VVPro via VVengine (Log attached below).

My issue is that I also run other backups after VV completes, but am starting to run out of backup window time....

Thanx

Dave


C:\Documents and Settings\All Users\Application Data\ViceVersa PRO 2\log1.log
Extract of File: C:\Documents and Settings\All Users\Application Data\ViceVersa PRO 2\log1.log
2007-09-28 22:00:03 : START ID: 20079281400-2639734-6092
2007-09-28 22:00:03 : ************************************************************
2007-09-28 22:00:03 : Profile Started.
2007-09-28 22:00:03 : ************************************************************
2007-09-28 22:00:03 : Version: ViceVersa PRO 2 Build 2014 (SN78)
2007-09-28 22:00:03 : Command Line: "C:\Program Files\ViceVersa Pro 2\ViceVersa.exe" "C:\Documents and Settings\administrator.NEXUSRISK\My Documents\ViceVersa PRO 2\GeroClients.fsf" /hiddenautosync /vveng
2007-09-28 22:00:03 : System: Windows Server 2003 (Domain Controller), (Terminal Services), (Terminal Services in Remote Admin Mode), (Multiprocessor Free), (x86-32 Processor) v5.2 Build:3790 Service Pack:2
2007-09-28 22:00:03 : User: NEXUSRISK.LOCAL\administrator
2007-09-28 22:00:03 : Computer: exch1-ge.nexusrisk.local
2007-09-28 22:00:03 : **** Profile: C:\Documents and Settings\administrator.NEXUSRISK\My Documents\ViceVersa PRO 2\GeroClients.fsf
2007-09-28 22:00:03 : **** - Folders -
2007-09-28 22:00:03 : **** Source: "E:\Data\Clients\"
2007-09-28 22:00:03 : **** Target: "\\Ts1-pe\Client Data Backups\Geraldton\"
2007-09-28 22:00:03 : **** Include Subfolders: Yes
2007-09-28 22:00:03 : **** Exclude files: "~*.*"
2007-09-28 22:00:03 : **** Exclude files: "thumbs.db"
2007-09-28 22:00:03 : **** Exclude files: "*gsdata*"
2007-09-28 22:00:03 : **** - Comparison Settings -
2007-09-28 22:00:03 : **** Comparison Type: Size and Timestamp
2007-09-28 22:00:03 : **** Ignore any second difference
2007-09-28 22:00:03 : **** - Execution Settings -
2007-09-28 22:00:03 : **** Method: Replication (Update Target)
2007-09-28 22:00:03 : **** Log File: "C:\Documents and Settings\All Users\Application Data\ViceVersa PRO 2\log1.log" (max 10000 KB)
2007-09-28 22:00:03 : **** Log only summary and errors
2007-09-28 22:00:03 : **** No archive for deleted/replaced source files
2007-09-28 22:00:03 : **** No archive for deleted/replaced target files
2007-09-28 22:00:06 : Checking source E:\Data\Clients\ ...
2007-09-28 22:03:03 : Checking target \\Ts1-pe\Client Data Backups\Geraldton\ ...
2007-09-29 02:49:51 : Comparing ...
2007-09-29 02:49:54 : Comparing done.
2007-09-29 02:49:55 : - Status Summary -
2007-09-29 02:49:55 : - Total Files - Source: 64,529 (8.91GB) - Target: 68,015 (9.41GB)
2007-09-29 02:49:55 : - Excluded Files - Source: 395 (4.24MB) - Target: 422 (4.65MB)
2007-09-29 02:49:55 : - Matched Files - Source: 64,126 (8.91GB) - Target: 64,126 (8.91GB)
2007-09-29 02:49:55 : - Single Files - Source: 7 (680.03KB) - Target: 3,466 (505.46MB)
2007-09-29 02:49:55 : - Newer Files - Source: 0 (0) - Target: 1 (2.57MB)
2007-09-29 02:49:55 : - Older Files - Source: 1 (2.46MB) - Target: 0 (0)
2007-09-29 02:49:55 : - Total Subfolders - Source: 54,675 - Target: 57,509
2007-09-29 02:49:55 : - Excluded Subfolders - Source: 0 - Target: 0
2007-09-29 02:49:55 : - Matched Subfolders - Source: 54,673 - Target: 54,673
2007-09-29 02:49:55 : - Single Subfolders - Source: 2 - Target: 2,836
2007-09-29 02:49:55 : - To Do - Replication (Update Target)
2007-09-29 02:49:55 : - Files to Add - Source: 0 (0) - Target: 7 (680.03KB)
2007-09-29 02:49:55 : - Files to Update - Source: 0 (0) - Target: 0 (0)
2007-09-29 02:49:55 : - Files to Delete - Source: 0 (0) - Target: 0 (0)
2007-09-29 02:49:55 : - Subfolders to Add - Source: 0 - Target: 2
2007-09-29 02:49:55 : - Subfolders to Delete - Source: 0 - Target: 0
2007-09-29 02:49:55 :
2007-09-29 02:49:55 : ---- Start Execution ----
2007-09-29 02:49:55 : ---- Replication (Update Target) (Source: E:\Data\Clients\ Target: \\Ts1-pe\Client Data Backups\Geraldton\) ----
2007-09-29 02:49:55 : ---- Start copying ----
2007-09-29 02:49:55 : -- Copying to target...
2007-09-29 02:50:31 : -- Done: 9 (680.03KB) Err: 0 (0) Skipped: 0 (0) Tot: 9 (680.03KB) --
2007-09-29 02:50:31 : -- Average Transfer Rate: 20.26KB/Sec -- Elapsed Time: 33 sec
2007-09-29 02:50:31 : Re-comparing ...
2007-09-29 02:50:31 : ---- Copying finished ----
2007-09-29 02:50:31 : ---- End ----
2007-09-29 02:50:31 : - Execution Summary -
2007-09-29 02:50:31 : - Added Files - Source: 0 (0) - Target: 7 (680.03KB)
2007-09-29 02:50:31 : - Updated Files - Source: 0 (0) - Target: 0 (0)
2007-09-29 02:50:31 : - Deleted Files - Source: 0 (0) - Target: 0 (0)
2007-09-29 02:50:31 : - Added Subfolders - Source: 0 - Target: 2
2007-09-29 02:50:31 : - Deleted Subfolders - Source: 0 - Target: 0
2007-09-29 02:50:31 : - Status Summary -
2007-09-29 02:50:31 : - Total Files - Source: 64,529 (8.91GB) - Target: 68,022 (9.41GB)
2007-09-29 02:50:31 : - Excluded Files - Source: 395 (4.24MB) - Target: 422 (4.65MB)
2007-09-29 02:50:31 : - Matched Files - Source: 64,133 (8.91GB) - Target: 64,133 (8.91GB)
2007-09-29 02:50:31 : - Single Files - Source: 0 (0) - Target: 3,466 (505.46MB)
2007-09-29 02:50:31 : - Newer Files - Source: 0 (0) - Target: 1 (2.57MB)
2007-09-29 02:50:31 : - Older Files - Source: 1 (2.46MB) - Target: 0 (0)
2007-09-29 02:50:31 : - Total Subfolders - Source: 54,675 - Target: 57,511
2007-09-29 02:50:31 : - Excluded Subfolders - Source: 0 - Target: 0
2007-09-29 02:50:31 : - Matched Subfolders - Source: 54,675 - Target: 54,675
2007-09-29 02:50:31 : - Single Subfolders - Source: 0 - Target: 2,836
2007-09-29 02:50:32 : Exit Code: 0. OK.
2007-09-29 02:50:32 : ************************************************************
2007-09-29 02:50:32 : END ID: 20079281400-2639734-6092

[TGRMN Software]

Hi, VV needs to rescan to decide which files to copy... this will be optimized in the next release of VVEngine and made much faster, but I do not have a release date for that yet, thanks
_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com

[thor]

Try rsync instead of vice versa pro. We are syncing our image directory with rsync (about 63000 images for a shop) from a windows client to a unix machine on a daily basis. Only very few images are always changed. The update takes... about 30 seconds! It's amazing.

rsync should also work between windows machines.

BTW Of course, am using ViceVersaPro for synchronisation on my local pc because it's easier to configure than rsync...

Thorsten

[TGRMN Software]

The issue is with the scanning of the remote target:

2007-09-28 22:03:03 : Checking target \\Ts1-pe\Client Data Backups\Geraldton\ ...
2007-09-29 02:49:51 : Comparing ...

It takes almost 5 hours, while for the local source it takes 3 minutes!

This will be solved in the future, when we will have a ViceVersa remote agent to install on the remote machine, which will do the scanning remotely.
_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com