Author |
Message |
ASPweb
Joined: 30 Jul 2015 Posts: 4
|
Posted: Thu Jul 30, 2015 9:16 pm Post subject: ViceVersa stuck "preparing display..." |
|
|
We are trying to backup 16.1 million files containing 645GB of data from one server to another. It took just over 3 hours to scan the source disk and almost another hour to compare 16.1 million files on the source to 21k files on the target, then it was stuck for just over an hour with some message about preparing a display, with instruction to switch to a flat panel display that couldn't be followed because the menu items did not exist.
We really need this product to work for us, but I must say, getting started has been a frustrating experience, particularly when popups like "preparing display" are hung for more than an hour. Had we been impatient, one might have aborted prematurely.
For your help, here are the logs while we wait yet again for the execution of the file copy to begin.
2015-07-30 16:01:38 :
2015-07-30 16:01:38 : -
2015-07-30 16:01:38 : ************************************************************
2015-07-30 16:01:38 : Profile Started.
2015-07-30 16:01:38 : ************************************************************
2015-07-30 16:01:38 : Version: ViceVersa PRO 2.5 Build 2515 64-bit (SN392)
2015-07-30 16:01:38 : Command Line: "C:\Program Files\ViceVersa Pro\ViceVersa.exe"
2015-07-30 16:01:38 : System: Windows Server 2003, (Standard Edition), (64 Bit Edition), (x86-64 Processor) v5.2 Build:3790 Service Pack:2
2015-07-30 16:01:38 : User: xxxx
2015-07-30 16:01:38 : Computer: xxxx
2015-07-30 16:01:38 : **** Profile: d:\viceversa\files.fsf
2015-07-30 16:01:38 : **** - Folders -
2015-07-30 16:01:38 : **** Source: "D:\files\"
2015-07-30 16:01:38 : **** Target: "\\xxxx\D$\Files\"
2015-07-30 16:01:38 : **** Include Subfolders: Yes
2015-07-30 16:01:38 : **** Create sources/targets that do not exist
2015-07-30 16:01:38 : **** - Comparison Settings -
2015-07-30 16:01:38 : **** Comparison Type: Size and Timestamp
2015-07-30 16:01:38 : **** - Execution Settings -
2015-07-30 16:01:38 : **** Method: Backup (Mirror Source to Target)
2015-07-30 16:01:38 : **** No overwrite / read-only / error confirmations
2015-07-30 16:01:38 : **** Tracking Database: "D:\ViceVersa\vvdb1.tdb"
2015-07-30 16:01:38 : **** Do not rescan source
2015-07-30 16:01:38 : **** Log File: "D:\ViceVersa\log1.log" (max 10000 KB) (empty log file each run)
2015-07-30 16:01:38 : **** Log only summary and errors
2015-07-30 16:01:38 : **** No archive for deleted/replaced source files
2015-07-30 16:01:38 : **** No archive for deleted/replaced target files
2015-07-30 16:01:38 : **** Process Priority: High
2015-07-30 16:01:38 : **** Use Windows native file copy
2015-07-30 16:01:38 : **** Network friendly folder scanning
2015-07-30 16:01:38 : Using network friendly folder scanning.
2015-07-30 16:01:38 : Checking source D:\files\ ...
2015-07-30 16:01:38 : [INFO] Rescanning needed for source folder D:\files\. Tracking database does not contain no-rescan info yet.
2015-07-30 19:09:51 : Total found: 16,389,181
2015-07-30 19:16:34 : Checking target \\xxxx\D$\Files\ ...
2015-07-30 19:16:35 : Total found: 21,982
2015-07-30 19:16:36 : Comparing ...
2015-07-30 19:57:31 : Comparing done.
2015-07-30 21:01:49 : - Status Summary -
2015-07-30 21:01:49 : - Total Files - Source: 16,156,212 (645.13GB) - Target: 21,473 (4.61GB)
2015-07-30 21:01:49 : - Excluded Files - Source: 0 (0) - Target: 0 (0)
2015-07-30 21:01:49 : - Matched Files - Source: 21,456 (4.49GB) - Target: 21,456 (4.49GB)
2015-07-30 21:01:49 : - Single Files - Source: 16,134,739 (640.52GB) - Target: 0 (0)
2015-07-30 21:01:49 : - Newer Files - Source: 17 (120.96MB) - Target: 0 (0)
2015-07-30 21:01:49 : - Older Files - Source: 0 (0) - Target: 17 (116.84MB)
2015-07-30 21:01:49 : - Total Subfolders - Source: 232,969 - Target: 509
2015-07-30 21:01:49 : - Excluded Subfolders - Source: 0 - Target: 0
2015-07-30 21:01:49 : - Matched Subfolders - Source: 509 - Target: 509
2015-07-30 21:01:49 : - Single Subfolders - Source: 232,460 - Target: 0
2015-07-30 21:01:49 : Saving tracking database...
2015-07-30 21:02:43 : Tracking database saved.
Just as this topic was being created, the process of preparing the display ended, but there is still some explanation needed to understand why these processes take so long.
The server has two Intel E4750 2.0Ghz cores and 8GB of RAM with a 48 disk RAID10 SAN with over 6,000 IOPS available. While the processors were reach 70-80% use, there was still plenty of performance available.
Please advise. |
|
Back to top |
|
|
ASPweb
Joined: 30 Jul 2015 Posts: 4
|
Posted: Thu Jul 30, 2015 10:56 pm Post subject: Still churning away... |
|
|
Since I made my last post, with the source and target panes being displayed and having click execute, the system has been churning away at the CPU, all of the 8GB of system RAM and not a single file has been copied to the target server.
I really don't understand what could be happening. We already scanned the source file system of 16.1 million files, compared to the target server, created the vvdb database, so why is this program using CPU and RAM and not copying a single file?
2015-07-30 19:57:31 : Comparing done.
2015-07-30 21:01:49 : - Status Summary -
2015-07-30 21:01:49 : - Total Files - Source: 16,156,212 (645.13GB) - Target: 21,473 (4.61GB)
2015-07-30 21:01:49 : - Excluded Files - Source: 0 (0) - Target: 0 (0)
2015-07-30 21:01:49 : - Matched Files - Source: 21,456 (4.49GB) - Target: 21,456 (4.49GB)
2015-07-30 21:01:49 : - Single Files - Source: 16,134,739 (640.52GB) - Target: 0 (0)
2015-07-30 21:01:49 : - Newer Files - Source: 17 (120.96MB) - Target: 0 (0)
2015-07-30 21:01:49 : - Older Files - Source: 0 (0) - Target: 17 (116.84MB)
2015-07-30 21:01:49 : - Total Subfolders - Source: 232,969 - Target: 509
2015-07-30 21:01:49 : - Excluded Subfolders - Source: 0 - Target: 0
2015-07-30 21:01:49 : - Matched Subfolders - Source: 509 - Target: 509
2015-07-30 21:01:49 : - Single Subfolders - Source: 232,460 - Target: 0
2015-07-30 21:01:49 : Saving tracking database...
2015-07-30 21:02:43 : Tracking database saved.
2015-07-30 21:12:48 :
2015-07-30 21:12:48 : ---- Start Execution ----
2015-07-30 21:12:48 : ---- Backup (Mirror Source to Target) (Source: D:\files\ Target: \\xxxx\D$\Files\) ----
2015-07-30 21:12:48 : ---- Start copying ----
It's now 22:54 server time... seriously, what is this software doing? Rescanning? Is so, why? |
|
Back to top |
|
|
TGRMN Software Site Admin
Joined: 10 Jan 2005 Posts: 8758
|
|
Back to top |
|
|
ASPweb
Joined: 30 Jul 2015 Posts: 4
|
Posted: Fri Jul 31, 2015 8:01 pm Post subject: |
|
|
Thanks for the recommendation, however, this doesn't really make sense to me.
Why would I want multiple profiles to run when one profile is already using up too much processing and RAM?
It's not practical to do this because my client has an images directory that contains tens of thousands of customerID folders with their images. I can't easily split them up into jobs/profiles, even using a RegEx style parameter as you show in your example. I would have to create hundreds of jobs/profiles.
I also don't understand why your program can't keep track of the files with one profile using your database. I have tried repeatedly to get the database populated and use it with the norescan option and it simply can't handle this many files. If there is some secret setting to make it work, I would appreciate knowing about it.
Fortunately, I am able to copy all 16.1m files with RoboCopy, which is using minimal RAM and CPU resources. While this is a great tool, I need your software to be able to keep the files in real-time sync without the need to rescan every time.
Once the initial RoboCopy backup is done, I'll give your program one more shot to see if it can handle the 16.1 million files and half a million folders and keep them in sync.
By the way, you never really explained this whole long delay with the preparing display issue. |
|
Back to top |
|
|
TGRMN Software Site Admin
Joined: 10 Jan 2005 Posts: 8758
|
Posted: Sat Aug 01, 2015 2:26 am Post subject: |
|
|
Hi, ViceVersa and Robocopy do work differently because Robocopy does not have a user interface so it does not need to keep all files in memory and create the graphical comparison. That's where the time is spent for the message you are seeing. If you split the job into multiple profiles and run them sequentially the problem is solved and you still have a visual feedback that source and target are really matching. thanks _________________ --
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com |
|
Back to top |
|
|
ASPweb
Joined: 30 Jul 2015 Posts: 4
|
Posted: Sat Aug 01, 2015 5:58 pm Post subject: |
|
|
I really don't care about a graphical display.
I only want to scan the file system once, sync the files, then keep them in sync in real time. I really don't mind an occasional rescan, particularly if I can say when and how often, but it is taking 3 hours to scan, plus the time to build the graphical display that I don't need.
Can I not do this without a graphical display? If so, how? |
|
Back to top |
|
|
TGRMN Software Site Admin
Joined: 10 Jan 2005 Posts: 8758
|
Posted: Mon Aug 03, 2015 12:52 am Post subject: |
|
|
Hi, the only way is to split the job into multiple profiles and run them sequentially. This also helps that if a full rescan is needed it can be limited to one profile and can be much quicker. thanks _________________ --
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com |
|
Back to top |
|
|
|
|
You can post new topics in this forum You can reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © phpBB Group
|