Smarter Synchronisation Please!

[sebvincent]

Imagine this scenario.

My Progs - 10GB
My Music - 20GB

You set up VV to sync these to a backup drive. No problem. But then you decide to rename 'My Progs' to 'Progs' and 'My Music' to 'Music'. So you'd think VV would be smart enough to just rename the two corresponding folders on the backup drive when you next do a backup.

But oh no. VV will detect a folder difference, and rewrite all 30GBs of data!! I can't believe this product has such a major flaw.

So any time you organise your files, by restructuring them and renaming them, in your working folder and you then do a sync - VV overwrites all the original data in the backup, instead of being smart enough to just rename and restructure the corresponding data in the backup. What an inefficient and time consuming way of doing things.

Synctoy 2.0 is free and is capable of doing this, so why on earth can VV not do it?! I would really like to hear from the developers on this.

Thanks in advance.

[LUIGI]

Yes, it'would be really a great improvement.

[jormartr]

I never heard about a software able to do this, and I've used so many...

It would be a nice feature, yes! I always have to remember to rename the other side of the backup when I rename on the source a big structure.

It is a flaw: obviously not.

[Matz]

Oh yes. I am also looking for that feature long time. Again and again I have the same problem like you when I re-arranged when cleaning up my data on the source drive from time to time.

[teknikool]

Matz, here we are nearly a year later since you've made that request. I just requested exactly this on another page. I'll have to give SyncToy a second chance. I love the features in VVP and VVE, but this is my biggest issue with it. It took me 35 minutes to shift around half a terabyte of data, and in VVP, here we are 2 DAYS later. With another 2-3 DAYS more... Thankfully it's the weekend? I'll check it on Monday.

Please TGRMN engineers!!! HEAR OUR CRY!

[Guest]

I also think this is a reat idea. I am faced with even more data (2TB), I also always do a manual rename of both folder sides to avoid that problem, but than you can't really have the sync happening automatically or need to disable it since it might start right in the middle when you are doing somehting.
Suggestion 2 sound like a good idea!
In this way you at least wouldn't need to get out of the software and you would have both both fodlers right next to each other making it a much quicker work!

[Josh18615]

I have seen people ask for "Smarter Synchronization". I would like this too and I have detailed suggestions and worflows in mind.

People don't like to wait for copy operations, however, there is a more important reason to implement this feature. I want to reorganize my entire hard drive, but still have CRC assurance that nothing got lost or damaged. I also would like a backup solution I trust (ViceVersa) to help me manage the process effectively.

I feel good about my data because of ViceVersa, but I CAN NOT ever reorganize no matter how silly and outdated my file structure becomes. I am afraid to reorganize, because then VV Synch can't compare my files for me. If it can't compare, I've just lost the assurance ViceVersa gives me that nothing was lost or damaged. And seriously, when do you need that assurance more than after a big reorganization effort?

I would happily pay again or buy an add-on for this kind of "file reorganization assurance". Add-on is fine, but I think these features need to have some integration with existing VV features. Here are some workflows/user stories of how I would work this in the VV UI:

1) I should just turn on the "REORGANIZE" feature with a simple menu command.
--- a) This would cause VV to store and track changes to many statistics about all files and folders (filename, size, CRC, full directory path). This would not be a full backup, but a database/file of statistics about each file and folder. These statistics would allow intelligent reorganization help and assurance. (I already have 10 off-site backups that are synchronized on a rotating basis to satisfy the full backup need so this is different).
--- b) I would run a synch against as many of my backups as possible to add their respective statistics before a major reorganization effort. VV should track change history stats about each backup set their comparison to other data sets.
--- c) Change statistics should go with each backup set and synchronize between instances of VV. For instance, some of my backup sets are two-way dynamic. Like synchronization between my laptop and desktop, therefore data can change on either side of the synch. I should be able to “reorganize” on either side and VV should still give me “assurance”.
--- d) For backup sets without these new statistics included, VV should assume the organization matched the original organization. Some thoughtful algorithms could find matching files that have moved even without the stored statistics. For instance if CRC matches, it should be treated as the same file regardless of locations. Same could be true of filename/size/modified date. And possibly a few more combinations.

2) There should be a new filter option to show/hide “moved/renamed” files and folders. It should also somehow indicate by color or a special icon that were moved and another for renamed. Please add separate filters too. This is in addition to the filters that already show/hide files as either identical, newer/older, or single. It would just do these things regardless of where files/folders were moved or renamed.

3) VV should CRC files against one another regardless of whether their directory paths match. This option should be available BEFORE synchronizing the “reorganization”. I always like to synch in a few steps so I can quickly scan through the affected files. (I frequently use the existing VV filters for this)

4) When ready, I should be able to use VV to intelligently "move" rather than copy files within each of my backup drives. It should allow similar control over what is modified on each execution like it does now (by using filters, collapsing directories, changing the synch option, etc).

5) I would also like to find duplicate files by name OR name/size OR name/size/CRC and so on. I have often wondered how many duplicated files I have. Tracking more file statistics may make this possible too.

This “reorganization assurance” is a critical feature for me and apparently for many other people too. Not only would this save on copying time, it also would offer more assurance that there were no mistaken deletions or corruptions in my reorganization effort. Every file still could have CRC comparisons and other checks after moving to its new location. If possible I want to CRC after a rename too (though I realize this may be a bit difficult to implement if filename is part of the CRC value).

I hope to see many “file reorganization assurance” features soon. I am already a VV Pro and VVEngine customer and I would gladly pay again for a new version with these features.

Thank you.

[P.Bristow (just visiting)]

[TGRMN Software]

To identify files that have been moved is very risky or very slow.
If a file has been renamed, its name has changed but timestamp and file size are still the same. But how to be sure that the file is really the same file, renamed, and not a new file, with different content? For that yoy would need to check the file content itself which is very slow...
_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com

[Really]

Admin, it's not that difficult, VV already has got a sync database, in which VV could store name changes and from which it would then check those.

Anyway, much more important than file renames are FOLDER renames and folder moves, i.e. you have some files in folder A which is in folder B, but you move them into folder C which is in folder D which is in folder E, and you then rename folder D and folder B or whatever. A good sync program is able to store these changes, and to check for them.

Of course, such a program must continuously run in the background in order to store any such changes within its database.

There are LOTS of such threads here, so, Admin, you must be aware that people REALLY WANT this feature, and VV being one of the more expensive offerings, I think people are entitled to want it indeed.

As I've said in another thread, I'm going to buy VV as soon as it will get this feature, since, as said there, I think VV has the most pleasant and clear comparison table of them all.

[pb]

In the case of renamed or moved folders there are numerous approaches that can be useful.

For example, if a folder is renamed or moved, there will be a folder in source that is not in target, and vice versa. Only such pairs need to be compared. All the filename/size/timestamp info is already available for such folders, so this will typically not be an expensive comparison. If there are sufficient identical files, then a rename is in order. If the comparison for moved/renamed folders is done so that it works outward from nearby places in the directory tree, that will probably also save time in the typical case.

Of course, whether to look for such directory changes would be a user option.

A harder problem is what to do about archived files that have siimply been moved, or appear to have moved due to a moved or renamed folder. In that case, if you simply move the archived file to its new place, you have lost the information of where the file was previously. In that situation, it is probably better to use a scheme where the old location is replaced with a symlink (junction) pointing to the new location, which should also somehow encode when the change took place.

--peter