What is the practical limit of count-of-files?

[bjogen]

The forum doesn't allow me to type the full sentence in the subject. Here it comes:

What is the practical limit of count-of-files within a single VV-profile?

I'm considering using VV for "extra-safe-disaster-backup" of our office files. My idea is to run a single VV-profile for all my files at fairly seldom schedule (maybe once a month) towards a cheap-style NAS. This job can (will be allowed) to take as long as it wish - it doesn't matter (I will address a "yesterday-snapshot", so it will be able to read files without problem).

All my files are about 7 Tb. Several millions of files. So here is the question:

Will this be any problem for VV?

You guys out there - how big profiles (file-count-wise) have you ran?

[Alice]

4,500,000 files in a single job without it crashing, from a Windows Server 2008 R2 8GB Dual-core 2.13Ghz 64-bit

In my experience the more physical memory the better, server was also dedicated to the task.

Tend to deal with smaller profiles though or VVEngine. Regularly my jobs are circa 1,500,000 files, good compromise of speed / re-compare trawl / hassle creating and managing multiple jobs :

[TGRMN Software]

Yes, I think around 1 million files per profile is probably best: you can create multiple profiles and run them sequentially with VVEngine. However on 64-bit there's no real limit, but the more the files in the profile the more memory is used.
_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com

[bjogen]

Thanks for promt reply!

The reason I want to put this into one profile is this:

By creating a single profile which address the root of our main file storage, I will make sure that any modifications of the file structure is maintained.

Meaning - if new root folders are created, the profile will automatically handle this. With several profiles I need to administrate this, creating new profiles if new root folders are created.

Now I hear what you think: Why doesn't he have control of root folder creations? Well, I do. Root folders can only be created by me (IT-guys) But I also know myself... I realise that there is a risk that I would forget to create a new profile... So a single profile would "remedy" that, so to say.

I will make a test run to see if it works!

[TGRMN Software]

You can also have multiple profiles that include only root folders starting with A to M, N to P, Q to Z etc. so new folders are automatically included.

More info:
http://www.tgrmn.com/web/kb/item104.htm

But if you can have the file replication done with just one profile it's even easier!
_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com

[bjogen]

Hi again.

Some info on this, if it is of value for someone else:

I did run an initial run for my whole system. It ended up 11 million files (8 Tb). I started to copy, but I cancelled the job due to other reasons.

Bottom line: It seems to handle that amount too.