Small File copy performance?

[swyck]

I've run some tests on WAN file copies (Backups). When copying large files, at least 1 MB in size, I can get nice bandwidth performance up to 10 MB\sec when using larger buffer sizes.

However when I copy numerous small files, 1KB in size or so, I see the bandwidth drop to around 5-6 KB/sec. That is a very large performance hit for small files.

Is there anything that can be done to increase performance when copying smaller files? I don't have much control over the makeup of the source files, and there may be larger numbers of small files or small numbers of very large files (>1GB).

Here are the results of copying about 1MB worth of files using 2097152 buffer size. Using smaller buffer sizes does not increase performance of smaller file sizes but does decrease performance for larger sizes.

1kb 5.3kb/sec
10kb 38kb/sec
100kb 420kb/sec
1mb 5200kb/sec
50mb 10520kb/sec

[TGRMN Software]

Hello swyck,

it's normal to get slower speed while copying very small files. The overhead is with opening, closing, changing the timestamp of files. For very small files that overhead may account to up to 99% of the total copy time!

The ViceVersa PRO copy speed should be comparable to what you get with Windows Explorer.

thanks!
_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com

[swyck]

[swyck]