Copy Buffer size in VV6

[Alice]

From the helpfile :

Buffer size (bytes) : This is the size of the memory buffer used to copy files. You can adapt this value to the type of network you are using. Usually larger values will result in faster copy performance.

When I set this to the maximum

8388608

the copy speed across my cabled Ethernet is much faster (W11 ► Server 2025) than any other setting. Presumably that's an 8 MegaByte Buffer ?

Both my Desktop and Server have 32GB RAM each, would there be any advantage in VV6 supporting a much larger Buffer in that scenario...? Is there a 'sweet spot' at 8MB ?

[TGRMN Software]

Hi Alice, we can build a new version of ViceVersa PRO 6 with the option of increasing the buffer even further if interested. There will be a limit where increasing the buffer does not help.

According to AI:

When copying files, using an optimized copy buffer size can significantly accelerate the process. The choice of the buffer size depends on the context (e.g., file sizes, storage medium, and operating system). Here are some tips:

General Recommendations:

Buffer Size Range:

• A buffer size of 4 KB (4096 bytes) to 1 MB (1048576 bytes) is commonly effective.
  
• For most systems, a buffer of 64 KB to 256 KB often provides a good balance between speed and memory usage.
  

Larger Buffers for Larger Files:

• For copying large files (e.g., multi-gigabyte files), larger buffer sizes (e.g., 1 MB or higher) can reduce the number of read/write operations, improving performance.
  

Medium-Specific Considerations:

• HDDs: Medium-sized buffers (64 KB to 256 KB) work well due to spinning disk latency.
  
• SSDs: Larger buffers (e.g., 1 MB) may optimize performance due to faster read/write speeds and low latency.
  
• Network Transfers: Test different sizes, but 8 KB to 64 KB is often recommended to reduce network overhead.
  

_________________
--
TGRMN Software Support
http://www.tgrmn.com
http://www.compareandmerge.com

[Alice]

A 'test' version would be great, thank you

[Alice]

Further bit of musing on this subject.

I just created a RAM Disk (similar principle to 'buffer' maybe), then used VV6 to copy data to it.....er wow :

-- Done: 45,992 (23.02GB) Err: 0 (0) Skipped: 0 (0) Tot: 45,992 (23.02GB) --
-- Average Transfer Rate: 1.54GB/Sec -- Elapsed Time: 14 sec

RAM in that PC is DDR5 4800MHz, SSD PCIe Gen4 x4.

[TGRMN Software]

RAM disks as super fast!!

Regarding the copy buffer size: I discovered that you can actually use a 16 MB buffer, bypassing the limit of 8,388,608 bytes. This capability is undocumented, but you can test it using the following method:

1. Create a profile with an 8,388,608-byte copy buffer in ViceVersa and save it.
2. Open the corresponding .fsf file in a text editor and change 8388608 to 16777216.
3. Save the changes and reopen the profile in ViceVersa. The application will now use a 16 MB copy buffer.

Important Notes:
- The maximum copy buffer size is 16 MB. Bigger than that will not be allowed in ViceVersa.
- The copy buffer is applied per thread. For instance, using 128 threads with a 16 MB copy buffer will require 128 × 16 MB = 2 GB of memory just for the copy.
- When testing, ensure you clear the disk cache after each copy to prevent caching from interfering with the results. A tool like RAMMap can help: Download RAMMap.
https://learn.microsoft.com/en-us/sysinternals/downloads/rammap

cheers

--
www.tgrmn.com

[Alice]

Ok thank you.

Tried the Buffer @ 16777216 with 128 Threads in VV6 Build 6011 under W11 24H2 with 32GB RAM, not too much of a difference I have to say. Maybe 10MB/sec faster which could help some people.

For me it only shortened the copy by 15 seconds overall, which I'll still take !

Noticed when editing the .fsf within VV it did indeed prompt as follows :

'Copy Buffer 0 out of bounds (32-16777216)'