The TFS FAQ mentions that the server creates "reverse delta" of both binary and text files "asynchronously as a background task".
I'm unable to find any other information on how or when this reverse delta creation happens, what factors affect how long it takes, and if it can be controlled. Is there any documentation on this
In addition to just wanting to know, I'm interested in understanding reverse delta creation as I am experimenting with using TFS as a large binary data file version control system. I'm working with a company that has to manage large binary data files (not documents and which cannot be easily converted to relational data); most of their files are 10-60MB in size, but some reach from .5GB up to 30GB in size. I realize that this may be a stretch in using TFS, but I've seen VSS used for binary data file version control.
I wasn't able to find specific file maximums or limits for TFS version control. I'm guessing that files added to TFS version control are limited to 2GB is size (since I believe the max for binary datatypes in SqlSvr2005 is 2GB). Given that SqlSvr log files are limited to 2TB and datafiles to 16TB, there should be plenty of room for large binary 'files'. (Of course, I'm sure there are performance considerations.)
When do reverse deltas take place?
Reese Bird
gripusa
The file size limit for "deltafication" is 16 MB.
When an edited file is checked in, it is queued for deltafication. The queue is maintained in the SQL db, so it won't get lost if you restart the server. There is a background thread that periodically checks the queue and computes the reverse delta. If the reverse delta is smaller, it is kept. If not, the file content is kept as it is. Prior to uploading, the client compresses all files that are smaller than 4 GB (limitation of the .NET 2.0 GZipStream class). If the compressed file is larger than the uncompressed file, the client will upload the uncompressed file.
As Richard and Chris have stated, file sizes are only limited by the disk space on your SQL server.
Buck
h3mp
There is no 2GB limit. Files bigger than 4GB are not compressed over the wire due to a Zip limitation, but that only affects upload/download time, not scalability.
Geeee
The deltification process cannot be manually controlled (it happens in the background on a low-priority thread so as not to adversely impact performance).
There IS an upper limit on how a big a file will be "deltified" - but it's pretty big, I want to say it was 40GB but I'm not sure. Brian did some work to find out how big a file was still practical for deltas, but this was months back, so I don't have all the details "in memory" :)
We most definitely support even individual revisions of files in excess of 2GB - we store them as multiple file content chunks so we're not limited by the limit in SQL server. The only practical (current) limit is the size of your hard disk, though there are probably some theoretical limits in the 2TB range.
As far as uploading/downloading multi-gigabyte files, yes you'll definitely feel the pain, but we've worked hard to make this as "doable" as possible, and we've tested the various version control operations with multi-gig files.
Hope that helps!
Vijay Guru Prasadh
server has been operational since 2004. The # of file versions and work
item versions are the key things to watch. In RTM those scale into the
millions; SP1 should add another order of magnitude or two.
The largest file I’ve personally tried was somewhere in the 30GB range.
I saw no indication that it couldn’t handle 300GB or 3TB if we had
enough disk & network resources.
Fyrus
I have a similar requirement to the one you mention; that is, using TFS VC to manage very-large binary files. Do you have any additional experience to share in this area I'm wondering how your stated performance has held up over time. I'm also wondering how large is the largest file you've source-managed. Any info is appreciated.
adorer
Yes, this definitely helps - Thank you.
As to performance, the simple tests I've run show it to be quite reasonable on our network, at the order of <10 minutes for adding/checking in a 500MB file (using the command line using a virtual server.) The next step is to build a front-end for managing our data files outside of Team Explorer (or use something like TortoiseVSTS when it comes out.)