Re: Handling large files with GIT

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Florian Weimer
Date: Wednesday, February 8, 2006 - 2:20 pm

* Martin Langhoff:


To my surprise, it's not that bad.  The Debian testing-security team
uses a single 1.8 MB file (400 KB compressed) to keep vulnerability
data.  Most changes to that file involve just a few lines.  But even
in this extreme case, git doesn't compare too badly against Subversion
if you pack regularly (but not too often).  Disk usage is actually
*below* Subversion FSFS even with --depth=10 (the default,
unfortunately a bit hard to override).

I plan to do another experiment for GCC, which contains marvels such
as:

  35905  126056 1379093 gcc/ChangeLog-2005
  12610   61215  417584 gcc/combine.c

But the outcome will likely be quite similar to the secure-testing
case: comparable disk space usage, not a difference in the order of
one or more magnitudes.

But Subversion still has got a significant adventage: I can get a
working copy without downloading full history (several gigabytes in
GCC's case).  There's also the slight drawback that you shouldn't pack
too often, otherwise you'll reduce its effectiveness.  You can always
run "git-repack -a -d", but it's rather expensive.  This means that
you need to keep compressed fulltexts from a few dozen revisions, but
I don't think this is a huge burden.  All in all, the compressed
fulltexts/packs model is a pretty good trade-off between disk usage,
end user usability nad code complexity.

In your mbox case, you should simply try Maildir.  The tree object
(which lists all files in the Maildir folder) will still be rather
large (about 40 to 50 bytes per message stored), though.
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Handling large files with GIT, Martin Langhoff, (Wed Feb 8, 2:14 am)
Re: Handling large files with GIT, Johannes Schindelin, (Wed Feb 8, 4:54 am)
Re: Handling large files with GIT, Linus Torvalds, (Wed Feb 8, 9:34 am)
Re: Handling large files with GIT, Linus Torvalds, (Wed Feb 8, 10:01 am)
Re: Handling large files with GIT, Junio C Hamano, (Wed Feb 8, 1:11 pm)
Re: Handling large files with GIT, Florian Weimer, (Wed Feb 8, 2:20 pm)
Re: Handling large files with GIT, Martin Langhoff, (Wed Feb 8, 3:35 pm)
Re: Handling large files with GIT, Greg KH, (Wed Feb 8, 9:54 pm)
Re: Handling large files with GIT, Martin Langhoff, (Wed Feb 8, 10:38 pm)
Re: Handling large files with GIT, Ben Clifford, (Sun Feb 12, 6:26 pm)
Re: Handling large files with GIT, Linus Torvalds, (Sun Feb 12, 8:42 pm)
Re: Handling large files with GIT, Martin Langhoff, (Sun Feb 12, 9:40 pm)
Re: Handling large files with GIT, Linus Torvalds, (Sun Feb 12, 9:57 pm)
Re: Handling large files with GIT, Linus Torvalds, (Sun Feb 12, 10:05 pm)
Re: Handling large files with GIT, Jeff Garzik, (Sun Feb 12, 10:55 pm)
Re: Handling large files with GIT, Keith Packard, (Sun Feb 12, 11:07 pm)
Re: Handling large files with GIT, Linus Torvalds, (Mon Feb 13, 9:19 am)
Re: Handling large files with GIT, Ian Molton, (Mon Feb 13, 4:17 pm)
Re: Handling large files with GIT, Martin Langhoff, (Mon Feb 13, 4:19 pm)
Re: Handling large files with GIT, Martin Langhoff, (Mon Feb 13, 5:07 pm)
Re: Handling large files with GIT, Johannes Schindelin, (Tue Feb 14, 11:56 am)
Re: Handling large files with GIT, Linus Torvalds, (Tue Feb 14, 12:52 pm)
Re: Handling large files with GIT, Sam Vilain, (Tue Feb 14, 2:21 pm)
Re: Handling large files with GIT, Linus Torvalds, (Tue Feb 14, 3:01 pm)
Re: Handling large files with GIT, Junio C Hamano, (Tue Feb 14, 3:30 pm)
Re: Handling large files with GIT, Sam Vilain, (Tue Feb 14, 5:40 pm)
Re: Handling large files with GIT, Junio C Hamano, (Tue Feb 14, 6:39 pm)
Re: Handling large files with GIT, Linus Torvalds, (Tue Feb 14, 7:05 pm)
Re: Handling large files with GIT, Martin Langhoff, (Tue Feb 14, 7:07 pm)
Re: Handling large files with GIT, Linus Torvalds, (Tue Feb 14, 7:18 pm)
Re: Handling large files with GIT, Linus Torvalds, (Tue Feb 14, 7:33 pm)
Re: Handling large files with GIT, Linus Torvalds, (Tue Feb 14, 8:58 pm)
Re: Handling large files with GIT, Sam Vilain, (Tue Feb 14, 9:03 pm)
Re: Handling large files with GIT, Junio C Hamano, (Wed Feb 15, 2:54 am)
Re: Handling large files with GIT, Linus Torvalds, (Wed Feb 15, 8:44 am)
Re: Handling large files with GIT, Linus Torvalds, (Wed Feb 15, 10:16 am)
Re: Handling large files with GIT, Linus Torvalds, (Wed Feb 15, 8:25 pm)
Re: Handling large files with GIT, Junio C Hamano, (Wed Feb 15, 8:29 pm)
Re: Handling large files with GIT, Fredrik Kuivinen, (Thu Feb 16, 1:32 pm)