Re: [PATCH] Teach "git add" and friends to be paranoid | KernelTrap

Re: [PATCH] Teach "git add" and friends to be paranoid

view
thread

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

[view in full thread]

From: Paolo Bonzini

Subject: Re: [PATCH] Teach "git add" and friends to be paranoid

Date: Monday, February 22, 2010 - 5:59 am

On 02/18/2010 06:36 AM, Junio C Hamano wrote:
quoted text> Nicolas Pitre<nico@fluxnic.net>  writes:
>
>> It is likely to have better performance if the buffer is small enough to
>> fit in the CPU L1 cache.  There are two sequencial passes over the
>> buffer: one for the SHA1 computation, and another for the compression,
>> and currently they're sure to trash the L1 cache on each pass.
>
> I did a very unscientific test to hash about 14k paths (arch/ and fs/ from
> the kernel source) using "git-hash-object -w --stdin-paths" into an empty
> repository with varying sizes of paranoia buffer (quarter, 1, 4, 8 and
> 256kB) and saw 8-30% overhead.  256kB did hurt and around 4kB seemed to be
> optimal for my this small sample load.
>
> In any case, with any size of paranoia, this hurts the sane use case

Because by mmaping + memcpying you are getting the worst of both cases: 
you get a page fault per page like with mmap, and touch memory twice 
like with read.

Paolo
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:

Re: Bug#569505: git-core: 'git add' corrupts repository if ..., Jonathan Nieder, (Thu Feb 11, 5:27 pm)

Re: Bug#569505: git-core: 'git add' corrupts repository if ..., Zygo Blaxell, (Thu Feb 11, 6:23 pm)

Re: 'git add' corrupts repository if the working directory ..., Jonathan Nieder, (Sat Feb 13, 5:12 am)

Re: 'git add' corrupts repository if the working directory ..., Ilari Liusvaara, (Sat Feb 13, 6:39 am)

Re: 'git add' corrupts repository if the working directory ..., Thomas Rast, (Sat Feb 13, 7:39 am)

Re: 'git add' corrupts repository if the working directory ..., Ilari Liusvaara, (Sat Feb 13, 9:29 am)

Re: 'git add' corrupts repository if the working directory ..., Dmitry Potapov, (Sat Feb 13, 3:09 pm)

Re: 'git add' corrupts repository if the working directory ..., Zygo Blaxell, (Sat Feb 13, 3:37 pm)

[PATCH] don't use mmap() to hash files, Dmitry Potapov, (Sat Feb 13, 6:18 pm)

mmap with MAP_PRIVATE is useless (was Re: Bug#569505: git- ..., Paolo Bonzini, (Sat Feb 13, 6:36 pm)

Re: [PATCH] don't use mmap() to hash files, Junio C Hamano, (Sat Feb 13, 6:37 pm)

Re: mmap with MAP_PRIVATE is useless, Junio C Hamano, (Sat Feb 13, 6:53 pm)

Re: [PATCH] don't use mmap() to hash files, Johannes Schindelin, (Sat Feb 13, 6:53 pm)

Re: [PATCH] don't use mmap() to hash files, Junio C Hamano, (Sat Feb 13, 7:00 pm)

Re: mmap with MAP_PRIVATE is useless, Paolo Bonzini, (Sat Feb 13, 7:11 pm)

Re: [PATCH] don't use mmap() to hash files, Dmitry Potapov, (Sat Feb 13, 7:18 pm)

Re: [PATCH] don't use mmap() to hash files, Dmitry Potapov, (Sat Feb 13, 7:42 pm)

[PATCH v2] don't use mmap() to hash files, Dmitry Potapov, (Sat Feb 13, 8:05 pm)

Re: [PATCH] don't use mmap() to hash files, Junio C Hamano, (Sat Feb 13, 8:14 pm)

Re: [PATCH] don't use mmap() to hash files, Jakub Narebski, (Sun Feb 14, 4:07 am)

Re: [PATCH] don't use mmap() to hash files, Thomas Rast, (Sun Feb 14, 4:14 am)

Re: [PATCH] don't use mmap() to hash files, Junio C Hamano, (Sun Feb 14, 4:46 am)

Re: [PATCH] don't use mmap() to hash files, Paolo Bonzini, (Sun Feb 14, 4:55 am)

Re: [PATCH] don't use mmap() to hash files, Johannes Schindelin, (Sun Feb 14, 11:10 am)

Re: [PATCH] don't use mmap() to hash files, Dmitry Potapov, (Sun Feb 14, 12:06 pm)

Re: [PATCH] don't use mmap() to hash files, Johannes Schindelin, (Sun Feb 14, 12:22 pm)

Re: [PATCH] don't use mmap() to hash files, Johannes Schindelin, (Sun Feb 14, 12:28 pm)

Re: [PATCH] don't use mmap() to hash files, Dmitry Potapov, (Sun Feb 14, 12:55 pm)

Re: [PATCH] don't use mmap() to hash files, Dmitry Potapov, (Sun Feb 14, 12:56 pm)

Re: [PATCH] don't use mmap() to hash files, Avery Pennarun, (Sun Feb 14, 4:13 pm)

Re: [PATCH] don't use mmap() to hash files, Zygo Blaxell, (Sun Feb 14, 4:52 pm)

Re: [PATCH] don't use mmap() to hash files, Nicolas Pitre, (Sun Feb 14, 9:16 pm)

Re: [PATCH] don't use mmap() to hash files, Avery Pennarun, (Sun Feb 14, 10:01 pm)

Re: [PATCH] don't use mmap() to hash files, Nicolas Pitre, (Sun Feb 14, 10:05 pm)

Re: [PATCH] don't use mmap() to hash files, Nicolas Pitre, (Sun Feb 14, 10:48 pm)

Re: [PATCH] don't use mmap() to hash files, Paolo Bonzini, (Mon Feb 15, 12:48 am)

Re: [PATCH] don't use mmap() to hash files, Dmitry Potapov, (Mon Feb 15, 5:23 am)

Re: [PATCH] don't use mmap() to hash files, Dmitry Potapov, (Mon Feb 15, 5:25 am)

Re: [PATCH] don't use mmap() to hash files, Avery Pennarun, (Mon Feb 15, 12:19 pm)

Re: [PATCH] don't use mmap() to hash files, Nicolas Pitre, (Mon Feb 15, 12:29 pm)

[PATCH] Teach "git add" and friends to be paranoid, Junio C Hamano, (Wed Feb 17, 6:16 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Junio C Hamano, (Wed Feb 17, 6:20 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Jeff King, (Wed Feb 17, 6:38 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Nicolas Pitre, (Wed Feb 17, 9:55 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Junio C Hamano, (Wed Feb 17, 10:36 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Wincent Colaiuta, (Thu Feb 18, 12:27 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Thomas Rast, (Thu Feb 18, 3:14 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Zygo Blaxell, (Thu Feb 18, 8:32 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Zygo Blaxell, (Thu Feb 18, 9:18 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Jonathan Nieder, (Thu Feb 18, 11:12 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Junio C Hamano, (Thu Feb 18, 11:16 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Junio C Hamano, (Thu Feb 18, 11:35 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Nicolas Pitre, (Thu Feb 18, 12:58 pm)

16 gig, 350,000 file repository, Bill Lear, (Thu Feb 18, 1:11 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Peter Harris, (Thu Feb 18, 1:14 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Junio C Hamano, (Thu Feb 18, 1:17 pm)

Re: 16 gig, 350,000 file repository, Nicolas Pitre, (Thu Feb 18, 1:58 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Nicolas Pitre, (Thu Feb 18, 2:30 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Jonathan Nieder, (Thu Feb 18, 6:04 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Dmitry Potapov, (Fri Feb 19, 1:28 am)

Re: 16 gig, 350,000 file repository, Erik Faye-Lund, (Fri Feb 19, 2:27 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Zygo Blaxell, (Fri Feb 19, 8:26 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Junio C Hamano, (Fri Feb 19, 10:51 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Junio C Hamano, (Fri Feb 19, 10:52 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Junio C Hamano, (Fri Feb 19, 10:52 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Zygo Blaxell, (Fri Feb 19, 12:08 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Junio C Hamano, (Sat Feb 20, 12:23 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Dmitry Potapov, (Sun Feb 21, 12:21 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Junio C Hamano, (Sun Feb 21, 12:32 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Dmitry Potapov, (Sun Feb 21, 8:35 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Junio C Hamano, (Sun Feb 21, 11:59 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Dmitry Potapov, (Mon Feb 22, 5:25 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Paolo Bonzini, (Mon Feb 22, 5:59 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Dmitry Potapov, (Mon Feb 22, 6:33 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Nicolas Pitre, (Mon Feb 22, 8:40 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Dmitry Potapov, (Mon Feb 22, 9:01 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Zygo Blaxell, (Mon Feb 22, 10:31 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Nicolas Pitre, (Mon Feb 22, 11:01 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Dmitry Potapov, (Mon Feb 22, 11:05 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Nicolas Pitre, (Mon Feb 22, 11:14 am)

Re: [PATCH] Teach "git add" and friends to be paranoid, Junio C Hamano, (Mon Feb 22, 12:56 pm)

Re: [PATCH] Teach "git add" and friends to be paranoid, Nicolas Pitre, (Mon Feb 22, 1:52 pm)

Re: 16 gig, 350,000 file repository, Bill Lear, (Mon Feb 22, 3:20 pm)

Re: 16 gig, 350,000 file repository, Nicolas Pitre, (Mon Feb 22, 3:31 pm)

Navigation

Popular discussions


linux-kernel:
Greg KH	Og dreams of kernels
Jens Axboe	[PATCH 31/33] Fusion: sg chaining support
Arnd Bergmann	Re: finding your own dead "CONFIG_" variables
Mark Brown	[PATCH 2/2] Subject: natsemi: Allow users to disable worka