On Jan 16, 2008, at 6:38 PM, Linus Torvalds wrote:

quoted text
> On Wed, 16 Jan 2008, Kevin Ballard wrote:
>>
>> There's a difference between "looks similar" as in "Polish" vs =20
>> "polish", and
>> actually is the same string as in "Ma<UMLAUT MODIFIER>rchen" vs =20
>> "M<A WITH
>> UMLAUT>rchen". Capitalization has a valid semantic meaning, =20
>> normalization
>> doesn't.
>
> That simply isn't true.
>
> Normalization actually has real semantic meaning. If it didn't, there
> would never ever be a reason why you'd use the non-normalized form =20
> in the
> first place.

My understanding is that normalization is there to help the computer. =20=

That doesn't give it any semantic meaning, because all normal forms of =20=

a given string still represent the exact same string to the user.

quoted text
> Others have argued the exact same thing for capitalization. "A" is the
> same letter as "a". Except there is a distinction.

The argument for case insensitivity is different than the argument for =20=

normalization. I certainly hope you understand why they are different =20=

arguments, or there's really no point in going further.

quoted text
> The same is true of "a<UMLAUT MODIFIER>" and "<a WITH UMLAUT>". Yes, =20=

quoted text
> it's
> the same "chacter" in either case. Except when there is a distinction.
>
> And there *are* cases where there are distinctions. Especially inside
> computers. For one thing, you may not be talking about "characters on
> screen", but you may be talking about "key sequences". And suddenly
> "a<UMLAUT MODIFIER>" is a two-key sequence, and "<a WITH UMLAUT>" is a
> single-key sequence, and THEY ARE DIFFERENT.
>
> See?
>
> "a" and "A" are the same letter. But sometimes case matters.
>
> Multi-character UTF-8 sequences may be the same character. But =20
> sometimes
> the sequence matters.
>
> Same exact thing.

You're right, sometimes the sequence matters. As in key sequences. But =20=

we're not talking about key sequences, we're talking about strings. =20
Just because it matters sometimes doesn't mean it matters all the time.


quoted text
>> 	The only way to argue that normalization is wrong is by =
providing a
quoted text
>> good reason to preserve the exact byte sequence, and so far the =20
>> only reason
>> I've seen is to help git.
>
> Git doesn't care. Just use the *same* sequence everywhere. Make sure
> something doesn't change it. Because if something changes it, git will
> track it.

And how am I supposed to use the same sequence everywhere? When I type =20=

"M=E4rchen", I don't know which form I'm typing, nor should I. It's not =20=

something that I, as a user, should have to know. Especially if I pass =20=

this name through various other utilities before using it - I have no =20=

idea if another utility is going to end up normalizing the name, and =20
it shouldn't matter, as they are equivalent strings.

quoted text
>> How do you figure? When I type "M=E4rchen", I'm typing a string, not =20=

quoted text
>> a byte
>> sequence. I have no control over the normalization of the characters.
>> Therefore, depending on what program I'm typing the name in, I =20
>> might use the
>> same normalization as the filename, or I might miss. It's =20
>> completely out of my
>> control. This is why the filesystem has to step in and say "You =20
>> composed that
>> character differently, but I know you were trying to specify this =20
>> file".
>
> Pure and utter garbage.
>
> What you are describing is an *input method* issue, not a filesystem
> issue.
>
> The fact that you think this has anything what-so-ever to do with
> filesystems, I cannot understand.
>
> Here's an example: I can type M=E4rchen two different ways on my =20
> keyboard: I
> can press the '=E4' key (yes, I have one, I have a Swedish keyboard), =20=

quoted text
> or I
> could press the '=A8' key and the 'a' key.
>
> See: I get '=E4' and '=E4' respectively.

On a US keyboard I only have one way of typing =E4, and I have no idea =20=

whether it ends up precomposed or decomposed in the resulting byte =20
stream. And I don't care. Because I'm typing characters, not bytes. I =20=

could be typing in a file in ISO-Latin-1 and I still wouldn't care, =20
because it looks the same to me. If my filesystem did make a =20
distinction between the normal forms, and I see that I have a file =20
named "M=E4rchen", how am I supposed to type that at my keyboard? I =20
don't know which normal form it's using.

The fact that you think the normalization of the string matters, I =20
don't understand.

quoted text
> And as I send this email off, those characters never *ever* got =20
> written as
> filenames to any filesystem. But they *did* get written as part of
> text-files to the disk using "write()", yes.
>
> And according to your *insane* logic, that write() call should have
> converted them to the same representation, no?
>
>
> Hell no! That conversion has absolutely nothing to do with the =20
> filesystem.
> It's done at a totally different layer that actually knows what it is
> doing, and turned them both into \xc3\xa4 (and then, the email client
> probably will turn this into Latin1, and send it out as a single-byte
> '\xe4' character).
>
> See? Putting the conversion in the filesystem IS INSANE. You =20
> wouldn't make
> the filesystem convert the characters in the data stream (because it =20=

quoted text
> would
> cause strange data conversion issues) AND FOR EXACTLY THE SAME =20
> REASON it
> shouldn't do it for filenames either!

What a fabulous straw man argument you just put together. I hope you =20
don't need me to point out why this argument is fundamentally flawed.

quoted text
> And your claim that "you have no control over the normalization of
> characters" is simply insane. Of course you have. It's just not =20
> supposed
> to be at the filesystem level - whether it's a write() call or a =20
> creat()
> call!

I'm speaking as a user, and as such, I shouldn't even have to know =20
that it's possible to write the same character in multiple different =20
ways. As a user, HFS+ behaves exactly the way I want it to. You were =20
talking earlier about not messing with the "user data", but what is =20
the "user data"? It's the string, not the byte sequence. That's all I =20=

care about - the string. That's all the OS cares about, that's all any =20=

application I use cares about, and that's all git should care about.

-Kevin Ballard

--=20
Kevin Ballard
http://kevin.sb.org
kevin@sb.org
http://www.tildesoft.com