Doug S (foo.delete@this.bar.bar) on July 15, 2020 10:46 am wrote:
> Michael S (already5chosen.delete@this.yahoo.com) on July 15, 2020 2:26 am wrote:
> > Linus Torvalds (torvalds.delete@this.linux-foundation.org) on July 15, 2020 1:37 am wrote:
> > > Maynard Handley (name99.delete@this.name99.org) on July 14, 2020 12:40 pm wrote:
> > > >
> > > > Let's ask some simpler questions.
> > >
> > > Here's a simple question for you: can you never really admit that Apple does something wrong?
> > >
> > > They literally picked the worst possible thing you could do with filenames.
> > > Really. They corrupt fundamental user data. And you want to ask "simple questions"
> > > when nobody else has ever done anything as stupid as they did..
> > >
> > > I knew you were likely going to do the apologia thing, but Christ... It would be nice if you once
> > > in a while could avoid disappointing people with your incessant and rabid one-sided fanboism.
> > >
> > > > Are you against the concept of filename normalization generically?
> > >
> > > I will state unequivocally that it's both unnecessary and horribly stupid to do at an OS level, yes.
> > >
> > > And if you have to do it (hint: you don't. Really. Not even if you want to be case-insensitive), for the
> > > love of all that is holy, don't pick the absolute worst possible kind of normalization like Apple did.
> > >
> > > Others have done it better. Literally everybody else did. Decades before Apple screwed it up.
> > >
> > > > So you don't just want no normalization
> > > > of capitalization, you want name+' to be treated a
> > > > separate filename from namé ?
> > >
> > > You are apparently too ignorant and stupid to even understand the difference
> > > between comparing something as equal and considering them she same name, and
> > > forcing a particular normalization (and a particularly stupid one) on people.
> > >
> > > Here's a hint: an OS can choose to say that it considerers the file names "And" and "and" to be the same.
> > >
> > > That's case-insensitive and it's a big pain, but there are
> > > (bad) historical reasons to do so. Capitalization matters.
> > >
> > > But notice how case-insensitive comparisons are not "normalization". You
> > > can still call your file "And", and the system preserves the case.
> > >
> > > Internally, to do the case-insensitive compare, there may be hidden normalization
> > > going on (ie the comparison may be done by comparing every character turned to
> > > upper-case), but that normalization isn't visible or forced on the user.
> > >
> > > So the user can choose to have CamelCase file names. Sounds good? Sure, you can also
> > > then open it using "camelcase", and there are problems with that too (really), but
> > > this is at least considered generally fairly acceptable and understandable.
> > >
> > > Notice: the above does not normalize the names. It preserves the case, it just doesn't
> > > consider it important for name comparisons. Two different things entirely.
> > >
> > > Now, as the ignoramus you are, you go "but that's what Apple does!".
> > >
> > > Would you admit that Apple did something wrong if it always turned all filenames into ALL UPPERCASE?
> > >
> > > You'd find that really horrible, wouldn't you?
> > >
> > > And that is exactly what Apple is doing, just at a different level. Apple is indeed
> > > case preserving, but not normalization-preserving. And they not only mess with normalization,
> > > they literally pick the worst possible one that nobody sane would pick.
> > >
> > > And exactly like you don't need to turn all filenames into UPPER-CASE in order to compare
> > > them, you absolutely don't need to turn all utf-8 into NFD to compare them.
> > >
> > > It really is kind of the ASCII equivalent to not just always picking the upper-case name (which
> > > is kind of reasonable and historic if you want to normalize case), but literally then pickibg
> > > some other internal format that everybody will have to convert back from when using it.
> > >
> > > It's really not that different from saying "not only will we turn your "Abc" filename
> > > into "ABC", but we'll do it in EBCDIC because that's what we do internally", and
> > > making the argument that EBCDIC is very traditional and thus sensible.
> > >
> > > But you fundamentally physically cannot understand that this would be a problem, can you?
> > > Because in your world Apple can never make a mistake. So you must come up with these "simple
> > > questions" to try to show how what Apple did was right and proper, don't you?
> > >
> > > Linus
> >
> > Your rant would be much more understandable to people like me that never wrote a single
> > OS X program (or even to those who wrote OS X programs, but were never particularly concerned
> > with handling of file names) if you give just one example of of what's wrong.
> > That is, I can google NFD, but want to understand why OS X use of NFD is bad for git.
> >
>
>
> I agree, I'm curious about this but the rant to signal ratio is too high to get meaningful information.
> Like you I saw the reference to 'NFD', wondered "what the hell is that?" (is it obvious I know
> nothing about UTF-8 yet?) and learned that Linux and most everyone uses NFC encoding (which I
> also had never heard of before) by default while OS X uses NFD encoding by default.
>
> I have no idea what the difference is, why there are two choices if there's a case to be made that choosing
> one of the choices is a bad idea (other than "everyone else chose the other one so you should too") and if
> there are any good reasons why someone might make the other choice despite the compatibility issues.
>
> In my googling I saw multiple claims that HFS+ is NFD only (and NFC is converted to NFD)
> while APFS can use either NFC or NFD. Not sure the ability to use both makes the problem
> go away in Apple's newer filesystem, doesn't help, or makes things worse however.


Let's say you have a file with a name like "Amélie" or "Ångström". I OS-X that gets converted to NFD form, so the things above the letters get converted into separate Unicode code-points.

The point Linus is making is that this conversion is not two-way. That means that two different words can get converted into the same word. That also means that two different filenames could be converted to the same filename. So, in Linux you can have a file called Ångström.txt where the first letter is "ANGSTROM SIGN". In the same directory you can have a file with the same name where the Å is "LATIN CAPITAL LETTER A WITH RING ABOVE". In HFS+ you can't you can only have the A with the ring-above as two separate codepoints. So, you can only have one file. When you see an Å which form did it originate from? You can't know.

A version control system has to compare each part of a file to the original. Not just that contents but also metadata like the name. So, what happens when a file with one of the Ås I mention above is copied down to a HFS+ file-system? In that case it gets converted. Then what is supposed to happen when it gets copied back the other way? There is now no information to convert back to the right character. That's especially true on a distributed version-control system. Conceptually there is no central server, everything is on the same level.