Oxia vs tonos accent

[Jonathan Robie]

So if indeed these characters are not “deprecated”, is the fact that these oxia characters are not the normalized form a sufficient reason not to use them? Normalization is for comparing characters, but that does not mean that the non-normalized form is necessarily wrong does it? Where might it become a problem down the road?

It becomes a problem when someone does a search and normalizing the search string does not make it match the corresponding data. And this might happen with many different systems, where the normalization is done in various layers.

And it becomes a problem when fonts display the two equivalent forms differently and people wonder why the same character displays two different ways.

If there is any data where the distinction really matters, then it becomes a problem when you actually intended the oxia. But it's not clear to me whether such cases exist.

[daveburt]

I don't believe the distinction should ever matter, because Unicode is not intended to convey subtle semantics, but rather simply to produce glyphs, to draw the letters' shapes correctly.

There are two problems here:
* Unicode mistakenly created redundant codepoints for the 'tonos' which should just be 'oxia'
* A number of fonts mistakenly render 'tonos' as vertical rather than right-slanting (as did an earlier edition of the Unicode standard[1]).

They should be 'underlyingly treated as equivalent to the multiscript acute accent ... since letters with oxia decompose to letters with tonos, which decompose in turn to base letter plus multiscript acute accent.'[2] But the font issue means using the 'oxia' codepoints might produce better results in contexts where you don't have control over the font.

Regarding deprecation, they are not deprecated, but the 'oxia' codepoints are officially 'discouraged': "Certain characters cannot occur at all in text that is in normalization form NFC. This effectively discourages the use of those characters, but does not formally constitute deprecation, nor does this PRI suggest that they all be given the Deprecated property."[3]

The best solution is probably using 'tonos' and good fonts everywhere (can the forum use a nice, serif, web font for body text?) and 'oxia' as a workaround only in contexts where a poor font is inescapable.

[1] http://citeseerx.ist.psu.edu/viewdoc/do ... df#page=11
[2] https://en.wikipedia.org/wiki/Greek_diacritics#Unicode
[3] http://www.unicode.org/review/pr-122.html

[Eeli Kaikkonen]

I don't believe the distinction should ever matter, because Unicode is not intended to convey subtle semantics, but rather simply to produce glyphs, to draw the letters' shapes correctly.

No, no, producing glyphs and drawing shapes is the job of a font rendering engine and font files. Unicode is a set of codepoints and abstract characters and the standard includes e.g. encodings and rules for normalization.

[daveburt]

I don't believe the distinction should ever matter, because Unicode is not intended to convey subtle semantics, but rather simply to produce glyphs, to draw the letters' shapes correctly.

No, no, producing glyphs and drawing shapes is the job of a font rendering engine and font files. Unicode is a set of codepoints and abstract characters and the standard includes e.g. encodings and rules for normalization.

The Unicode Standard is a character coding system designed to support the worldwide interchange, processing, and display of the written texts of the diverse languages and technical disciplines of the modern world. In addition, it supports classical and historical texts of many written languages.

As a summary, your statement is truer and more complete than mine, but I'd still argue that Unicode's purpose is differentiating characters that should look different, not characters that should never look different (such as 'tonos' and 'oxia'!). Glyphs are still the goal, even if fonts fulfil that role. That is why the standard comes with model glyphs, and that's why the greek question mark (for example) decomposes into the (semantically distinct but visually identical) ASCII semicolon.

[Stephen Carlson]

As a summary, your statement is truer and more complete than mine, but I'd still argue that Unicode's purpose is differentiating characters that should look different, not characters that should never look different (such as 'tonos' and 'oxia'!).

Who decided that tonos and oxia "should never look different"? They are two different (though related) things and, in my view, useful to distinguish. Just like the de-unification of Coptic, which was (also) prematurely unified.

[daveburt]

Who decided that tonos and oxia "should never look different"? They are two different (though related) things and, in my view, useful to distinguish. Just like the de-unification of Coptic, which was (also) prematurely unified.

The Unicode Consortium has decided that, which is clear in Unicode's model glyphs, decomposition/normalization rules, and equivalent characters[1] -- and in doing so it follows the Greek government's position, which is the authority for the monotonic orthography. A resident of Greece in the 1980s when the legislation was passed writes:

The [pre-standard] dot or wedge continued in use long enough to be confusing to early versions of Unicode, which differentiated the “tonos” (accent) from the acute. Since 1986, the monotonic accent has officially been the acute.[2]

In the Unicode standard itself:

The basic Greek accent written in modern Greek is called tonos. It is represented by an acute accent (U+0301). ...[which] in earlier editions of this standard was mistakenly shown as a vertical line

By denotation, they are two names for the same thing, the acute accent, even though it might seem that 'tonos' implies a monotonic and 'oxia' implies a polytonic context.

[1] http://www.unicode.org/charts/PDF/U0370.pdf
[2] https://www.quora.com/Whats-the-history ... OY?share=1
[3] http://www.unicode.org/versions/Unicode ... df#page=18

[Stephen Carlson]

Polytonic users have special needs and deferring to a montonic authority resulted in a sub-optimal solution. I'm still convinced they botched it.

[daveburt]

In what way, Stephen? Or what would you propose to fix it?

[jtauber]

Polytonic users have special needs and deferring to a montonic authority resulted in a sub-optimal solution. I'm still convinced they botched it.

The Unicode Technical Committee _may_ agree. When I requested precomposed characters be added for cases where there is a macron AND accent (e.g. in dictionaries) they told me NONE of the precomposed characters should ever have made it in.

Here's a quote from Ken Whistler on why there are precomposed characters in Greek at all:

It happened back in 1993 and was the result of standards politics -- the influence
of the Greek national body in developing the initial drafts of 10646
and their interest in promoting polytonic Greek. They were working
in an outdated model that assumed all accented stacks had to be
encoded as individual characters -- and that resulted in a compatibility
mess for Greek that endures today.

There is zero chance that the UTC will add to that mess by encoding
more precomposed polytonic Greek with macron accents.

Not the exact topic of the thread but I think relevant nonetheless and an example of the politics that goes on in character encoding.

See https://jktauber.com/2016/02/09/updated ... -problems/ for some more context.

[Stephen Carlson]

Interesting. Thanks.