Posted: Sun Jan 27, 2008 2:57 pm Post subject: Strange diacritics behaviour of OT font in Neooffice
(first post, hello there!)
I've got an OpenType-font, where pre-combined accented glyphs (base letter glyph + one or more diacritical marks) are defined in the "liga" feature as ligatures of the corresponding sequence of base glyph + the diacritical marks.
For example a sequence "r" "uni0325" "uni0304" (i.e. an "r" with a ring below and a macron on top) is replaced by a pre-combined glyph located in the Private Use Area (PUA) of that font.
Now, for some sequences NeoOffice does exactly the right thing and outputs beautiful complicated pre-combined glyphs, and for others it absolutely doesn't. The OpenType-code is correct though, since in TextEdit all works well.
Here's an example (on the left Neooffice, on the right TextEdit).
[img=http://img104.imageshack.us/img104/6217/diacriticsprobwm8.th.png]
As you can see in the middle example, Neooffice even seems to reorder the diacritical marks so that the acute is placed on the "r" first even if it was entered after the ring below!
Posted: Sun Jan 27, 2008 3:07 pm Post subject: Re: nope
dobby wrote:
No, I am using MacOSX Tiger 10.4.11.
OK. So what happens if you create some text that looks correct in the TextEdit application, save it as HTML, and then open the HTML file in Safari? Does it look correct in Safari or does Safari display the same errors?
Posted: Sun Jan 27, 2008 3:21 pm Post subject: I have a suspicion (before I go to sleep) ...
I just tried your test with Safari: Safari isn't able to render the accented characters at all, it simply displays the base character and boxes. (Firefox displays the diacritics in succession, at least.) But nowhere are there the neat pre-combined glyphs as in TextEdit.
On the behaviour in Neooffice: It seems to me that Neooffice is a bit too clever in this case as it seems to "fall back" on pre-combined accents which are in Unicode. So in examples 2 and 3 it "falls back" on "r+acute" and "a+macron" which both are defined in Unicode -- the more complex sequence of the first example however contains no sequence interpretable as a Unicode character (?). It being unknown, it is then correctly handled. (That's my suspicion, at least, does it make any sense?)
Actually, your case is a known bug in Mac OS X's text layout functions. A very wordy description of why this works only in TextEdit is in bug 1199, but the short answer is that TextEdit has some proprietary functionality that can handle OpenType fonts whereas Apple's public text layout functions only handle AAT fonts.
Posted: Thu Jan 31, 2008 4:39 am Post subject: Problem solved!
It might interest some of you, that for this particular problem there is a solution, at least for those of you who have access to an OpenType-savvy font editor. (The details of why Neooffice does what it does with the accent chars in the first place remains obscure to me though, explain!)
When I wrote the code for the "liga" feature, I substituted the whole sequence of source characters (base + diacritics) directly by the target pre-combined glyph in the Private Use Area.
Thus, eg.
Code:
sub a uni0304 acutecomb by uniF674; # a + COMBINING MACRON + COMBINING ACUTE
This, as you can gather from the screenshot in my first post, didn't work. I then augmented the OpenType code by additional intermediate substitution statements, that is to say:
Code:
sub a uni0304 by amacron; # a + COMBINING MACRON
sub amacron acutecomb by uniF674; # a WITH MACRON + COMBINING ACUTE
sub a uni0304 acutecomb by uniF674; # as before
In this way, Neooffice does the right thing, I get beautiful pre-combined accented characters, the whole thing is in accordance with the Unicode, and I am one happy fella! Thank you, and have a nice day.
dobby
Last edited by dobby on Thu Jan 31, 2008 12:59 pm; edited 1 time in total
Posted: Thu Jan 31, 2008 10:08 am Post subject: Re: Problem solved!
dobby wrote:
It might interest some of you, that for this particular problem there is a solution, at least for those of you who have access to an OpenType-savvy font editor. (The details of why Neooffice does what it does with the accent chars in the first place remains obscure to me though, explain!)
I am not a font developer so I cannot tell you exactly why things don't work, but what I can tell you is what I have seen as the behavior limits of Mac OS X's ATSUI text layout functions:
1. In Mac OS X 10.4.x and earlier, ATSUI has only limited supported for reading OpenType font tables. ATSUI prefers to read AAT font tables. If there isn't an AAT font table in your font, ATSUI will try to use the OpenType table but in many cases it will render only the most basic entries in that table.
2. To get a combining diacritical to render properly that aren't ligatures, your font table entries should hint that the combining diacritical will have zero advance and be placed in front of the glyph it is modifying.
dobby wrote:
In this way, Neooffice does the right thing, I get beautiful pre-combined accented characters, the whole thing is in accordance with the Unicode, and I am one happy fella! Thank you, and have a nice day.
Glad this worked. I suspect that you found a combination of OpenType table entries that ATSUI can process whereas before they were being ignored by ATSUI. It is too bad that your font editor tool can't also generate an AAT table for your font, but I suspect that Windows tools only deal with OpenType since OpenType is Microsoft's standard and AAT is only used by Mac OS X. Gotta love competing standards.
BTW, can you try the test patch listed at the end of bug 2893 and see if your font still renders OK?
This test patch contains a fix for a bug in NeoOffice's underlying OpenOffice.org code that appears to cause an infinite loop in complex documents when there are glyphs with zero advance. To avoid this nasty bug, I had to implement shuffling of such glyphs by one pixel to ensure that the OpenOffice.org code thinks that all glyphs have advances of no less than 1 pixel.
IMO, messing with the kerning of some glyphs makes our rendering worse, but then precise rendering doesn't help if NeoOffice goes into an infinite loop when you open your document.
Posted: Thu Jan 31, 2008 1:32 pm Post subject: results
I tried the patch, and this is what I found.
With the zoom set to far away, like 100%, a small gap is clearly visible between an accented letter and the following letter (as expected). It seems that the more accents I put on a base letter the greater the gap will be. With greater zoom, 200%, 300%, 400%, the gap gets smaller, this doesn't come unexpected either.
However, for some combinations of base + diacritics, diacritics miraculously disappear at the greater zoom ratios, so that they're only visible from far off! That's strange.
A pdf-printout confirms that these diacritics are still there and do in fact print. I think I can see a very slightly larger gap even in the pdf, though, which is not good.
Posted: Thu Jan 31, 2008 2:33 pm Post subject: Re: results
dobby wrote:
A pdf-printout confirms that these diacritics are still there and do in fact print. I think I can see a very slightly larger gap even in the pdf, though, which is not good.
This is as it should be as NeoOffice's underlying OpenOffice.org code really likes to control placement of glyphs and each of the combining diacriticals needs to have a single pixel or more of advance to prevent OpenOffice.org's document layout code from spinning into an infinite loop.
I had enabled zero advance glyphs a few patches ago after some testing indicated that the OpenOffice.org code didn't have problems. However, it looks like they are still there in OpenOffice.org 2.2.1 (the code we base NeoOffice on) so I have to pollute kerning to prevent hanging and data loss from a forced quit.
I am hoping the person who filed bug 2893 can confirm my changes stop the hanging as then I can try a less radical approach of only disallowing the last glyph from having zero advance. This would reduce the kerning pollution considerably.
Posted: Thu Jan 31, 2008 3:14 pm Post subject: one more thing
One more bit of info about glyph placement:
I found that the gap isn't because there's space added on the right of the base glyph (to be accented), but rather, the base glyph is shifted to the left into the preceding glyph!
Posted: Thu Jan 31, 2008 3:21 pm Post subject: Re: one more thing
dobby wrote:
I found that the gap isn't because there's space added on the right of the base glyph (to be accented), but rather, the base glyph is shifted to the left into the preceding glyph!
That is a key piece of data. I will look at my recent code changes and see if I am shuffling a pixel from the wrong glyph.
FYI. Good news for font kerning in NeoOffice. My code that forces advance glyphs to always be greater than 0 was found to be unnecessary so I have reverted the code back to allow zero advance glyphs.
You can get this latest code by in the test patch listed at the end of bug 2889.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You cannot download files in this forum
