Joined: Jun 04, 2007 Posts: 14 Location: Reno, NV, USA
Posted: Fri Jan 06, 2012 3:08 pm Post subject: Missing unicode characters
On another issue, I recently uploaded a file in Palatino done in V. 3.1.2 on a G5 running OS 10.4.11. I used a unicode character shown in the attached screen shot, highlighted in blue. When I view this file on a MacBook air with V. 3.2.1 installed under OS 10.6.7, not only this special character, but the whole row of characters and the one below it, are missing. I have the same issue with my G5 at home, also running OS 10.4.11 and same version of NeoOffice.
I found Palatino was in both installed in only the main library on hard disk, so I also installed it in user library. Even tried copying a folder off the desktop onto the MacBook air, which has an older Palatino font- 9/2000 vs. 3/2007. Even after restarting computer, these two rows of Unicode characters are missing.
I suspect that Palatino does not have a glyph for that particular Unicode character (most fonts only cover a small subset of all possible Unicode characters) and so when NeoOffice searches for a "fallback" font to use as a substitute for that Unicode character, it does not find any other fonts on your machine that have a glyph for those Unicode characters.
To test if my theory is true or not, can you use the instructions below to open the Mac OS X Character Viewer window's Unicode tables and scroll down to the "0080" line? If my theory is correct, you will only see empty rectangles like as shown in the attached screen snapshot from my Mac OS X machine.
To open the Mac OS X Character Viewer window on Mac OS X 10.5 (I believe 10.4 steps are the same or very similar), launch the Mac OS X System Preferences application, click on the Language & Text icon, and then click on the Input Menu tab. In that tab, check the "Character Palette" checkbox.
After checking the "Character Palette" checkbox, a "flag" icon will appear in the Mac OS X menubar. Click on that icon and select the Show Character Palette menu. In the "Characters" window that appears, set the "View" option to "Code Tables". You can then see the fonts listed by Unicode character in the window like in my screen snapshot.
If you actually see glyphs in the "0080" or "0090" lines on a machine that NeoOffice does not display a character, can you click on the "0086" character and tell us which fonts are listed as having a glyph for that characters?
Joined: Jun 04, 2007 Posts: 14 Location: Reno, NV, USA
Posted: Fri Jan 06, 2012 5:37 pm Post subject:
Brilliant! Yes, you reach the character palette in OS 10.4 via the "International" option in System Preferences. What is curious is that when I do what you say on both machines, in the one that is displaying the character correctly, I find it is in a Microsoft true type font, PMingLiU.
Curiously, this font is also being used in the OS 10.6 machine for the same character, except it is NOT displaying. The font is in the fonts folder in main library, I also added it to my user library; still won't display the special character. Also curiously, this is NOT such an uncommon character- it is found in all kinds of fonts in my old Word Perfect 3.5, which still runs fine on 10.4 (see screen shot).
Curiously, this font is also being used in the OS 10.6 machine for the same character, except it is NOT displaying. The font is in the fonts folder in main library, I also added it to my user library; still won't display the special character. Also curiously, this is NOT such an uncommon character- it is found in all kinds of fonts in my old Word Perfect 3.5, which still runs fine on 10.4 (see screen shot).
On Mac OS X 10.5 and higher, adding fonts to specific folders will not work until the next reboot. Instead, try launching the /Applications/Font Book application, select the File :: Add Fonts menu, and select the font file that you want to add. After you add the font using the Font Book application, does the font appear in NeoOffice? You may need to restart NeoOffice after adding the font, but maybe not.
Joined: Jun 04, 2007 Posts: 14 Location: Reno, NV, USA
Posted: Fri Jan 06, 2012 6:10 pm Post subject:
Patrick-
I had already rebooted. Thing that is perplexing is font DOES show in NeoOffice, but cannot see the characters in the 10.6 machine, but can in 10.4. Will try my other 10.4 machine when I go home.
I had already rebooted. Thing that is perplexing is font DOES show in NeoOffice, but cannot see the characters in the 10.6 machine, but can in 10.4. Will try my other 10.4 machine when I go home.
If Smokey's suggestions for finding and eliminating duplicate instances of the font do not change anything, can you post a screen snapshot of the character palette window on your Mac OS X machine with the "0086" unicode character selected like in my previous screen snapshot?
I ask this because Apple replaced the entire text rendering functions in Mac OS X 10.5. Applications like NeoOffice 3.1.2 and Microsoft Office that work on Mac OS 10.4 use Apple's older "ATSUI" text rendering functions. In contrast, application like NeoOffice 3.2.1 use Apple's newer "CoreText" functions. So, while the font may be listed, it is possible that some very old fonts like your version of Palatino may no longer be supported by Apple's newer CoreText functions.
Joined: Jun 04, 2007 Posts: 14 Location: Reno, NV, USA
Posted: Fri Jan 06, 2012 6:44 pm Post subject:
I did try font book; could not find PMingLiu in my user library because it has already been moved to a Disabled Fonts folder on the 10.4 machine which IS showing the character.
I have taken a screen shot on the MacBook Air OS 10.6.7, which is NOT displying the character, although it shows in the NeoOffice font menu.
I did try font book; could not find PMingLiu in my user library because it has already been moved to a Disabled Fonts folder on the 10.4 machine which IS showing the character.
I have taken a screen shot on the MacBook Air OS 10.6.7, which is NOT displying the character, although it shows in the NeoOffice font menu.
So do you know where you obtained the PMingLiu font that is on your Mac OS X 10.6.7 machine? I ask because cannot find that font in my Microsoft Office 2008 installation or my Mac OS X 10.6.8 machine so, for me, there are still no fonts that have a glyph for the "0086" unicode character.
Ignore my last post. I found the PMingLiu font that Microsoft Office 2008 installs and I can now see its glyph for the 0086 unicode character in the character paletter and, like you, I cannot see it in NeoOffice's special characters dialog.
I will start investigating this to see why this range of PMingLiu glyphs are not displaying in NeoOffice. I will post again when I have some news to report.
I have found the reason why NeoOffice cannot display the PMingLiU font's glyphs for the 0080 through 009F range of unicode characters and the bad news is that this is not a NeoOffice bug. Instead, it is a Mac OS X bug.
What I found is that while the Mac OS X character palette can display these PMingLiU glyphs, Apple's text layout functions will always fail and will return the "no glyph available" glyph for these unicode characters even when the font is explicitly set to PMingLiU. Even worse, in Mac OS X 10.7, I could not even get the Font Book application to install the PMingLiU font so it is clear that newer versions of Mac OS X no longer fully support this particular font.
FWIW, I found that this was a Mac OS X bug by doing the following:
1. Launch TextEdit and then open the character palette and select the 0086 unicode character and press the character palette's Insert button. Nothing will get inserted in TextEdit for this unicode character, but for other unicode characters below 0080, inserting the PMingLiU font will work fine.
2. I put debug code in NeoOffice to print out what glyph number Apple's CoreText functions return for the 0086 unicode character and PMingLiU font and those functions return the 0xffff font which, if displayed, is an empty rectangle (that is, the "no glyph available" glyph).
I suspect that you will need to find a different font to use for the symbols in the 0080 through 009F unicode character range. Unfortunately, I cannot recommend any fonts as my quick Google search and my search of sil.org's sites did not find any free fonts that have these glyphs for that unicode character range.
I have found the reason why NeoOffice cannot display the PMingLiU font's glyphs for the 0080 through 009F range of unicode characters and the bad news is that this is not a NeoOffice bug. Instead, it is a Mac OS X bug.
No, I think it's actually an "old pre-Unicode font" bug Sorry, I didn't think to look at the Unicode glyph number before your last post.
pluby wrote:
I suspect that you will need to find a different font to use for the symbols in the 0080 through 009F unicode character range. Unfortunately, I cannot recommend any fonts as my quick Google search and my search of sil.org's sites did not find any free fonts that have these glyphs for that unicode character range.
I don't think that will work; the only fonts that will have glyphs in that range are going to be old, pre-Unicode fonts, and as you've demonstrated, the OS is going to return a "no glyph found" result for any codepoints in that range, in any font (even, I suspect, LastResort). Why? Because the U+0080 to U+009F range is reserved for control characters (and has been since Unicode 1.1): http://unicode.org/charts/PDF/U0080.pdf (I'm a bit surprised it's showing up in the Character Palette screenshots on 10.6, actually.)
DavidS's best option, I think, is to find the Unicode codepoint where that glyph has moved (I think U+2020) and find a fully-Unicode-compliant font that displays the glyph as he likes it.
Smokey _________________ "[...] whether the duck drinks hot chocolate or coffee is irrelevant." -- ovvldc and sardisson in the NeoWiki
DavidS's best option, I think, is to find the Unicode codepoint where that glyph has moved (I think U+2020) and find a fully-Unicode-compliant font that displays the glyph as he likes it.
Smokey's advice is definitely the best route for DavidS so please ignore my suggestion in my last post.
Also, thank you Smokey for catching that the U+0080 to U+009F range is reserved for control characters. I had forgotten about that special range and I didn't make the connection that "Latin-1 Supplement" was synonymous for "control characters".
Also, thank you Smokey for catching that the U+0080 to U+009F range is reserved for control characters. I had forgotten about that special range and I didn't make the connection that "Latin-1 Supplement" was synonymous for "control characters".
FWIW, all of the Latin-1 Supplement isn't control characters, just those first 32 codepoints. (It's kind-of like the control characters were shoved in to some previously-empty spaces between in-use characters and code blocks.) The rest of the Latin-1 Supplement contains various symbol glyphs and multilingual variants of standard ASCII/Roman letters.
Smokey _________________ "[...] whether the duck drinks hot chocolate or coffee is irrelevant." -- ovvldc and sardisson in the NeoWiki
Joined: Jun 04, 2007 Posts: 14 Location: Reno, NV, USA
Posted: Fri Jan 20, 2012 4:37 pm Post subject:
Patrick-
Have been on vacation close to two weeks. I fear I do not buy the explanation that it is a Mac OS coding problem. On the plane last night, I went to insert a degree symbol that was repeatedly used in a document I was correcting. For various reasons, I decided to do this in text edit. By god, there in text edit, when I went to insert the degree, I found preselected the dagger from PMingLiu (see attached screenshot). To be doubly sure, I tried this also in Text Wrangler, an industrial strength text editor used for bioinformatics due to its ability to open monster (500mb) genome data files. It is also there in Text Wrangler. This was done with a MacBookAir, OS 10.6.7, running the latest version of NeoOffice with patch 3 installed.
I argue this points the finger back at NeoOffice, and again raises the nasty question of why this works on my office G5 DP running OS 10.4.11, and not on my home G5Dp. One clue I think is that the PMingLiu is an font folder labeled "Fonts disabled" on the office G5 (runing Neoffice 3.1.2.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You cannot download files in this forum