Welcome to NeoOffice developer notes and announcements
NeoOffice
Developer notes and announcements
 
 

This website is an archive and is no longer active
NeoOffice announcements have moved to the NeoOffice News website


Support
· Forums
· NeoOffice Support
· NeoWiki


Announcements
· Twitter @NeoOffice


Downloads
· Download NeoOffice


  
NeoOffice :: View topic - LanguageTool
LanguageTool
 
   NeoOffice Forum Index -> NeoOffice Beta Releases
View previous topic :: View next topic  
Author Message
Gust
Councilperson


Joined: Oct 09, 2007
Posts: 137

PostPosted: Wed Feb 18, 2009 7:51 am    Post subject:

pluby wrote:
Can you attach a sample document that I can use to try to reproduce this behavior?
I managed to trace the issue by manually removing code from the xml files till I found the trigger. You can reproduce the error starting with the alfa-beta example above and adding a few steps:
  1. Restart NeoOffice
  2. In the blank Writer document, type two words on two separate lines (e.g. alfa on the first line and beta on the second)
  3. Select the word on the second line (beta in our example) and change the language for that word to something else.
  4. Insert :: Header :: Default
  5. Save the document (in odt format) and quit NeoOffice
  6. Open the saved file by double clicking on it in the Finder
You should get two error messages like the one quoted above. Although I didn't test for it, I expect the error to happen with Footers alike.
Back to top
narf
The Anomaly


Joined: Jan 21, 2007
Posts: 1075

PostPosted: Wed Feb 18, 2009 8:53 am    Post subject:

Thank you for the steps. I can reproduce the error messages with the steps you listed.

I am running Mac OS X 10.4.11 on a PowerPC machine.

--fran
Back to top
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Wed Feb 18, 2009 11:49 am    Post subject:

After spending several hours on this last night and this morning, I finally figured out what is causing this bug. The clue was that this bug occurs (although much frequently than in NeoOffice) in OpenOffice.org 3.0.1.

Unfortunately, it is not a NeoOffice bug but is a bug in the morfologik code that Language Tool uses. Since I have already invested a lot of time debugging this problem, I went ahead and determined the fix for their code and filed the following bug with the Language Tool developers:

https://sourceforge.net/tracker/index.php?func=detail&aid=2613574&group_id=110216&atid=655717

Since my last few test patches were failed attempts at working around their bug and those attempts introduced some hacky code, I have removed the hacks and upload the following test patch. Please install the following test patch so that you are using the same code that we are:

Intel:
http://joe.neooffice.org/test/NeoOffice-3.0_Early_Access_2-Patch-1-Test-7-Intel.dmg

PowerPC:
http://joe.neooffice.org/test/NeoOffice-3.0_Early_Access_2-Patch-1-Test-7-PowerPC.dmg

Patrick
Back to top
Gust
Councilperson


Joined: Oct 09, 2007
Posts: 137

PostPosted: Wed Feb 18, 2009 12:16 pm    Post subject:

pluby wrote:
After spending several hours on this last night and this morning, I finally figured out what is causing this bug. The clue was that this bug occurs (although much frequently than in NeoOffice) in OpenOffice.org 3.0.1.

Unfortunately, it is not a NeoOffice bug but is a bug in the morfologik code that Language Tool uses. Since I have already invested a lot of time debugging this problem, I went ahead and determined the fix for their code and filed the following bug with the Language Tool developers:

https://sourceforge.net/tracker/index.php?func=detail&aid=2613574&group_id=110216&atid=655717
Ok. At first I was a bit reluctant in raising the issue (also see my first message) as I already feared that we might end up at the other side of the line. And indeed was debugging a pain, I spent hours in determining reproducible errors. So thank you very much for your flexibility in taking up the issue and even taking it that little bit further.

That said, I think that full support of LanguageTool would contribute great value to NeoOffice. This may be less obvious in an US-based environment, but in Europe it is common to write texts in different languages on a daily base. Especially for a number of smaller languages, open source projects have delivered best of breed tools. And I must add that even for English, the performance of LanguageTool (when it worked) easily surpassed everything I'd seen till now.
Back to top
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Wed Feb 18, 2009 12:21 pm    Post subject:

Gust wrote:
That said, I think that full support of LanguageTool would contribute great value to NeoOffice. This may be less obvious in an US-based environment, but in Europe it is common to write texts in different languages on a daily base. Especially for a number of smaller languages, open source projects have delivered best of breed tools. And I must add that even for English, the performance of LanguageTool (when it worked) easily surpassed everything I'd seen till now.


NeoOffice already has full support for LanguageTool. The problem is that LanguageTool has a bug. Their code makes an assumption about Java that is not always ture. I have given them two possible fixes so lobby them to fix their bug.

Patrick
Back to top
Gust
Councilperson


Joined: Oct 09, 2007
Posts: 137

PostPosted: Wed Feb 18, 2009 12:26 pm    Post subject:

pluby wrote:
NeoOffice already has full support for LanguageTool. The problem is that LanguageTool has a bug. Their code makes an assumption about Java that is not always ture. I have given them two possible fixes so lobby them to fix their bug.
Ok. I wanted to say full functionality rather than full support--my mistake.
Back to top
Gust
Councilperson


Joined: Oct 09, 2007
Posts: 137

PostPosted: Wed Mar 04, 2009 12:01 pm    Post subject:

A new version 0.9.7 of LanguageTool was released, but unfortunately it seems not to include the suggested fix for the problem at hand. We'll have to wait somewhat longer...
Back to top
Gust
Councilperson


Joined: Oct 09, 2007
Posts: 137

PostPosted: Mon Apr 27, 2009 1:04 pm    Post subject:

And now there is LanguageTool 0.9.8 which has the bug corrected.

I'm wondering if we should spend a couple of words on it in NeoWiki. More general there may be a case to group spell and grammar check information by language. For some languages the best source available is OS X, for other it third party (open source) dictionaries are far better and so on.

This makes me remember that we used to have at some point a discussion on the use of the Opentaal dictionary rather than OS X spell check services for NL. The behaviour of NeoOffice (2.X) was adapted to meet this consideration, in such that it would first check for a dialect-specific list and only in a subsequent step for a language-specific one. This would result in using the Opentaal list (specified as NL-NL) if setting the language to NL-NL, and the OS X services (specified as NL) is setting the language to NL-BE. The full discussion is here:
https://trinity.neooffice.org/modules.php?name=Forums&file=viewtopic&t=5085

It seems to me that this behaviour is unchanged in NeoOffice 3. It is described here:
http://neowiki.neooffice.org/index.php/Troubleshooting_Spellcheck
For NeoOffice 3 it however bears the note FIXME, so I am not sure in how far it still accurately describes the behaviour.

The link between the behaviour and the settings in the new Language Settings :: Writing Aids section of the NeoOffice preferences is ambiguous to say the least. To take the Opentaal case as an example: looking up the language specific settings (through the edit button in that section) for NL-NL it reports the OS X Spellchecker to be activated, while effectively it uses the installed Opentaal list. Although this behaviour is exactly what I want, it may be clearer presented to the user.

I'll see if I can come up with some suggestions for NeoWiki.
Back to top
ovvldc
Captain Naiobi


Joined: Sep 13, 2004
Posts: 2352
Location: Zürich, CH

PostPosted: Mon Apr 27, 2009 1:08 pm    Post subject:

It is a good thing that the languagetool bug is fixed.

As for the wrong indication of what spellchecker is in use, that would be a bug in NeoOffice. I trust Patrick can mend it.

best wishes,
Oscar

_________________
"What do you think of Western Civilization?"
"I think it would be a good idea!"
- Mohandas Karamchand Gandhi
Back to top
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Mon Apr 27, 2009 1:22 pm    Post subject:

Gust wrote:
The link between the behaviour and the settings in the new Language Settings :: Writing Aids section of the NeoOffice preferences is ambiguous to say the least. To take the Opentaal case as an example: looking up the language specific settings (through the edit button in that section) for NL-NL it reports the OS X Spellchecker to be activated, while effectively it uses the installed Opentaal list. Although this behaviour is exactly what I want, it may be clearer presented to the user.


Sorry, but this is not something we can change as only Apple's Mac OS X's spellchecking code knows which dictionary is being used. All that NeoOffice can report is the setting that we send to the Mac OS X spellchecking service.

I think what you are really asking for is for NeoOffice to reproduce the drop down list box of dictionaries in Apple's native spellchecking dialog. However, that list is not accessible by an public Apple functions. Hence, you choose a locale (like "NL-NL") and the Mac OS X spellchecking service functions only return that that is a valid locale.

Patrick
Back to top
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Mon Apr 27, 2009 2:13 pm    Post subject:

pluby wrote:
I think what you are really asking for is for NeoOffice to reproduce the drop down list box of dictionaries in Apple's native spellchecking dialog. However, that list is not accessible by an public Apple functions. Hence, you choose a locale (like "NL-NL") and the Mac OS X spellchecking service functions only return that that is a valid locale.


Oops. I had confused the Opentaal dictionary issue with the cocoASpell dictionaries.

In the case of Hunspell dictionaries like Opentaal, the issue is different but the result is still same: you can only see whether a locale is supported or not and the list of locales is a pre-defined list of locales hardcoded in NeoOffice's underlying OpenOffice.org code.

What we have done is tweak the OpenOffice.org dictionary loader code to handle loading failures more gracefully by loading the first Hunspell dictionary that matches the locale if the Mac OS X spellchecker service says a locale is not supported.

Since NeoOffice's underlying OpenOffice.org dialogs expect only a "yes/no" answer to whether or not a particular locale is supported, any dictionary substitution that we can do cannot be communicated to the dialog. As a result, when the dialog asks the our dictionary loading code if the "NL-NL" locale is supported, the answer in your case is yes because we found either that the Mac OS X spellchecking service or a Hunspell dictionary supports "NL-NL" or at least "NL-<some>".

Given that the OpenOffice.org dialog is designed to only determine if a locale is supported or not by the spellchecking code, the only other option we have in your case is to return "no" for "NL-NL". That, however, results in no spellchecking for any documents that have text set to "NL-NL" as the OpenOffice.org will disable spellchecking for all such text.

While our approach has some amibiguity, I believe it is the best solution we could implement without rewriting OpenOffice.org spellcheck pipeline from top to bottom. Unfortunately, that is not something we can fund as the current scope of the NeoOffice project is limited by resources to keeping a native version of OpenOffice.org running on Mac OS X and fixing OpenOffice.org feature bugs is outside that scope.

Patrick
Back to top
Gust
Councilperson


Joined: Oct 09, 2007
Posts: 137

PostPosted: Mon Apr 27, 2009 2:39 pm    Post subject:

pluby wrote:
While our approach has some amibiguity, I believe it is the best solution we could implement without rewriting OpenOffice.org spellcheck pipeline from top to bottom. Unfortunately, that is not something we can fund as the current scope of the NeoOffice project is limited by resources to keeping a native version of OpenOffice.org running on Mac OS X and fixing OpenOffice.org feature bugs is outside that scope.
I completely agree that we have the best achievable solution here, and I am pleased to learn that we stuck to it in NeoOffice 3.0. I was more thinking about how we could improve the documentation of the current language-related behaviour in NeoWiki. I'll check out how I can contribute to it.
Back to top
ovvldc
Captain Naiobi


Joined: Sep 13, 2004
Posts: 2352
Location: Zürich, CH

PostPosted: Mon Apr 27, 2009 2:40 pm    Post subject:

Waitaminute Shocked.

Surely, there is a fallback mechanism that ensures that if no NL-NL or NL-BE spellchecker is available, it will check if there is an NL (general Dutch) spellchecker?

Best wishes,
Oscar

_________________
"What do you think of Western Civilization?"
"I think it would be a good idea!"
- Mohandas Karamchand Gandhi
Back to top
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Mon Apr 27, 2009 2:53 pm    Post subject:

ovvldc wrote:
Waitaminute Shocked.

Surely, there is a fallback mechanism that ensures that if no NL-NL or NL-BE spellchecker is available, it will check if there is an NL (general Dutch) spellchecker?


Yes. It is in the same code but it is only used if no other "NL-<some>" native or Hunspell dictionary is found. More details about the fallback to "language only" dictionaries is in this forum post.

Edit: correct URL in link

Patrick
Back to top
sardisson
Town Crier
Town Crier


Joined: Feb 01, 2004
Posts: 4588

PostPosted: Mon Apr 27, 2009 5:37 pm    Post subject:

Gust wrote:
And now there is LanguageTool 0.9.8 which has the bug corrected.

I'm wondering if we should spend a couple of words on it in NeoWiki. More general there may be a case to group spell and grammar check information by language. For some languages the best source available is OS X, for other it third party (open source) dictionaries are far better and so on.

I think it would be good to mention LanguageTool in the main spellcheck article. Right now it doesn't sound like we have enough grammar check options to warrant a separate page (nor, for the moment, do I think we have enough information about "writing tools" options in any language to split the information up by language, but I'll keep that in mind).

Gust wrote:
It seems to me that this behaviour is unchanged in NeoOffice 3. It is described here:
http://neowiki.neooffice.org/index.php/Troubleshooting_Spellcheck
For NeoOffice 3 it however bears the note FIXME, so I am not sure in how far it still accurately describes the behaviour.

The FIXME comment is about the manual override of Patrick's fallback system, where in Neo 2 you could edit the list of installed dictionaries in order to make sure you didn't use a locale-specific OOo dictionary before a more general locale Mac OS X dictionary. In Neo 2 one could do this by editing a simple text file; I'm not sure how it's done (if it can be done at all) in Neo 3....

I clarified the FIXME comment to make sit clear it applies to that work-around, not to the way the fallback mechanism functions.

Gust wrote:
I'll see if I can come up with some suggestions for NeoWiki.

Sounds good!

Smokey

_________________
"[...] whether the duck drinks hot chocolate or coffee is irrelevant." -- ovvldc and sardisson in the NeoWiki
Back to top
Display posts from previous:   
   NeoOffice Forum Index -> NeoOffice Beta Releases All times are GMT - 7 Hours
Goto page Previous  1, 2, 3  Next
Page 2 of 3

 
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum

Powered by phpBB © 2001, 2005 phpBB Group

All logos and trademarks in this site are property of their respective owner. The comments are property of their posters, all the rest © Planamesa Inc.
NeoOffice is a registered trademark of Planamesa Inc. and may not be used without permission.
PHP-Nuke Copyright © 2005 by Francisco Burzi. This is free software, and you may redistribute it under the GPL. PHP-Nuke comes with absolutely no warranty, for details, see the license.