Joined: May 31, 2003 Posts: 219 Location: French Alps
Posted: Sat Jan 24, 2004 12:53 pm Post subject: Java system charset, JDBC driver
Neo/J 0.7.1 patch-5
Patrick,
I'm trying to connect Neo/J to a MySQL database.
I downloaded the JDBC driver from MySQL site. I had to put the .jar class file in the <NeoOfficeJ.app>/Contents/MacOS/classes folder, make it root property and set the rights accordingly.
Note: Do you already take care of this kind (classes) of customization? If not, it may come in the way at next release.
Anyway, I'm able to set-up a database connection to the MySQL server. The tables shows up and display.
But something is wrong with the charset. None of the text fields display correclty. All chars are square, as usal with this kind of problem. The database use latin1. I tried all combinations between the Neo/J setting of the database charset and the JDBC driver setting (using url options), without finding a suitable one. Other type of fields display correctly.
What charset is used by the Java 1.3 machine?
Do you have any hint ?
Joined: May 31, 2003 Posts: 219 Location: French Alps
Posted: Sat Jan 24, 2004 1:31 pm Post subject: self followup
I thought it might be usefule to insert this excerpt from the JDBC MySQL driver Readme :
Quote:
CHARACTER SETS
MySQL Connector/J now handles character sets in a more transparent way
than MM.MySQL did. The driver attempts to determine the character
set of the MySQL server you are connecting to, and map it to the
Java character set that is appropriate. This mapping is kept up to
date with new server releases. In almost all cases you will not
need to set the "useUnicode" or "characterEncoding" parameters,
as the driver will do this for you. Because of this the "useUnicode"
parameter has the default value of "true". If you need to force
the old behavior of the driver, you need to add "useUnicode=false"
to your URL.
For single-byte character sets, the driver uses an optimized
character translation scheme that yields performance that is as
good or better than MM.MySQL or the first 3.0 version of
MySQL Connector/J. Multi-byte character sets use your JVM's built
in character translation support, and on average this is 5-10%
slower than single-byte character support.
If you have a character set that was not created by MySQL AB, a
driver version older than the version of MySQL server you are using,
or for some reason can not get the mapping to work, you can force
the character set using the "useUnicode=true" and
"characterEncoding=some_charset" parameters in your URL to tell
the driver which character set to use, where "some_charset" is
a character set that your JVM supports.
You can also store UTF-8 encoded data in MySQL server versions
prior to 4.1 by using 'useUnicode=true&characterEncoding=UTF8'
as parameters in your JDBC URL. You might encounter wierd collation
results (sorting/comparing) because UTF-8 encompasses all Unicode
character sets, so this is best used when you sort/search on
keys in a normal (latin1) character set.
I don't know much about the MySQL JDBC driver, but you may want to try that "useUnicode" option with both "false" and "true" that the documentation mentioned.
Since Java strings are Unicode, my thinking is that the MySQL driver performing some custom character to Unicode translation that is confusing OOo's JDBC code so maybe setting "useUnicode" to "false" will allow Java to do the translation instead of the MySQL driver.
BTW, Java uses your system character set which is set from your preferred language in the System Preferences application. For French, I think you get MacRoman which is nearly the same as Latin1 so I don't think that the problem is the Java charset.
Joined: May 31, 2003 Posts: 219 Location: French Alps
Posted: Sun Feb 01, 2004 10:12 am Post subject: MySQL VARCHAR vs TEXT
I tried every possible setting without success, until I realized, by displaying other tables that VARCHAR fields displays without problems. Even accented characters are OK, with the Mac setting on the OOo side of the API. The problem occurs only with TEXT fields. I barely see why.
I see nothing about this on the Connector/J site/FAQ so this must be a bug in the API implementation. On which side?
To check if the Java nature of Neo/J does interact with this, I must try to connect the database to OOo/X11. So far I didn't succeed (driver not found error). I'll try further but I post here, just in case someone comes with some advice.
Joined: May 31, 2003 Posts: 219 Location: French Alps
Posted: Mon Mar 01, 2004 8:07 am Post subject: upgrading does not preserve custom Java classes
When upgrading from 0.8 to 0.8.1, the update process did not preserve the custom Java classes I added in "<Neo/J>/Contents/MacOS/classes".
As I anticipated it, this is not a major trouble, but I think this should be taken care for future release. That's why I'll file the same remark to Bugzilla. Feel free to close it as "not a bug", especially if there is another place for a user to store custom classes (which one then?).
BTW, the charset bug that initiated this thread is still there. I verified the same behavior in OOo/X11, as well as with OOo/Linux. And this with two different (while recent) version of MySQL.
It remains to find out if the bug is in the Java layer of OOo or in the Connector/J JDBC driver. I'll post on Connector/J forum someday. If the bug is OOo's, where should I report it?
What sound curious is that I remember (and found in archives) of some folks reporting in OOdocs to be able to use the MySQL JDBC driver?
I have everything except TEXT fields and unsigned ints working. VARCHARS, for example, come through fine. If you're having trouble with all fields, or just connecting, I can tell you how I'm set up. But I think we have the same problem, right?
Posted: Tue May 11, 2004 10:51 am Post subject: Possible fix for encoding issue
I had the same problem. I have just made it work by putting useUnicode=true into my connection string, but without specifying characterEncoding=UTF8 (or any other value).
I got the idea from this clue in the MySQL Connector changelog:
Quote:
10-24-01 - Version 2.0.7
- Character sets read from database if useUnicode=true and characterEncoding is not set. (thanks to Dmitry Vereshchagin)
I'd be interested to know if this works for you too, as my circumstances may not be identical.
Joined: May 31, 2003 Posts: 219 Location: French Alps
Posted: Thu May 13, 2004 2:41 pm Post subject: Nope,...
...for me neither.
I did try all possible setting on this two parameters before my first post.
Moreover the charset setting should affect both char/varchar and text fields, and only for special/accented chars. As this is not the case, I suspect the problem to be in the handling of data format. Not one single character displays...
Joined: May 31, 2003 Posts: 219 Location: French Alps
Posted: Mon Aug 16, 2004 1:35 pm Post subject: OK in OOo 1.1
This is a repost, due to hacker's way.
I recently tested more on MySQL connectivity and upgraded my linux box to OOo 1.1.
The text fields now display as expected.
The recent (>4.1) versions of MySQL which give more precise control on charset and know about unicode are easier to use, with or without JDBC URL parameter.
Hope Patrick will be able to release a 1.1 codebase version of Neo/J before I need this connectivity for one of my works. I'm too comfortable with Neo/J, now, to keep a working copy of OOo/X11 for OSX (sorry Ed).
But if someone needs this feature on Mac NOW this is the way to go.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You cannot download files in this forum
')