Strange problem with german umlauts in testcases

[Kai Moritz]

Hi Folks,

I have this really strange problem with german umlauts in my testcases.

On my development system (Ubuntu feisty with Sun JDK 6.0
and "LANG=***@euro") all my testcases work fine. But when I check out the
sources on my production system (Debian etch with Sun JDK 5.0
and "LANG=en_US.ISO-8859-15") "mvn test" failes with a weired errors:

-------------------------------------
[INFO] Error executing database operation: CLEAN_INSERT

Embedded error: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too
long for column 'description' at row 1
-------------------------------------
The mentioned column holds an "ä" and the error vanishes, if I remove that
character. But even more strange: another column of the same row can hold any
number of german umlauts without any error! The only difference between the
two rows is, that the one with the error was maped with type="text" in
Hibernate (which results in "text") and the one without the error with
type="string" (which results in "varchar(255)") ...

When I remove the "ä" CLEAN_INSERT works, but all my testcases, which are
comparing strings that contain german umlauts, are failing with errors like:
----------------------------------------
junit.framework.ComparisonFailure: null expected:<S[Ì]dring 2> but
was:<S[ü]dring 2>
----------------------------------------


I am hopelessly stuck! I hope someone has an idea or can give me a clue!

Greetings kai

[Peter Schneider-Manzell]

Hi Kay!

Maybe you should check the encoding of your sources.
Make sure all text files (e.g. xml / java) are encoded with utf8.

Bye,

Peter

Post by Kai Moritz
Hi Folks,
I have this really strange problem with german umlauts in my testcases.
On my development system (Ubuntu feisty with Sun JDK 6.0
sources on my production system (Debian etch with Sun JDK 5.0
-------------------------------------
[INFO] Error executing database operation: CLEAN_INSERT
Embedded error: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too
long for column 'description' at row 1
-------------------------------------
The mentioned column holds an "ä" and the error vanishes, if I remove that
character. But even more strange: another column of the same row can hold any
number of german umlauts without any error! The only difference between the
two rows is, that the one with the error was maped with type="text" in
Hibernate (which results in "text") and the one without the error with
type="string" (which results in "varchar(255)") ...
When I remove the "ä" CLEAN_INSERT works, but all my testcases, which are
----------------------------------------
junit.framework.ComparisonFailure: null expected:<S[Ì]dring 2> but
was:<S[ü]dring 2>
----------------------------------------
I am hopelessly stuck! I hope someone has an idea or can give me a clue!
Greetings kai
---------------------------------------------------------------------

[Kropp, Henning]

Hi,

you probably have umlaute in your data-sample.xml. To use umlaute in
this file please make sure it is encoded in utf-8.

regards
Henning

Post by Peter Schneider-Manzell
Hi Kay!
Maybe you should check the encoding of your sources.
Make sure all text files (e.g. xml / java) are encoded with utf8.
Bye,
Peter
Hi Folks,
I have this really strange problem with german umlauts in my testcases.
On my development system (Ubuntu feisty with Sun JDK 6.0
sources on my production system (Debian etch with Sun JDK 5.0
-------------------------------------
[INFO] Error executing database operation: CLEAN_INSERT
Embedded error: com.mysql.jdbc.MysqlDataTruncation: Data
truncation: Data too
long for column 'description' at row 1
-------------------------------------
The mentioned column holds an "ä" and the error vanishes, if I remove that
character. But even more strange: another column of the same row can hold any
number of german umlauts without any error! The only difference between the
two rows is, that the one with the error was maped with type="text" in
Hibernate (which results in "text") and the one without the error with
type="string" (which results in "varchar(255)") ...
When I remove the "ä" CLEAN_INSERT works, but all my testcases, which are
----------------------------------------
junit.framework.ComparisonFailure: null expected:<S[Ì]dring 2> but
was:<S[ü]dring 2>
----------------------------------------
I am hopelessly stuck! I hope someone has an idea or can give me a clue!
Greetings kai
---------------------------------------------------------------------

[Kai Moritz]

Post by Kropp, Henning
Hi,
you probably have umlaute in your data-sample.xml. To use umlaute in
this file please make sure it is encoded in utf-8.

It is!
The source-code should be in UTF-8 as well. But I will doublecheck this this
evening...

Perhaps I should try to encode the sample-data.xm
l as ISO-bla-15 and give it a
try. I guess that DBUnit counts the characters in the row, without being
aware, that the umlaut has more than 1 byte, whicht then leads to the
turncate...

Greetings kai

[Kai Moritz]

Post by Kai Moritz
Perhaps I should try to encode the sample-data.xm
l as ISO-bla-15 and give it a
try. I guess that DBUnit counts the characters in the row, without being
aware, that the umlaut has more than 1 byte, whicht then leads to the
turncate...

Changing the encoding of sample-data.xml to "latin1" (via vim) and the
XML-prolog to <?xml version="1.0" encoding="ISO-8859-1"?> makes no
difference :(
The strange DBUnit-Error remains!
Here is the stack-trace:
------------------------------------------------
[INFO] [dbunit:operation {execution: test-compile}]
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error executing database operation: CLEAN_INSERT

Embedded error: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too
long for column 'description' at row 1
[INFO] ------------------------------------------------------------------------
[INFO] Trace
org.apache.maven.lifecycle.LifecycleExecutionException: Error executing
database operation: CLEAN_INSERT
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:564)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:480)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:459)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:311)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:278)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:143)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:333)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:126)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:282)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.apache.maven.plugin.MojoExecutionException: Error executing
database operation: CLEAN_INSERT
at org.codehaus.mojo.dbunit.OperationMojo.execute(OperationMojo.java:110)
at
org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:447)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:539)
... 16 more
Caused by: org.dbunit.DatabaseUnitException:
com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for
column 'description' at row 1
at org.dbunit.ant.Operation.execute(Operation.java:187)
at org.codehaus.mojo.dbunit.OperationMojo.execute(OperationMojo.java:101)
... 18 more
---------------------------------------------

Greetings Kai

[Kropp, Henning]

Post by Kai Moritz

Post by Kai Moritz
Perhaps I should try to encode the sample-data.xm
l as ISO-bla-15 and give it a
try. I guess that DBUnit counts the characters in the row, without being
aware, that the umlaut has more than 1 byte, whicht then leads to the
turncate...

Changing the encoding of sample-data.xml to "latin1" (via vim) and the
XML-prolog to <?xml version="1.0" encoding="ISO-8859-1"?> makes no
difference :(
The strange DBUnit-Error remains!
------------------------------------------------
[INFO] [dbunit:operation {execution: test-compile}]
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error executing database operation: CLEAN_INSERT
Embedded error: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too
long for column 'description' at row 1
[INFO] ------------------------------------------------------------------------
[INFO] Trace
org.apache.maven.lifecycle.LifecycleExecutionException: Error executing
database operation: CLEAN_INSERT
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:564)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:480)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:459)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:311)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:278)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:143)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:333)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:126)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:282)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.apache.maven.plugin.MojoExecutionException: Error executing
database operation: CLEAN_INSERT
at org.codehaus.mojo.dbunit.OperationMojo.execute(OperationMojo.java:110)
at
org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:447)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:539)
... 16 more
com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for
column 'description' at row 1
at org.dbunit.ant.Operation.execute(Operation.java:187)
at org.codehaus.mojo.dbunit.OperationMojo.execute(OperationMojo.java:101)
... 18 more
---------------------------------------------
Greetings Kai
---------------------------------------------------------------------

I am not sure, but maybe using <![CDATA[]]]]> for descriptions helps.

regards

[Kai Moritz]

Post by Kropp, Henning
I am not sure, but maybe using <![CDATA[]]]]> for descriptions helps.

I've tried that. But it does not help :(

Looking for differences between the two systems, I remebered, that Ubuntu
comes with Java 6.0 and Etch only with 5.0.

The development systems, where the tests run with no problems, has:
java version "1.6.0"
Java(TM) SE Runtime Environment (build 1.6.0-b105)
Java HotSpot(TM) Client VM (build 1.6.0-b105, mixed mode, sharing)

The production system, where the strange DBUnit error occures and where
strings with german umlauts do not match, has:
java version "1.5.0_14"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_14-b03)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_14-b03, mixed mode)

I'm really running out of ideas, what the heck causes the tests to fail on the
production system...

[Kropp, Henning]

Post by Kai Moritz

Post by Kropp, Henning
I am not sure, but maybe using <![CDATA[]]]]> for descriptions helps.

I've tried that. But it does not help :(
Looking for differences between the two systems, I remebered, that Ubuntu
comes with Java 6.0 and Etch only with 5.0.
java version "1.6.0"
Java(TM) SE Runtime Environment (build 1.6.0-b105)
Java HotSpot(TM) Client VM (build 1.6.0-b105, mixed mode, sharing)
The production system, where the strange DBUnit error occures and where
java version "1.5.0_14"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_14-b03)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_14-b03, mixed mode)
I'm really running out of ideas, what the heck causes the tests to fail on the
production system...
---------------------------------------------------------------------

What about listing the files inside the pom.xml for the native2ascii
codehaus plugin?

regards

[Kai Moritz]

Post by Peter Schneider-Manzell
Hi Kay!
Maybe you should check the encoding of your sources.
Make sure all text files (e.g. xml / java) are encoded with utf8.

I think, I've checked that already. I am editing all my sources with Eclipse,
which is tuned to use UTF-8 as default encoding. But I will doub
le-check that
issue again this evening ;)

Greetings kai

[Kai Moritz]

Post by Peter Schneider-Manzell
Hi Kay!
Maybe you should check the encoding of your sources.
Make sure all text files (e.g. xml / java) are encoded with utf8.

I've double-checked this now. All files (xml and java) are encoded in utf8, so
this cannot be the reason for the strange error...

By the way:
If that would have been the cause of the error, the error should have shown on
both systems, shouldn't it?

Greetings kai