Categories
Bugzilla for Java CSV Java RS Library

Multiple Releases

The last few days I released new versions of three of my projects.

All projects are available through Maven Central.

 

Categories
CSV Java

CSV/Excel Utility improves performance

I recently stumbled across a performance test of Java CSV libraries (can’t remember where). To my surprise, someone tested multiple Open Source products including my own CSV/Excel Utility Package. And even more surprising to me, mine was the worst. By far! It took 4 times as much than others spent in parsing a CSV file. Embarrasing! Why did I never spent effort in measuring performance?

Anyway. I took the time and wrote a comparable JUnit perfromance test for all major CSV libraries and gave them a 150MB file to read. Analyzing the results of my own library with JProfiler, I found a very stupid performance eater (simplified here):

1
2
3
4
5
6
String s = "";
for (char c : anotherString.toCharArray()) {
   ...
   s += c;
   ...
}

The “addition” of line 6 was called 150 million times – for each single character in the file. Replacing this by a StringBuilder construct, the performance rose near the other major CSV libraries:

1
2
3
4
5
6
StringBuilder s = new StringBuilder();
for (char c : anotherString.toCharArray()) {
   ...
   s.append(c);
   ...
}

So two things to learn here: (1) Do not under-estimate performance test before releasing something, (2) Take care when using the “addition” operator for strings. 🙂

PS: CSV/Excel Utility Package 1.7 will contain the fix.
PPS: StringBuilder is preferred against StringBuffer as it is not synchronized and therefore faster. Most use cases allow this simplifications.

Categories
RsBudget

RsBudget Release Candidate

RsBudgetIt’s done. The last five months I have been preparing an old RCP application for release. I never thought that this would require so much effort in addition to the already existing functionality. I was used to the behaviour and configuration. So I never needed things such as a Preferences Dialog, a Help system, Installation procedure and many other things. In fact, they do exist now for the only purpose to allow every person to use the application.

So here it is. It is the Release Candidate for version 2.0. (The old private application was 1.0 and it will never get released 😉 ) Now I ask you to help me in making it bug-free. Please visit the RsBudget Homepage, download and install it (there are videos available explaining how to install it correctly), and then finally try to use it on a daily basis.

Whatever you think is wrong or not behaving right or not explained correctly: Please tell me! The application is released under GPL, and it’s free of charge. Thank you all for your support – and spread the word 🙂

Categories
Upload Maven Plugin

Plugin to Publish Eclipse P2 Repositories

I currently work to publish my first RCP application, based on Eclipse/E4 (Kepler). One of the major topics is to automate the complete build and publish process. Tycho does a good job there already, and in fact it works perfect :). I setup my Bamboo instance to build it without any interaction.

However, Tycho does not offer yet any possibility to finally publish P2 repositories. I understand this as publishing can be a difficult job when it comes to various P2 repository flavours (combined, single etc). The Deploy plugin of Maven 3 is not good for this job as far as I have discovered. So I faced the problem of somehow to publish Snapshots and  final releases automatically. The net didn’t come up with any automated solution. Most folks just recommended to perform that step manually.

So I created the Upload Files Maven Plugin. It’s job is to upload any file(s) to the repository valid for an artifact. Users simply define their repositories as they did previously (in <distributionManagement> section of their POM), and then add the plugin to their lifecycle. Here is an example of how to do this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
   <build>
      <plugins>
         <plugin>
            <groupId>eu.ralph-schuster</groupId>
            <artifactId>uploadfiles-maven-plugin</artifactId>
            <version>1.1.0</version>
            <executions>
               <execution>
                  <goals>
                     <goal>upload</goal>
                  </goals>
                  <phase>deploy</phase>
               </execution>
            </executions>
            <configuration>
               <path>target/repository</path>
               <targetPath>.</targetPath>
            </configuration>
         </plugin>
         ...
      </plugins>
      ...
   </build>

This will upload the P2 repository (created by Tycho) to your defined server repository. As the plugin uses Wagon for this task, you can use protocols such as SCP, FTP, WebDAV etc.

You also might also want to disable default install and deploy targets of Maven for your project:

1
2
3
4
   <properties>
      <maven.install.skip>true</maven.install.skip>
      <maven.deploy.skip>true</maven.deploy.skip>
   </properties>

One nice feature is the execution of pre and post commands on your remote server (tested with SCP only, other protocols might not support this). That can be used e.g. to clean your server repository before uploading the new one:

1
2
3
4
5
6
            <configuration>
               ...
               <preCommands>
                  <preCommand>rm -rf /path/to/repository/*</preCommand>
               </preCommands>
            </configuration>

These commands are executed in the user’s home directory, so please be careful :). There exists several possibilities to handle errors of such commands, so just check out the Goal documentation. It also lists other options you might find useful.

Version 1.1.1 will add a small variable substitution for commands, e.g. you can access the user name and base path of your server repository. This would allow a configuration such as:

1
2
3
4
5
6
            <configuration>
               ...
               <preCommands>
                  <preCommand>rm -rf $repository.basepath/*</preCommand>
               </preCommands>
            </configuration>

However, if you want to use that feature by now, you must use the 1.1.1-SNAPSHOT version. It is not officially released yet as I still want to get more experience with publishing P2 repositories before finally releasing it.

Feedback is welcome…

PS: Maven 3.1.1 is required to use this plugin!
PS: Version 1.1.1 was released meanwhile.

Categories
Java RS Library

RS Library 1.2.2 released

A new version of the RS Library was released. Version 1.2.2 fixes many bugs found by unit tests. The previous version just had 12 unit tests, the new version got 190 unit tests to ensure quality of the code. A complete list of changes can be found here.

The Maven repositories shall be up-to-date by now. The Maven coordinates are:

   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>baselib</artifactid>
      <version>1.2.2</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data</artifactid>
      <version>1.2.2</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data-file</artifactid>
      <version>1.2.2</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data-hibernate</artifactid>
      <version>1.2.2</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>templating</artifactid>
      <version>1.2.2</version>
   </dependency>
Categories
Java RS Library

The Class.forName() problem

In earlier Java days, it was easy to get a Class object when you had its name only. You simply called:

1
2
3
4
5
   // Without parametrized
   Class c = Class.forName("java.lang.String");
 
   // With parameters
   Class<String> = (Class<String>)Class.forName("java.lang.String");

This worked as long as there have been only one class loader. Long time ago :). The world kept spinning and concepts of separated applications in a single Java engine appeared. And so did OSGI. OSGI specified its container in a way that you can have a class being loaded in multiple versions at the same time. Furthermore, an OSGI bundle (effectively a JAR) can describe what other bundles can see their classes and what not. OSGI containers implement these behaviour through different ClassLoader instances responsible for the respective bundles.

However, this introduces a problem on Class.forName(String) calls. The method uses the ClassLoader of the “caller”. That means that it will be restricted to the bundle’s definition. As long as you use this method within bundle A to find a class in bundle A, everything works fine. It will even find a class in bundle B when B exported correctly and A defined B as a pre-requisite.

However, modular code encapsulates common functions. That’s why it can happen that a Class.forName(String) call is implemented in bundle C and doesn’t know about bundle A at all. When bundle A now calls bundle C which in turn calls Class.forName(String) to load a class from bundle A, then it fails. The reason is that ClassLoader C is not allowed to access classes in bundle A.

Several solutions are possible. The OSGI solution for this problem is the friend. You declare bundle C as friend of bundle A and class loading will succeed. However, this works only as long as you don’t try to load a class from bundle B. Now B would be required to declare C as a friend. If B is a 3rd party bundle then this solution is out of reach.

Another solution is the iteration on all OSGI bundles and trying to find the class via their class loaders. However, this is error-prone. Unless you don’t know exactly what bundle contains your required class you might get in trouble. Several versions of the same class can be present (or just define a class with the same name). You would need some heuristics to identify the correct class.

There is a better and most-likely the best solution. Each thread in Java can have a Context Class Loader. This will allow us to resolve all the dependency problems in OSGI. We will define the method in bundle C as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
	/**
	 * Load a class by first checking the thread's class loader and then the caller's class loader.
	 * @param className name of class to be loaded
	 * @return the class found
	 * @throws ClassNotFoundException when the class cannot be found
	 */
	public static Class<?> forName(String className) throws ClassNotFoundException {
		return forName(className, null);
	}
 
	/**
	 * Load a class by directly specifying a class loader.
	 * @param className name of class to be loaded
	 * @param classLoader the class loader to be used - if null the thread's class loader will be  used first
	 * @return the class found
	 * @throws ClassNotFoundException when the class cannot be found
	 */
	public static Class<?> forName(String className, ClassLoader classLoader) throws ClassNotFoundException {
		if (classLoader == null) try {
			// Check the thread's class loader
			classLoader = Thread.currentThread().getContextClassLoader();
			if (classLoader != null) {
				return Class.forName(className, true, classLoader);
			}
		} catch (ClassNotFoundException e) {
			// not found, use the class' loader
			classLoader = null;
		}
		if (classLoader != null) {
			return Class.forName(className, true, classLoader);
		}
		return Class.forName(className);
	}

The method will (if no specific class loader was given) first test the Context Class Loader (lines 21-24). Only if this fails or is (for whatever reasons) not available, then the original Class.forName(String) method will be asked (line 30).

As the Context Class Loader is defined in the beginning of the executing thread, it is usually the class loader of Bundle A which has access to all other dependent bundles. Therefore we will be able to load classes from bundle A and B with code written in bundle C.

1
2
3
4
5
6
7
8
   // Code in bundle C
   public Class<?> getClass(String name) {
      return BundleC.forName(name);
   }
 
   // Code in bundle A
   Class<?> c1 = BundleC.forName("bundleA.className");
   Class<?> c2 = BundleC.forName("bundleB.className");

I added the helper methods in my RS Base Classes Library class LangUtils where you can use it directly from (V1.2.2 – available very soon) in your OSGI projects.

Categories
Programming languages RS Library

The Importance of Unit Tests

I am not a close friend of unit tests. In fact, as a programmer I regard them as annoying. However, when I started to write unit tests for my RS Library classes, I discovered many bugs that I didn’t see before. Those bugs never occurred in production yet mainly because the code was used in a slightly different way and bypassed the buggy parts. Some of the bugs are very stupid so that I wondered how they could ever get into a release.

The experience shows me now that it is important to write unit tests for all classes and their non-trivial methods. This is the only way to write better code. The self-confidence that I have in my own code is not enough. So there will be a lot of unit testing effort within the next weeks. And the next RS library code is most-likely the best-tested code ever.

Nobody is perfect. 😉

Categories
Java RS Library

RS Library 1.2.1 released

A major milestone of RS Library version 1.2 is out. I already discovered a critical bug in 1.2.0. That’s why the latest version is 1.2.1 now. The most important changes are:

However, I already discovered a minor bug in V1.2.1: RSLIBS-37 describes an incorrect behaviour of SoftMapCache.containsValue(Object) and WeakMapCache.containsValue(Object) methods which will be fixed in next version. V1.2.1 does not use this methods anywhere, so you would need to work-around this function in case you require it.

I also added more documentation now so you will be able to understand and use the libraries. It can be found either through Javadocs or the appropriate module homepages:

The Maven coordinates are:

   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>baselib</artifactid>
      <version>1.2.1</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data</artifactid>
      <version>1.2.1</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data-file</artifactid>
      <version>1.2.1</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data-hibernate</artifactid>
      <version>1.2.1</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>templating</artifactid>
      <version>1.2.1</version>
   </dependency>
Categories
Java

Eclipse/E4: Problem with Key Bindings

I recently updated one of my E4 applications from Juno to Kepler (V4.3) release. Beside a few things (e.g. adding the HandlerProcessingAddOn to e4xmi configuration) I suddenly lost my customized key bindings. Unfortunately, there was not even a slight hint anywhere in the net why this happened. Most of the web sites were just referring to the afore mentioned add-on. But this didn’t bring back the missing key bindings.

So the only solution was to debug the bootstrap of RCP/E4 and find out where the key bindings disappeared. I noticed very early that my Binding Table was read but didn’t contain the correct key bindings. Although, an earlier breakpoint revealed that they have been read correctly. However, somehow they were removed and replaced by other ones.

Debugging in Eclipse/E4 is quite exhausting escpecially when you try to trace a problem in the bootstrap mechanism. But finally I saw an extension point (a “processor” to be precise) as root cause. This extension just threw away my key bindings. It eventually turned out to be the org.eclipse.ui.workbench plugin which I required in my application. This plugin defines a BindingToModelProcessor that simply throws away all key bindings within the org.eclipse.ui.contexts.dialogAndWindow Binding Context except those tagged with type:user.

That’s the solution. Simple as that. I added the tag to all my key bindings and they were not removed and working again. So be aware when adding org.eclipse.ui.workbench as a dependency plugin to your E4 application. 😉

Categories
Java

Hibernate 4 with JTA Transactions

The following problem occurred when I was migrating from Hibernate 3 to Hibernate 4.2.3. Scrollable result sets were suddenly throwing exceptions:

org.hibernate.exception.GenericJDBCException: could not advance using next()
	at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110)
	at org.hibernate.internal.ScrollableResultsImpl.next(ScrollableResultsImpl.java:121)
...
Caused by: java.sql.SQLException: Operation not allowed after ResultSet closed
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920)
	at com.mysql.jdbc.ResultSetImpl.checkClosed(ResultSetImpl.java:804)
	at com.mysql.jdbc.ResultSetImpl.next(ResultSetImpl.java:6986)
	at org.hibernate.internal.ScrollableResultsImpl.next(ScrollableResultsImpl.java:116)

It turned out after 3 hours of debugging and research that Hibernate’s 4 JTA implementation relies on having a transaction started not via JTA User Transaction but through Hibernate’s own Session object. The correct configuration in such cases is simply to use CMTTransactionFactory as transaction factory class:

1
2
3
4
5
6
7
8
9
10
11
12
13
<hibernate-configuration>
   <session-factory>
      <!-- How to find the Transaction -->
      <property name="hibernate.transaction.factory_class">org.hibernate.engine.transaction.internal.jta.CMTTransactionFactory</property>
 
      <!-- How to produce transaction -->
      <property name="hibernate.transaction.jta.platform">org.hibernate.service.jta.platform.internal.JOTMJtaPlatform</property>
 
      <!-- Session context with JTA -->
      <property name="current_session_context_class">jta</property>
      ...
   </session-factory>
</hibernate-configuration>

As explanation: The error is primarily caused by a bug in JTA implementation within Hibernate (it does not synchronize Hibernate’s with JTA’s transaction object). It occurs mainly when you use the Entity Manager. In above’s exception, a second retrieval was executed within object iteration which caused Hibernate to think that there is no active transaction and hence, close any previous statements. That’s why the Scrollable Result loses its session.

The solution was published first by Koen Serneels. Many thanks!