Just an ordinary Java blog

VisualVM and JFreeChart

2010-03-07T12:59:00.000-08:00

I work in an European project in which we are developing a set of VisualVM plugins to construct a management application. It's the same principle I've used on the VisualVM OSGi plugin: a set of MBeans in the application, and a nice GUI in the VisualVM side that uses JMX to access the MBeans.

We used JFreeChart to visualize (in the VisualVM) data captured by sensors (in the managed application).

I had some minor problems for using JFreeChart due to class loading that I detail next.

Importing JFreeChart the wrong way

If you create a regular Netbeans Module project you won't be able to import libraries. So you need to create a Module Suite (File > New Project > Netbeans Module > Module Suite), and then add the Module project of you plugin to that Module Suite (see picture below).

Then, you can go right click the project, select Properties > Libraries and get the below pop up window to add your library.

What I initially did was importing in two separate steps. First the JFreeChart library, and then JCommons which is necessary for JFreeChart. After executing my code I got a java.lang.ClassNotFoundException: org.jfree.date.MonthConstants. Weird. Everything was imported. Well, aparently we had some classloading visibility issues here.

As previously described, I had configured the following dependencies:

MyModule -> JFreeChart

MyModule -> JCommons

Doing it the right way (well, at least in a way that works)

Adding the libraries two steps gives me two Library Wrapper Modules, which will belong to the same Module Suite. But when I added both libraries at once (resulting in just one Library Wrapper Module) and tried executing the JFreeChart code, my chart appeared and I had no longer a ClassNotFoundException. Thus, the depencency became:

MyModule -> {JFreeChart, JCommons}

I believe the above issues are the same for any other Netbeans RCP application when trying to use JFreeChart.

Just to have a sample of the nice charts that you can add, here goes a screenshot with some example charts. Library download and more info at the JFreeChart website

VisualVM OSGi Plugin available at the plugins center

2009-11-22T14:40:00.000-08:00

The VisualVM OSGi plugin has become part of the OW2 Chameleon project, which tries to deliver a set of pluggable components for OSGi frameworks, allowing to create sorts of lightweight OSGi-based application servers with just the needed functionality. The VisualVM OSGi plugin was the initial step for a set of VisualVM plugins that will be provided in the Chameleon project.

Recently Jiri Sedlacek and Tomas Hurka have proposed to add a link to the OSGi plugin in the VisualVM 1.2 plugins center. Now you can install the plugin from the VisualVM 1.2 or above, in the menu Tools > Plugins. Don'f forget to download and install the appropriate OSGi bundle in your OSGi application (this is how the plugin will get the info about the OSGi application).
Detailed installation instructions can be found on the plugin page in Chameleon's wiki

Many thanks to my thesis advisor, Didier Donsez, for the idea of the plugin.
Thanks to Clément Escoffier, for inviting the VisualVM OSGi Plugin to the Chameleon project.
Thanks to Jiri and Tomas for all the tips, testing and for adding the plugin to the VisualVM plugins center, and thanks in advance for anyone who would eventually use the tool and hopefully provide any comments for improving it.

Meeting of the OSGi users group France

2009-10-16T13:45:00.000-07:00

The 9th meeting of the OSGi users group France took place today, at the Grenoble University Campus. People from different places in France have attended. The central theme of the talks was about isolation, mostly concentrated in recent research efforts. In the morning we had three talks:

Laurent Daynès, researcher from Sun Microsystems at Grenoble, gave the talk “Lightweight Isolation with a multi-tasking JVM” where he talked about motivations for multitasking, lightweight isolation in the JVM (obviously! It is the presentation's title), Java Isolates, the Multitasking Virtual Machine (MVM) and a lot of interesting stuff concerning isolation.
Nicolas Geoffray, a fresh PhD from Université Paris 6, presented the "I-JVM: a Java Virtual Machine for Component Isolation in OSGi", which he developed during his thesis under the co-direction of Gaël Thomas who was also there. He shows how he implemented lightweight Java Isolates in the VM level, and how he associated them with OSGi bundles allowing the safe termination and fine grained resource control of bundles.
Kiev Gama (sounds familiar to me...) presented the initial efforts of my PhD’s first year, under the direction of Didier Donsez, in the talk entitled "Towards Dynamic Component Isolation in a Service Oriented Platform". In our work we provide a sort of fault contained sandbox for OSGi components by means of Java Isolates.

In the afternoon, four more talks:

My ex-labmate on the Adèle team Clément Escoffier, from Akquinet and leader of the iPOJO project, presented directly from Berlin via DimDim the talk "Chapter 4.2, Verse 138 : The Framework Multiplication". The RFC 138 is an ongoing specification at the OSGi alliance for allowing different OSGi framework instances to be hosted as nested frameworks isolated from each other. Although it addresses more issues, the basis for this specification seems to be quite similar to what Stéphane Frénot and Yvan Royon did a few years ago with V-OSGi (V for Virtual) by virtualizing different OSGi platform instances on one framework.
Stéphane Frénot, professor from INSA Lyon, presented the ROCS (Remote OSGi Caching Service) which targets resource constrained devices allowing OSGi applications to load classes from a remote cache, saving space on the device's persistent memory.
Next, Pierre Parrend, researcher from FZI Karlsruhe, in another live broadcasted session via DimDim talked about "Software Security Models for Service-Oriented Programming", where he presents different security threats that may be presented by OSGi bundles. He showed also how we a hardened OSGi can be implemented.
Another ex-member of my team, Mikaël Désertot, professor from Université de Valenciennes, presented "Vehicular network: The VESPA use case" where OSGi is used in a vehicular project.

Well, I must admit that during this last presentation I could not pay too much attention because I was showing to Nicolas Le Sommer and Laurent Daynès a snippet for crashing a JVM. During my presentation in the morning I had mentioned that and told everyone to try it home, but Nicolas had the initiative to try it already in the afternoon :)

Here is the snippet that crashes the JVM, taken from here:

public class Crash {
public static void main(String[] args) {
  Object[] o = null;
  while (true) {
     o = new Object[] {o};
  }
}
}

Nicolas tried the above code which crashed the Sun JVM. However the GNU Classpath and IBM JVMs did not crash…

Laurent tested it in his machine, also crashing the Sun JVM. He debugged it quickly and could precise that it happened when a full GC took place, but someone needs to fix it :)

The presentation abstracts and slides can be found here.

VisualVM OSGi plugin

2009-06-16T02:51:00.000-07:00

I wrote a simple OSGi management console as a VisualVM plugin, based on a concept similar to the OSGi JConsole plugin that my advisor asked me to do last year.

In this post I will describe the main steps for constructing the OSGi bundle and the VisualVM plugin, including the usage of a table from the JFluid API (so you plugin can look pretty much like the standard VisualVM stuff) and how simple it is to use JMX from your VisualVM plugin.

If you want to skip the development steps, you can install and try the plugin by downloading the built sfuff here.

If you want to check the step-by-step, please continue reading.

There is not too much advanced stuff here, but there are three pre-requisites for a complete understanding of this post:
- A minimum of familiarity with VisualVM plugin development (You may want to try the Hello VisualVM plugin before)
- Basic JMX knowledge (JMX Tutorial here)
- Basic OSGi knowledge (This main purpose of the plugin is not very useful to you if you don’t use OSGi, but you still can check out the JMX + VisualVM stuff).

I’ll show in this post:
- Brief description of the plugin functionality
- How I used Netbeans to create an OSGi bundle exposing some management functionality via JMX
- An overview of how I developed VisualVM plugin to access the JMX stuff above for accessing/managing OSGi applications
- Some of my findings that I’ve think useful to describe here

What I’ve used to develop, build and run the code show here:

Java 1.6
Netbeans 6.5
Apache Felix (you can use another OSGi distribution like Equinox, Knopflerfish)
VisualVM 1.1.1

1. Plugin Functionality Overview
This plugin can be an alternative to the OSGi consoles that exist, or yet another toy for your VisualVM plugin set.
The functionality of the plugin (screenshot below) is simple:

Listing of all bundles in an OSGi application
Installation of bundles
Perform a start/stop/update/uninstall in a bundle from a list
Visualize bundle information (Manifest headers, list of services, navigate through bundle contents).

2. Accessing OSGi via JMX
In order to have a plugin in another process (the VisualVM) to access the OSGi platform, we needed to expose some interfaces for that. JMX is an easy way to expose management interfaces for Java applications, and there is built-in support in the VisualVM for using it.

In this section I describe the development of the OSGi bundle that exposes this funcionality.

I created a Netbeans project as a Java class library (I have called mine OSGiJMX)
- Next I needed to reference a jar with the OSGi classes.
- Right click on the project, go to project properties > libraries > add libraries, I needed to create a new one pointing to the felix.jar which contained the needed OSGi API
- OSGi bundles need special manifest attributes. So I’ve added my own manifest to src/META-INF/MANIFEST.MF with the following content:

Manifest-Version: 1.0
Bundle-Version: 1.0.0
Bundle-Name: JMX OSGi
Bundle-Description: A JMX OSGi Example
Bundle-ManifestVersion: 2
Bundle-Vendor: Kiev Gama
Bundle-SymbolicName: osgijmx
Bundle-Activator: osgijmx.impl.Activator
Import-Package: org.osgi.framework,javax.management
Export-Package: osgijmx, osgijmx.impl

- I needed to change the project ant build file so it would use my manifest instead of the default one. On the files tab of your project (usually on the left side of the Netbeans IDE), open the build.xml

<import file="nbproject/build-impl.xml"/>
<!--ADD LINE BELOW -->
<property name="manifest.file" value="src/META-INF/MANIFEST.MF"/>

Basic operations should be made available via JMX MBeans, so our VisualVM plugin can access information concerning an executing OSGi platform and its bundles.
The MBean that my advisor provided me for the JConsole plugin previously mentioned could be used, but it was a single MBean instance that mixed bundles and platform methods. At that time a friend from my lab suggested to create an MBean for each bundle instance. That’s what I finally did now. So, I’ve defined two MXBean interfaces:

One for the OSGi Platform methods (kinda like a proxy to BundleContext which gives access to the OSGi platform)
Another for bundle related methods (worked like a proxy to a Bundle instance).

The code for the interfaces:
BundleMXBean and OSGiPlatformMXBean
The code of their implementation:
BundleMXBeanImpl and OSGiPlatformMXBeanImpl
the bundle Activator

The OSGiPlatformMXBean will hold a list to all Bundle MXBean instances (thanks to Eammon McManus’ blog).
OSGi is a dynamic platform: bundles can change their state, new services may arrive, etc. So, this JMX bundle should listen to Bundle events in order to add or remove the bundle MBeans in case of bundle installation or uninstallation, respectively, as well as notify (via JMX) any changes of bundle state so the VisualVM plugin GUI could be updated and always have consistent information. Currently the code does not have JMX notification of service events (a temporary workaround would be select/deselect the bundle from the list so its services list would be updated).

If you’ve followed all the steps you can build the project and test the bundle (or maybe download it from here).
Run Felix, and type start .
If you start JConsole and connect it to Felix, you should be able to visualize all tof our OSGi MBeans in the MBeans tab.

3. OSGi VisualVM Plugin Development
The VisualVM plugin will depend on the JMX interfaces of the project described on the previous section, so it would need to be built and deployed with the plugin as well. My choice was to create a Module Suite, for aggregating the library and the VVM plugin.

So, you'll end up with four Netbeans projects:

The OSGi bundle from previous step
The Module Suite (just an umbrella project for the two below)
A library wrapper for the OSGi bundle
The VisualVM plugin

3.1 Steps for creating the Module Suite:
-New Project > Netbeans Modules > Module Suite

- Next, choose the target Netbeans platform (in our case is VisualVM 1.1.1)

3.2 Creating the Library Wrappper module

- Choose a Netbeans Project > Library Wrapper Module
- Select the libraries (the jar files) to be wrapped. In this case I've wrapped the OSGiJMX.jar and the felix.jar in the same module. - In the next step you must indicate which Module Suite will contain your wrapped library (as indicated in the figure below). Then you can go to the next and final step where you define your codebase.

3.3 Creating the VisualVM Plugin

- Create a new project > Netbeans modules > Module
- Important: Choose the module suite create before, so your plugin module will be able to resolve the classes of the wrapped library under the same Module Suite (but we still need to configure that)

After this step was finished, I've created a module installer:
- Right click on the plugin project, New > Module Installer
- I've named it installer and added the code that follows

public class Installer extends ModuleInstall {
@Override
public void restored() {
BundlesViewProvider.initialize();
}
}

Besides the Installer and the BundlesViewProvides, the other classes for the VVM plugin were:

Dependencies
After this, we would need to add the module dependencies concerning the imported types. Right click on the project > Properties > Libraries. Then you select the dependencies (check the "Show non-API modules" on the pop-up dependency window) listed in the figure below:

At this point, trying to run/build the plugin gives the following error:

"The module osgijmx.vvm is not a friend of \profiler3\modules\org-netbeans-lib-profiler-ui.jar"

We can solve this by editing the JFluid module dependency and setting it as an "Implementation Version"

Ok, but why depend on the JFluid-UI classes?
Well, these are the classes used for developing the pretty Visual VM built-in plugins. So, in order to get our plugin to look similar to them we used the JFluid-UI. The result can be seen in the table on the figure below where we inspect Fuji.

Here's the code for creating the table (JExtendedTable from JFluid UI):

        bundleTable = new JExtendedTable(tableModel);
bundleTable.setBorder(BorderFactory.createEmptyBorder());

bundleTable.setRowSelectionAllowed(true);
bundleTable.setSelectionMode(ListSelectionModel.SINGLE_SELECTION);
bundleTable.setGridColor(UIConstants.TABLE_VERTICAL_GRID_COLOR);
bundleTable.setSelectionBackground(UIConstants.TABLE_SELECTION_BACKGROUND_COLOR);
bundleTable.setSelectionForeground(UIConstants.TABLE_SELECTION_FOREGROUND_COLOR);
bundleTable.setShowHorizontalLines(UIConstants.SHOW_TABLE_HORIZONTAL_GRID);
bundleTable.setShowVerticalLines(UIConstants.SHOW_TABLE_VERTICAL_GRID);
bundleTable.setRowMargin(UIConstants.TABLE_ROW_MARGIN);

Using JMX from a VisualVM plugin

The usage of JMX was straightforward, without any problems. From our DataSourceView the Application instance can be used to retrieve the MBeanServerConnection:

public BundlesView(Application application) {
super(application, "OSGi", new ImageIcon(Utilities.loadImage(IMAGE_PATH, true)).getImage(), 60, false);
mbs = JmxModelFactory.getJmxModelFor(application).getMBeanServerConnection();
...

After that, the MBeanServerConnection can be used to retrieve a proxy to the MXBean

ObjectName objName = new ObjectName("osgijmx:type=framework");
OSGiPlatformMXBean osgiProxy = JMX.newMXBeanProxy(mbs, objName, OSGiPlatformMXBean.class);

The invocations of the methods were ok, but I had some problems with a MXBean method that returned

a Map[] :
java.lang.ClassCastException: sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl cannot be cast to java.lang.Class

But I still had errors changing the return type to a Map of Map.
Surely issues with JMX’s Open type translation. The above methods did not work called from my proxy (any help from JMX experts would be great…), but the Map getBundleHeaders() did work…

I tried using an MBeanServerConnection.getAttribute and it did work. However it obliged me to navigate through TabularData instead of a java.util.Map, just the way JConsole does.

Building and deploying
Inside the Netbeans we can run the plugin, but if we want to deploy it in the VVM we need to build the nbm files. Just right click the Module suite project, and hit Create NBMs.
Going to the updates folder of the Module Suite project we can find the two generated files.
For their deployment on the VVM, the steps are the same from here.

I've tested it just in Felix-based applications like Glassfish and Fuji, and it worked fine.

For deploying in Glassfish the OSGi bundle with the JMX probes:

GLASSFISH_HOME\bin\asadmin deploy --type osgi YOUR_PATH\OSGiJMX.jar

This is it. Maybe a long post, but this is the end of it...

Suggestions for new features are welcome, as well as source code for doing so.

References:
Getting Started Extending the Visual VM
JMX Tutorial
OSGi Javadoc
Inter-MXBean references

A scripting bundle for OSGi

2009-02-12T08:39:00.000-08:00

I've wrapped the scriptconsole4j as an OSGi bundle (tested in Apache Felix 1.4.0 and Equinox 3.4.0).

This console is a simple GUI that allows using the JSR-223 compatible scripting languages available in you Java 6 VM. The Rhino Engine (Javascript) is available by default.

If you want to execute scripts accessing the OSGi framework to retrieve services in adhoc scripts, you may find this version of the scriptconsole4j tool useful. When the bundle is started a scripting window is automatically displayed. Upon bundle termination, the window is disposed and vice-versa.

How to use it:

Just download it.
Install it on your favorite OSGi framework
Start the bundle
And its ready to be used

You can, for example, register listeners to framework objects:



  var o = new Object();
  o.bundleChanged = function(e) {
      message = "An event happened to bundle "
                + e.getBundle().getBundleId();
      output.println(message);
      //Writing in the standard output
      java.lang.System.out.println(message);
  };

  ctx.addBundleListener(new org.osgi.framework.BundleListener(o));

Since the bundle uses DynamicImport-Package, you can use it with any exported class, so you can dynamically implement and register services:

var o = new Object();
o.sayHello = function() { java.lang.System.out.println("closed")};
  
r = new com.foo.HelloServerService(o);
//registering the service in the OSGi Service Registry
ctx.registerService("foo.BarService",r,null);
//retrieving the service instance
service = ctx.getService(ctx.getServiceReference("foo.BarService"));
service.myMethod();

Take a look at the Rhino page showing how to use Java from Javascript, and use your imagination for the scripts that may help you.

Recently I needed the scripting functionality in OSGi on a Java 5 Virtual Machine, which does not come with the JSR-223. I've removed the JSR-223 dependant stuff from the scriptconsole4j and embedded the beanshell core with it. It worked fine but without dynamic class generation. I guess by adding the bsh-classgen it "would" work. Since we are running on OSGi, classloading is always an issue when dynamically generating stuff...
I'll test it and leave it available soon (I hope so).

Withdrawn JSRs

2009-01-14T07:36:00.000-08:00

I've checked the status of JSR-277, which was announced as being halted by Sun, but its status did not change to withdrawn. That makes sense, since it was announced as temporarily halted.
So I went to check the list of all withdrawn JSRs, just for curiosity, and the number surprised me: 52. A lot of those were concerning JAIN (which I haven't heard of until now). Some of the JSRs had comments like "withdrawn at the request of the submitter".

If you look at the number of all JSRs (355) we can do simple math and find that 15% of all JSRs have been withdrawn. It is true that they vary a lot in size, as there are umbrella JSRs that have a lot of stuff behind and we have some successful efforts that are heavyweight. Also, among that list there are "spinoffs", like the JSR-8 which originated OSGi. But, is 15% a high rate or considering the fact that standardization is a complex effort that involves people from different organizations which may have different interests, could we say that it is a fairly acceptable percentage ?

Can AOP be some sort of elegant workaround?

2008-11-24T13:45:00.000-08:00

In the pursuit of problem solving (or at least trying to) in my research, I guess I am taking AOP as a strategy to give me what I don't have. Maybe I am becoming too addicted to pointcuts...

I have used Aspect Oriented Programming for intercepting method calls to the OSGi framework and for adding my "concerns". These interceptions allowed me to, mainly, add hooks to service registration and retrieval. I think the good thing is that the solution was elegant enough and enabled portability accross different OSGi implementations, since the aspects targeted the API. The bad thing is that it does not rely in a standard mechanism supported by the platform, and it needs an additional building step (i.e. weaving the framework).

After taking a look at the draft OSGi spec 4.2, I've checked the Service Registry Hooks with the hope that my weaved hooks could be replaced by a standard mechanism. They have exactly proposed the hooks I wanted but sadly that alternative solution had been rejected. The hooks they will provide still do not allow me to do what I need. I'll still be doing the AOP trick...

I just can think of more pointcuts to add to my future solutions/attempts in order to have other types of hooks. As I said previously, this approach is useful to avoid changing the source code of a chosen OSGi implementation. There are lots of people in the research area who usually customize an OSGi impl to their needs (I still see recent articles talking about customizations of OSCAR). One disadvantage is that if you touch the source code directly, you're stuck with a given version which makes it difficult to follow the evolution of the chosen impl. (either Equinox, KF or Felix). You would need constant effort for merging/synchronization your branch with the current version. I believe they could avoid that by doing the AOP trick as well. However, sometimes (like while writing this) I am not sure if this a true elegant solution or if I've just found a nice and powerful workaround for adding my hooks to the OSGi core.

How to open source an OSGi targeted tool?

2008-09-18T09:21:00.000-07:00

I was working on a tool (with a funny name, or a ridiculous name if you will) for the diagnosis of stale references in the OSGi platform. It was part of my master thesis at Grenoble 1 university. The tool has been presented in an OSGi event in june, and in an academic conference in the beginning of this month. Among our “future plans” is to open source this tool. I am starting the PhD in the same university, and this tool will not be my main activity. I will have plenty of other things to do also. However we would like to continue evolving it.

But, I have several questions for all this process of releasing such a thing as open source. Maybe an inner self crisis!

1. Is this tool useful to anyone?

This tool is no rocket science and it does not cure problems, it only points a few of them.

My first thought is that the thing is useless, because I will always have the thought that what I’ve coded sucks and I could have done better…

But, if we do not make it available for the developer community it will surely be useless, since it would not be accessible to others. We never know if any a brave (and maybe a little insane) person would like to help evolving that.

2. How (and where) to do open source it?

Initially I may think to put it in sourceforge or google code. But my thesis advisor says it would be useless and without a good visibility for the target audience: developers of OSGi based applications. I agree. I’ve made available a small “useless” scripting console at google code, but the download count is insignificant, thus nobody is using it. We can imagine that a more specific project would fall into oblivion.

So, my thesis advisor, who is a committer in an Open Source OSGi implementation, has contacted its project leader. The guy said ok, but no more open source progress for our tool after that. I am not sure if : 1- he was just being polite and said ok; or 2- he is so busy that he can’t get into too much details on all mail that he receives. We also have not insisted since other stuff came up.

3. What if the code is not ready yet?

I am sure it is not ready. I feel like I’m leaving my house and while I'm not there people come in and see all the mess: dirt dishes, laundry, underwear in the floor. I would need to polish it. And after that, the project will no longer be “my house”, since it would belong to the community. Then the dirty laundry will be everyone's business.

But, that leads to another question: when would it be ready for that? I guess the polishing is just removing the really ugly hardcoded stuff, a few bad practices (did I just said that?). The GUI is complicated for the end user to understand it. It would definitely need some good work. I confess that in a 5 month project, with a bunch of articles to read, and also the master’s thesis to write I’ve neglected in some software engineering and also human interaction practices. If anybody has never done that before, cast the first stone.
Ah! Mavenization also. That's part of the polishing. For simplicity, I’ve used ant. Decent projects these days at least use maven for managing dependencies.

4. What if they find bugs in my code?

Normal. Just open an issue. I am far away from being a bug free coder and I guess 99% of the developers out there are not bug free either.

5. When to do the open sourcing?

Good question. I guess ASAP before I have even less time for polishing the code.

6. Volunteers for continuing the tool development?
I believe we have at least one: me :)

Well, these are basically my doubts that I share here in this post based on the little spark of dellusion that my master thesis result would be useful to someone.

Exclusive OSGi session at Euromicro SEAA

2008-09-05T08:10:00.000-07:00

I was at the 34th Euromicro conference on Software Engineering and Advanced Applications (SEAA) to present an article about the tool that we have developed for detecting stale references in OSGi applications.
The conference had a session dedicated to OSGi in the Component-Based Software Engineering Track. Somehow it shows the industry/academia recognition of the important role that OSGi plays in the development of component-based appplications.

There were three articles presented in that session:

"Enhanced OSGi Bundle Updates to Prevent Runtime Exceptions"
Premysl Brada
"Method for resource monitoring of OSGi-based software components"
Tuukka Miettinen, Daniel Pakkala, Mika Hongisto
"Service Coroner: A Diagnostic Tool for locating OSGi Stale References"
Kiev Gama, Didier Donsez

The other two presentations were pretty interesting. Each one of us showed little "hacks" for addressing some issues in the OSGi platform :)

Benchmarking the cost of dynamic proxies

2008-08-10T11:01:00.000-07:00

I needed to quickly compare the cost of dynamic proxies versus direct access to objects, and also position it in relation to RMI. In theory we know that they add overhead, but how much? I just wanted more precise arguments to reinforce a rather obvious statement: “Dynamic proxies are not that bad. Take RMI for example...” :)

I could not find exact and up to date info saying that RMI is X times slower than direct method calls, nor dynamic proxies. In this link they’ve said that RMI is at least 1000 times slower than local calls. Since that appears to be some info from 2001, I imagine that we’ve had some nice optimizations in JVMs in the last 7 years.

I’ve changed a benchmark code that has been used in my lab, so I could have an approximate value of each method call in different approaches. This post (which actually points to here) that I’ve just read this Sunday helped me a lot in adapting the code.

My benchmark consisted of calling a million times a method in a given object which implements the tested "dummy" interface. The method was void with no parameters, and the implementation had just one line of code assigning an integer variable. The idea was to get the closest possible to the actual invocation time, with the minimum possible for method execution time.

I’ve executed the benchmark a few times and I’ve always got the same minimum values for direct calls and dynamic proxy, respectively. However the minimum time for RMI calls varied a lot.

Dynamic proxies: 1.63 times slower than direct calls, which is not bad.
RMI calls: At least 200 times slower.

For info, here is my not so performing platform:
JVM: Sun Hotspot/JRE 1.6.0 07.
OS: Windows XP SP2.
Hardware: Pentium 1.7 GHz 1GB RAM

The cost of dynamic proxies surprised me. This article from developerWorks shows similar conclusions. I thought they would be more expensive. However, we are talking about a minimal overhead level here, as in actual usages of proxies we usually add more code for decoration, verifications, etc which would obviously lead to more than the 1.63 times that I’ve measured. Well, this info provided here cannot considered sufficient as more tests would need to be performed in order to be more precise.

JSR 296 in coma ?

2008-07-20T23:47:00.000-07:00

I’ve been checking the upcoming features of Java 7 and found a cool detailed list here. Also, some Java One 2008 slides from Danny Coward’s presentation show a little on that too.

Among the JSRs mentioned in the first link, I already had checked some stuff from JSRs 277, 294, concerning Java Modules, JSR 284, concerning resource consumption and JSRs 295, 296 and 303 concerning Swing.

I’ve known these swing-related JSRs since, more or less, they became available as JSRs. I had special interest on them at the time because I was working in a project where we needed a product built on top of a Rich Client Platform, to enable the development of plugins from third parties. The idea was to have a simple and pluggable structure.

For us, Eclipse RCP was overkill so was the Netbeans platform. Spring RCP was in its initial stages. We’ve decided to go from scratch, using OSGi as the base for a pluggable architecture, and build our own straightforward RCP. The JSR 296 (Swing Application Framework) proposes most of what we needed (and developed) as a Swing foundation for our RCP:

Resource management for i18n
Task services/monitoring
Storage for session state
Events/actions framework
Managed application exiting (exit listeners), startup, shutdown

I can't say that this is a PCP (Poor Client Platorm). Maybe an ARCP (Almost Rich Client Platform). But, as I said, it does most of what we needed at that time. And I believe there are hundreds or thousands of other applications that don't need all the heavy richness provided by the RCPs.

So I decided to take a look at JSR 296 (Swing Application Framework) to see what they have for us so far. I’ve downloaded it from the project site. (You should also need to download the SwingWorker since they do not use the SwingWorker delivered with Java 6)

This tutorial has good examples concerning the features of the JSR 296, like the magic stuff for saving session state (components dimensioning, positioning, etc):

//The non-qualified getters below are inherited from the SingleFrameApplication class
//
//Saving session State
getContext().getSessionStorage().save(getMainFrame(), sessionFile);
...
//Restoring session State
getContext().getSessionStorage().restore(getMainFrame(), sessionFile);

Lots of weeks of modelling and coding can be saved by using that framework.

There was a weird thing about the last available version: it was from November 2007. It seems that it is not evolving anymore. I’ve read a few months ago in blogs that I don’t recall, that some senior engineers from the Swing team were leaving Sun Microsystems. I could find this blog entry mentioning some of them.

The guys that took care of JSR 296 (Hans Muller) and 295 (Scott Violet) are gone. However, I think that there are no worries with JSR 295 and 303 which are just “standardizing” the concepts used in JGoodies by Karsten Lentz, who is a member of both expert groups. But the JSR 296 is apparently a home grown implementation from Sun. The guy that was handling that is gone, and there are no new versions for 8 months.

Weird… It appears that the project is in some sort of "coma".

Don’t know if it will make it until Java 7

Heap snapshot analysis and objects with different class versions

2008-07-06T23:01:00.000-07:00

I’ve been working lately on the detection of a particular problem, called stale references, that may happen on OSGi based applications. It is a consequence of bad OSGi programming practices that may lead to memory retention and the utilization of inconsistent objects.

As a part of the diagnosis process I need to analyze heap snapshots to find the referrers of services that have been unregistered. I’ve found some interesting stuff (at least for me) concerning memory inspection and classloading. Each module (bundle) in OSGi is provided with its own class loader instance. If you replace (update) a module during runtime, it will basically stop, refresh and restart; and get a new class loader. Objects and classes from the previous class loader must not be referenced anymore, and that previous CL is supposed to be “discarded” and GC’d.

In our custom tool I was seeing that objects from “discarded” class loaders were still being referenced by other modules. However, when I used JHat (either embedded or standalone) to inspect the heap snapshots by performing queries like “select x from com.foo.TheClassOfTheReferencedObject” the result set listed only one object instance (from the running module) when I was supposed to get two instances (one loaded by the old class loader and other by the new one).

I thought my tracking code was falsely accusing the service object from the old bundle version. After patiently and manually (maybe “stupidly”) verifying each class loader instance, I found the two different class loaders that referred to the same bundle ID, and could also found the “lost” object.

After a few weeks I’ve tried to track the same problem with the heap inspection provided in the VisualVM. It worked like a charm! I could see all object instances of the same class name, no matter what class loader provided the object class. It does not have advanced queries like JHat, but in this case I only needed a filter to find the class instances that interest me.

It’s a pity. JHat has powerful queries to analyze heap snapshots, but it can’t deal clearly with the same class names loaded by different class loaders (i.e. different versions of the same class). VisualVM has much more limited power that JHat, but allows to see attribute values, referrers and referring trees just like JHat. In addition, VisualVM deals with no problem with classes carrying the same name but loaded by different class loaders, and that what’s more interesting for me, at least for now.

How can we developers (not the companies) do off-shore jobs ?

2008-06-16T12:46:00.000-07:00

Recently (actually just 40 minutes ago) I got an “almost certain” temporary home-office contract refused because I am overseas.
Huh ?
I’ll explain it… But skip to (2) if you’re in a hurry…

(1) The company is located in Brazil, and some people who know my job (and apparently trust on it) wanted to hire me for a temporary home-office job in that company. No development services with them so far, but as a freelance trainer last year I’ve even taught Java classes in this company which is also a faculty and tech training school, so I’m not a complete stranger for them.

I've quit my job 10 months ago and came to France to enroll in a computer science master’s degree. Ok so far. That's my choice. After knowing that some friends had done home-office for them, and since I was about to take summer vacations I’ve contacted them one month ago and said “hey if you need somebody for home-office development I’m here”.

Before leaving I left a legal representative in Brazil who can sign papers in my name, open bank accounts, etc. I’ve informed the company that I was going to be able to give legal receipts, invoices or whatever, which was what they have asked for.

But their legal department said NO…

(2) To make this story shorter:

The receipt/invoice would be issued by the town hall of my home city in Brazil (this is the type of receipt they asked for)
The money would be deposited in my bank account in Brazil
The job was going to be performed by a brazilian citizen with brazilian documents who would sign a contract with them, but with a TEMPORARY address in France and a PERMANENT address in Brazil.

However, as I am not residing in the country, the legal department refused to issue the temporary contract.

I have no clue how to deal with such situation. Neither these guys who contacted me have. They had good intention but the big guys said no…

My question is: how to deal with such issues related to off-shore jobs? Usually companies are hired to do off-shore. But what about people? Developers like me, like you.
Does anybody have ever faced similar situations?

OSGi Community Event

2008-06-02T23:10:00.000-07:00

The second annual OSGi Community Event will take place next week (10-11 June), in Berlin, Germany. The event will have presentations and demos concerning both business and research projects.

It is a little expensive (550 euros for non-members) but it is a good opportunity to business and research personnel gather together and exchange ideas and present results on OSGi technology usage.

More on the "OSGi hype"

2008-05-25T05:31:00.000-07:00

I’ve seen more and more people talking about OSGi lately. Some are saying that we are going through “OSGi hype”…

Well, I’ll talk about the OSGi hype here too :)

OSGi is being around for almost ten years. Maybe it gained (more) momentum after Eclipse adopting it since version 3. I’m not sure, I have no "authority" to say that. That's just what I think as an outsider.

A non-exhaustive list of OSGi advantages: no more classpath hell, modules life cycle, dynamic updates with no application reboots, package visibility, decoupling through services.

Several important organizations are seeing OSGi’s value and are investing on it. This guy posted a list of applications using OSGi. We can add Apache Sling , Apache ServiceMix and Project Fuji to that list too, as well as Glassfish. I've worked as part of the team that developed a now extinct product built on top of OSGi. There must be a lot of other not so popular stuff built with OSGi.

Maybe some recent events caused all this hype:

JOnAS 5 released as the first (I guess it was the first one) JEE application server to run on top of OSGi.
Recently we’ve seen Glassfish using it.
JBoss is talking about OSGi.
BEA (now Oracle) is using OSGi in its microservice architecture.

I guess OSGi has become the de facto module system for Java. It seems that JSR 277 has a risk of falling into oblivion when it becomes released if it does not converge with OSGi. Since so many important announcements about OSGi adoption are taking place, we are seeing this “OSGi hype”.

In my opinion, this is not hype. It is industry evolution. But those that still do not get it may see it only as hype.

Technology hypes may sometimes lead to some naïve decisions like “let’s use it because it’s a cool trend and everybody is using it. It's a new trend”. Maybe that was a naïve example too...
Before choosing to use stuff like OSGi try to evaluate if it fits your needs. Get to know it better. Maybe taking a look at the docs in the OSGi alliance website or looking for tutorials. Decisions like this involve the architecture of your project.

Whoever tries to use OSGi must be careful with the sometimes steep learning curve. Try to take advantage of existing component models (OSGi declarative services, Service Binder, Spring DM, iPOJO) that handle hard dynamic aspects, service location, dependency injection, and do all the dynamic voodoo magic. There was a time when all that stuff needed to be made by hand. You don’t need to do that anymore.

There is some stuff that is painful in the beginning like “bundlizing” a jar. But constructing an OSGi compliant manifest may be simplified by automated stuff like a bundle plugin for maven. Well, there is plenty of stuff (like here) to minimize efforts concerning OSGi development.

I'm away from the market for almost a year and started (or continued, if you will) to do research that involves OSGi. Each time I see news about its adoption in industry I get even more motivated about my choice.

Even if you don’t buy the idea or if it is useless for you, it is worth taking a look. OSGi is not just a buzzword. Try to use it not because it is hype, check its benefits first.

Do you know weak references? Have you ever used them?

2008-05-14T14:16:00.000-07:00

I’ve been using a lot of weak references lately. A couple of years ago I’ve read a blog entry that said that a lot of experienced Java developers did not know what they were. I’m glad I already knew it at the time I’ve read that :)

I won’t go into details of weak references. I’ll just say that an object is not prevented to be garbage collected if it only has weak references pointing to it. You can get a nice overview here.

In my current project I’m tracking garbage collection of special types of objects by using weak references. Maybe in another post I’ll provide more info on that.

Throughout my career I’ve used that nice resource of Java a few times. The most important ones that I can remember are:

Weak Listeners
Memory Sensitive Caches

Weak Listeners

Memory leaks are a painful problem that sometimes we don’t have a clue how to find them. I’ve worked in a desktop project where we had lots of instantiations of GUI panels, models, etc. That implied in a bunch of model listeners, property listeners, and so forth. We had a major memory leak due to a simple problem in most of the panels: objects from each created panel were being added as a listener of the model, but after the panel was discarded they did not remove themselves from the listeners list of the model. So, they kept being reference by the listeners list on the model and could not be garbage collected.

A naïve example tries to illustrate that:

public void setActivePanel() {
MyPanel thePanel = new MyPanel();
ApplicationModel.getInstance().addXYZListener(thePanel);
this.activePanel = thePanel;
}

public void releaseActivePanel() {
this.activePanel = null;
this.clearSelection();
}

One solution was to provide a way to unsubscribe "thePanel" from the XYZListener list:

public void releaseActivePanel() {
ApplicationModel.getInstance().addXYZListener(this.activePanel);
this.activePanel = null;
this.clearSelection();
}

But.... you could also use weak references. They would avoid such problems. At the time I found good stuff that I’ve used as a reference.

The only issue that I see in his solution is that we may have sometimes a bunch of empty WeakPropertyChangeListeners (pointing to null) when the referred object is GC’d. He provides a lazy approach for clean up. However, if you use your own ReferenceQueue you can provide a thread that once in a while polls the queue and does the sweeping of listener lists. Or also you can use a WeakHashMap which does the dirty work of clean up.

Memory sensitive caches

There are several caching strategies (e.g. LRU, MRU) but this one is very simple. It caches everything, but when the system runs out of memory the cache is cleaned up.

Depending on the memory footprint of your system this strategy may be useless and your cache will not work really well.

This memory sensitive cache relies on SoftReferences. They are special types of WeakReference which hang around for a while.

The following class is not an optimal implementation of a cache, but it works…

public class ImageCache {
private HashMap<String, SoftReference<ImageIcon>> map = new HashMap<String, SoftReference<ImageIcon>>();
private static final String IMAGE_FOLDER = "/images/";

public ImageIcon getImage(String imageFile) {
ImageIcon icon = null;
if (map.containsKey(imageFile)) {
icon = map.get(imageFile).get();
if (icon == null) {
icon = cacheImage(imageFile);
}
} else {
icon = cacheImage(imageFile);
}
return icon;
}
private ImageIcon cacheImage(String imageFile) {
ImageIcon icon = new ImageIcon(this.getClass().getResource(
imageFile));
map.put(imageFile, new SoftReference<ImageIcon>(icon));
return icon;
}
}

Another cool stuff that can be done (although I've never done it in a "real life" project as I did with the other stuff) is to provide an alternative way to finalization using PhantomReferences. You can use them combined with a ReferenceQueue to provide your own finalization mechanism avoiding the error-prone finalize method. You could write your own finalizer thread :)

There is a good reference here.

I guess that it for the moment.

Scriptconsole4j: Embed a scripting console in your Java App.

2008-04-23T07:21:00.000-07:00

Using scripting languages in Java became very easy. The JSR 223 standardizes the usage of scripting languages in the Java Platform; and Java 6 already ships with built-in support to JavaScript (Mozilla Rhino engine).

If you need to embed a multi-language enabled scripting “visual console” to your application you may try the scriptconsole4j.

I was developing a Java application where I needed to do some coding on the fly during runtime, change variable values, run snippets of code, etc. So, I’ve created a simple scripting console to use on that application. I don’t know if something like that already existed, but my code and its usage are really simple.

I’ve extracted the classes and made a few changes on it and created a separate reusable jar library (the scriptconsole4j). It is hosted at http://code.google.com/p/scriptconsole4j/

There are two ways to use it:

A JPanel with scripting functionality that can be embedded in another application (e.g. you can add a scripting window, or a scripting console to your app and access some of your application variables through the scripting window)
A standalone frame that embeds the above scripting console (e.g. you can practice the scripting language of your choice with it)

This scripting console provides a standard output variable, but allows variables from your own application to be available through the scripting console.
The combobox shows all available scripting engines on the running JVM. By default, Java 6 comes with JavaScript.

By doing the following as in the example, it is possible to add you custom(s) variable(s):

import scriptconsole4j.*;
...

JFrame frame = new JFrame("[Scriptconsole4j] Default Window");
//No textual description available. The console will just display the classname as info
ScriptContextVariable myObjectVar = new ScriptContextVariable(new MyObject(),"myobj","");
//The scripting panel can take several ScriptContextVariable as parameter
ScriptingPanel panel = new ScriptingPanel(myObjectVar);
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.setSize(400, 400);
frame.add(panel);
frame.setVisible(true);

You can add any variable that you want and use it from the scripting panel. In the above code we add an object of type MyObject that will be available in the console with the alias of "myobj". There is no big deal here, since the JSR 223 allows context objects. However, we wrap it and put some additional info so other can see what are the variables that we intentionally want to let available to the console, and hopefully some (optional) textual description of it.

Other JSR 223 compliant engines of existing scripting languages are available at http://scripting.java.net .The download on that site provides the JSR 223 engines but it is still necessary to make additional downloads to the scripting libraries themselves.

Several improvements still need to be done in the console (e.g. ability to load scripts, script embelisher, undo support), but this little panel brought a lot of flexibility for me during development time of my apps.

The first release can be downloaded here.

I hope you can give it a try to see if it helps you too.

"Hacking" JHat API to integrate it to my application

2008-04-02T16:05:00.001-07:00

Those that already used jhat may say: "JHat API? I thought it was just a java tool that comes with JDK 6."
I'll explain.

JHat (Java Heap Analysis Tool) is an experimental tool that comes with JDK 6. It can parse a heap dump file (HPROF format) and provide a simple http server (port 7000 as default) where we can execute queries in a simple SQL like language that has some predefined functions.

No big deal so far. All that is documented. A while ago I needed to integrate a memory inspection tool in a Java application that I was developing. I know there are few tools to do that externally but I wanted do memory inspection from a menu option in my application.

Although jhat is available as a command line tool, actually its classes are in the tools.jar file. So, I thought I could try do integrate that with my app. I tried to google some javadocs on jhat. Couldn’t find it. The tool is experimental, I forgot that.

Thanks to decompilers I could see how to start up the query engine. I extracted the code and could adapt it to my needs in my application.

I’ve tried to write the examples in a more didactic method in order to be more understandable. I hope I’ve put it in an easy way.

In order to run the example you would need an HPROF file. There are some ways to obtain a heap dump. The easiest (I think) is to run jmap and get the file dump.

Memory inspection done this way is not real time, since we are analyzing a snapshot of memory. But that helped me a lot. Remember that this is a heavy process that may consume lots of memory.

Run jps to find out the PID of the Java application to take the memory snapshot
Now run: jmap –dump:fomat=b,file=myheapdump.bin
Now: java JHatExample myheapdump.bin

In case of out of memory errors when running the example use the –Xmx option:

java –Xmx512m JHatQueryExample myheapdump.bin

The example consists in three classes:

JHatExample: The GUI code
JHatWrapper: A class that wraps the access to the jhat API
MyQueryVisitor: Implementation of the jhat’s ObjectVisitor interface that provides a call back method on the results of the query.

Running the code will show the following GUI:

With the same query from the picture, if you click execute it will show all referrers to all instances of java.io.File.
Actually the way I print the result of the query is quite confusing...
If anybody ever happen to try to run the code, there is a good link on constructing queries in Sundarajan's blog with nice examples. The code I provided makes some calls to the javascript library that is provided with JHat.
If you uncompress the tools.jar file you may find that library and the help page in:
tools.jar\com\sun\tools\hat\resources\

The purpose of this code is to show the possibility to integrate jhat usage into your application. Normally it is used as a stand alone tool. The stand alone tool is ready to use, but is a side tool that you have to open. What I did in my app (not the one shown here) was to automate the memory dump via a Runtime.exec call to jmap, and then the resulting file was used as a parameter to instatiate the jhat engine.

Since jhat is an experimental tool, it may completely change or become unavailable on Java 7. Use it like I did at your own risk.

Have fun playing with the code!

The GUI Code:


import java.awt.*;
import java.io.File;
import javax.swing.*;

public class JHatExample extends JFrame {
private JHatWrapper jhat;

public JHatExample(JHatWrapper jhat) {
 super("JHat wrapper");
 this.jhat = jhat;
 initComponents();
}


private void initComponents() {
 final JTextArea queryText = new JTextArea();
 JPanel buttonPanel = new JPanel();
 JButton executeBtn = new JButton("Execute");
 buttonPanel.add(executeBtn);
 this.add(new JScrollPane(queryText),BorderLayout.CENTER);
 this.add(buttonPanel,BorderLayout.SOUTH);
 executeBtn.addActionListener(new ActionListener() {
  public void actionPerformed(ActionEvent e) {
   try {
    jhat.query(queryText.getText());
   } catch (Exception e1) {
    JOptionPane.showMessageDialog(JHatExample.this, "Error executing query");
   }
  }
 });
}

public static void main(String[] args) throws Exception {
 if (args.length == 0 || ! new File(args[0]).exists()) {
  System.out.println("The program must take a valid file path as parameter");
  return;
 }
 JHatWrapper jhat = new JHatWrapper(args[0]);
 JHatExample frame = new JHatExample(jhat);
 frame.setSize(400,200);
 frame.setVisible(true);
 frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
}
}

Now the wrapper class:

import java.io.*;
import com.sun.tools.hat.internal.model.Snapshot;
import com.sun.tools.hat.internal.oql.OQLEngine;
import com.sun.tools.hat.internal.parser.HprofReader;
import com.sun.tools.hat.internal.parser.PositionDataInputStream;

public class JHatWrapper {
private OQLEngine engine;

public JHatWrapper(String s) throws Exception {
 PositionDataInputStream positiondatainputstream =
 new PositionDataInputStream(new BufferedInputStream(new FileInputStream(s)));
  int l;
  Snapshot snapshot;
  l = positiondatainputstream.readInt();
   //I "inferred" the meaning of the parameters below by comparing the decompiled code
//and jhat's command line parameters
  int debuglevel = 0;
  //Don't know what this is...
  int j =1;
  HprofReader hprofreader = new HprofReader(s, positiondatainputstream,
j, true, debuglevel);
  snapshot = hprofreader.read();
  positiondatainputstream.close();
  //resolve references
  snapshot.resolve(true);
  engine = new OQLEngine(snapshot);
}

public void query(String query) throws Exception {
  engine.executeQuery(query, new MyQueryVisitor(engine) );
}
}

And the Visitor code:


import java.util.Enumeration;

import com.sun.tools.hat.internal.model.JavaHeapObject;
import com.sun.tools.hat.internal.oql.OQLEngine;
import com.sun.tools.hat.internal.oql.ObjectVisitor;

public class MyQueryVisitor implements ObjectVisitor {
private OQLEngine engine;

public MyQueryVisitor(OQLEngine engine) {
 this.engine = engine;
}

public boolean visit(Object obj) {
 try {
  Object iterator = engine.call("wrapIterator",new Object[]{obj});
  //Handles result items that are instance of enumeration. Ex: select referrers(x) from java.io.File x
  if (iterator instanceof Enumeration) {
   System.out.println("enum");
   handleEnumeration((Enumeration)iterator);
  } else {
   //handles single result items. Ex: select x from java.io.File x
   JavaHeapObject javaObj = (JavaHeapObject)engine.call("unwrapJavaObject",new Object[]{obj});
   handleEnumeration(javaObj.getReferers());
  }
 } catch (Exception e) {
  e.printStackTrace();
 }
 return false;
}
void handleEnumeration(Enumeration en) {
 while (en.hasMoreElements()) {
  try {
   Object obj = en.nextElement();
   JavaHeapObject unwrapped = (JavaHeapObject)engine.call("unwrapJavaObject",new Object[]{obj});
   System.out.println(unwrapped);
  } catch (Exception e) {
   e.printStackTrace();
  }
 }
}
}

Yet another Java blog

2008-04-02T15:50:00.000-07:00

Ok, one grain of sand in the desert of blogs that talk about Java...
My intention is to leave some (useful) contribution. I frequently google for things to help me when I develop.
Sometimes I go to forums and mailing lists to make questions but usually I never contribute back helping other people. What a leech!
I hope my posts can help somebody.