VisualVM and JFreeChart

I work in an European project in which we are developing a set of VisualVM plugins to construct a management application. It's the same principle I've used on the VisualVM OSGi plugin: a set of MBeans in the application, and a nice GUI in the VisualVM side that uses JMX to access the MBeans.

We used JFreeChart to visualize (in the VisualVM) data captured by sensors (in the managed application).

I had some minor problems for using JFreeChart due to class loading that I detail next.

Importing JFreeChart the wrong way

If you create a regular Netbeans Module project you won't be able to import libraries. So you need to create a Module Suite (File > New Project > Netbeans Module > Module Suite), and then add the Module project of you plugin to that Module Suite (see picture below).

Then, you can go right click the project, select Properties > Libraries and get the below pop up window to add your library.

What I initially did was importing in two separate steps. First the JFreeChart library, and then JCommons which is necessary for JFreeChart. After executing my code I got a java.lang.ClassNotFoundException: org.jfree.date.MonthConstants. Weird. Everything was imported. Well, aparently we had some classloading visibility issues here.

As previously described, I had configured the following dependencies:

MyModule -> JFreeChart

MyModule -> JCommons

Doing it the right way (well, at least in a way that works)

Adding the libraries two steps gives me two Library Wrapper Modules, which will belong to the same Module Suite. But when I added both libraries at once (resulting in just one Library Wrapper Module) and tried executing the JFreeChart code, my chart appeared and I had no longer a ClassNotFoundException. Thus, the depencency became:

MyModule -> {JFreeChart, JCommons}

I believe the above issues are the same for any other Netbeans RCP application when trying to use JFreeChart.

Just to have a sample of the nice charts that you can add, here goes a screenshot with some example charts. Library download and more info at the JFreeChart website

VisualVM OSGi Plugin available at the plugins center

The VisualVM OSGi plugin has become part of the OW2 Chameleon project, which tries to deliver a set of pluggable components for OSGi frameworks, allowing to create sorts of lightweight OSGi-based application servers with just the needed functionality. The VisualVM OSGi plugin was the initial step for a set of VisualVM plugins that will be provided in the Chameleon project.

Recently Jiri Sedlacek and Tomas Hurka have proposed to add a link to the OSGi plugin in the VisualVM 1.2 plugins center. Now you can install the plugin from the VisualVM 1.2 or above, in the menu Tools > Plugins. Don'f forget to download and install the appropriate OSGi bundle in your OSGi application (this is how the plugin will get the info about the OSGi application).
Detailed installation instructions can be found on the plugin page in Chameleon's wiki

Many thanks to my thesis advisor, Didier Donsez, for the idea of the plugin.
Thanks to Clément Escoffier, for inviting the VisualVM OSGi Plugin to the Chameleon project.
Thanks to Jiri and Tomas for all the tips, testing and for adding the plugin to the VisualVM plugins center, and thanks in advance for anyone who would eventually use the tool and hopefully provide any comments for improving it.

Meeting of the OSGi users group France

The 9th meeting of the OSGi users group France took place today, at the Grenoble University Campus. People from different places in France have attended. The central theme of the talks was about isolation, mostly concentrated in recent research efforts. In the morning we had three talks:

• Laurent Daynès, researcher from Sun Microsystems at Grenoble, gave the talk “Lightweight Isolation with a multi-tasking JVM” where he talked about motivations for multitasking, lightweight isolation in the JVM (obviously! It is the presentation's title), Java Isolates, the Multitasking Virtual Machine (MVM) and a lot of interesting stuff concerning isolation.
• Nicolas Geoffray, a fresh PhD from Université Paris 6, presented the "I-JVM: a Java Virtual Machine for Component Isolation in OSGi", which he developed during his thesis under the co-direction of Gaël Thomas who was also there. He shows how he implemented lightweight Java Isolates in the VM level, and how he associated them with OSGi bundles allowing the safe termination and fine grained resource control of bundles.
• Kiev Gama (sounds familiar to me...) presented the initial efforts of my PhD’s first year, under the direction of Didier Donsez, in the talk entitled "Towards Dynamic Component Isolation in a Service Oriented Platform". In our work we provide a sort of fault contained sandbox for OSGi components by means of Java Isolates.

In the afternoon, four more talks:

• My ex-labmate on the Adèle team Clément Escoffier, from Akquinet and leader of the iPOJO project, presented directly from Berlin via DimDim the talk "Chapter 4.2, Verse 138 : The Framework Multiplication". The RFC 138 is an ongoing specification at the OSGi alliance for allowing different OSGi framework instances to be hosted as nested frameworks isolated from each other. Although it addresses more issues, the basis for this specification seems to be quite similar to what Stéphane Frénot and Yvan Royon did a few years ago with V-OSGi (V for Virtual) by virtualizing different OSGi platform instances on one framework.
• Stéphane Frénot, professor from INSA Lyon, presented the ROCS (Remote OSGi Caching Service) which targets resource constrained devices allowing OSGi applications to load classes from a remote cache, saving space on the device's persistent memory.
• Next, Pierre Parrend, researcher from FZI Karlsruhe, in another live broadcasted session via DimDim talked about "Software Security Models for Service-Oriented Programming", where he presents different security threats that may be presented by OSGi bundles. He showed also how we a hardened OSGi can be implemented.
• Another ex-member of my team, Mikaël Désertot, professor from Université de Valenciennes, presented "Vehicular network: The VESPA use case" where OSGi is used in a vehicular project.

Well, I must admit that during this last presentation I could not pay too much attention because I was showing to Nicolas Le Sommer and Laurent Daynès a snippet for crashing a JVM. During my presentation in the morning I had mentioned that and told everyone to try it home, but Nicolas had the initiative to try it already in the afternoon :)

Here is the snippet that crashes the JVM, taken from here:

public class Crash {
public static void main(String[] args) {
  Object[] o = null;
  while (true) {
     o = new Object[] {o};
  }
}
}

Nicolas tried the above code which crashed the Sun JVM. However the GNU Classpath and IBM JVMs did not crash…

Laurent tested it in his machine, also crashing the Sun JVM. He debugged it quickly and could precise that it happened when a full GC took place, but someone needs to fix it :)


The presentation abstracts and slides can be found here.

VisualVM OSGi plugin

I wrote a simple OSGi management console as a VisualVM plugin, based on a concept similar to the OSGi JConsole plugin that my advisor asked me to do last year.

In this post I will describe the main steps for constructing the OSGi bundle and the VisualVM plugin, including the usage of a table from the JFluid API (so you plugin can look pretty much like the standard VisualVM stuff) and how simple it is to use JMX from your VisualVM plugin.

If you want to skip the development steps, you can install and try the plugin by downloading the built sfuff here.

If you want to check the step-by-step, please continue reading.

There is not too much advanced stuff here, but there are three pre-requisites for a complete understanding of this post:
- A minimum of familiarity with VisualVM plugin development (You may want to try the Hello VisualVM plugin before)
- Basic JMX knowledge (JMX Tutorial here)
- Basic OSGi knowledge (This main purpose of the plugin is not very useful to you if you don’t use OSGi, but you still can check out the JMX + VisualVM stuff).

I’ll show in this post:
- Brief description of the plugin functionality
- How I used Netbeans to create an OSGi bundle exposing some management functionality via JMX
- An overview of how I developed VisualVM plugin to access the JMX stuff above for accessing/managing OSGi applications
- Some of my findings that I’ve think useful to describe here

What I’ve used to develop, build and run the code show here:

• Java 1.6
• Netbeans 6.5
  
• Apache Felix (you can use another OSGi distribution like Equinox, Knopflerfish)
• VisualVM 1.1.1
  

1. Plugin Functionality Overview
This plugin can be an alternative to the OSGi consoles that exist, or yet another toy for your VisualVM plugin set.
The functionality of the plugin (screenshot below) is simple:

• Listing of all bundles in an OSGi application
• Installation of bundles
• Perform a start/stop/update/uninstall in a bundle from a list
• Visualize bundle information (Manifest headers, list of services, navigate through bundle contents).

2. Accessing OSGi via JMX
In order to have a plugin in another process (the VisualVM) to access the OSGi platform, we needed to expose some interfaces for that. JMX is an easy way to expose management interfaces for Java applications, and there is built-in support in the VisualVM for using it.

In this section I describe the development of the OSGi bundle that exposes this funcionality.

I created a Netbeans project as a Java class library (I have called mine OSGiJMX)
- Next I needed to reference a jar with the OSGi classes.
- Right click on the project, go to project properties > libraries > add libraries, I needed to create a new one pointing to the felix.jar which contained the needed OSGi API
- OSGi bundles need special manifest attributes. So I’ve added my own manifest to src/META-INF/MANIFEST.MF with the following content:

Manifest-Version: 1.0
Bundle-Version: 1.0.0
Bundle-Name: JMX OSGi
Bundle-Description: A JMX OSGi Example
Bundle-ManifestVersion: 2
Bundle-Vendor: Kiev Gama
Bundle-SymbolicName: osgijmx
Bundle-Activator: osgijmx.impl.Activator
Import-Package: org.osgi.framework,javax.management
Export-Package: osgijmx, osgijmx.impl

- I needed to change the project ant build file so it would use my manifest instead of the default one. On the files tab of your project (usually on the left side of the Netbeans IDE), open the build.xml

<import file="nbproject/build-impl.xml"/>
<!--ADD LINE BELOW -->
<property name="manifest.file" value="src/META-INF/MANIFEST.MF"/>

Basic operations should be made available via JMX MBeans, so our VisualVM plugin can access information concerning an executing OSGi platform and its bundles.
The MBean that my advisor provided me for the JConsole plugin previously mentioned could be used, but it was a single MBean instance that mixed bundles and platform methods. At that time a friend from my lab suggested to create an MBean for each bundle instance. That’s what I finally did now. So, I’ve defined two MXBean interfaces:

• One for the OSGi Platform methods (kinda like a proxy to BundleContext which gives access to the OSGi platform)
• Another for bundle related methods (worked like a proxy to a Bundle instance).

The code for the interfaces:
BundleMXBean and OSGiPlatformMXBean
The code of their implementation:
BundleMXBeanImpl and OSGiPlatformMXBeanImpl
the bundle Activator

The OSGiPlatformMXBean will hold a list to all Bundle MXBean instances (thanks to Eammon McManus’ blog).
OSGi is a dynamic platform: bundles can change their state, new services may arrive, etc. So, this JMX bundle should listen to Bundle events in order to add or remove the bundle MBeans in case of bundle installation or uninstallation, respectively, as well as notify (via JMX) any changes of bundle state so the VisualVM plugin GUI could be updated and always have consistent information. Currently the code does not have JMX notification of service events (a temporary workaround would be select/deselect the bundle from the list so its services list would be updated).

If you’ve followed all the steps you can build the project and test the bundle (or maybe download it from here).
Run Felix, and type start .
If you start JConsole and connect it to Felix, you should be able to visualize all tof our OSGi MBeans in the MBeans tab.

3. OSGi VisualVM Plugin Development
The VisualVM plugin will depend on the JMX interfaces of the project described on the previous section, so it would need to be built and deployed with the plugin as well. My choice was to create a Module Suite, for aggregating the library and the VVM plugin.

So, you'll end up with four Netbeans projects:

• The OSGi bundle from previous step
• The Module Suite (just an umbrella project for the two below)
  
• A library wrapper for the OSGi bundle
• The VisualVM plugin
  

3.1 Steps for creating the Module Suite:
-New Project > Netbeans Modules > Module Suite


- Next, choose the target Netbeans platform (in our case is VisualVM 1.1.1)

3.2 Creating the Library Wrappper module

- Choose a Netbeans Project > Library Wrapper Module
- Select the libraries (the jar files) to be wrapped. In this case I've wrapped the OSGiJMX.jar and the felix.jar in the same module. - In the next step you must indicate which Module Suite will contain your wrapped library (as indicated in the figure below). Then you can go to the next and final step where you define your codebase.


3.3 Creating the VisualVM Plugin

- Create a new project > Netbeans modules > Module
- Important: Choose the module suite create before, so your plugin module will be able to resolve the classes of the wrapped library under the same Module Suite (but we still need to configure that)


After this step was finished, I've created a module installer:
- Right click on the plugin project, New > Module Installer
- I've named it installer and added the code that follows

public class Installer extends ModuleInstall {
@Override
public void restored() {
BundlesViewProvider.initialize();
}
}

Besides the Installer and the BundlesViewProvides, the other classes for the VVM plugin were:

• BundlesView
• BundlesTableModel

Dependencies
After this, we would need to add the module dependencies concerning the imported types. Right click on the project > Properties > Libraries. Then you select the dependencies (check the "Show non-API modules" on the pop-up dependency window) listed in the figure below:


At this point, trying to run/build the plugin gives the following error:

"The module osgijmx.vvm is not a friend of \profiler3\modules\org-netbeans-lib-profiler-ui.jar"

We can solve this by editing the JFluid module dependency and setting it as an "Implementation Version"


Ok, but why depend on the JFluid-UI classes?
Well, these are the classes used for developing the pretty Visual VM built-in plugins. So, in order to get our plugin to look similar to them we used the JFluid-UI. The result can be seen in the table on the figure below where we inspect Fuji.

Here's the code for creating the table (JExtendedTable from JFluid UI):

        bundleTable = new JExtendedTable(tableModel);
bundleTable.setBorder(BorderFactory.createEmptyBorder());

bundleTable.setRowSelectionAllowed(true);
bundleTable.setSelectionMode(ListSelectionModel.SINGLE_SELECTION);
bundleTable.setGridColor(UIConstants.TABLE_VERTICAL_GRID_COLOR);
bundleTable.setSelectionBackground(UIConstants.TABLE_SELECTION_BACKGROUND_COLOR);
bundleTable.setSelectionForeground(UIConstants.TABLE_SELECTION_FOREGROUND_COLOR);
bundleTable.setShowHorizontalLines(UIConstants.SHOW_TABLE_HORIZONTAL_GRID);
bundleTable.setShowVerticalLines(UIConstants.SHOW_TABLE_VERTICAL_GRID);
bundleTable.setRowMargin(UIConstants.TABLE_ROW_MARGIN);

Using JMX from a VisualVM plugin

The usage of JMX was straightforward, without any problems. From our DataSourceView the Application instance can be used to retrieve the MBeanServerConnection:

public BundlesView(Application application) {
super(application, "OSGi", new ImageIcon(Utilities.loadImage(IMAGE_PATH, true)).getImage(), 60, false);
mbs = JmxModelFactory.getJmxModelFor(application).getMBeanServerConnection();
...

After that, the MBeanServerConnection can be used to retrieve a proxy to the MXBean

ObjectName objName = new ObjectName("osgijmx:type=framework");
OSGiPlatformMXBean osgiProxy = JMX.newMXBeanProxy(mbs, objName, OSGiPlatformMXBean.class);

The invocations of the methods were ok, but I had some problems with a MXBean method that returned a Map[] :
java.lang.ClassCastException: sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl cannot be cast to java.lang.Class

But I still had errors changing the return type to a Map of Map.
Surely issues with JMX’s Open type translation. The above methods did not work called from my proxy (any help from JMX experts would be great…), but the Map getBundleHeaders() did work…

I tried using an MBeanServerConnection.getAttribute and it did work. However it obliged me to navigate through TabularData instead of a java.util.Map, just the way JConsole does.

Building and deploying
Inside the Netbeans we can run the plugin, but if we want to deploy it in the VVM we need to build the nbm files. Just right click the Module suite project, and hit Create NBMs.
Going to the updates folder of the Module Suite project we can find the two generated files.
For their deployment on the VVM, the steps are the same from here.

I've tested it just in Felix-based applications like Glassfish and Fuji, and it worked fine.

For deploying in Glassfish the OSGi bundle with the JMX probes:

GLASSFISH_HOME\bin\asadmin deploy --type osgi YOUR_PATH\OSGiJMX.jar

This is it. Maybe a long post, but this is the end of it...

Suggestions for new features are welcome, as well as source code for doing so.

References:
Getting Started Extending the Visual VM
JMX Tutorial
OSGi Javadoc
Inter-MXBean references

A scripting bundle for OSGi

I've wrapped the scriptconsole4j as an OSGi bundle (tested in Apache Felix 1.4.0 and Equinox 3.4.0).

This console is a simple GUI that allows using the JSR-223 compatible scripting languages available in you Java 6 VM. The Rhino Engine (Javascript) is available by default.

If you want to execute scripts accessing the OSGi framework to retrieve services in adhoc scripts, you may find this version of the scriptconsole4j tool useful. When the bundle is started a scripting window is automatically displayed. Upon bundle termination, the window is disposed and vice-versa.

How to use it:

• Just download it.
  
• Install it on your favorite OSGi framework
• Start the bundle
• And its ready to be used

You can, for example, register listeners to framework objects:

  var o = new Object();
  o.bundleChanged = function(e) {
      message = "An event happened to bundle "
                + e.getBundle().getBundleId();
      output.println(message);
      //Writing in the standard output
      java.lang.System.out.println(message);
  };

  ctx.addBundleListener(new org.osgi.framework.BundleListener(o));

Since the bundle uses DynamicImport-Package, you can use it with any exported class, so you can dynamically implement and register services:

var o = new Object();
o.sayHello = function() { java.lang.System.out.println("closed")};
  
r = new com.foo.HelloServerService(o);
//registering the service in the OSGi Service Registry
ctx.registerService("foo.BarService",r,null);
//retrieving the service instance
service = ctx.getService(ctx.getServiceReference("foo.BarService"));
service.myMethod();

Take a look at the Rhino page showing how to use Java from Javascript, and use your imagination for the scripts that may help you.

Recently I needed the scripting functionality in OSGi on a Java 5 Virtual Machine, which does not come with the JSR-223. I've removed the JSR-223 dependant stuff from the scriptconsole4j and embedded the beanshell core with it. It worked fine but without dynamic class generation. I guess by adding the bsh-classgen it "would" work. Since we are running on OSGi, classloading is always an issue when dynamically generating stuff...
I'll test it and leave it available soon (I hope so).

Withdrawn JSRs

I've checked the status of JSR-277, which was announced as being halted by Sun, but its status did not change to withdrawn. That makes sense, since it was announced as temporarily halted.
So I went to check the list of all withdrawn JSRs, just for curiosity, and the number surprised me: 52. A lot of those were concerning JAIN (which I haven't heard of until now). Some of the JSRs had comments like "withdrawn at the request of the submitter".

If you look at the number of all JSRs (355) we can do simple math and find that 15% of all JSRs have been withdrawn. It is true that they vary a lot in size, as there are umbrella JSRs that have a lot of stuff behind and we have some successful efforts that are heavyweight. Also, among that list there are "spinoffs", like the JSR-8 which originated OSGi. But, is 15% a high rate or considering the fact that standardization is a complex effort that involves people from different organizations which may have different interests, could we say that it is a fairly acceptable percentage ?

Can AOP be some sort of elegant workaround?

In the pursuit of problem solving (or at least trying to) in my research, I guess I am taking AOP as a strategy to give me what I don't have. Maybe I am becoming too addicted to pointcuts...

I have used Aspect Oriented Programming for intercepting method calls to the OSGi framework and for adding my "concerns". These interceptions allowed me to, mainly, add hooks to service registration and retrieval. I think the good thing is that the solution was elegant enough and enabled portability accross different OSGi implementations, since the aspects targeted the API. The bad thing is that it does not rely in a standard mechanism supported by the platform, and it needs an additional building step (i.e. weaving the framework).

After taking a look at the draft OSGi spec 4.2, I've checked the Service Registry Hooks with the hope that my weaved hooks could be replaced by a standard mechanism. They have exactly proposed the hooks I wanted but sadly that alternative solution had been rejected. The hooks they will provide still do not allow me to do what I need. I'll still be doing the AOP trick...

I just can think of more pointcuts to add to my future solutions/attempts in order to have other types of hooks. As I said previously, this approach is useful to avoid changing the source code of a chosen OSGi implementation. There are lots of people in the research area who usually customize an OSGi impl to their needs (I still see recent articles talking about customizations of OSCAR). One disadvantage is that if you touch the source code directly, you're stuck with a given version which makes it difficult to follow the evolution of the chosen impl. (either Equinox, KF or Felix). You would need constant effort for merging/synchronization your branch with the current version. I believe they could avoid that by doing the AOP trick as well. However, sometimes (like while writing this) I am not sure if this a true elegant solution or if I've just found a nice and powerful workaround for adding my hooks to the OSGi core.