Monday, August 3, 2009

An iPOJO 1.4.0 overview



Finally! The last week, the iPOJO team released the iPOJO 1.4.0 version of the framework. This new version fixed several bugs but also provides new features. This post gives on overview of the novelties provided in this opus.


The API to describe iPOJO component type in Java


iPOJO supports XML & annotation based descriptions. The iPOJO API provides a way to create /declare new component types & instances at runtime. This is a very convenient way against annotations and XML allergies! This API was designed to be easy to use, avoids redundancies… Here a small example creating a service provider and a service consumer:


public void create(BundleContext bc) throws Exception {
new PrimitiveComponentType()
.setBundleContext(bc)
.setClassName(HelloServiceProvider.class.getName())
.addService(new Service()
.addProperty(
new ServiceProperty().setName("language")
.setType(String.class.getName())
.setValue("en")
))
.createInstance();


new PrimitiveComponentType()
.setBundleContext(bc)
.setClassName(HelloServiceConsumer.class.getName())
.addDependency(new Dependency().setField("hello"))
.setValidateMethod("start")
.createInstance();
}


The POJO classes are manipulated at runtime if required.
An interesting property of the API is the iPOJO composite support. So you can easily describe applications natively dynamic. To illustrate that, let’s imagine a pretty simple application compared by 2 instances from 2 types (potentially described with the API). This application is described and instantiated as follow:


new CompositeComponentType()
.setBundleContext(bc)
.addInstance(new Instance("MyProviderType"))
.addInstance(new Instance("MyConsumerType"))
.createInstance();


Last but not least, the API is bound with the introspection and reconfiguration mechanism of iPOJO. For reconfiguration adepts (Autonomic, context-aware applications), it simplifies the reconfiguration process. Thanks to this feature, it is pretty easy to declare a component type, create an instance of this type, and then, later, reconfigure its service dependency:


Dependency dependency = new Dependency().setField("hello");
ComponentInstance instance = new PrimitiveComponentType()
.setBundleContext(bc)
.setClassName(HelloServiceConsumer.class.getName())
.addDependency(dependency)
.setValidateMethod("start")
.createInstance();

dependency.getDependencyDescription(instance).setFilter(bc.createFilter("(language=en)"));


Interesting, isn’t it? And that’s just an overview!

Online Manipulator


The second great novelty is the online manipulator. Let’s sum up the story. A lot of users were complaining about the additional packaging step to create iPOJO bundle. This step prepares the bundle to be managed by iPOJO. So despite tools such as Ant and Maven doing this preparation seamlessly (once configured), it’s trickier with tools as Eclipse. The online manipulator just removes this additional step. In fact, it does this process during the deployment of the bundle. To use, the online manipulator, just prefix your bundle URL with ipojo://. That’s it. It looks for the metadata.xml file inside the bundle and executes manipulation. You can also specify or override the metadata in the URL:


install ipojo:file:/tmp/mybundle.jar!file:/tmp/mymetadata.xml


Thanks to the online manipulator, you can use any tools to develop your bundle such as the Eclipse PDE. Just awesome!

Web Console Plugin


Finally, the last novelty is really interesting for those development complex applications with iPOJO. The Web Console Plugin is an Apache Felix Web Console extension giving data about your iPOJO applications.
It’s a Web based alternative to the «arch» command. The plugin displays the created instances, available factories and handlers. For all those entities, details are given such as the current state, provided services (and properties), used services… The model is navigable! So, for a service dependency, you can directly navigate to the used iPOJO instance, from an instance, you can switch to the factory (i.e. component type) and see all the instances from this type. For «arch» command user, the « raw » architecture is also given. This plugin definitively helps debugging iPOJO applications by giving everything required to understand the reasons to unsatisfied service dependencies, invalid instances or factories.






Conclusion


That’s all for the biggest novelties of iPOJO 1.4.0. It’s not only improvements. See the release notes for more details.
Moreover, the road map of the next version is also promising: proxy support, constructor injection, transaction support and persistence. And that’s just a couple of example. So one advice, stay tuned!

Friday, June 26, 2009

Android talk at Linuxtag

Yesterday I gave with Karl Pauls a talk about Android at Linuxtag.
It was great: 70 to 80 people, a lot of questions ...
I just updated the slide, so, if you're interested by Android and maybe OSGi, just have a look.

Tuesday, June 23, 2009

OSGi devcon Talks

Just back from OSGi Dev Con Europe. The conference was great and the talk quality was very good. Here are my presentations:








You can look at the others presentation on OSGi devcon Europe 2009 group

Sunday, May 3, 2009

The Junit4OSGi - Pax:Exam mix

Half of both, shake, enjoy …

In the OSGi world, there is different ways to test “bundles” / “systems” / “applications”.

Spring Integration Testing fit very well with Spring applications, but in all the others cases it doesn’t work very well.

BND Testing Harness is a testing framework integrated with BND. Despite very powerful to generate bundle on the fly, it sounds not very adequate when test must be integrated inside a build process expect Ant and Eclipse (such as Maven).

Junit4osgi is a very simple test framework, running junit 3 tests on OSGi. The goal was to provide a simple way to write tests checking services … junit4osgi also provides high-level method simplifying the OSGi test writing.

Finally, pax:exam is a test framework for OSGi allowing to finely configure the test framework.

Recently, I discuss with the pax:exam developers about providing THE test framework for OSGi. In fact, I was considering moving the iPOJO test suites and the application test suites that I developed to pax:exam. After trying pax:exam, I was finally able to understand what pax:exam provides and what would be great…

Requirements
First what are the requirements of a cool test framework for OSGi.

To begin, it has to reuse existing “test engine” such as junit or testNG. Java developers have already developed tests with those frameworks (I hope so…), and so this should ease the OSGi test writing.

Then, as you know, testing is never a very exiting task. So, the test framework should provide high-level methods to ease the OSGi integration. Methods such as “getService”, “waitForService” … should be provided and so the developer can focus on testing its service and not how to get the service.

Third important point: the configuration. The application/service have to be deployed and run is a “real” OSGi platform. This platform has to be configured. This configuration contains obviously the bundles to deploy but can also contain other properties (extra packages, system configuration…). Supporting several OSGi implementation and several dependent bundle package is also a good property.

Another important aspect related to the configuration is the dispatching test / platform. This should allow configuring which tests have to be deployed and execute on the SAME frameworks. Despite this can create side effects; sometimes we are looking for those side effects.

The test framework must provide a “seamless” integration with build process and development tools such as Ant, Maven, Eclipse… Tests should be executed when the project is built, and developer should be able to execute quickly their tests from their favorite IDE.

OSGi defines Execution-Environment. The minimum environment is J2ME Foundation Profile 1.1. The test framework should provide a mode allowing executing test on this environment. Indeed, sometimes we want to check that everything works on a production-like environment or on different Java Virtual Machine.

Finally, the test framework should provide a way to execute the test manually on a “regular” platform. This way allows debugging what does not work. The developer recreates the environment or uses the production environment, deploys its tests and executes them manually.

Junit4osgi / pax:exam : The comparison

Test Engine

  • Junit4OSGi: junit3 only
  • Pax:Exam: sub-set of junit4 (no afterClass and beforeClass), possibility to support others test engines.
High-Level method
  • Junit4OSGi: Helper pattern (osgi (bundle, service, package admin), ipojo, config admin, event admin…)
  • Pax:Exam: just the bundle context
Configuration
  • Junit4OSGi: all maven dependencies that are bundles + system properties. Use Felix only
  • Pax:Exam: @Configure method allowing to finely configure that framework (provisioning, Maven helper…). Allow running test on several configuration (OSGi implementation…).
Platform / Test
  • Junit4OSGi: one framework per Maven project
  • Pax:Exam: One framework per @Test method
Build Process / IDE integration
  • Junit4OSGi: Maven integration, possibility to manually launch the tests
  • Pax:Exam: Maven, any IDE (at least Eclipse)
Execution-Environment
  • Junit4OSGi: J2ME Foundation Profile 1.1
  • Pax:Exam: Java 5
Manual launching
  • Junit4OSGi: Yes, all the tests are inside a bundle
  • Pax:Exam: No, must use the regular test runner
Diving into pax:exam
I recently tried pax:exam and developed a couple of tests with it. It is really a promising framework, and the integration of the junit4osgi features is now on the roadmap. However, I have a couple of complaints and solutions:
  • pax:exam fits very well to test OSGi applications, but to test framework we need a way to create on the fly bundles. Pax:exam does not provide natively such kind of feature. However, Tony Menzel developed tinybundle, a small and simple API to create bundle on the fly. I extended it a little bit to support iPOJO bundles. So, I’m able to create iPOJO bundles on the fly inside my tests! Definitely great.
  • The pax:exam development model relies on the bundle context. As a junit4osgi power user, I want high-level method. I just externalized junit4OSGi helpers, and so was able to reuse all the junit4osgi methods.
  • Pax:Exam is definitely slow… As it creates one framework per @Test method, it’s really inefficient when your test suite contains thousands of tests. I have 2000 tests with junit4osgi. Executing all of them takes almost 5 minutes (I have very expensive test with asynchronous event handling…). I also have a test suite developed with pax:exam of 20 tests. Such suite is executed in 3 minutes. So, there is definitely a performance / time issue. However, the pax:exam team is developing a Felix container as the junit4osgi one. This will improve performance.
  • A way to execute junit3 tests (working on Foundation Profile 1.1) and TestNG tests would be great.
  • A way to create test bundles “deployable” manually on a manually configured platform

So, in conclusion, pax:exam is a promising framework. I’m looking forward the next improvements and hope that junit4OSGi features will be provided soon.

Resources
Junit4osgi: http://felix.apache.org/site/apache-felix-ipojo-junit4osgi.html
Pax:exam: http://wiki.ops4j.org/display/paxexam/Pax+Exam
BND Testing Harness: http://felix.apache.org/site/bnd-testing-harness.html
Spring Integration Testing: http://springosgi.googlepages.com/ch04.html
Junit 3: http://junit.sourceforge.net/junit3.8.1/index.html
Junit 4: http://www.junit.org
TestNG: http://testng.org
Helper class to ease test writing: http://svn.apache.org/viewvc/felix/trunk/ipojo/tests/api/src/test/java/org/apache/felix/ipojo/tests/api/
Tinybundle: https://scm.ops4j.org/repos/ops4j/projects/pax/swissbox/pax-swissbox-tinybundles/
iPOJO extension to tinybundle: http://svn.apache.org/viewvc/felix/sandbox/clement/BundleAsiPOJO/

Tuesday, April 14, 2009

Customized service object creation with iPOJO

The last week, I attended to the OSGi in Action talk organized by the Java User Group Berlin – Brandenburg. Karl did a very nice talk, but I was definitely more interested by the questions ☺
One of the questions was about the service object creation policy. The person asking for this, was a little worried by the default OSGi policy: publishing the service object itself in the service registry. I can understand his issue: why creating something not necessary needed (lazy creation), and how can I deal with multiple service objects.

So, the answer is definitely the Service Factory. When publishing a service factory instead of the service object, the framework notifies you each time a NEW bundle asks for a service object. So, you can create the service object at the first call or create one service object each time a new bundle asks for the service. In fact, the OSGi framework caches the returned object in order to improve performance.

However, it’s weird to add this “bundle” concept there. How I can create one service object per call or per user, or per thread? Those creation policies make sense in a lot of context. The traditional OSGi answer relies on the “Facade” design pattern. You registers an object that delegate the invocation on the “good” object. You can create the “good” object regarding to your creation policy.

However, I was never really happy to implement this facade pattern in the code. Why? Just because it’s typically something that:
  1. Pollute your code
  2. Can be externalized
Inside iPOJO, I tried to remove this limitation by adding a third creation policy. It is possible to provide a service and to create one service object per asking instance. So, two instances will use two different service object despite they are in the same bundle!

In fact, iPOJO provides a way to specify your own creation strategy. Imagine that you have a specific creation policy. Just implement the desired behavior by creating a class extending CreationStrategy. Moreover, iPOJO brings a new concept called IPojoServiceFactory closed to OSGi Service Factory but based on instance. So, if an iPOJO instance requires a service implementing this interface, the iPOJO container will call a special method to get the service object. This method will be called every time the service is get (so not only once per instance). However the creation strategy may decide to return always the same service object for one instance (per instance strategy).

So, how I can create my own fine-grained creation policy?

Just create a class extending the CreationStrategy class and implements the IPOJOServiceFactory interface.

So, let’s say that your OSGi Service Factory getService method is called. So you return a proxy/facade extending the IPOJO Service Factory interface like in

public Object getService(Bundle arg0, ServiceRegistration arg1) {
Object proxy = Proxy.newProxyInstance(getInstanceManager().getClazz().getClassLoader(),
getSpecificationsWithIPOJOServiceFactory(m_serviceSpecification, m_handler.getInstanceManager().getContext()), this);
return proxy;
}

When another iPOJO instance will asks for your service, it will call the iPOJO Service Factory getService method. Then, you intercept this method and apply your creation strategy. For example, the next snippet shows the per instance creation strategy:

public Object invoke(Object arg0, Method arg1, Object[] arg2) {
if (isGetServiceMethod(arg1)) {
return getService((ComponentInstance) arg2[0]);
}

if (isUngetServiceMethod(arg1)) {
ungetService((ComponentInstance) arg2[0], arg2[1]);
return null;
}

throw new UnsupportedOperationException("This service requires an advanced creation policy. "
+ "Before calling the service, call the getService(ComponentInstance) method to get "
+ "the service object. ");
}


public Object getService(ComponentInstance instance) {
Object obj = m_instances.get(instance);
if (obj == null) {
obj = m_handler.getInstanceManager().createPojoObject();
m_instances.put(instance, obj);
}
return obj;
}

public void ungetService(ComponentInstance instance, Object svcObject) {
Object pojo = m_instances.remove(instance);
m_handler.getInstanceManager().deletePojoObject(pojo);
}


You can easily change this code to implement your own strategy like one per thread... As you get the ComponentInstance object, you can easily look for properties… and so create very sophisticated strategies.

So, thanks to this mechanism, you can creation you own policy and refer to it from your provides metadata:

<component
classname="…FooBarProviderType">
<provides strategy="….MyCreationStrategy">
</component>


That’s it!

The main and major limitation of these strategies is that they rely on iPOJO. So both the provider and the consumer have to be developed with iPOJO to deal correctly with the new service object creation policy. So, just use iPOJO ;-)

Wednesday, April 1, 2009

iPOJO and File Install : configuring iPOJO instances with 'cfg' files

FileInstall is a great bundle to provision your system. It watches a directory and installs every bundles contained inside the directory. Moreover, it’s fully support the configuration admin. So, it reads .cfg files contained in the folder and pushes the read configuration into the configuration admin.

iPOJO also supports the configuration admin. iPOJO instances can be created, deleted and reconfigured from the configuration admin.

So, why not creating, reconfiguring and deleting iPOJO instances from the cfg files.

File Install philosophy


File install is great and simple. It just has one general idea to understand:

The content of the watched folder is your system configuration

So, if you create cfg files inside this folder, those files will be analyzed and the configuration pushed in the configuration admin. To remove the configuration, just remove the file from the folder!

Note: Don’t forget to install the configuration admin

Using File install to create instances


When you declare an iPOJO component, this will expose a ManagedServiceFactory service. [Except if you voluntary set the component type to private]. This service can be used to create / reconfigure/ delete instances of this component type.

So, let’s say that we have the following bar.Foo class:

@Component
public class Foo {
@Property
private String name;


}

This declare an iPOJO component type (named bar.Foo (i.e. the classname) as no name is specified). This component type also declares a property (name).

So, once deployed, iPOJO will exposed a ManagedServiceFactory service with the ‘bar.Foo’ service pid. The configuration admin will push all matching configuration to this ManagedServiceFactory.

Now, let’s say that File install watches the ‘load’ folder. If you create a file named bar.Foo-1.cfg containing the following content:
name = the name
Once saved, File Install will find it, analyzes it and pushes the configuration to the configuration admin. The configuration admin will pushes this configuration to our ManagedServiceFactory, and so we just create an instance of our type!

So, cfg files completely replaces the tag of iPOJO metadata files. You no more need them (if you use annotations).

Let’s create a second file named bar.Foo-2.cfg containing the following content:
name = the second name
This will create another instance.

Reconfiguring an instance


To reconfigure an instance, just edit the cfg file. For example, edit the bar.Foo-2.cfg and change the name property to ‘a third name’. The instance will be automatically reconfigured.

Deleting an instance


To delete an instance, just remove the cfg file (containing the instance configuration) from the file install folder. That’s it! The watched folder is your system configuration.

The ManagedService case


So, let’s imagine that you don’t want to create instances from cfg file but just reconfiguring them. It’s also possible. File install also supports Managed Service, and so, will pushed a Managed Service configuration inside the configuration admin.

If you use this way, you have to create the instance yourself (from the metadata.xml file for example).
First, let’s modify our component type:

@Component(managedservice="MyPID")
public class Foo {
@Property
private String name;


}

and create the metadata.xml file declaring an instance of this type:

<ipojo>
<instance component="bar.Foo"/>
</ipojo>

The “managedservice” attribute of the @Component annotation allows setting the service pid of the Managed Service. So, iPOJO will expose a ManagedService allowing reconfiguring your instance with the “MyPID” pid. [You can also set the managedservice pid from the instance configuration].
So, to reconfigure the instance, create a cfg file named MyPID.cfg containing the configuration such as:
name = a name

When you save the file in the watched folder, File install will pushes the configuration to the configuration admin. Then, the configuration admin will look for the matching ManagedService (the one with the MyPID service.pid) and pushed the configuration. So, your instance will receives the configuration!

To edit the configuration, just edit the cfg file. To remove it, just remove the file from the folder.

Conclusion


File install is definitely a great tool to provision your system. The configuration admin supports can be useful to create iPOJO applications “on the fly” and will allows you reconfiguring your system dynamically!

Sunday, March 29, 2009

Back from Amsterdam

So, I’m back from ApacheCon. It was a great conference with high quality talks. OSGi was really a hot topic.

I organized an Apache Felix BOF with Felix, Carsten and Marcel. 30 people attend this BOF. This definitely illustrates the OSGi wave. I posted a BOF report on the Felix dev mailing list.

Another surprise is the feedbacks I get about junit4OSGi. It definitely seems to be the missing piece in OSGi development! So, my goal is now to release it soon (April). I recently improved the doc (now available from the iPOJO web site).

I have a couple of ideas about junit4osgi evolution. I will work on a better maven integration (site generation), in a way to do in-container tests, supporting other test ‘frameworks' such as junit4 or testNG… My discussions about these leads with junit4osgi users definitely interest them. So, here we go!

To summary ApacheCon in three words: “it was awesome!”

Monday, March 16, 2009

ApacheCon EU / EclipseCon

The next week, I will be at ApacheCon EU in Amsterdam from Wednesday to Friday. If you have questions about iPOJO, pervasive or mobile applications, or if you just want to drink a beer, feel free to contact me.

If you prefer the sunny California to Amsterdam, Karl Pauls and Richard Hall will be at EclipseCon. Don’t hesitate to ask them iPOJO related questions. Moreover, do not miss their "OSGi in Action" tutorial.

Friday, March 6, 2009

Maven, OSGi, and native libraries on Mac OS

I’m currently involved in a project dealing with native library. As a software engineering fan, I decide to set a project containing both unitary tests and integration test (thanks to the maven-junit4osgi-plugin, of course ☺).

However yesterday a strange issue appeared. The integration tests using the native library were executed “successfully”, but when I ran the same bundles in our application, I get UnsatisfiedLinkError. For all of you that already meet this Exception, it is always a bad thing.

What really does it means? No native libraries match with the actual architecture/os/… or if one matches it cannot be loaded for an obscured reason.

Hum, but I ran junit4osgi and the regular application on the same machine, my Mac book pro with Mac OS X! So, why junit4osgi was able to load the library and not a normal Felix platform (using exactly the same bundles)?

It takes at least 3 hours of debugging to understand what happened. First, I claimed against Maven ☺ (as everyday!), but in fact Maven was not guilty (just a little bit, because it is Maven). I also tried the application on Felix and Equinox. Despite Equinox was a little more verbose on the issue, the library was not loaded. I try different version of Felix, and always the same error: UnsatisfiedLinkError ! The worst thing happened when one of my co-worker executed the application on Windows correctly!

WTF????

Junit4OSGi test reports contain the system properties, and so I decided to compare the properties between junit4osgi and the normal runtime. And here arrived the light: maven and the runtime does not use the same JVM.

The OS provides the ‘mvn’ executable.
Clement-Escoffier:integration-test clement$ which mvn
/Users/clement/dev/apache-maven-2.0.9/bin/mvn

This executable uses Java 5 (the default JVM on Mac). So when running maven-junit4osgi-plugin, it uses this 32bits VM. In this context, the native library was loaded correctly.

However, when I ran the application, I use a Java 6 VM (64bits architecture). In this case, the library cannot be loaded. Despite it match the native library clause of my bundle; the library was not compiled to run on a 64bits CPU.

So finally, three hours spend for just a JAVA_HOME issue…

Monday, February 23, 2009

iPOJO available on Module Fusion

Module Fusion is an OSGi based stack for enterprise applications. The goal of ModuleFusion is to help programmers to use the OSGi Service Platform as their underlying runtime environment.

ModuleFusion contains a full stack typical for Java enterprise applications. This stack currently consists of best-of-breed open source frameworks from the Java ecosystem. Additionally, ModuleFusion contains the necessary glue code to easily use these frameworks within OSGi.
Probably the most important principle of ModuleFusion is to base every integration, glue code and additional functionality on pure OSGi constructs. Therefore, ModuleFusion does not rely on proprietary features offered by some OSGi framework implementations.

The last week, I discuss with Roman about the possibility to integrate iPOJO in Module Fusion. So, now you can get an iPOJO feature pack, containing:
  • iPOJO 1.2.0
  • The ‘arch’ command
  • The temporal dependency handler
You can also get some examples about how to use iPOJO in Module fusion. More specially, a complete example about how using JPA (Hibernate) with iPOJO is provided.

For iPOJO early adopter using iPOJO 1.3.0-SNAPSHOT, a Wicket web application using iPOJO dependencies injection is also provided. This is for early adopter only because of the Felix-936 feature. This new feature is used to exposed the web application in the OSGi registry and so allows Pax:Web to publish it.

So, if you’re interested in OSGi on the server side, or on domain-specific OSGi based application servers, have a look to Module Fusion and see how iPOJO can help you in your daily development tasks.