Pentaho 6 – OSGi what?

I’m going to direct this post for those of you who are more Pentaho people rather than Java developers. This won’t be an in-depth article. My intention here is to shed light on a key change that Pentaho 6 brings that will improve the whole system modularity. I’ll also show a really basic example to demonstrate how things can glue together.

Modularity now

So Pentaho BI Server has its own plugin system that is pretty handy and allows us to do a great variety of things. Even core functionalities have been delivered as plugins so that the core platform code doesn’t need to be changed and, therefore updates and migrations be more specific to plugins or components.

Due to the use of Spring and to the whole architecture of the platform, you can influence and extend the system from a plugin. Each plugin has its own factory and all of its classes are loaded in an isolated classloader with their own context.

In some cases this is good. However, things start to get a little tricky when we talk about sharing, cohesion and maintenance.

Imagine that you’re developing two plugins that both need one component you developed – a template factory. Let’s call Plugin-A, Plugin-B and template-component. What happens now is that, if you want both plugins to use factory-component, you have a couple of options:

  • Place template-component.jar on each plugins lib folder and use them on each of their contexts normally;
  • Place template-component.jar into webapp lib – you just won’t do that.
  • Place template-component.jar into Plugin-A lib, declare Plugin-A as a dependency of Plugin-B and use hacks to access the template-component classes such as:
    • web-services;
    • content-generators;
    • Java reflection.

You’ve noticed that is definitely not a nice-to-have scenario for core functionalities, right? All the presented options have clear problems – that i don’t think i need to number – and that’s why guys at Pentaho came out with OSGi.

OSGi in a nutshell

OSGi is a specification of a modular system – that’s the tagline. But how does it look like?

Very shallow explanation ahead.

Every module – a bundled jar with metadata – from a OSGi system, explicitly defines:

  • Its identity – name + version. Yes, different versions of the same module can coexist on the same environment;
  • Externals – identity, packages and services it provides. The externals definition sets the boundaries around what’s implemented on the module, making implementations safer; services allows loose coupling between modules.
  • Dependencies – identities, packages and services it needs to run.

Instead of having nested hierarchical classloaders, OSGi has an environment that puts all the modules on the same level. And because modules define exactly what they provide, the search domain is considerably narrowed.

Some other environmental key-features that need to be highlighted, are:

  • Dependency resolving – in some implementations dependencies can be transitively resolved with maven repositories.
  • Hot deployment – you can deploy/update modules without restarting the server.
  • Encapsulation – more than encapsulating your code at class level, OSGi brings encapsulation at jar level, by declaring externals.

Now let’s skip back to Pentaho and the problem we’ve described above.

Reality check

So, if Plugin-A and Plugin-B both need the same template-component, in this scenario, this component can be a OSGi module and therefore be shared between them and with whoever needs it. There will be no need for workarounds like web services or reflection. And if our template-component needs a bug correction, we’ll be able to do it without restarting the server and in a more elegant fashion – like a boss.

Plugins

The plugins system that still exists to the present – and will be there forever for a long time – on BI Server has a lot of dev-features that maybe aren’t yet fully translated to OSGi – but I’m sure Pentaho is working on this as I write. I am only saying this because of the status of some key JIRAS like this, this and this. Right now, you can already do important stuff on Pentaho’s OSGi structure, like creating perspectives, overlays, webservices, components and so on.

The whole Marketplace plugin have been fully translated into a OSGi plugin with a maven multi-modules archetype. As we can see of this JIRA issue it also knows how to properly install OSGi modules.

Example

I am more of a practical kind of person and I like to show and say things by experimenting. For this reason, I am going to guide you through a very small implementation to demonstrate how to develop a small OSGi bundle that will be deployed into Pentaho BI Server.

Environment

I want to give you a very important information before we start – it might serve also for your further exploration. From the version 6.0 on, Pentaho brings Apache Karaf as the OSGi runtime, along with Apache Felix, that is an OSGi framework/service platform and much more such as Pax Web Extender and Spring.

Tools

For this example, I’ll use Eclipse – Version: Luna Service Release 2 (4.4.2) – with Bndtools 3.0.0.REL – installed.

“Bndtools is an Eclipse-based development environment for OSGi bundles and applications.” Bndtools page

You will also need a ssh tool – get one if you’re on windows – to connect to Pentaho’s karaf instance and deploy our bundle.

UPDATE: You can also drop your bundle into …/karaf/deploy :) no restart needed!

Preparing the environment

Make sure you have Eclipse installed. I didn’t test this in other versions different than Luna, but you should be good to go.

Bndtools

To install Bndtools, you just need to go on Eclipse and click Help > Eclipse Marketplace. There you should type Bndtools on the search field to find it. After installing it you will be prompted to restart your Eclipse. Do it.

Enabling ssh on karaf

You can skip enabling ssh if you want to deploy your bundle directly into .../karaf/deploy.

Even though our Catalina logs on Pentaho shows the Karaf Port, the instance is not actually listening to that. Ssh doesn’t come as an enabled feature on startup.

*******************************************************************************
*** Karaf Instance Number: 1 at /.../biserver-ce/pentaho-solutions/system/k ***
***   araf//data1                                                           ***
*** Karaf Port:8801                                                         ***
*** OSGI Service Port:9051                                                  ***
*** JMX RMI Registry Port:11098                                             ***
*** RMI Server Port:44444                                                   ***
*******************************************************************************

To enable ssh so we can connect to our instance, you’ll need to change the file pentaho-solutions/system/karaf/etc/org.apache.karaf.features.cfg and add ssh to the list of featuresBoot like this:

featuresBoot=config,management,ssh, ...

Once you start your BI Server – again – you’ll be able to connect to your Karaf instance using ssh – and your usual server credentials: admin/password.

$ ssh admin@localhost -p 8801

If everything went as expected, you should see this welcome message after entering your password:

        __ __                  ____      
       / //_/____ __________ _/ __/      
      / ,<  / __ `/ ___/ __ `/ /_        
     / /| |/ /_/ / /  / /_/ / __/        
    /_/ |_|\__,_/_/   \__,_/_/         

  Apache Karaf (3.0.3)

Hit '<tab>' for a list of available commands
and '[cmd] --help' for help on a specific command.
Hit 'system:shutdown' to shutdown Karaf.
Hit '<ctrl-d>' or type 'logout' to disconnect shell from current session.

admin@root()> 

Let’s rock ‘n’ roll.

Implementation

We’re going to do something very simple. A perspective that will render a static html page. Everything will be contained into a single bundle.

For those of you that don’t know what a User Console perspective is:

“A perspective changes the behavior and appearance of PUC by taking over certain areas of the interface. The PUC main toolbar and main menubar are now easily setup with XUL overlays. The content area of PUC can also be completely owned by a perspective. In this way, PUC can be dramatically customized. Switching perspectives is done by clicking on them in the upper right hand corner.” Pentaho Wiki

Project creation

It’s time to create a new Maven Project on our workspace.

  1. Right-click the Project Explorer and select New > Other…;
  2. Find and choose Maven Project;
  3. Click Next > on the first screen – we’re going to leave everything default here;
  4. On the archetype selection, find and choose maven-archetype-quickstart;
  5. Give your project a Group Id – I’ll use com.oncase;
  6. Give your project an Artifact Id – I’ll use sample-perspective;
  7. You can still change version and package information, but I’ll leave the defaults there.

pom.xml

Now that our project has been created, you can see there’s a generated pom.xml. The first thing we’re gonna need to do there is change the value to bundle, instead of jar.

<packaging>bundle</packaging>

This will help us build the metadata of a OSGi bundle. Every time you see some weird alerts on our Eclipse project, make sure to right-click the project and go to Maven > Update Project. This will have to be done a couple of times during the development of this bundle.

To help us with writing the OSGi metadata, we’re also going to need maven-bundle-plugin. We can make use of it by inserting in our pom.xml, the following code:

  <packaging>bundle</packaging>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.felix</groupId>
        <artifactId>maven-bundle-plugin</artifactId>
        <version>2.3.7</version>
        <extensions>true</extensions>
        <configuration>
          <instructions>
            <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
            <Bundle-Version>${project.version}</Bundle-Version>
            <Export-Package>com.oncase.sample.perspective</Export-Package>
            <Import-Package>!javax.xml.namespace, org.apache.felix.http.api, *</Import-Package>
          </instructions>
        </configuration>
      </plugin>
    </plugins>
  </build>

Note that into <Export-Package>, we’re using our package name. You should put there the packages you want to be visible outside of that bundle. This content addresses the jar encapsulation I talked about earlier.

For developing our perspective class we depend on two packages that need to be declared as dependencies in our pom.xml: pentaho-platform-api and pentaho-platform-core. If you don’t have Pentaho repository configured, you’ll also need to add it on your pom or settings file.

So the whole thing would look like this:

...
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <dependency.pentaho.platform.version>6.0.0.0-353</dependency.pentaho.platform.version>
    </properties>
    <repositories>
        <repository>
            <id>pentaho-omni</id>
            <name>pentaho omni</name>
            <url>http://repository.pentaho.org/content/groups/omni</url>
        </repository>
    </repositories>
    <dependencies>
        <dependency>
            <groupId>pentaho</groupId>
            <artifactId>pentaho-platform-api</artifactId>
            <version>${dependency.pentaho.platform.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>pentaho</groupId>
            <artifactId>pentaho-platform-core</artifactId>
            <version>${dependency.pentaho.platform.version}</version>
            <scope>provided</scope>
        </dependency>
        ...
    </dependencies>
...

The perspective

I’ll now create a new class com.oncase.sample.perspective.StaticPerspective that will implement org.pentaho.platform.api.engine.perspective.pojo.IPluginPerspective.

package com.oncase.sample.perspective;

import java.util.ArrayList;

import org.pentaho.platform.api.engine.perspective.pojo.IPluginPerspective;
import org.pentaho.ui.xul.XulOverlay;

public class StaticPerspective implements IPluginPerspective {

    private static final long serialVersionUID = 1280540339612668867L;
    private String contentUrl;
    private String id;
    private int layoutPriority;
    private String resourceBundleUri;
    private ArrayList<XulOverlay> overlays;
    private ArrayList<String> requiredSecurityActions;
    private String title;


    public String getContentUrl() {
        return this.contentUrl;
    }

    public String getId() {
        return this.id;
    }

    public int getLayoutPriority() {
        return this.layoutPriority;
    }

    public ArrayList<XulOverlay> getOverlays() {
        return this.overlays;
    }

    public ArrayList<String> getRequiredSecurityActions() {
        return this.requiredSecurityActions;
    }

    public String getResourceBundleUri() {
        return this.resourceBundleUri;
    }

    public String getTitle() {
        return this.title;
    }

    public void setContentUrl(String contentUrl) {
        this.contentUrl = contentUrl;
    }

    public void setId(String id) {
        this.id = id;
    }

    public void setLayoutPriority(int layoutPriority) {
        this.layoutPriority = layoutPriority;
    }

    public void setOverlays(ArrayList<XulOverlay> overlays) {
        this.overlays = overlays;
    }

    public void setRequiredSecurityActions(ArrayList<String> requiredSecurityActions) {
        this.requiredSecurityActions = requiredSecurityActions;
    }

    public void setResourceBundleUri( String resourceBundleUri ) {
        this.resourceBundleUri = resourceBundleUri;
    }

    public void setTitle(String title) {
        this.title = title;
    }

}

At this point, I suggest you to go ahead and try a maven install to test if everything is doing ok. Just right-click your pom.xml file and choose Maven > Install.

If you have some trouble there, the console messages are usually pretty clear about what’s going on. If you get it right, a BUILD SUCCESS will show up at the end. Something like this:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.792 s
[INFO] Finished at: 2015-11-02T12:46:31+01:00
[INFO] Final Memory: 14M/222M
[INFO] ------------------------------------------------------------------------

After doing this first build, you’ll also notice that the maven-bundle-plugin also wrote on our target/sample-perspective-0.0.1-SNAPSHOT.jar/META-INF/MANIFEST.MF the proper OSGi headers:

Manifest-Version: 1.0
Bnd-LastModified: 1446465082027
Build-Jdk: 1.7.0_79
Built-By: marcellopontes
Bundle-ManifestVersion: 2
Bundle-Name: sample-perspective
Bundle-SymbolicName: sample-perspective
Bundle-Version: 0.0.1.SNAPSHOT
Created-By: Apache Maven Bundle Plugin
Export-Package: com.oncase.sample.perspective;uses:="org.pentaho.platfor
 m.api.engine.perspective.pojo,org.pentaho.ui.xul";version="0.0.1.SNAPSH
 OT"
Import-Package: org.apache.felix.http.api,org.pentaho.platform.api.engin
 e.perspective.pojo,org.pentaho.ui.xul
Tool: Bnd-1.50.0

The web folder

We need to create a folder where we’re going to place our html sample file and some more static resources we want to provide.

I am going to place mine under src/main/resources/web.

There, i’ll put a index.html file with a single <h1>Hello OSGi</h1>.

Under web/lang, i’ll create a messages.properties file to put some messages variables in. I’m gonna leave mine empty.

The whole thing should look like this:

* src/
  * main/
      * java/
      * resources/
          * web/
              * index.html
              * lang/
                  * messages.properties

Wiring things up

To serve our static web folder and also to tell the system we have a perspective, we’re gonna have to write a blueprint file. This xml flie will be placed at src/main/resources/OSGI-INF/blueprint/sample-blueprint.xml and will have the following structure:

<?xml version="1.0" encoding="UTF-8"?>

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0"
           xmlns:cxf="http://cxf.apache.org/blueprint/core"
           xmlns:jaxrs="http://cxf.apache.org/blueprint/jaxrs" xmlns:jaxws="http://cxf.apache.org/blueprint/jaxws"
           xsi:schemaLocation="
  http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
  http://www.osgi.org/xmlns/blueprint-ext/v1.1.0 https://svn.apache.org/repos/asf/aries/tags/blueprint-0.3.1/blueprint-core/src/main/resources/org/apache/aries/blueprint/ext/blueprint-ext.xsd
  http://cxf.apache.org/blueprint/jaxws http://cxf.apache.org/schemas/blueprint/jaxws.xsd
  http://cxf.apache.org/blueprint/jaxrs http://cxf.apache.org/schemas/blueprint/jaxrs.xsd
  http://cxf.apache.org/blueprint/core http://cxf.apache.org/schemas/blueprint/core.xsd
  http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd
  http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0 http://aries.apache.org/schemas/blueprint-cm/blueprint-cm-1.1.0.xsd
  ">

  <!-- HttpService registration -->
  <service id="oncaseWeb"
           interface="org.ops4j.pax.web.extender.whiteboard.ResourceMapping">
    <bean class="org.ops4j.pax.web.extender.whiteboard.runtime.DefaultResourceMapping">
      <property name="alias" value="/sample-perspective/web"/>
      <property name="path" value="/web"/>
    </bean>
  </service>

  <!-- perspective registration -->
  <service interface="org.pentaho.platform.api.engine.perspective.pojo.IPluginPerspective">
    <bean class="com.oncase.sample.perspective.StaticPerspective">
      <property name="id" value="sample.perspective.osgi"/>
      <property name="title" value="Oncase"/>
      <property name="contentUrl"     value="osgi/sample-perspective/web/index.html"/>
      <property name="resourceBundleUri" value="osgi/sample-perspective/web/lang/messages"/>
      <property name="requiredSecurityActions">
        <list>
          <value>org.pentaho.security.administerSecurity</value>
        </list>
      </property>
      <property name="layoutPriority" value="-1"/>
    </bean>
  </service>


</blueprint>

In the first section, where it says HttpService registration, we’re mapping our internal /web folder to an address that will be mounted at http://<BISERVER-DOMAIN>/pentaho/osgi/. So our index.html will be accessible at http://<BISERVER-DOMAIN>/pentaho/osgi/sample-perspective/web/index.html.

On the perspective registration section, we’re registering our class to serve a perspective. This is a traditional bean declaration in which we initialize the variables we’ve declared in our class. Note that we are listing the org.pentaho.security.administerSecurity to explicitly say that this perspective will only be available to platform administrators.

build & go!

Our project is now ready to go. You should do our last maven install execution to generate your bundle. Again:

  • Right-click over pom.xml;
  • Maven > Install

This should export our file to the target/ folder. Get that file’s absolute path and copy that somewhere.

Go back to your ssh console $ ssh admin@localhost -p 8801 and type:

admin@root()> bundle:install file:///<PATH-TO-YOUR-PROJECT>/target/sample-perspective-0.0.1-SNAPSHOT.jar
Bundle ID: 228
admin@root()> bundle:start 228

UPDATE: You can also drop your bundle into …/karaf/deploy :) no restart needed!

Done! You’re now good to go.

After starting the bundle, you’ll note that the perspective is now available on BI Server, but the URL you’ve configured isn’t. I won’t cover how to refresh the system so that your URL would be available. Just restart your server now and you’ll see that working.

Perspective menu

Hello OSGi

Final Thoughts

I can resume all of my impressions about OSGi on Pentaho as: I am happy to see this.

I am happy with the fact that the guys have pushed it. As a developer, I like it for the the reasons I mentioned above, especially for not having to work things around. I think that the whole Pentaho suite will benefit from it – as already seen on the marketplace project.

Infrastructure maintainers should benefit from this push as lower impact updates will come; Business users will probably benefit from having more specific upgrades. I see OSGi as a way to bring faster updates as bundles are more independent and isolated.

Now I’ll try to push as many internal demands as we have to the OSGi world and keep you guys posted about tips&tricks.

The working project is available at github.com/oncase/sample-perspective.

A few more references:

See you soon at the #PCM15 in London!

Marcello Pontes
Analytics Architect & Front-ender at Oncase |

  • rn82497

    Checkout Neil Bartlett’s talk at the OSGi community event this week. Paremus have been pushing forwards with enhanced Maven support in bndtools.