So long jroller

Off to greener pastures - this blog has now officially moved to mrdonbrown.blogspot.com

Posted at 12:59AM Jul 18, 2009 by Don Brown in General  | 

Clarifications re JavaOne "Web on OSGi" slides

It has been interesting watching the reactions to my JavaOne "Web on OSGi: Here's How" slides. A couple of bits of additional information you don't see in just reading the slides:

• Very little of the talk was actually about how Atlassian uses OSGi, as it was more general than that. In fact, slideshare seems to not respect hidden slides, so several hidden slides about Atlassian Plugins are shown that a) are confusing without additional information (which is why they were hidden in the first place) and b) make it seem like Atlassian Plugins was more the point of the talk, which it wasn't.
• Peter Kriens, the Technical Director of OSGi and a pretty nice bloke I chatted with at the OSGi BOF later that day, made the good point that I should have clarified that our classloader issues were not because OSGi was faulty, but because the third-party libraries just weren't written for modularity. Looking back, the vast majority of our issues with OSGi have been because of a poorly written (from a modularity perspective) library whether it was Jersey, Rome, JAXB, or even JAXP. Unfortunately, the average developer probably doesn't care who's fault it was - just that it worked before, and now it doesn't. I'm hoping Peter's excellent work in JSR-294 and the like will convince developers to put a higher priority on modularity in library design.
• Atlassian Plugins is the plugin framework for Atlassian applications that has been around for over 5 years now, so this wasn't something that was cooked up just to hide OSGi. In it, we did have a simple classloader-per-plugin model that just wasn't powerful enough, prompting our investigation into OSGi. As I also mentioned in the talk, though it isn't reflected in the slides, our plugin developers were used to writing plugins a certain way using things like dependency injection, atlassian-plugin.xml as the sole configuration file, bundling jars inside the plugin jar, etc., so a lot of the details about how OSGi was fit into it are done in such a way to keep that feature set and development experience.

I guess this means I'll have to do a proper writeup/talk/both on how and why Atlassian uses OSGi in Atlassian Plugins, so again, if that is what you were looking for, you won't find a satisfying answer in just these slides. The takeaway that I was going for was not that OSGi is just too hard, but that make sure you go into with both eyes open.

We (Atlassian) had some hard problems that OSGi solved, and I've been very happy with it. OSGi has let us write big features in a portable, modular way across products, and has dramatically sped up the development process via its hot deployment capability. Along the way, we've had some issues, particularly with libraries that don't play well in modular environment, but they certainly haven't outweighed the benefit IMO; OSGi certainly does rock.

Posted at 03:53PM Jun 09, 2009 by Don Brown in Java  |  Comments[4]

Slides from my JavaOne "Web on OSGi - Here's How" talk and Atlassian Summit

I've uploaded my slides from JavaOne talk titled, "The Web on OSGi - Here's How", to slideshare. The code from the demo, where I put OSGi, via Atlassian Plugins, into JForums is available via the GitHub project jforum-plugins.

While I was at it, I uploaded the slides from my two talks at Atlassian Summit. I believe the videos of all sessions, including mine, will be available via the summit site soon.

• Plugins 2.0: The Overview
• REST

Posted at 05:09PM Jun 06, 2009 by Don Brown in Java  |  Comments[1]

Adventures in game programming

I've been playing trading games a lot lately, namely Tradewinds Legends and Space Rangers 2, and after a good gaming session, all I want to do is hack on the game to add this feature or tweak this formula. Building with my long term addiction to writing TradeWars 2002 helpers, I set out this time to write my own trading game. Along the way, I decided to brush up on OpenGL and dive into Groovy.

The result is the soon-to-be-named "Trader" game, now with a 0.1 release. The experience has been a blast getting out of the J2EE world and into gaming and Groovy. I've been trying to see how it works mixing Groovy and Java, and while admirable, the process isn't yet seamless, but man do closures, named constructor arguments, and sane property accessors make a difference. Rounding out the experience, I've been trying to do my own graphics so I've been following a few Gimp and Inkscape tutorials, which have been quite humbling.

Coming from the J2EE world, Java gaming programming is somewhat rough with many outdated libraries and little build/Maven infrastructure. Adding a GUI to the game was particularly interesting, as the minimal GUI frameworks actually made me miss Swing.

All in all, its been a lot of fun and something I'd encourage anyone else who has always wanted to get into gaming. Slick2D is quite well done and FengGUI, while seemingly abandoned, takes care of a lot of the grunt work.

Posted at 11:49PM May 03, 2009 by Don Brown in Java  |  Comments[3]

Maven less ugly, part 2

Just about two years ago, Eric Redmond showed off code to make Maven less ugly, inspiring me to turn his Ruby script into the Maven YamlPom Plugin. Now that Maven 2.1 is out (thanks Brett, et al) with parallel artifact retrieval, I've been thinking about what else really bugs me about Maven, and near the top of the list is its extra-verbose POM XML files.

The Maven YamlPom Plugin supports the use of YAML to implement the Maven POM. It keeps both versions in sync, allowing other tools to continue to use XML while giving the developer the simplicity of a human-readable, YAML pom. In test conversions of open source projects and those internal at Atlassian, I've found the YAML version to be 2-4 times smaller than the original XML version, both in number of lines and total size. Furthermore, it makes poms so easy, you can write them from scratch, and the power of YAML allows advanced users to do things like node references, as demonstrated in Eric's blog post. Enough talk, here's an example:

groupId: org.twdata.maven
artifactId: maven-yamlpom-plugin
version: 1.0-SNAPSHOT
packaging: maven-plugin
name: YAML POM Plugin
dependencies:
  - { groupId: org.apache.maven, artifactId: maven-plugin-api, version: 2.0 }
  - { groupId: SnakeYAML, artifactId: SnakeYAML, version: 1.1 }
  - { groupId: commons-io, artifactId: commons-io, version: 1.4 }
  - { groupId: dom4j, artifactId: dom4j, version: 1.4 }
  - { groupId: junit, artifactId: junit, version: 3.8.1, scope: test }
  - { groupId: xmlunit, artifactId: xmlunit, version: 1.2, scope: test }
build:
  plugins:
    - artifactId: maven-compiler-plugin
      configuration:
        source: 1.5
        target: 1.5
repositories:
  - id: snakeyaml
    name: SnakeYAML repository
    url: snakeyamlrepo.appspot.com/repository

To my eyes, anyway, this is soo much cleaner and readable. The best part of how the plugin works is your tools and other developers can continue to use the XML version, and the plugin will try to keep them in sync for you. Think of it as a POM management tool that is there when you need it. From what I understand, Maven 3 will support pluggable presentations, which will make it even easier.

Posted at 12:40AM Mar 29, 2009 by Don Brown in Java  |  Comments[1]

Trick of the day: find an open TCP port

At Atlassian, every project, where applicable, will have a matrix of integration tests running our code on different application servers, databases, JDK version, etc. Our Bamboo build boxes will run multiple builds at the same time leading to contention over what TCP port to listen to that isn't being used by another build. Our usual solution is a system property we set in the build, but with this code, Java can handle the tedious work for you.

    private int pickFreePort()

    {

        ServerSocket socket = null;

        try

        {

            socket = new ServerSocket(0);

            return socket.getLocalPort();

        }

        catch (IOException e)

        {

            throw new RuntimeException("Error opening socket", e);

        }

        finally

        {

            if (socket != null)

            {

                try

                {

                    socket.close();

                }

                catch (IOException e)

                {

                    throw new RuntimeException("Error closing socket", e);

                }

            }

        }

    }

Apparently if you pass 0 into the ServerSocket constructor, Java will pick a free port, so it is just a matter of retrieving this value. Obviously, not something I'd use in production code, but for testing where you need to eliminate conflicts without increasing tedious configuration, this does the trick.

I should also mention, the immediate reason I wanted this was for my Maven IT Blast Plugin, which runs your integration tests over multiple application servers in one go with minimal Maven config. It works great for easily integrating functional tests for smaller projects.

Posted at 10:43PM Feb 03, 2009 by Don Brown in Java  |  Comments[4]

Maven without all the slowness - now with IDEA support

Maven 1 had this nice feature called the console, which allowed you to execute goals very quickly in a running Maven instance, as it bypassed all the startup slowness. I wanted the same thing for Maven 2, so a while back, I wrote the Maven CLI Plugin. Over the weekend, I wrote an Intellij IDEA plugin so that I can execute Maven goals quickly via a keybinding from within IDEA.

My primary motivation was to minimize the code-build-deploy-test cycle, which is particularly painful for multi-module web apps. In my case, I'm usually working on Atlassian plugins, which is a plugin container for Atlassian web applications that uses OSGi to manage the plugins at runtime. The framework allows me put upgrade a plugin via our Maven PDK plugin. The Maven CLI plugin, particularly when combined with its IDEA plugin, now let's me upgrade plugins in one or two seconds on a running instance of an Atlassian application like Confluence, thereby saving me hours, if not days, in the long run.

Here is the Maven CLI plugin in action:

The plugin can be configured to also listen on a TCP port for commands, which is how the IDEA plugin sends the goals over. The IDEA plugin contains three configurable actions to execute Maven commands, which can be bound to any keystroke:

More information can be found in the User Guide.

I should also mention this plugin was made possible via the Mojo Executor library I wrote that allows you to programmatically execute Maven goals, also used for plugins like the Maven IT Blast plugin.

Posted at 01:44PM Jan 30, 2009 by Don Brown in Java  |  Comments[4]

Git in real life

I've been using git on and off during the last year, but never really saw enough concrete value to fully make the switch. I like how you can use git but still interact with your company's subversion server, but if the features aren't worth the extra complexity, the point is moot.

Admittedly, I never really sat down with the git manual and studied all it had to offer, but this description of git in real life has convinced me to give it a go. Forget its distributed nature, forget neat features like cherry-picking revisions for merging - oh git-stash, where you have you been all my life? I'm forever experimenting with different features and implementations, and with our subversion server sitting on the other side of the globe, I've ended up with bunches of uncommitted code and specially named zip files littering my hard drive.

To quote the blog post:

Every now and then I’ll get distracted by a quick bug fix when I have a bunch of code I don’t want to commit yet. That’s where stashing comes in.

• git stash (all my changes are saved away, and I have a clean tree)
• Fix the bug
• git commit -a -m "bugfix"
• git stash pop (changes are back)
• Continue where I left off

Neat.

Posted at 12:39AM Jan 26, 2009 by Don Brown in Java  |  Comments[5]

Stylish makes DZone usable

DZone can be made usable on devices with small screens, namely the Eee PC, using the very useful Stylish Firefox plugin and this stylesheet: @namespace url(www.w3.org/1999/xhtml); @-moz-document domain("dzone.com") { #left { display:none !important; } #head { margin : 0px !important; px !important; top: 0px !important; } #content { left: 0px !important; top: 80px !important; } #pageTitle { margin: 0px !important; top: 0px !important; } }

Specifically, this stylesheet removes the left bar, which I've never found any use for, and collapses the top nav a bit to give you more usable space when browsing content. The 7" Eee PC has a whole 800x480 resolution, leaving about an inch for the actual DZone content. This fixes that.

Posted at 06:14PM Oct 22, 2008 by Don Brown in Java  | 

Next frontier for JSON - Telnet?

The TradeWars bug is back, and this time, I want to solve the problem of writing clients for text-only telnet applications like TradeWars or MUDs. Since the only way to interact with the application is a textual terminal, any scripting or data gathering attempts have to resort to screen-scraping, which is very brittle and tedious to write.

Borrowing a page from microformats, I came up with the Hidden-In-Plain-Sight (HIPS) protocol, which embeds structured data within a telnet data stream using telnet option negotiation and ANSI escape codes. To encode the message, I needed a textual data format that was compact yet easy to read, so I picked JSON. JSON is great because it is incredibly easy to write and there are libraries and marshaling frameworks for every language, and as you'd expect, Java has a bunch. Here is an example:

Esc[8mlogin={"name":"John Doe"}Esc[28m

For the marshaling library, I wanted one that used annotations to be able to map arbitrary JSON text into a POJO, and since the library would be included in a client-side application, I wanted one with minimal dependencies. Well, I found a library that met criteria #1, but not #2 as much as I'd like. JSON Marshaller excels at being easy to use and configure, but for some reason, feels the need to bring along the ASM jars and Google Collections. Why oh why do these little libraries feel the need to include five times their weight in dependencies? More people need to use tools like JarJar to keep their libraries slim and developer-friendly.

Posted at 01:18AM Aug 17, 2008 by Don Brown in Java  |  Comments[1]

Speed up Maven through the Maven CLI Plugin

Despite how many extra flags you throw at Maven (-o -Dmaven.test.skip), it still grates on my nerves, waiting five or more seconds for a simple compilation or jar build, especially when a day may be filled with hundreds of such builds.

To turn a five second task into a few hundred milliseconds and perhaps shave hours off my development time a week, I created the Maven CLI Plugin, which provides an interactive command line interface to Maven for executing specific plugins.  By leaving Maven running, you save the JVM startup time and all that XML POM parsing and resolution, and by being able to execute specific plugins, you can skip goals that may be necessary for your full build, but you don't need in day-to-day development.

If you have the plugin installed in your local repository, you can execute the cli from the command line:

mvn org.twdata.maven:maven-cli-plugin:execute

Here is sample output:

mrdon@makoa:~/dev/maven-cli-plugin$ mvn org.twdata.maven:maven-cli-plugin:execute
[INFO] Scanning for projects...
WAGON_VERSION: 1.0-beta-2
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven CLI Plugin
[INFO]    task-segment: [org.twdata.maven:maven-cli-plugin:execute]
[INFO] ------------------------------------------------------------------------
[INFO] [cli:execute]
[INFO] Waiting for commands

maven2> compile
[INFO] Executing: org.apache.maven.plugins:maven-compiler-plugin [compile]
[INFO] [compiler:compile]
[INFO] Nothing to compile - all classes are up to date
[INFO] Execution time: 255 ms

maven2> clean compile
[INFO] Executing: org.apache.maven.plugins:maven-clean-plugin [clean]
[INFO] [clean:clean]
[INFO] Deleting directory /home/mrdon/dev/maven-cli-plugin/target
[INFO] Execution time: 335 ms
[INFO] Executing: org.apache.maven.plugins:maven-compiler-plugin [compile]
[INFO] [compiler:compile {execution: virtual-execution}]
[INFO] Compiling 1 source file to /home/mrdon/dev/maven-cli-plugin/target/classes
[INFO] Execution time: 352 ms

maven2> exit
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2 minutes 
[INFO] Finished at: Sat Aug 02 21:44:09 GMT+10:00 2008
[INFO] Final Memory: 10M/135M
[INFO] ------------------------------------------------------------------------
mrdon@makoa:~/dev/maven-cli-plugin$

The project is, again, hosted at GitHub at github.com/mrdon/maven-cli-plugin/wikis

Posted at 10:23PM Aug 02, 2008 by Don Brown in Java  |  Comments[12]

Run integration tests on multiple servers in 200^H^H^H 13 lines of XML

Maven has a nice built-in capability, via Cargo and Surefire, to run integration tests easily as part of your build.  Unfortunately, you need a separate Maven run per server and heaps of XML incantations to get it working.  My new maven-itblast-plugin helps to solve these problems to make integration tests even easier.

The Maven itblast (Integration Test Blast) plugin uses a handy little library I wrote called Mojo Executor to programmatically execute other plugins inside another plugin, making the creation of composite plugins trivial.  The new plugin combines a lot of the XML configuration needed to configure and run cargo and surefire with explicit executions of both plugins, looping through the configured servers.  This means you can execute:

mvn integration-test -Dcontainers=tomcat5x,jboss42x,jetty6x,resin3x

and the itblast plugin will run your integration tests against those four servers in one Maven run.  If you are using JUnit to execute the tests, it will even try to combine the tests in such a way that each test has a unique name for easy reporting in Continuous Integration (CI) servers like CruiseControl or Bamboo.  My first use of the plugin was to start to add integration tests to Struts 2, and already it found several bugs that I wouldn't have found testing on Tomcat alone.

For more information, see the project wiki, hosted by GitHub: github.com/mrdon/maven-itblast-plugin/wikis/home

Posted at 12:42AM Aug 02, 2008 by Don Brown in Java  | 

Maven-enabled project hosting with GitHub

To my dismay, I discovered Google Code only allows 10 projects, and as I'm up to number 9, I decided it was time to find a new host, and apparently GitHub is what the cool kids use, so I'm giving it a go.  While GitHub wasn't built with Java and Maven in mind, with a little work, it turns out to be an excellent fit.  Here is how I did it:

Obviously Java can use any version control system and Maven generally ignores it, so there is nothing inherently incompatible between Java, Maven, and Git.  Where it gets interesting is when you want to using the Maven release plugin, which handles not only changing version numbers in pom.xml files but committing them, tagging trunk, and releasing the artifacts to a Maven repository.

The key problem to solve was how do I make Maven aware of Git so that it could detect outstanding changes that should hold up releases, and then tag and commit files.  The good news is the next release of Maven SCM, the bit that handles source code management for Maven, has support for git.  The bad news is it is only in trunk.  Time to fork another Maven project, I guess :)  I branched the git provider so I could create a release of the git module, since the release plugin will fail if there are any snapshots in your project.  To use my "1.1-db-1" release, I added this Maven repository to my pom.xml:

<repositories>
    <repository>
        <id>don-asf-repository</id>
        <url>people.apache.org/~mrdon/repository/</url>
    </repository>
</repositories>

Now, I needed to tell the scm and release plugins to use my new git provider:

    
    <build>
        <extensions>
            <extension>
                <groupId>org.jvnet.wagon-svn</groupId>
                <artifactId>wagon-svn</artifactId>
                <version>1.8</version>
            </extension>
        </extensions>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-release-plugin</artifactId>
                <dependencies>
                  <dependency>
                    <groupId>org.apache.maven.scm</groupId>
                    <artifactId>maven-scm-provider-gitexe</artifactId>
                    <version>1.1-db-1</version>
                  </dependency>
                </dependencies>
            </plugin>
           <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-scm-plugin</artifactId>
                <dependencies>
                  <dependency>
                    <groupId>org.apache.maven.scm</groupId>
                    <artifactId>maven-scm-provider-gitexe</artifactId>
                    <version>1.1-db-1</version>
                  </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>

The wagon-svn extension is to support deploying the artifacts to a Maven repository accessible via Subversion.  I've found Subversion to be an excellent tool for managing a Maven repository, and since there didn't seem to be a nice way to do this in GitHub (they don't host files, just git repositories), I created a Google Code project just for being a Maven repository.  For more information about hosting a Maven repository with Google Code, see my previous writeup.

Now that Maven is git-aware, I needed to tell it where to find my remote git repository.  Remember, git is a distributed version control system, so I want it to not only commit the files and tag my local git repository, but push the changes up to the "official" repository, the one hosted at GitHub.  I added this scm configuration, again to my pom.xml:

    
    <scm>
        <connection>scm:git:git://github.com/mrdon/maven-itblast-plugin.git</connection>
        <url>scm:git:git://github.com/mrdon/maven-itblast-plugin.git</url>
    </scm>

That's it!  I can now execute 'mvn release:prepare' and 'mvn release:perform' and everything works perfectly.  It even handles pushing the git repository changes to GitHub transparently:

sierra-nevada:~/dev/maven-itblast-plugin dbrown$ mvn release:prepare
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'release'.
WAGON_VERSION: 1.0-beta-2
[INFO] ------------------------------------------------------------------------
[INFO] Building maven-itblast-plugin Maven Mojo
[INFO]    task-segment: [release:prepare] (aggregator-style)
[INFO] ------------------------------------------------------------------------
[INFO] [release:prepare]
[INFO] Resuming release from phase 'map-release-versions'
What is the release version for "maven-itblast-plugin Maven Mojo"? (org.twdata.maven:maven-itblast-plugin) 0.1: : 
What is SCM release tag or label for "maven-itblast-plugin Maven Mojo"? (org.twdata.maven:maven-itblast-plugin) maven-itblast-plugin-0.1: : 
What is the new development version for "maven-itblast-plugin Maven Mojo"? (org.twdata.maven:maven-itblast-plugin) 0.2-SNAPSHOT: : 
[INFO] Transforming 'maven-itblast-plugin Maven Mojo'...
[INFO] Not generating release POMs
[INFO] Executing goals 'clean verify'...
[INFO] Executing: mvn clean verify --no-plugin-updates -P foobar,foobar
[INFO] Scanning for projects...
        WAGON_VERSION: 1.0-beta-2
        [INFO] ------------------------------------------------------------------------
        [INFO] Building maven-itblast-plugin Maven Mojo
        [INFO]    task-segment: [clean, verify]
        [INFO] ------------------------------------------------------------------------
        [INFO] artifact org.apache.maven.plugins:maven-scm-plugin: checking for updates from central
        [INFO] snapshot org.apache.maven.plugins:maven-scm-plugin:1.1-SNAPSHOT: checking for updates from central
        [INFO] snapshot org.apache.maven.scm:maven-scm:1.1-SNAPSHOT: checking for updates from central
        [INFO] [clean:clean]
        [INFO] Deleting directory /Users/dbrown/dev/maven-itblast-plugin/target
        [INFO] [plugin:descriptor]
        [INFO] Using 2 extractors.
        [INFO] Applying extractor for language: java
        [INFO] Extractor for language: java found 1 mojo descriptors.
        [INFO] Applying extractor for language: bsh
        [INFO] Extractor for language: bsh found 0 mojo descriptors.
        [INFO] [resources:resources]
        [INFO] Using default encoding to copy filtered resources.
        [INFO] [compiler:compile]
        [INFO] Compiling 1 source file to /Users/dbrown/dev/maven-itblast-plugin/target/classes
        [INFO] [resources:testResources]
        [INFO] Using default encoding to copy filtered resources.
        [INFO] [compiler:testCompile]
        [INFO] No sources to compile
        [INFO] [surefire:test]
        [INFO] No tests to run.
        [INFO] [jar:jar]
        [INFO] Building jar: /Users/dbrown/dev/maven-itblast-plugin/target/maven-itblast-plugin-0.1.jar
        [INFO] [plugin:addPluginArtifactMetadata]
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD SUCCESSFUL
        [INFO] ------------------------------------------------------------------------
        [INFO] Total time: 15 seconds
        [INFO] Finished at: Mon Jul 28 17:23:19 EST 2008
        [INFO] Final Memory: 18M/33M
        [INFO] ------------------------------------------------------------------------
        [INFO] Checking in modified POMs...
[INFO]