Posted on April 25, 2013 by tom

Reply

When most folks were out enjoying the fine weekend weather last Saturday morning in Arlington, the County Board met and approved letting government developers publish software as open source. Whether software the county shares with the open source community proves to be useful to other developers is a secondary issue. The exciting news here is that the county board’s unanimous vote is a good idea both for the open source community and the Arlington County government itself. 

Before I get into the details of why open-sourcing is a good idea for the county, let me explain what the county actually did and intends to do. The county is in the midst of redesigning and converting its main website from Active Server Pages, a commercial technology from Microsoft, to WordPress, a free and open source website management tool. During the conversion, county technology staff will or has already made modifications to parts of WordPress or perhaps some of the publicly available WordPress plugins that add features to the basic content management features that come with vanilla WordPress. Since WordPress is licensed under the GNU General Public License, version 2, the county cannot distribute its changes without making those changes open source under the GPLv2 license. What the county board did on Saturday was to approve releasing county-developed source code under the GPLv2 license. Since county developers are using WordPress plugins licensed under version 3 of the GPL license, the board also approved releasing source code under GPLv3. 

So why is releasing county-developed website code as open source a good idea?

First, it will cost the county close to nothing. There are free code-sharing repositories available, like GitHub, the county can use to host its open source code. The time developers will spend uploading code to a public repository is insignificant.

Second, changes to WordPress or its plugins that county developers make might actually catch on in the open source community. If the county has a need for an enhancement in the code, it seems likely someone else will, too. If other developers pick up some of the county’s changes and run with them, those developers might make their own open source improvements (and bug fixes) that can feed back into the county’s website. 

Third, if county developers find and fix bugs in WordPress or its plugins, they previously could not contribute those fixes back to the original developers. When county developers create software, the county owns it. Developers were not at liberty to give their code away to others, even to fix bugs. With the county board’s action, they can now release their bug fixes and improvements back to the open-source community. These contributions not only should be good for the open source community, the county won’t have to deal with the software upgrade dilemma: if you upgrade to the next version, but you have made changes to that version, your changes (and bug fixes) will be lost, which pressures you to stagnate with old but working software. If the county can contribute its changes back to the original source, then new upgrades will include the county’s changes, making the upgrade path much easier.

Fourth, the county loses nothing. The changes the county makes to GPL software cannot be shared with others (like other local governments) without also licensing their changes under the GPL license. (This is the viral nature of the GPL license that some businesses bemoan and the open source community cheers.) Since the county is using GPL software, the changes it makes to the software can’t be leveraged somehow into an income-producing revenue stream to benefit taxpayers. As the county’s staff report to the board put it, “Since County code is partially derived from open-source code, it cannot be released under any other license other than an open-source license. The County’s choices are to release the code under an open-source license, or to not release the code at all.” So why not go ahead and release it.

Fifth, the county should actually save money. Assuming Arlington’s new website has nice features, other local governments around the country might take notice. They then might email the Arlington County technology division asking for copies of their WordPress customizations. Each of these requests would need to be dealt with by some staffer, and the source code zipped up and emailed to the requesting government agency. Now, with the open source policy, Arlington can just refer interested government agencies to its public source code repository. Easy.

Sixth, and this is the political upside, the taxpayers have already paid for the source code. By having the county release the code to the public, the county is saying, “Here is software we created for the purpose of serving you. If you can further reuse it for your own personal benefit, go for it.” Since WordPress is one of the most popular web content management systems out there, it isn’t unreasonable to assume that a few Arlington County taxpayers (including businesses) might actually derive benefit from the county’s source code.

That last point complements a trend among governments to release information that was gathered at taxpayer expense. In February, the White House announced that federally funded scientific research would be made available free to the public — although waiting one year to allow subscriber-funded academic journals to make money from the research as well. The trend is a good sign that more people, and more people in government, are believing that what the government creates in the name of the public interest should be owned by the people.

So, this month my local government has gone open source — or at least is starting in that direction. Has yours?

Posted on April 17, 2012 by tom

Reply

Here are some of the photos I captured today of Space Shuttle Discovery flying over D.C. today on board its 747 carrier. The shuttle circled around D.C. a few times before heading for its final resting place at the National Air and Space Museum Steven F. Udvar-Hazy Center next to Dulles International Airport.

Posted on March 28, 2011 by tom

vs.
EclipseLink is the default JPA provider on the GlassFish JEE server. As such, I figured EclipseLink would work well as the persistence provider. However, we began a recent JEE 6 project using GlassFish v3 and chose Hibernate as the JPA provider because of the team's familiarity with Hibernate 3. We later switched to EclipseLink to be compatible with another application running on the server -- and immediately encountered two annoying problems that never occurred with Hibernate. This article is meant to document those EclipseLink problems in case it helps others with similar EclipseLink issues on GlassFish.

The first problem we saw was a sporadic ClassCastException after a module redeploy. The second problem we found was that entity methods marked with @PrePersist were not being called when the entity was being saved to the database. Instead, those fields would remain null when EclipseLink executed the INSERT SQL statement, resulting in constraint violations for our non-nullable columns.

The ClassCastException occurs at a line in our code that assigns an entity to a reference variable declared as the type of its parent class, which is a JPA mapped superclass. A widening cast like that should cause no problem, so this was a head-scratcher. Adding to the mystery is we had no problem with this code when using Hibernate JPA. The entity class in question, ActionTypeLookup, as shown in the stack trace below, directly extends the parent class type that it is being assigned to.

Here are the partial class definitions. The parent class type, CodeLookupValues,

@MappedSuperclass
public abstract class CodeLookupValues implements Serializable {
...
}

is a @MappedSuperclass that the entity ActionTypeLookup directly extends:

@Entity
@Table(name="ACTION_TYPE_LOOKUP")
public class ActionTypeLookup extends CodeLookupValues implements Serializable {
...
}

The persistence code is defined in a module (an EAR file) containing the JPA persistence unit definition. The ClassCastException occurs only if that same module is unloaded and reloaded from the server several times. That is, something you do a lot of during development. We had loaded and unloaded this EAR file many times on GlassFish with no problem when using Hibernate. Shortly after the switch to EclipseLink, we would see this stack trace in the log at deploy time:

javax.enterprise.system.core.com.sun.enterprise.v3.server Exception while loading the app
javax.ejb.EJBException: javax.ejb.CreateException: Initialization failed for Singleton LookupSessionFacade
at com.sun.ejb.containers.AbstractSingletonContainer$SingletonContextFactory.create(AbstractSingletonContainer.java:695)
at com.sun.ejb.containers.AbstractSingletonContainer.instantiateSingletonInstance(AbstractSingletonContainer.java:444)
at org.glassfish.ejb.startup.SingletonLifeCycleManager.initializeSingleton(SingletonLifeCycleManager.java:213)
at org.glassfish.ejb.startup.SingletonLifeCycleManager.initializeSingleton(SingletonLifeCycleManager.java:174)
at org.glassfish.ejb.startup.SingletonLifeCycleManager.doStartup(SingletonLifeCycleManager.java:152)
at org.glassfish.ejb.startup.EjbApplication.start(EjbApplication.java:150)
... [removed several classes involved in installing the EAR] ...
at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:57)
at com.sun.grizzly.ContextTask.run(ContextTask.java:69)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
at java.lang.Thread.run(Thread.java:619)
Caused by: javax.ejb.CreateException: Initialization failed for Singleton LookupSessionFacade
at com.sun.ejb.containers.AbstractSingletonContainer.createSingletonEJB(AbstractSingletonContainer.java:525)
at com.sun.ejb.containers.AbstractSingletonContainer.access$100(AbstractSingletonContainer.java:74)
at com.sun.ejb.containers.AbstractSingletonContainer$SingletonContextFactory.create(AbstractSingletonContainer.java:693)
... 36 more
Caused by: java.lang.ClassCastException: my.customer.package.shared.datamodel.reference.ActionTypeLookup cannot be cast to my.customer.package.shared.datamodel.reference.CodeLookupValues
at my.customer.package.shared.datamodel.persistence.LookupSessionFacadeBean.reloadCommonLookupValues(LookupSessionFacadeBean.java:127)
at my.customer.package.shared.datamodel.persistence.LookupSessionFacadeBean.readLookupTables(LookupSessionFacadeBean.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.ejb.containers.interceptors.BeanCallbackInterceptor.intercept(InterceptorManager.java:1006)
... [removed several more classes involved in installing the EAR] ...
at com.sun.ejb.containers.interceptors.CallbackChainImpl.invokeNext(CallbackChainImpl.java:61)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:390)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:373)
at com.sun.ejb.containers.AbstractSingletonContainer.createSingletonEJB(AbstractSingletonContainer.java:518)
... 38 more

We see the error at deploy time because the application has a @Singleton EJB that also is annotated with @Startup so the server loads it at deploy time. I don't know if having a non-startup EJB or a non-singleton EJB would make a difference.

It turns out this was the easiest problem to work around -- the ClassCastException goes away if you restart the domain. But I never discovered what causes the exception. At first, I thought the problem must be caused by the two classes being loaded by separate classloaders due to the way we had structured the modules in our application. However, both classes are:

• Deployed in the same Java package
• Inside the same JAR
• Packaged and deployed inside the same EAR.

The ClassCastException does not occur all the time. It just suddenly pops up after a series of deploy/undeploy cycles and does not go away until we restart the GlassFish domain.

My best theory so far as to what causes the ClassCastException is it is a bug in GlassFish's OSGi bundle class loading. The problem appears as if some GlassFish classloader has the parent MappedSuperclass class loaded even after module undeploy but not the child class. Subsequently, when the updated EAR file is deployed again, the new module's newly assigned Archive classloader sees that the parent class is already loaded but needs to load the child classes from the new EAR files's JAR. When the code tries a widening assignment of a child entity to a reference type of the parent class -- kaboom! -- a ClassCastException because the two classes are now unrelated because of the different classloaders.

The GlassFish OSGi implementation gets my suspicion only because EclipseLink is deployed in GlassFish as an OSGi bundle, and the ClassCastException never occurred when using Hibernate, which is deployed as a set of jar files inside the server's shared library classpath.

Better theories are welcome in the comments. I poked around but found no others reporting the same bug or that this problem has been fixed in a GlassFish update (we are not running the latest updates of v3). This ClassCastException is only an annoyance because the workaround is to restart the GlassFish domain/server. A restart always fixes the problem and it goes away, until after another series of deploy/undeploy cycles.

@PrePersist problem

The second and more vexing problem is with the way EclipseLink synchronizes its entity session cache with the database. If I understand the problem correctly from reading various forum postings, if you instantiate a new entity object and call an EntityManager.find() method before the new, detached entity has been persisted with EntityManager.persist(), EclipseLink synchronizes all the detached entity states with the database before performing the query. That synchronization makes sense: the database is going to need to know about the new rows in the tables if the SELECT query is going to find the correct data.

The problem, at least for me, is that when EclipseLink performs the INSERT statement to persist the unsaved entities, it does not call the @PrePersist methods on the entity. I was counting on @PrePersist methods to set values like created-date and last-updated-date, which cannot be null in our schema. OK, I can understand that EclipseLink opted not to call @PrePersist methods when it is merely inserting the data for its convenience and not because EntityManager.persist() was called. But the unexpected behavior required some logic change in our entity code because Hibernate always seemed to call our @PrePersist methods before performing an INSERT on the database. After the switch to EclipseLink, we started seeing database constraint violation errors on these @PrePersist columns.

This second problem with EclipseLink, then, cannot be called a bug, but it was unexpected behavior worth documenting here. I assumed incorrectly that @PrePersist methods, by definition, always would be called before an entity got persisted. Wrong.

Those are the only two unusual problems we saw after switching from Hibernate to EclipseLink. The other problems that occurred were some HQL-specific queries failed, but those could all be converted to standard JPQL with a little alteration.

After complaining about EclipseLink issues, I will profess one preference for EclipseLink: The debugging log messages and error log messages seemed clearer and more straightforward with EclipseLink. When one our our HQL queries failed or I needed to trace what queries were being called and when, EclipseLink's debugging messages just seemed clearer and easier to understand than many of Hibernate's logging messages. I could see queries more clearly being translated from JPQL to Oracle SQL, and see what values were being bound to query substitution parameters. I do hand it to EclipseLink for making its database interactions more transparent than I am used to with Hibernate. So when we had the @PrePersist problem, at least I was able to debug and diagnose them fairly easily.

Posted on March 16, 2011 by tom

The full moon will rise this weekend at nearly 90 degrees azimuth for those in Washington, D.C. That means the moon will be almost directly to the east. Since the National Mall and many of its famous monuments and buildings align along an east-west axis, the astronomical phenomenon promises stunning moonrises from places like the Washington Monument, the Lincoln Memorial, and the Netherlands Carillon as the moon slowly rises behind or next to the U.S. Capitol. If the expected clouds abate on Friday and the weather holds out, that is.

As you can see from these photos of near-90 azimuth moonrises last year, the moon looks great near the horizon when looking east across the Mall. Here are the stats for the weekend:

On Friday, the nearly-full moon will rise in D.C. at 6:23 p.m. EDT at 89 degrees azimuth (source). On Saturday, the full moon rises at 7:39 p.m. at 97 degrees azimuth.

And if the full moon due east isn't enough to pique your geeky astronomical interest, this weekend's moon will be a big and bright moon. Saturday, the full moon is at perigee -- its closest approach to earth. This perigee is the closest the moon will get to the Earth for all of 2011: 221,575 miles (356,575 kilometers). That's 31,064 miles closer than the moon was on March 6, according to EarthSky. EarthSky says Saturday's full moon will be its closest encounter with the Earth since Dec. 12, 2008, and the closest it will be until Nov. 14, 2016.

For photographers interested in capturing the event, outside the Lincoln Memorial and the Netherlands Carillon should be good places to set up your tripod. (Tip: You might want to get to the Carillon early to stake out an unobstructed spot for your tripod. It's a popular place for full moon photos. Note that the path in front of the bronze lions is a popular one with joggers, bikers and pedestrians, so get your siteline set up with that in mind.) As far as weather goes, the current forecast calls for partly cloudy Friday around moonrise with a slight chance of rain. But sometimes a low cloud layer can make for great photos if the clouds aren't dense. Saturday also promises to be partly cloudy with some rain possible in the morning. But the clouds are supposed to clear by moonrise. Saturday promises to be the better day for weather but with the 97 degrees azimuth, perhaps less-stunning photos (more like the one below). I'm looking forward to some beautiful moonrises this weekend. Share the moment with someone you love -- but hey, take the camera. If you grab some good photos, please send me a link in the comments.

Posted on November 7, 2010 by tom

I have had continuing problems with ddclient being able to connect to the network and make an http call to check my current IP address. If you use ddclient and also see this problem, this workaround might work for you, too.

The ddclient bug exhibits itself with two errors I would see in the system log and also kindly emailed to me by the ddclient daemon itself:

WARNING:  cannot connect to checkip.dyndns.org:80 socket: IO::Socket::INET: Bad hostname 'checkip.dyndns.org'

or the more generic error:

WARNING:  cannot connect to 192.168.0.1:80 socket: IO::Socket::INET: connect: Network is unreachable

The issue seems to be that ddclient, a Perl client that talks to dynamic DNS services like dyndns.org, has problems either making network connections or perhaps caches a bad address at system start when networking services might not yet be up. This problem with ddclient seems longstanding, with a bug filed in 2003 on the Debian list and a bug filed in 2009 on the Red Hat list.

The Red Hat bug was closed May 29 with a fix (ddclient-3.8.0.2) posted to update sites for Fedora 11 and later. But if you have not or cannot update, or still see the bug, here's my workaround: Instead of using ddclient's built-in web client to connect to your dynamic-DNS service, call a shell script that uses curl to make the network call. Specifically, I replaced this line in my /etc/ddclient.conf configuration file:

use=web, web=checkip.dyndns.org/, web-skip='IP Address' # found after IP Address

with this line:

use=cmd, cmd=/home/tom/bin/checkip.sh, cmd-skip='IP Address' # found after IP Address

Here is my checkip.sh shell script, stored in my home "bin" directory:

#!/bin/sh
#
# A script to fill in for what ddclient
# can't seem to do: reliably connect to checkip.dyndns.org.
curl checkip.dyndns.org/

That's it. The only extra steps you need to take are to ensure the user that runs your ddclient daemon (typically user "ddclient") has access to the script. That means in my case making sure the script itself is executable, e.g. chmod 755 ~/bin/checkip.sh, and that my home directory and bin directory are world executable, e.g. chmod --recursive o+x ~/bin/checkip.sh

When I eventually upgrade my system and use version 3.8.0.2 of ddclient, I look forward to seeing if this longstanding networking bug really got fixed.

Posted on March 18, 2010 by tom

Rich Hickey, inventor of the Clojure language, and Stuart Halloway, author of "Programming Clojure," presented introductory and advanced concepts of the young JVM language at Wednesday's Northern Virginia Java Users Group. These are some of my notes from the meeting. The session served to whet interest in learning Clojure, thus these notes do not include a lot of code or explain Clojure's unusual syntax. There are many other sources for that.

Clojure, a Lisp-like language that compiles to Java byte code and runs on the Java virtual machine, was created as a general-purpose programming language that embraces a functional style of software design, rather than the imperative style typical in languages like Java -- and most other general purpose languages in use today. Functional programming languages like Clojure, Scheme and Erlang have been getting a lot of attention at technology conferences over the last few years, which first brought my attention to Clojure. Its functional style and its ability to run alongside and integrate with existing Java code interested me in learning more about Clojure. The fact that its inventor and a technology instructor I highly respect were presenting a free session on Clojure compelled me to attend the JUG meeting.

Rich Hickey released the first version of Clojure in October, 2007, with version 1.0 released May 1, 2009. We are talking about a young language. Still, from what I learned last night, it looks like a powerful language with potential. Clojure is released as open source under the Eclipse Public License 1.0, which makes it easy to use in a non-open source commercial environment.

Stu Halloway, co-founder of the top-notch professional training and agile consulting company Relevance Inc., began with an introduction to Clojure's features and why a Java developer might want to learn it. Rich then took over and introduced three new features of Clojure (Protocols, Reify and Datatypes) that can be downloaded from the latest source tree but are not part of the current 1.1 release of the language.

According to Stu, some of the compelling features of Clojure are its:

• Easy interoperability with Java
• Lisp syntax
• Functional style
• Ability to run in a multi-threaded environment with no coding overhead

To demonstrate the syntax benefit, Stu "refactored" the StringUtils.isBlank method from the Apache Commons lang library. He started by showing the full Java source code and then removing all the ceremonial scaffolding code to expose the core logic, then simplified the Java code into the definition of an equivalent Clojure function:

(defn blank? [s]
(every? #(Character/isWhitespace %) s))

I'm not a Clojure programmer (yet) but I think I captured the above syntax correctly. Like Ruby, Clojure uses the question mark to replace the traditional "is" prefix in boolean functions. The # symbol introduces an anonymous function. From what Stu described, the functional programming paradigm in Clojure handles most (all?) corner cases for you. There is no need to write special-case "if" statements to deal with a null parameter, for instance.

For Clojure's interoperability with Java, Clojure code can call Java, and Java code can call Clojure functions. (According to Rich, the integration is implemented with little or no need to use Java reflection at runtime, adding less runtime overhead.)

For Clojure's advantage by using Lisp syntax, Stu referred everyone to Paul Graham's 2001 article, "What Made Lisp Different" as the best explanation. Most languages have "special forms" like imports, scopes, protection definitions, metadata, keywords. These special forms are language features you can use, but you cannot create them yourself and add them to the language. These language features are thus unavailable for reuse. Lisp abandoned this restriction. In a Lisp-like language, special forms of the language look like anything else in the language. All forms of the language are created equal. In Lisp (and Clojure), defining scope, the control flow, method calls, operators, functions, import mechanisms -- they are all lists. Stu said a language's "special forms" restrictions cause a programming language to "crap out," and joked that the restrictions bring about magical cut-and-paste reuse workarounds we call "design patterns."

For Clojure's advantage by being a functional language, Clojure encourages you to write small pieces of code that work well together. Good code has the same shape as pseudo code, he said, and Clojure's functional style lets you create more pseudo-code looking real code. According to Stu, functional languages are simpler to understand. They let you write code that eliminates or reduces what he called "incidental complexity" required by non-functional languages:

• Corner cases
• Class definitions
• Internal exit points
• Variables
• Branches

The resulting code is less complex, he said, and simpler to understand by orders of magnitude.

The final benefit he talked about is Clojure's inherent ability to run in a multi-threaded environment with no special concurrency-handling code from the developer. Clojure and other functional programming languages perform this feat by treating data as immutable and producing a new copy of a data structure when data needs to be changed. Two threads never look at the same data at the same time, so there is never any need to synchronize access to code that reads and writes data. Clojure's solution, Stu said, is to separate identify from value. He went on to explain what this means, but maybe the late hour caused me to miss the details.

Rich Hickey

After Stuart set the stage for why learn and use Clojure, Rich Hickey took over to talk about new features he is adding to the language. He said, quite truthfully, that for those in the audience who don't already know Clojure, what he was about to say would not make a lot of sense. These features are Protocols, Reify and Datatypes. As a result of my newness to Clojure, I will pass along what I thought Rich said and hope he and the Clojure crowd forgive my ignorance.

Rich, for an open source programming language inventor, was a refreshingly clear advocate for his new language. Maybe I'm jaded from years of slogging through open-source code, but from my experience, most open source projects release their code with little explanation of how or even why to use it, and then treat users like they are the ones who failed if they misunderstand how to use the code correctly. Rich actually understood where most of us in the audience were coming from. "I know it's a big deal to try to learn a new programming language," he said, but he believes Clojure is worth taking the time to learn and will make our jobs as programmers easier.

Before delving into the new features he is adding, Rich provided a summary of how Clojure is implemented. Part of it is written in Java for performance, and the rest is written in Clojure itself. He said his goal is to eventually write most of Clojure in Clojure once he can get performance boosted to an equivalent level.

Clojure is built using abstractions, with those abstractions written as Java interfaces. The fundamental implementation objectives of Clojure (or at least the ones I picked up on), he said, are to leverage high-performance polymorphism mechanisms of the host environment, to write to abstractions not concrete types, and to enable extension and interoperability with Java.

From what I understood of the new language features, Protocols are named sets of generic functions. Reify allows developers to use the "cool code generation" in the built-in fn function. "I put a lot of work into 'fn' and I wanted to make it reusable," he said. Even though it went over my head, Rich said Reify allows developers to create an instance of an unnamed type that implements protocols, like proxy for protocols. For the new Datatypes feature, if I understood correctly, he said he added a new construct, deftype, to define a name for a type and list of fields in that type.

Additional details that might make sense if you know Clojure:

• Datatypes fields can be primitives
• Datatypes support metadata and value-based equality by default
• In-line method definitions are true methods, no in