Invariant Properties

Notebook: Development Tips

Various development tips that I have accumulated over the years.

Use heart beats in long-running applications

Silence is usually golden (you don’t want to hide important information in the clutter) but it’s also important to know that a process is still alive.  Answer: a heart beat thread that does nothing but periodically writes a “everything’s okay!” message to the log.  That message could be a simple fixed string or it could provide self-diagnostics results.

In the production environment you can take this a step further and have a monitoring tool watching for these heart beats.  Any missing heart beats results in an alert.

Log system information on startup

On application startup log the basic system information.  What is the computer’s name?  What operating system?  What java version?  Who is the user running the application?  Really basic information that can be invaluable when you’re looking at a log file a month later and trying to recreate the problem.

This can usually be handled by a very simple Spring Framework/EJB3 bean that’s auto-loaded.  (That is, set ‘lazy-init’ to ‘false’.)  You should never have to make any change to the application itself.

(Aside: you probably want to repeat this information on a periodic heartbeat so that it will be caught on rotating log files.)

Log database connection information on startup

On application startup log the database connection information provided in the DatabaseMetaData object.  I usually log the connection URL (for database server name), connection user name (since different users may have different rights), database server name and version (in case there’s something specific to that database server version), and the JDBC driver name and version (ditto).

If your database has a table containing the schema version you should log that as well.

Again you can do this in a standalone Spring Framework/EJB3 bean.

Have a ‘one-click’ way to create and restore snapshots.

You want to make it easy to capture details when something goes wrong.  Digging into documentation to see how to perform a database backup or restore is not easy.  Figure out what you need (e.g., database dump and configuration files) and capture it.  You also need an easy way to restore to a snapshot in the development environment.

Zip files are ideal for snapshots since they support compression and allow you to add an arbitrary amount of information.  XML files are either harder to change or so generic that you can’t make the best use of XML-based tools.  (I’m a big fan of zip files containing XML documents.)  Some users are nosy and would peek into a .zip file but it’s easy to give the files a different extension.

Several caveats:

• You can’t create large snapshots.  “Large” is in the eye of the beholder though – what about a 100MB snapshot?
• The backup and restore applications provided by the database vendor are the most powerful but they limit you to that database, maybe to that specific release.  They require you to know their location (to run as an external process).  They require you to have permissions to run them.  (E.g., a well-run system will only allow users in the ‘postgres’ group to run the backup and restore programs and the appserver will NOT be in that group.)  You can work around all of this but it requires customization – it’s not something you can toss into a class file and never think about again.
• On the other hand you can use DbUtil to dump the database into XML files. This is more flexible and you could, at least in theory, create a snapshot on one database server and restore it to a second.  However there are nasty dependency problems, esp. with circular references and self-referential tables, and you could miss advanced functionality.
• You need to support encryption if this code gets anywhere near live data. The standard JRE implementation does not support encryption although third-party implementations might.  Or you could use the standard JSSE libraries and a good open-source provider like BouncyCastle — but it’s nontrivial to properly manage your keys.  At least one common task is easy – deploy a keystore with an standard PK public key.  It can be used to encrypt the data before sending it to QA or the developers.  The PK private key can stay in-house.  This doesn’t answer the questions if the user wants the ability to create and restore their own snapshots.
• You need to decide what you need.   Obviously a database dump.  Probably any configuration files and System.properties.  What about information that may not be available to you, e.g., details of container managed resources?

Use separate ‘user’, ‘owner’ and ‘security’ roles in the database.

Modern jet aircraft do not go down because one thing breaks.  They don’t go down because two things break.  I think the current standard is six independent failures.  This is known as ‘defense in depth’ – do your best but don’t require any one thing to be perfect.

This tip is a classic example of defense in depth.  Something simple that is good design on its own but really shines in increasing overall security by providing a backstop to your anti-SQL injection code.

Create three roles for your application:  ‘owner’, ‘user’ and ‘security’.  The ‘owner’ role creates the tables but does not have any rights to edit, or even view, the data.  The ‘user’ role owns the data but does not have any rights to edit the schema.  The ‘user’ role can write to the audit logs but only the ‘security’ role can read them.  (This keeps an attacker from learning what we know about him.)  The ‘security’ role can read the application data but can’t edit it.

A good analogy is an apartment lease.  The landlord owns the building but doesn’t have the right to look around your apartment except as explicitly permitted by the lease.  The tenants own their own stuff but can’t make changes to the building itself.

The web application NEVER uses the ‘owner’ role.  Period.  It should only be used by upgrade scripts.

The public-facing web application should never use the ‘security’ role either.  Web apps for internal use can use it.

Use a JVM-based database for unit testing

This tip is for ORM managers.  Most of your database code will be very generic and the backend isn’t important.  So use a JVM-based database for unit testing.  (I like H2 in in-memory mode, other people like Derby or HSQL.) You don’t have any dependencies on external resources, you can control exactly what goes into the database (e.g., with DbUtil), you can create snapshots of the database if tests fail, and more.

I’ve even used JVM-based databases for mockups.  It gives you a ‘live’ system without requiring the full backend yet.

N.B., some things are database-specific and can’t be tested this way.  Fortunately this usually follows the 80-20 rule – you can test 80% of your code in a very generic manner in about half of your time.   The remaining 20% is harder but is simpler in ways since you’ve already gotten so much out of the way.

Verify java – database enumerations at startup

This tip is important when you use JDBC and iBatis mappings directly.  ORM managers like JPA and Hibernate may take care of it for us.

The design concept is to use a database table for each persistent java enum class.  All other tables then use foreign key references into these tables to ensure the values are valid.  You can’t guarantee this if you use a code without the foreign key reference.  There’s a bit of a performance hit but I think every experienced DBA and developer will gladly pay it to avoid dealing with corrupt data.

You will have a world of pain if the java code and the database tables get out of sync.  The solution is to verify that each java enum value is represented in the corresponding database table, and vice versa, to ensure the two are in sync.  It’s easy to do during startup.  Just be sure to do it intelligently via reflection, not by maintaining an explicit list of values to check!

Have an easy way to dump the actual DB schema

This is another tip that’s very important with JDBC and iBatis mappings, less so with ORM.  You want to have an easy way to dump the database schema into a usable format.  This information should come from the database metadata to capture the database as it is, not as you expect it to be.

If you have a tool that does this for you – great!

If you don’t I generally use Excel workbooks with

• one sheet listing all tables (and their primary key)
• one sheet listing all table.columns (a great way to verify consistency of like-named attributes)
• one sheet listing all foreign keys
• one sheet listing all other indexes
• one sheet per table listing column details
• one sheet per ‘interesting’ metadata lists, e.g., the list of data types and their precision.

This usually works out well but you have to be careful since some databases are chatty and  you’ll exceed Excel limitations.

Dump SQL results to Excel files

Yet another tip that mostly applies if you are using JDBC or iBatis mappings since you can get directly to the database, or at least the first-tier cache.  ORM usually has more levels of caching and the raw database may not be up to date.

In a day or so you can write a class that dumps a ResultSet to an Excel worksheet (using Apache POI). With a bit of work it can even honor attribute precision by setting the correct formatting for each column.  Once you have this it is easy to grab a snapshot of the pertinent data when there’s a problem.