Since the last release of Obviel 1.0b2, I've been plugging away at creating a new version. This version has a massive list of changes. Some highlights:

• a much nicer demo app, the inevitable todo list. This is a demo along the lines of the TodoMVC project. I haven't quite got it to the state where I think it's ready to submit to that project, but I'm getting there. One thing you'll notice is that the UI is in Dutch. By a one-line change in the JS code you can switch that UI to English. I still need change the UI so you can change the locale in the UI itself; next release!
• extensive improvements in the i18n infrastructure.
• pluralization support in templates. This brings Obviel Template's i18n support to a level slightly beyond what I'm familiar with as best-of-breed i18n in server-side templates (ZPT/Chameleon). In brief, you can have a UI that says "{count} cows", and when count is 1, it'll say "1 cow", and when count is 2, it'll say "2 cows", without the need for a conditional in the template. You'll note that in English there are two forms (one and many). Other languages have other rules and may have only one form for all numbers, or more than 2 forms. Obviel builds on gettext to support all these different pluralization rules.
• data-handler lets you bind a DOM event to a view method directly in a template.
• data-func can now also use methods of the view, which turns out very useful to support more complex attribute rendering logic which is better written in JavaScript instead of in clunky templates.
• some backwards incompatible changes: the great renaming (under_score to camelCase in the API, as that's more the JS way), and eliminated callbacks in favor of jQuery deferred. A bit late, but the installed base is still small, and I figured now is better than never!
• the docs got a freshening up, both in presentation as well in contents.
• many bugfixes and minor improvements.

Please give it a spin and give feedback!

A beta version of Plumi 4.4 was released recently, updating Plumi to Plone 4.2. Changes include deployment as a WSGI app and asynchronous execution of jobs via plone.app.async, separating the video uploading process from the database, which should mean stability improvements in the uploading system. The load balancer has been removed, buildouts have been simplified, and the beta also includes a new publish form with a video file upload progress indicator, a big step forward in terms of usability.

See the changelog here:
pypi.python.org/pypi/plumi.app/4.4b1

You can download the package from:
plone.org/products/plumi/releases/4.4b1
or via:
pypi.python.org/pypi/plumi.app/4.4b1#downloads

Please download and test the beta. The install.txt has changed, you can find it here:
https://github.com/plumi/plumi.app/blob/master/docs/INSTALL.txt

The changes can be seen in action at testing.plumi.org.

This release was brought to you by Unweb.me (https://unweb.me) and EngageMedia (https://engagemedia.org).

We’ve been having classic summer weather here in New England – gardens, fields and forests are exploding with lush vegetation. It brings out my inner botanist – something I actually have advanced training in. These days botany is usually a hobby for me, but two years ago my Jazkarta world and my botanist world came together on an amazing project – Go Botany.

In 2009 the New England Wild Flower Society was awarded a major grant from the National Science Foundation to develop an innovative suite of tools to teach botany and plant identification. This multi-year project has many deliverables, but the overarching goal is to develop radical new web-based tools for identifying plants that will improve botanical knowledge and science education for novices and citizen scientists. In other words, get more people hooked on plants!

Discovery

NEWFS selected Jazkarta to be their technology partner and we worked on the project throughout 2010. We started with a discovery phase, during which we sorted out the various tools, prioritized requirements, defined the technology platform, and created a plan for tackling development. Our plan relied on an agile, iterative approach; the project owner – Elizabeth Farnsworth, NEWFS Senior Research Ecologist and PI on the NSF grant – was always in charge of what we delivered next. This was essential since at the beginning of the project no one had a very clear idea of what the tools should look like. By working in an agile way we were able to easily adjust the plan as we learned more. This quote from the book The Art of Agile Development, by James Shore and Shane Warden (O’Reilly) summarizes the situation nicely.

The beginning of every software project is when you know the least about what will make the software valuable. You might know a lot about its value, but you will always know more after you talk with stakeholders, show them demos, and conduct actual releases. As you continue, you will discover that some of your initial opinions about value were incorrect. No plan is perfect, but if you change your plan to reflect what you’ve learned – if you adapt – you create more value.

Technology Platform

Python was a natural choice for implementation language because NEWFS IT staff were already using it. In addition to being mature, powerful, secure, open source, and object oriented, Python has a wide array of libraries available for scientific computing and GIS.

We considered several Python web frameworks that were popular at the time. Plone (the CMS in use at NEWFS) is not well suited to building the sort of custom applications we needed, but some of the other frameworks (like Grok and repoze.bfg) didn’t have enough functionality. We chose Django, which had gained traction in the Python community because of its simplicity, admin features, form technology, and the availability of numerous add-ons. Two of those add-ons were particularly critical to the success of the project: GeoDjango for adding GIS features, and Pinax for adding social networking features.

We had a lengthy in-house debate over the choice of database technology. PostGIS (the spatially enabled version of PostgreSQL, an industrial-strength open source relational database) was the clear winner for its compatibility with the Django ORM, and it was required for GeoDjango. But the data model for the botanical information was essentially a star schema (one table in the middle looking up information in dozens of auxiliary tables for attributes and vocabularies), which is much more efficient to implement in an object database than in a relational database. In the end, we chose to use PostgreSQL/PostGIS for everything, mostly because we knew it would make life easier for future developers and sysadmins on the project.

We needed one more component to implement the Go Botany tools, which were going to rely heavily on various types of search. We chose Solr, an open source search server based on the Lucene Java search library, to provide indexing and search services for all tools. It provides cross database search functionality over a REST interface, including features like custom query structures, common document type extraction, geographic searches, and search facets.

Implementation

By May we had put together a team and launched into release planning and implementation. The team consisted of Jazkarta developers and NEWFS developers working together on the project, with me as the PM and Elizabeth as the project owner. We like to have joint, collaborative teams with our clients whenever we can, it’s a great way to get their IT staff up to speed quickly. We also spent some time teaching Elizabeth about agile development practices, which helped her do a great job supplying us with the information we needed, when we needed it (user stories, feedback, etc.)

Database and API

Our first project was to design and implement the PostgreSQL database of botanical information that would drive all the Go Botany tools. NEWFS had previously built a Microsoft Access-based tool for entering the copious amounts of botanical data required for the project – dozens of characteristics for each of the thousands of species of New England plants. A time consuming part of building the database involved automating the import of all the MS Access data. We also implemented an API wrapper around the database that was optimized for the kinds of queries and transactions that would be required. The API calls were largely implemented as json web services for ease of integration into Javascript user interfaces.

The database plus API was the most technologically challenging part of the project. It had to provide a flexible and fast foundation for the entire family of Go Botany tools. It was also interesting to write “user stories” for this part of the project – the “users” were all the Go Botany tools, which would need to get information from the API.

Simple Key

Once the database and API foundation was laid, we began work on the Simple Key – an interactive, online guide to identifying 1,200 of the more common New England plants. This was the most challenging part of the project from a user interface point of view. It needed to be fun and exciting to seduce novices into learning about plants and botany, while at the same time being useful to pros. NEWFS engaged user experience designer Matt Belge (Visionlogic.com) to help them define the tool we would be building. Based on lots of interviews and other research, Matt produced wireframes for successive bits of Simple Key functionality, and we figured out how to implement them.

The result is unlike any previous plant identification tool. The Simple Key presents the user with a series of questions about their plant’s characteristics that are designed to home in on the species identification as efficiently as possible. It does this based on the questions that have already been answered and the features that the user is able to observe. The secret behind this behavior is an information gain algorithm that generates optimal decision trees.

When the user has identified their plant, they can go to a species page with a wealth of information – including maps of its geographic range, diagnostic characteristics, memorable facts, and gorgeous color photographs of the plant and its leaves, flowers, bark, etc. in different seasons. A fitting end to a successful quest.

And It’s Open Source

Because taxpayer dollars through the National Science Foundation supported this project, the software and data on plant characteristics will be open sourced. Details are still being worked out, but the goal is to enable others to contribute and improve the code over the long-term. Stay tuned to the Go Botany site for an announcement of the details. NEWFS staff are also engaged in adapting the database to allow other botanical institutions to customize the data for their particular region.

But This Is 2012!

Yes, all of our Go Botany work was done in 2010, so why am I blogging about this now?

We got Go Botany tool development off to a good start, but a lot of work remained after we were done. Seven botanical data specialists did a massive amount of manual data entry: scores of characteristics for thousands of plant species. Four image collectors scoured the web, publishers, and individual contacts for the gorgeous photographs used on the site. Botanists developed technical descriptions of the New England plant families and genera. Many NEWFS staff created videos, help pages, an illustrated glossary of terms, and other content. A web design firm, Fresh Tilled Soil, was hired to create the graphic design for the Go Botany website, including the Simple Key. NEWFS developers applied the theme to the Django site, as well as continuing to refine and fix the Python and Javascript code.

All of this hard work came to fruition in April: the Go Botany website is now live at gobotany.newenglandwild.org/!

So if you live in the northeast and you sometimes wonder what the name of a plant is, please try it out! Go Botany works best with Firefox, Safari, and Chrome on iPads, tablet and desktop computers (and the UI is being adapted for phones) so you can take it with you on walks and have an expert assistant for your botanizing. If you have comments, NEWFS would love to get your feedback at gobotany@newenglandwild.org.

The official announcement came this week, and the team here at Six Feet Up is excited!

The keyword for this release is: easy to use.

While Plone has many powerful features to manage content and provide a consistent look and feel, they might have been considered harder to use or had complicated steps to activate. This new release focuses on addressing some of these concerns.

"The keyword for this release is: easy to use."

Updated Collections

Plone's system of Collections have provided a powerful method of pulling together content into a single unified view. However, the interface for defining the criteria to collect the content can leave you scratching your head.

This release features a completely overhauled user interface that includes some very slick AJAX. This makes it very intuitive to manipulate the criteria with no distracting page reloads.

Bundled Theming

The Diazo theming system is a fresh new take on customizing the look and feel of your site. It can be used to unify the appearance of multiple underlying technologies. However, there have been some barriers to setting it up and configuring it correctly with Plone.

Plone 4.2 removes a few of these by bundling the integration package (plone.app.theming) with the core distribution and updating the installers to make it easy to use out of the box. Way to follow the Python batteries-included philosophy!

Upgraded Python

The framework team went to great lengths to ensure that this latest release functions well with Python 2.7, culminating in making it the version used by the installers. This will be a welcome improvement with our development and hosting teams, as it contains useful improvements to the standard library and performance enhancements.

Congratulations

Great job, Eric Steele, on getting this release out the door! It is good to see the continued focus on improving the user experience.

Keep your eye on the Plone Demo site: it will be updated to this latest release soon.

I have to hand it to the University of Minnesota Press, they do a great job keeping their homepage fresh. It has lots of portlets around the edges, and a big slot in the middle to hold a featured image.  Every few weeks they swap a new image into that slot, and voilà! new homepage. This week’s UMP homepage features kale. Really. It made me both laugh and click (which took me to a page featuring local food writers and their books.) Their first hompage featured rock music (captured in the client blurb on our website), and by the time you read this it will probably be featuring something else. But whatever is featured, you can bet it will be intriguing and will draw you into exploring UMP’s amazing catalog of books.

Kudos to our partners, Might & Main, for a compelling design (it even looks good on mobile devices) that can be refocused with a single image upload.  And kudos to the Plone CMS for making this so easy to implement. We used ContentWellPortlets to provide the bottom porlet slots. The central image goes into the homepage body, and the rest of the content comes from normal Plone text, RSS, and collection portlets. The site is themed using Diazo (via plone.app.theming), but the homepage magic is mostly CSS. The various portlets are styled depending on their location (right, left, bottom), plus there is extra CSS for the Twitter and New Releases portlets. This tyling is made possible by the fact that Plone provides CSS identifiers for just about everything on the page.

The result? A homepage that keeps surprising me.

A little know feature of zc.buildout can save you from such a dependency. 

The extends-cache.

In your buildout or in your global buildout (e.g. ~/.buildout/default.cfg") have something like the following

Ensure this directory is create and has the correct permissions.

Now when you run buildout, even if something you buildout.cfg extends from is not available on the network, your buildout will continue. What's more buildouts offline mode will now work, even if you extend from network .cfg files.

If you specifically want to get around the issue with the file download.zope.org/zopetoolkit/index/1.0.7/zopeapp-versions.cfg being unavailable due to a zope.org outage I've created a tar file which you can untar into your extends cache. This will should make your buildout work again.