I was trying to make a glossary of words grouped by their first letter, but I wanted words starting with the letter é grouped with words starting with the letter e. No small feat you might imagine. Wrong.

dict = words.inject({}) do |dict, word|
  letter = word.chars.decompose[0..0].downcase.to_s
  dict[letter] ||= []
  dict[letter] << word; dict
end

The reason this works is that letters like é have a decomposed form in Unicode, this form consists of a latin letter and a accent modifier. I’m not sure what happens if you run Arabic through this code, but we’ll cross that bridge when we get there.

If you’ve installed the Rubyforge gem, which is a dependency of just about every gem in the wild through Hoe, then you might have see the OpenSSL::SSL::SSLError when trying to connect to a server over SSL. This is caused by http_access2 because it refuses to connect with SSL without verifying the certificates.

The problem is a little bit tricky because SOAP4r abstracts you away from http_access2 and that makes it hard to set the proper configuration on the HTTP client. Fortunately SOAP4r has a way to configure it’s subsystem.

Let’s assume that you’ve defined a SOAP::RPC::Driver somewhere.

class WebrApi < ::SOAP::RPC::Driver
  DefaultEndpointUrl = "https://webr.com/api"
  [snip]
end

There are two ways of making the error go away, the first is to tell http_access2 to stop verifying certificates and just get on with the connection.

proxy = WebrApi.new
proxy.options['protocol.http.ssl_config.verify_mode'] = OpenSSL::SSL::VERIFY_NONE

This is not recommended because one of the ideas behind SSL is that you can always verify that you’re talking to the correct server. If you want to keep this feature you will have to tell http_access2 which certificates to use.

proxy = WebrApi.new
client.options['protocol.http.ssl_config.verify_mode'] = OpenSSL::SSL::VERIFY_PEER
client.options['protocol.http.ssl_config.ca_file'] = '/etc/ssl/certs/certification_authority.crt'
client.options['protocol.http.ssl_config.client_cert'] = '/etc/ssl/certs/client.cert'
client.options['protocol.http.ssl_config.client_key'] = '/etc/ssl/certs/client.key'

You can find more advanced configuration examples in the SOAP4r source in sample/soap/ssl.

Changeset 6185, or so we assume, broke our fix for Basic Authentication in Safari when using RESTful Rails routes.

Luckily it wasn’t too hard to fix the fix and we’ve decided to make a plugin out of it so we can easily keep the fix in sync with Rails for all our projects. You can find safari_basic_auth_fix in our Subversion repository. The current fix makes sure that semicolons are encoded for every outgoing URL and it decodes the semicolons for all the incoming URLs.

Update: Changeset 6485 removed semicolons as a action separator, so this shouldn’t be a problem anymore once you upgrade to Rails trunk.

As you may have noticed OpenStruct#id always returns the object id of the OpenStruct instance, even when you set id.

>> o = OpenStruct.new :id => 2
=> #<OpenStruct id=2>
>> o.id
(irb):4: warning: Object#id will be deprecated; use Object#object_id
=> 9850940

Fortunately there is a simple way around this.

OpenStruct.__send__(:define_method, :id) { @table[:id] }

Now OpenStruct behaves like we want.

>> o.id
=> 2

Note that hash and object_id still work fine. You probably want to keep in mind that we’ve redefined the default OpenStruct behaviour and it might cause problems elsewhere.

Again a great meeting. I was amazed to see how much Rails is being used these days.

A little less than a year ago at the first meeting, only a handful of the developers visiting were using Rails professionally. This time I not only wasn’t able to find anyone who was not yet getting paid to write Rails code, but I also talked with folks from three different startups using Rails; soocial, SugarStats, and Wakoopa.

The last few weeks we noticed the dreaded question marks on our sites running against MySQL 5.0. We thought we did everything to make sure our servers, databases, tables, clients and connections understood UTF-8, but somehow connections to the database were reset back to Latin1 after some time.

Instead of trying to fix the problem in Rails/Ruby/libmysql I decided to squash the problem in the MySQL server configuration. By default we were seeing this:

mysql> SHOW VARIABLES LIKE 'character\_set\_%';
+--------------------------+--------+
| Variable_name            | Value  |
+--------------------------+--------+
| character_set_client     | latin1 | 
| character_set_connection | latin1 | 
| character_set_database   | latin1 | 
| character_set_filesystem | binary | 
| character_set_results    | latin1 | 
| character_set_server     | latin1 | 
| character_set_system     | utf8   | 
+--------------------------+--------+

So I set the following in /etc/mysql/my.cnf:

[mysqld]
character-set-server = utf8

[client]
default-character-set = utf8

Which forces all the encoding to go to UTF-8 by default:

mysql> SHOW VARIABLES LIKE 'character\_set\_%';
+--------------------------+--------+
| Variable_name            | Value  |
+--------------------------+--------+
| character_set_client     | utf8   | 
| character_set_connection | utf8   | 
| character_set_database   | utf8   | 
| character_set_filesystem | binary | 
| character_set_results    | utf8   | 
| character_set_server     | utf8   | 
| character_set_system     | utf8   | 
+--------------------------+--------+

We’re very happy to welcome Norbert Crombach to the Fingertips team. He started this morning.

Norbert has been working with Ruby on Rails since the very first public release when a good friend – who has since moved to Io – convinced him Ruby really was better than Perl.

Norbert always struck us as a talented and extremely bright developer who seemed to be doing very well on his own. He did however get fed up with the business side of running his own company, which made him decide to join us so he could spend more time doing actual development work.

For the next few months Norbert will be working remotely from Eindhoven while he’s trying to find a place to live here in Amsterdam.

Time for another one. The goal remains unchanged: a good chat about our experiences with Rails, Django, Seaside and other next generation web frameworks over a strong cup of coffee.

Photo by Rodney Ramdas

Please leave a comment to tell us you’ll be there or if you have any questions.

I’m sure we’ve all been there; trying to save a list of associated objects together with the instance they belong to. Fortunately with the right tools this can be really simple.

Let’s assume we have a question in a questionnaire and need to specify an arbitrary number of options to this question. We could construct the following database schema.

create_table :questions do |t|
  t.column :stem,        :text
end

create_table :options do |t|
  t.column :question_id, :integer
  t.column :label,       :text
  t.column :feedback,    :text
  t.column :correct,     :boolean, :default => false
end

There are basically three ways I know about to assign multiple options to a question from one page. The first option is to create a Question instance in the database before rendering the form so we can use remote_form to send an Ajax request and directly associate the Option instance. An advantage of this solution is that all data is directly saved to the database, which makes losing any data less likely. Another advantage is that the functionality is spread nicely over the responsible controllers. First you perform a create on the QuestionsController, after that you perform multiple creates on the OptionsController. The disadvantage is that you have to create a Question with default values or by circumventing the initial validation. This could generate some ‘blank’ Question instances in the database which you will have to garbage collect.

The second option is a slight variation on the first option. Instead of creating a Question to link the Options to you keep all the generated Option id’s in the session and link them to the Question after validation. This solution also benefits from nice separation of concern in the controllers, but on the downside it also leaves you with unlinked Option objects you will have to garbage collect.

The third option is to keep all the information about the Questions and Options in the DOM tree of your from and post it all at once. This used to be hard, requiring a lot of custom JavaScript and browser tricks. Fortunately WHATWG is here to help; Web Forms 2.0 defines a repetition model for repeating form controls. Browser support for these interaction models is years away (except for Opera 9, which has an experimental implementation), that’s why there’s a JavaScript library to accelerate the adaptation.

It’s really easy to use the library in your Rails application. Download the repetitionmodel library, extract the zipfile and put the javascript files in public/javascripts. When you’re done with that we can get back to coding our application. Include the JavaScript file somewhere in your views.

<%= javascript_include_tag 'repetition-model-p' %>

We just mentioned that we want to send the entire form at once, we can use validates_associated to make sure the Question is never saved when the options aren’t valid. This allows you to easily validate all the information from the form at once. The models to go with the database look something like this.

class Question < ActiveRecord::Base
  has_many :options, :dependent => :delete
  validates_presence_of :stem
  validates_associated :options
end

class Option < ActiveRecord::Base
  belongs_to :question
  validates_presence_of :label
end

Build the new and edit forms for the QuestionController like you would normally do. A the bottom of the form we will add the HTML to edit the options.

<h2>Options</h2>
<ol id="options">
  <%= render :partial => 'options/option', :collection => @question.options %>
  <%= render :partial => 'options/option', :object => Option.new,
    :locals => {:option_counter => '[option]'} %>
</ol>
<p><button type="add" template="option">New option</button></p>

The first render is for all the existing options, the second render is for what the repetition model calls a template, the template should always be the last in the list of controls.

In app/views/options/_option.rhtml we put the following. This partial doubles as a template for new controls and as a partial for existing options in the database.

<% if option_counter == '[option]' -%>
<li id="option" repeat="template" repeat-start="0">
<% else -%>
<li repeat="<%= option_counter %>">
<% end -%>
  <div>
    <div><label>Label</label></div>
    <%= text_area_tag "options[#{option_counter}][label]", option.label,
      :rows => 1, :cols => 40 %>
  </div>
  <div>
    <div><label>Feedback</label></div>
    <%= text_area_tag "options[#{option_counter}][feedback]", option.feedback,
      :rows => 1, :cols => 40 %>
  </div>
  <div>
    <label><%= radio_button_tag "options[correct]", option_counter, option.correct?,
      :id => "options_correct_#{option_counter}" %> Correct</label>
  </div>
  <div>
    <button type="remove">Delete option</button>
  </div>
</li>

In the HTML you see some extra attributes used by the JavaScript to determine what to do with them. When the partial is rendered with an new Option, the template is flagged by setting the repeat attribute to template. The id of this element is used as a handle for our controls, in our case this is ‘option’. The repeat-start attribute tells the javascript how many empty controls to generate initially from the template, we don’t want any so we’ve set it to 0. Note that we explicitly set the option_counter to ‘[option]’, this is the variable notation for the repetition model. When a new control is instanciated from the template this variable is replaced by the the index of the new control. The first control gets index 0, the second gets index 1 and so forth.

When the partial is rendered with a collection, the magic variable option_counter is set to the index of the collection every time the partial is rendered. We use this index to set the repeat attribute of the list item, this signals to the JavaScript that this is an already instantiated control. The JavaScript will start counting from the largest index when it instantiates a new control.

Finally we want the user to add and remove option controls in our page, this is done with the ‘New option’ and ‘Delete option’ buttons. Their type attribute signals what we want the JavaScript to do with the repetition blocks. In case of the ‘New option’ button the template attribute tells the JavaScript which template to instantiate.

In addition to managing the controls and template in the DOM tree, the JavaScript also takes care of disabling buttons on appropriate times and setting the CSS display property of the template to none. They really thought of everything.

The advantage of this solution is that you can use all the standard Rails tricks to keep your database clean. The biggest disadvantage is that the create method in your QuestionsController becomes a lot more complex.

Let’s hope native browser implementations follow quickly.

Today the Rails Core Team released Ruby on Rails 1.2. The long awaited new version is of course full of features and fixes. The thing we’re most excited about is the inclusion of ActiveSupport::Multibyte. But it doesn’t stop with multibyte support, Rails now ships with UTF-8 as a default in all parts of the framework, something we couldn’t have dreamed of a year ago.

We’re looking for a full-time Ruby on Rails developer. If you’re interested, please send email to thijs@fngtps.com

A few weeks ago Amazon launched UnSpun, a web application to collectively manage lists of all sorts.

During signup I was presented with the following.

I know Internet Explorer fixes a lot of broken encoding by guessing the true encoding for just about everything, maybe that’s why they never noticed during development?

I’ve had this problem myself on a few occasions. Because geographical information is commonly extracted from text files and loaded into a database you always have to be really careful to transcode any data extracted from text files to the same encoding as the database. In the case of ISO-8859-1/15, which is commonly used in west-european countries, there is a really simple oneliner to transcode to utf-8.

source.unpack('C*').pack('U*')

From the Mac OS X Leopard Technology Overview:

Leopard Server features a built-in installation of the powerful and productive Ruby on Rails web application framework. Ruby on Rails is a full stack framework optimized for sustainable productivity. Leopard Server will ship with Mongrel for simplified development and deployment of web-based applications.

That’s great. I only hope this bug will be fixed too. It would be somewhat ironic if you can’t use HTTP Basic Authentication in Safari with the new RESTful Ruby on Rails urls.

If you have access to the latest pre-release version of Mac OS X Leopard, please visit onautopilot.com/test;webkit and let us know if you get asked for a username and password or if it’s still broken.

Update: Tim found that you can make this work by url-escaping the semicolon. Add the following to your ApplicationController in app/controllers/application.rb:

# make HTTP Authentication work on Safari for RESTful Rails
def url_for(options = {}, *parameters_for_method_reference)
  result = super(options, parameters_for_method_reference)
  if request.env['HTTP_USER_AGENT'].to_s.include? 'AppleWebKit' 
    result.is_a?(String) ? result.gsub(';', '%3B') : result
  else
    result
  end
end

This is from a nice new app we’ve been working on for the last month or so. Any idea what it is and who we’re building this for?

Update: First right answer gets two great books. Can’t tell you what they are about without giving away the answer, I’m afraid…

Last week I posted a short screencast to show some features of ActiveSupport::Multibyte. After typing through the entire screencast twice I decided to automate the process. Screenager, the automated screencast typer, was born.

Download screencast (QuickTime, 544KB)

You can download Screenager from my personal Subversion repository, you will also need a recent version of ActiveSupport.

svn export svn.dwerg.net/screenager/trunk screenager
cd screenager
svn export --force dev.rubyonrails.org/svn/rails/trunk/activesupport/lib
./screenager --speed 2 www.fngtps.com/files/2/2006/10/activesupport.rb

The version currently in SVN evaluates the Ruby code with eval using a clean binding every time you start a new screenplay. I really wanted to use the Freaky Freaky Sandbox for this, but it’s in heavy development and didn’t run at all when I was coding this. Sandbox and multiline Ruby statements are planned for future versions.

It was a lot of fun chatting with other developers at our second ‘morning coffee’ meeting today. Here’s a short video to give you a feel of the event:

Download video (Quicktime, 1.7MB)

As of revision #5223 ActiveSupport::Multibyte is part of Rails. Now everyone can enjoy multibyte safeness in their applications. Needless to say we are really happy. To show some of the features of Multibyte’s String#chars proxy I’ve put together a short screencast.

If you have any questions about ActiveSupport::Multibyte, please consult the API documentation and the Trac Wiki first. Enjoy.

Download screencast (QuickTime, 1.9MB)

My new favorite idiom to use in Ruby is the proxy pattern. But I’m not using it to reduce the memory footprint or to manage some complex resource. I’m using it to keep the original class clean of my torrent of methods and to make my API easy to use.

One of the biggest problems with adding methods to a core class is the chance of collision with other libraries or code, in order to keep this chance low you don’t want to flood classes with new methods. Imagine we want to perform some textual operations on a String, for example ‘Textilize’ it.

require 'redcloth'
RedCloth.new('h1. Proxies!').to_html

But that doesn’t look very nice, especially if we have to duplicate it. Duplication will get us in trouble when we want to change the parameters we send to RedCloth or when we want to switch to another textile processor.

class Formatters
  def self.textilize(str)
    RedCloth.new(str).to_html
  end
end

Formatters.textilize 'h1. Proxies!'

Even though we packaged the code nicely in a method, I’m still not satisfied. This will require us to type that long classname before every call to textilize. And what if we want to perform a number of operations on the same string? Brackets will have to come into play and Lisp our code.

module Formatters
  def textilized
    RedCloth.new(self).to_html
  end
  
  def unnewlined
    self.gsub!(/\r\n/, "\n")
  end
end

String.send :include, Formatters

"h1. Proxies!\r\n".unnewlined.textilized #=> "<h1>Proxies!</h1>"

Very nice! But there are still some problems with this solution. Thje formatting methods might accidentally override other methods on String, especially if we define even more of these formatting methods. Even though the methods work on the string, they don’t really have much to do with the String class.

This is where the proxy class comes into the picture, it will only take one method on the String class and work as a portal to all our formatting code. In the process the proxy class is even going to make our solution pluggable.

require 'rubygems'
require 'redcloth'

module Formatting
  class Formatter
    
    def initialize(str)
      @str = str
    end
    
    def to_str
      @str
    end
    alias_method :to_s, :to_str
  end
  
  module StringExtension
    def format
      Formatting::Formatter.new self
    end
  end
  
  def register(mod)
    Formatting::Formatter.send :include, mod
  end
  module_function :register
end

module Formatting::Textilize
  def textilized
    @str = RedCloth.new(@str).to_html
    self
  end
end
Formatting.register Formatting::Textilize

module Formatting::UnNewline
  def unnewlined
    @str.gsub!(/\r\n/, "\n")
    self
  end
end
Formatting.register Formatting::UnNewline

String.send :include, Formatting::StringExtension

See how the well placed self enables us to to chain methods?

"h1. Proxies!\r\n".format.unnewlined #=> #<Formatting::Formatter:0x1087c4c @str="h1. Proxies!\n">
"h1. Proxies!\r\n".format.unnewlined.textilized.to_s #=> "<h1>Proxies!</h1>"

There we go, a nice proxy implementation with all the benefits of the original solution. Unfortunately there are some drawbacks, instead of a String instance the methods return a Formatter instance. The to_str keeps us safe in most cases, like concatenation, but not in all cases. Further cloaking measures are left as an exercise for the reader (hint: method_missing, the boat and Comparable).

If you want to see a really cool proxy implementation, check out ActiveSupport::Multibyte.

Since we really enjoyed the first ‘morning coffee’ meeting, it is time for another one. The goal remains unchanged: a good chat about our experiences over a strong cup of coffee.

We have decided, however, to broaden the scope beyong Rails developers. Anyone working with next-generation web application frameworks, like Django or even Seaside, is invited too.

Please leave a comment to tell us you’ll be there or if you have any questions.

Today I’ve been merging the Rails multibyte support from Julik’s unicode hacks plugin into the current edge source. After a few testruns my Mac Mini started complaining about the normalization conformance tests…

Finished in 102.769516 seconds.

460 tests, 353652 assertions, 0 failures, 0 errors

I guess I’ll have to fix the Rakefile so I don’t overheat everyone’s computer.

URoR stands for ‘Unicode Ruby on Rails’ which is a series on using Unicode with Rails. In this second article I’ll show you how to enable the (somewhat limited) UTF-8 support in Ruby and Rails. (first article)

Let’s break a string

Suppose you’re using the truncate helper like this:

<p><%= truncate 'Iñtërnâtiônàlizætiøn', 12 %></p>

The result is something like:

Iñtërn …

Because the helper truncates the string to 12 bytes, it slices the codepoint for the ‘â’ in halve. The result is an invalid sequence which can’t be rendered.

Fix this by adding the following to the top of config/environment.rb:

# Add basic utf-8 encoding support 
$KCODE = 'UTF8'

And the result will be:

Iñtërnâti…

The string is now truncated to 12 codepoints instead of 12 bytes.

What’s happening here?

Setting KCODE to 'UTF8' tell Ruby that your source code is encoded as UTF-8. Some libraries like CGI and some parts of Rails look at KCODE to find out if they need to process strings in a UTF-8 friendly way.

You can now also require the jcode library you get some basic UTF-8 encoding support in Ruby. More about this in a future article.

Not all is good

Although it’s great that truncate has been