Globalizing Mephisto (Part 2)

Welcome to the second part of the Globalizing Mephisto article series where I discuss various deployment issues surrounding the use of this plugin.

• Part 1 – Installation and usage for single site mephisto
• Part 2 – Deployment and production issues
• Part 3 – Installation and usage for multi site mephisto
• Part 4 – Using a path parameter to define the locale

I’ll be walking you through, the deployment issues that arise with the setup enforced by the plugin.

Document Roots and Virtual Hosts

As I mentioned in my previous article, the mephisto_i18n plugin maintains the current locale of a session by assuming that the top level subdomain of the host indicates the language part of the locale.

e.g.

  example.com => base locale is currently active (e.g. English )
  es.example.com => 'es' locale is currently active (e.g. Spanish )
  fr.example.com => 'fr' locale is currently active (e.g. French )

So the web server needs multiple virtual hosts, one for each of the languages to be supported.

e.g

  * RAILS_ROOT/public               (Holds the base locale's doc root)
  * RAILS_ROOT/public/es             (Holds the 'es' locale's doc root)
  * RAILS_ROOT/public/fr             (Holds the 'fr' locale's doc root)

In the previous article we’ve seen how the rake tasks provided by the plugin, make setting up these doc roots a piece of cake, and from now on I’m assuming that you’ve got all that setup and mephisto is globalized and running well in development mode.

Setting up the web server

So we need to setup the web server to hande multiple document roots. You need to have a server that has support for name-based virtual hosts. These days any decent web server should have this baked in.

Mine’s a cup of nginx

My current preferred web server is Nginx (go here for details in other languages) so I’m going to show you the configuration I use for this particular web server.

Of course, any web server that can handle virtual hosts will do. e.g. Apache , Lighttpd

The nginx config file usually resides at /usr/local/nginx/conf/nginx.conf

The following nginx.conf assumes there are 4 different virtual hosts. One each for the en,es,ca and fr locales. It also assumes that Catalan (ca) is the base locale.

Additionally, it redirects all requests for any unrecognised subdomains (as well as the base locale subdomain) to www.

e.g. requests coming to somesubdomain.example.com will be redirected to www.example.com Also if we assume catalan is the base locale, requests for ca.example.com will be redirected to www.example.com

Just leave the last virtual host out if you don’t want this redirection to occur.

Also, I’m showing the whole nginx.conf here including the virtual hosts, but what i like to do for production is to put the different server directives for a specific site in their own file and use an include directive to pull it in.

e.g. /usr/local/nginx/conf/apps/saimonmoore.net.conf

and just add:

include apps/saimonmoore.net.conf;

within the http directive. (I usually add it at the bottom). Within each app config I also include the call to the upstream directive

upstream mongrel_saimonmoore.net {
   server 127.0.0.1:8080;
   server 127.0.0.1:8081;
}

Here is the full example nginx.conf file though:

  # user and group to run as
  #user  www-data www-data;
  # number of nginx workers
  worker_processes  2;

  # pid of nginx master process
  pid        /var/log/nginx/nginx.pid;

  # Number of worker connections. 1024 is a good default
  events {
      worker_connections  1024;
  }

  # start the http module where we config http access.
  http {
    # pull in mime-types. You can break out your config
    # into as many include's as you want to make it cleaner
      include       conf/mime.types;
      # set a default type for the rare situation that
      # nothing matches from the mimie-type include
      default_type  application/octet-stream;

      # configure log format
      log_format  main  '$remote_addr - $remote_user [$time_local] $status '
                        '"$request" $body_bytes_sent "$http_referer" '
                        '"$http_user_agent" "$http_x_forwarded_for"';

      # main access log
      access_log  /var/log/nginx/access.log  main;
      # main error log
      error_log  /var/log/nginx/error.log info;

      # no sendfile on OSX uncomment
      #this if your on linux or bsd
      #sendfile        on;

      # These are good default values.
      tcp_nopush     on;
      keepalive_timeout  65;
      tcp_nodelay        on;

      # this is where you define your mongrel clusters.
      # you need one of these blocks for each cluster
      # and each one needs its own name to refer to it later.
      # To determine how many mongrels are necessary follow:
      # mongrel.rubyforge.org/docs/how_many_mongrels.html
      upstream mongrel_example.com {
          server 127.0.0.1:8080;
          server 127.0.0.1:8081;
      }

      # output compression saves bandwidth
      gzip on;
      gzip_comp_level 2;
      gzip_min_length  1100;
      gzip_buffers     4 8k;
      #gzip_proxied any;
      gzip_types       text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;

      #server_names_hash_bucket_size 32/64/128
      server_names_hash_bucket_size 64;

      #vhosts for example.com
      #
      #root /var/www/apps/example.com/current/public;
      #Spanish locale: es subdomain
      server {
          listen       80;

          server_name es.example.com;

          root /var/www/apps/example.com/current/public/es;

          access_log  /var/log/nginx/host.access.log  main;
          rewrite_log on;

          if (-f $document_root/maintenance.html){
            rewrite  ^(.*)$  /maintenance.html last;
            break;
          }

          location /index2 {
            rewrite (.*) / permanent;
          }

          location / {
            proxy_set_header  X-Real-IP  $remote_addr;
            proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_redirect false;

            # If the file exists as a static file serve it directly without
            # running all the other rewite tests on it
            if (-f $request_filename) {
              break;
            }

            # check for index.html for directory index
            # if its there on the filesystem then rewite
            # the url to add /index.html to the end of it
            # and then break to send it to the next config rules.
            if (-f $request_filename/index.html) {
              rewrite (.*) $1/index.html break;
            }

            # this is the meat of the rails page caching config
            # it adds .html to the end of the url and then checks
            # the filesystem for that file. If it exists, then we
            # rewite the url to have explicit .html on the end
            # and then send it on its way to the next config rule.
            # if there is no file on the fs then it sets all the
            # necessary headers and proxies to our upstream mongrels
            if (-f $request_filename.html) {
              rewrite (.*) $1.html break;
            }

            if (!-f $request_filename) {
              proxy_pass mongrel_example.com;
              break;
            }
          }
      }

      #French locale: fr subdomain
      server {
          listen       80;

          server_name fr.example.com;

          root /var/www/apps/example.com/current/public/fr;

          access_log  /var/log/nginx/host.access.log  main;

          if (-f $document_root/maintenance.html){
            rewrite  ^(.*)$  /maintenance.html last;
            break;
          }

          location /index2 {
            rewrite (.*) / permanent;
          }

          location / {
            proxy_set_header  X-Real-IP  $remote_addr;
            proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_redirect false;

            if (-f $request_filename) {
              break;
            }

            if (-f $request_filename/index.html) {
              rewrite (.*) $1/index.html break;
            }

            if (-f $request_filename.html) {
              rewrite (.*) $1.html break;
            }

            if (!-f $request_filename) {
              proxy_pass mongrel_example.com;
              break;
            }
          }
      }

      #English locale: en subdomain
      server {
          listen       80;

          server_name en.example.com;

          root /var/www/apps/example.com/current/public/en;

          access_log  /var/log/nginx/host.access.log  main;

          if (-f $document_root/maintenance.html){
            rewrite  ^(.*)$  /maintenance.html last;
            break;
          }

          location /index2 {
            rewrite (.*) / permanent;
          }

          location / {

            proxy_set_header  X-Real-IP  $remote_addr;
            proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_redirect false;

            if (-f $request_filename) {
              break;
            }

            if (-f $request_filename/index.html) {
              rewrite (.*) $1/index.html break;
            }

            if (-f $request_filename.html) {
              rewrite (.*) $1.html break;
            }

            if (!-f $request_filename) {
              proxy_pass mongrel_example.com;
              break;
            }
          }
      }

      # Catalan locale: www
      server {
          listen       80;

          server_name www.example.com;

          root /var/www/apps/example.com/current/public;

          access_log  /var/log/nginx/host.access.log  main;

          if (-f $document_root/maintenance.html){
            rewrite  ^(.*)$  /maintenance.html last;
            break;
          }

          location /index2 {
            rewrite (.*) / permanent;
          }

          location / {
            proxy_set_header  X-Real-IP  $remote_addr;
            proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_redirect false;

            if (-f $request_filename) {
              break;
            }

            if (-f $request_filename/index.html) {
              rewrite (.*) $1/index.html break;
            }

            if (-f $request_filename.html) {
              rewrite (.*) $1.html break;
            }

            if (!-f $request_filename) {
              proxy_pass mongrel_example.com;
              break;
            }
          }
      }

      #Catalan locale: tld
      server {
          listen       80;

          server_name example.com;

          root /var/www/apps/example.com/current/public;

          access_log  /var/log/nginx/host.access.log  main;

          if (-f $document_root/maintenance.html){
            rewrite  ^(.*)$  /maintenance.html last;
            break;
          }

          location /index2 {
            rewrite (.*) / permanent;
          }

          location / {
            proxy_set_header  X-Real-IP  $remote_addr;
            proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_redirect false;

            if (-f $request_filename) {
              break;
            }

            if (-f $request_filename/index.html) {
              rewrite (.*) $1/index.html break;
            }

            if (-f $request_filename.html) {
              rewrite (.*) $1.html break;
            }

            if (!-f $request_filename) {
              proxy_pass mongrel_example.com;
              break;
            }
          }
      }

      #Catalan locale: any other subdomain (redirected to www)
      server {
          listen       80;

          server_name ca.example.com *.example.com;
          rewrite  ^(.*) www.example.com$1  permanent;
      }
  }

By the way, you can find this file within the resources/config directory in the mephisto_i18n plugin.

Here doggy…

So once we’ve got the web server all configured, it’s time to take the dog for a walk. :) Unfortunately, this pup is going to be a tad stroppy. Right, enough with the dog lingo, let’s get down to details.

Mephisto is great. However, it likes to serve files.

- That’s not bad is it, I hear someone say.

Not at all, except when you’ve got multiple document roots. Your web server should be configured to forward any request for non-static content to mongrel. That’s fine, mongrel receives the request, and creates a rails process to handle it.

However, by default Mongrel will also attempt to serve static content if it receives a request for something available in the RAILS_ROOT/public (your application’s document root).

This “feature” allows it to be used as is (without the need for a separate web server) while developing normal applications and works great.

But imagine the scenario where a request comes in for:

example.com

1. The web server checks the base locale’s doc root (RAILS_ROOT/public), doesn’t find any index.html and forwards the request to mongrel.

   Let me represent that as:

example.com => [nginx] RAILS_ROOT/public/index.html ? serve : forward (forward)

(Is that understandable?)

1. Mongrel also checks but doesn’t find any index.html in the application doc root (RAILS_ROOT/public – which is the only one mongrel knows about), so it creates a rails process which caches the result as RAILS_ROOT/public/index.html.

This is then returned to the web server for serving to the client’s browser. i.e.

example.com/index.html (via nginx) => [mongrel] RAILS_ROOT/public/index.html ? serve : pass_to_rails (pass_to_rails)

Ok, so now we’ve cached the home page for the base locale. Let’s refresh the browser and ask for the same page. In short hand:

example.com => [nginx] RAILS_ROOT/public/index.html ? serve : forward (serve)

i.e. the web server correctly found RAILS_ROOT/public/index.html in the doc root for the base locale and served it. “yoopee” for page caching.

But now our viewer wants to see the home page in spanish, so we clicks on the “switch locale” link in the home page which sends a request for:

es.example.com

The web server receives this and correctly identifies the doc root for the es domain as RAILS_ROOT/public/es. i.e.

example.com => [nginx] RAILS_ROOT/public/es/index.html ? serve : forward (forward)

index.html in the RAILS_ROOT/public/es doc root isn’t found so it forwards to mongrel.

Mongrel being a very alert puppy, checks to see if it can serve any static content.

Since it hasn’t got any idea about multiple doc roots, it checks the only doc root it knows about which is RAILS_ROOT/public. i.e.

es.example.com/index.html (via nginx) => [mongrel] RAILS_ROOT/public/index.html ? serve : pass_to_rails (serve)

Mongrel finds the index.html created for the base locale, and although alert, it’s not a very bright puppy so it goes ahead and serves that rather than as we’d hope; create a rails processes to cache:

es.example.com/index.html => RAILS_ROOT/public/es/index.html

So what has happened here is that the client asked for

es.example.com/index.html (RAILS_ROOT/public/es/index.html)

but actually got served

example.com/index.html (RAILS_ROOT/public/index.html)

Lay!

This is a bit of a problem :) and I’m sure there are a number of ways to solve this (.e.g. use fcgi/scgi) but if you want to keep using mongrel, then you need to be able to tell mongrel that you don’t ever want it to serve static content ever again.

This way the web server is responsible for serving existing static content and mongrel is responsible for creating the static content only.

Currently (as far as I know), mongrel doesn’t have this capability nor does it support virtual hosts.

Ideally it should support virtual hosts but as it was a lot simpler, I created my own hacked version of mongrel (and the mongrel_cluster gem) to allow for an extra configuration option (dont_serve_static default: false).

When this is set to true, mongrel will diligently create rails processes for every single request that it receives, which solves our little problem nicely.

You can find these hacked versions of mongrel here

To install:

#kill mongrel if its running
sudo /etc/init.d/mongrel_cluster stop
sudo gem uninstall mongrel mongrel_cluster
cd /tmp
wget www.webtypes.com/projects/files/basecamp/webtypes/webtypesthebusiness/mongrel_with_dont_serve_static.tar.gz
tar xvfz mongrel_with_dont_serve_static.tar.gz
cd mongrel_with_dont_serve_static
rake package
sudo gem install pkg/mongrel-1.0.1.gem
#Successfully installed mongrel, version 1.0.1

the same for mongrel_cluster :

cd projects/mongrel_cluster/
rake package
sudo gem install pkg/mongrel_cluster-0.2.1.gem
#Successfully installed mongrel_cluster, version 0.2.1

Once you’ve got that installed, then just modify you mongrel_cluster.yml to something like:

---
user: saimon
cwd: /var/www/apps/saimonmoore.net/current
port: "8080" 
environment: production
group: deploy
address: 127.0.0.1
pid_file: log/mongrel.pid
servers: 2
dont_serve_static: true

Note the dont_serve_static: true. That’ll tell mongrel to just do as it’s told and create rails processes.

Conclusion

Well, between these two articles, you should be able to globalize mephisto quite easily. If you have any doubts, questions, feedback or bug reports then feel free to leave a comment here.

Hope this provides a decent i18n solution for people using Mephisto.

P.S. I’m planning on writing a third article about tips an tricks with localising your templates and other tid bits. Perhaps, we could offer alternative globalized versions of the main mephisto themes?

Want a globalized multi site mephisto? Read Part 3

Saimon Moore (also in Greek, Spanish & Catalan :)