Scaling Web Applications with HMVC
by Sam de Freyssinet | 48 comments | February 22, 2010
The last decade has been witness to the second iteration of web design and development. Web sites have transformed into web applications and rarely are new projects commissioned that do not involve some element of interactivity. The increasing complexity of the software being developed for the internet fuelled a requirement for structured and considered application design.
Today the most common design pattern for web software is the Model-View-Controller (MVC) pattern. The widespread adoption of the MVC design pattern was supported, in part, by the success and popularity of the Ruby on Rails framework. MVC is now synonymous with web application development across all platforms.
With the rising complexity of projects developed for the web, modern software for the web increasingly relies on dedicated services to perform processor intensive tasks. This has been encouraged further by the introduction of cloud services from Amazon, Google and several others enabling developers to considerably reduce the processor load on their servers. Each service is usually designed as a separate piece of software that runs in its own domain using its own resources.
When working with small budgets, it is generally much harder to convince clients of the benefits of funding more than one complete piece of software. In these situations I have found that many clients conclude that scalability is not a concern. They "look forward to the day when they will have to worry about scaling".
To reduce the initial investment, usually it is decided that the application should designed to be one holistic piece of software containing all the required features. This represents a potential point of failure if the software becomes very popular in a short timeframe. I have painful memories of refactoring existing codebases that have not scaled well. It can also be very costly in time and resources to re-architect software that not scaled well. Ideally applications should grow organically as required and without large sums of money being exchanged in the process.
Hierarchical-Model-View-Controller pattern
The Hierarchical-Model-View-Controller (HMVC) pattern is a direct extension to the MVC pattern that manages to solve many of the scalability issues already mentioned. HMVC was first described in a blog post entitled HMVC: The layered pattern for developing strong client tiers on the JavaWorld web site in July 2000. Much of the article concentrates on the benefits of using HMVC with graphical user interfaces. There has been some suggestion that the authors where actually re-interpreting another pattern called Presentation-Abstraction-Control (PAC) described in 1987. The article in JavaWorld provides a detailed explanation of how HMVC can aid in the design of desktop applications with GUIs. The focus of this article is to demonstrate how HMVC can be used to create scalable web applications.
HMVC is a collection of traditional MVC triads operating as one application. Each triad is completely independent and can execute without the presence of any other. All requests made to triads must use the controller interface, never loading models or libraries outside of their own domain. The triads physical location within the hosting environment is not important, as long as it is accessible from all other parts of the system. The distinct features of HMVC encourages the reuse of existing code, simplifies testing of disparate parts of the system and ensures that the application is easily enhanced or extended.
To successfully design applications that implement the HMVC pattern, it is critical that all of the application features are broken down into systems. Each system is one MVC triad within the larger HMVC application, independently managing presentation and persistent storage methods. Presently few frameworks are available that support HMVC without additional extensions, or use inefficient Front Controllers and dispatching. Kohana PHP version 3 is a framework that was designed from the ground up with HMVC at the core. I will be using Kohana PHP 3 for all of the code examples in this document.
Kohana 3 uses the core request object to call other controllers. Requests can be made internally to application controllers or externally to web services transparently using the same request class[1]. If a MVC triad is scaled out, the request only requires modification of one parameter.
<?php class Controller_Default extends Controller { public function action_index() { // Internal request example $internal_request = Request::factory('controller/action/param') ->execute(); // External request example $external_request = Request::factory('www.ibuildings.com/controller/action/param') ->execute(); } } |
Requesting internally requires a valid route path targeting a controller and action. Creating a request to an external resource is as simple as supplying the full URL. This feature makes internal and external requests quickly interchangeable, ensuring that scaling out triads is a relatively simple task.
Using the Kohana Request class to provide data from internal controllers may seem similar to action forwarding in other frameworks, such as the Zend Framework. In reality the two methods are quite different. Kohana Requests have the ability to operate as unique requests in isolation. Forwarding actions do not operate in this way, each invoked controller action exists within the originating request. To demonstrate this, consider the example below.
Default Controller – /application/controllers/default.php
<?php // First request controller class Controller_Default extends Controller { public function action_index() { // If the Request was a GET request if ($this->request->method === 'GET') { // Output POST, will print array (0) { empty } var_dump($_POST); // Create a new request for another resource $log = Request::factory('/log/access/'.$page_id); // Set the request method to POST $log->method = 'POST'; // Apply some data to send $log->post = array( 'uid' => $this->user->id, 'ua' => Request::user_agent('browser'), 'protocol' => Request::$protocol, ); // Log the access $log->execute(); // Output POST, will still print array (0) { empty } var_dump($_POST); } } } |
Log Controller – /application/controllers/log.php
<?php // Second request controller class Controller_Log extends Controller { public function action_access($page_id) { // When requested from the index action above // will print the posted vars from the second request // array (3) {string (3) 'uid' => int (1) 1, string (2) 'ua' => string(10) 'Mozilla ... ... var_dump($_POST); // Create a new log model $log = new Log_Model; // Set the values and save $log->set_values($_POST) ->save(); } } |
The example above demonstrates the independence afforded to the Request object. The initial request invokes the Default controller index action from a GET request, which in turn invokes a POST request to the Log controller access action. The index action sets three post variables which are not available to the global $_POST variable from the first controller. When the second request executes, $_POST has the post variables we made available to that request. Notice how after $log->execute(); has finished within the index controller, the $_POST data is not there. To do dynamic interaction of this kind within other frameworks requires creating a new request using a tool like Curl.
Gazouillement, the status service with a continental twist
To demonstrate the power of Hierarchical-MVC lets look at an example of a hypothetical status update service called Gazouillement, which works in a similar way to Twitter. Gazouillement has been designed around a service-oriented-architecture (SOA) to ensure the message and relationship engines are disparate from the web interface.
Traffic to the server will be relatively light initially. The service is new with an audience completely unaware of its presence. So it is safe to allow all of the application logic to execute on the same server for the time being.
Lets implement the controller to display a users homepage. A user homepage will show their most recent status messages, plus a list of people the user is following.
Index Controller – /application/controllers/index.php
<?php // Handles a request to gazouillement.com/samsoir/ class Controller_Index extends Controller { public function action_index() { // Load the user (samsoir) into a model $user = new Model_User($this->request->param('user')); // If user is not loaded, then throw a 404 exception if ( ! $user->loaded) throw new Controller_Exception_404('Unable to load user :user', array(':user' => $this->request->param('user'))); // Load messages for user in xhtml format $messages = Request::factory('messages/find/'.$user->name.'.xhtml') ->execute() ->response; // Load relationships for user in xhtml format $relations = Request::factory($user->name.'/following.xhtml') ->execute() ->response; // Apply the user index page view to the response // and set the user, messages and relations to the view $this->request->response = View::factory('user/home', array( 'user' => $user, 'messages' => $messages, 'relations' => $relations, )); } } |
Now a review of what Controller_Index::action_index() is doing. Initially the action attempts to load a user based on the user parameter of the url. If the user fails to load, a 404 page is displayed. A new request for messages is created using the users name property as a parameter within the request uri, asking for a response in xhtml format. Another request for the users relations is made in a similar manner, again in xhtml format. Finally a view object is created with the user, messages and relations responses set to it.
As the existing messages and relations are loaded using a new request, the entire application logic for each service remains abstracted from the web site. This architecture provides two significant advantages over traditional controller execution;
- The controller is not concerned with any part of the messages service execution logic. The controller only requires that the result is xhtml formatted. No additional libraries or extensions were loaded within the controller execution.
- Each controller is only responsible for one particular task, ensuring that writing unit tests for controllers is significantly less complicated.
Due to the abstraction currently demonstrated, it is impossible to see what the services are doing. So lets look at the messages service controller, starting with the route defined in the bootstrap to internally handle requests for messages. The Kohana Route class handles internal url parsing, mapping supplied
Route setup – /application/bootstrap.php
Route::set('messages', 'messages/<action>/<user>(<format>)', array('format' => '.w+')) ->defaults(array( 'format' => '.json', 'controller' => 'messages', )); |
This sets a route for the messages service, presently located within the main application domain. The url request for messages/find/samsoir.xhtml will be routed to the messages controller, calling the find() action and passing 'user' => 'samsoir', plus 'format => '.json' as parameters.
Messages Controller – /application/controllers/messages.php
<?php class Controller_Messages extends Controller { // Output formats supported by this controller protected $supported_formats = array( '.xhtml', '.json', '.xml', '.rss', ); // The user context of this request protected $user; // this code will be executed before the action. // we check for valid format and user public function before() { // Test to ensure the format requested is supported if ( ! in_array($this->request->param('format'), $this->supported_formats)) throw new Controller_Exception_404('File not found'); // Next test the username is valid $this->user = new Model_User($this->request->param('user'); if ( ! $this->user->loaded()) throw new Controller_Exception_404('File not found'); return parent::before(); } // This finds the users messages public function find() { // Load messages user 1:M messages relation $messages = $this->user->messages; // Set the response, using a prepare method for correct output $this->request->response = $this->_prepare_response($messages); } // Method to prepare the output protected function _prepare_response(Model_Iterator $messages) { // Return messages formatted correctly to format switch ($this->request->param('format') { case '.json' : { $this->request->headers['Content-Type'] = 'application/json'; $messages = $messages->as_array(); return json_encode($messages); } case '.xhtml' : { return View::factory('messages/xhtml', $messages); } default : { throw new Controller_Exception_404('File not found!'); } } } } |
The detail of how user messages are retrieved is demonstrated within the Controller_Messages controller. All of the methods and properties are exclusively related to the messages context, including the relationship to users. Lets step through the messages controller code to understand what is happening.
The request object initially invokes the before() method ahead of the defined action. This allows code to execute ahead of any action, normalising common controller tasks. The before() method first tests the file format requested is supported, followed by a test to ensure the user is valid. Assuming before() completed without exception, the request object will invoke the find() action. find() loads the users messages as a Model_Iterator object. It is important to note that the iterator will be empty if no messages in the relationship were found. Finally the messages iterator is passed to a parser method _prepare_response() that correctly formats the data for output and sets any headers that are required.
The tw