Recently I fell in love with Pretender, the mock server library in Javascript, so I decided to record a screencast showing how to use it in an Ember.js integration test:

The source code for the login application is on github. The finished version is in the pretender branch.

I’ve found that a lot of people don’t know how powerful the Chrome Developer Tools are, especially when paired with the Ember Inspector. I recorded this short screencast to show a handful of tricks you can use when debugging Ember.js applications:

The source code to the Bug Tracker project is on github.

If you’re interested in learning Ember.js, I’m happy to announce that I’m partnering with my friends at Unspace to teach an introductory class on Saturday September 13th here in Toronto.

We did one of these last year and it was super fun. This time around we’re just focusing on the beginner stuff, so if you or anyone you know are new to Ember.js, you should check it out.

All the information you need is on the Embergarten website!

Once upon a time it used to be difficult to create integration tests in Ember.js. Fortunately, the framework has come a long way and it’s now really easy to get integration testing working in your application. This screencast shows how to set it up with ember-cli:

There is some boilerplate code required that you’ll need at the top of your integration test files if you want to do it yourself. Here it is:

import startApp from 'vault/tests/helpers/start-app';
var App;

module('Integration - Secret', {
  setup: function() {
    App = startApp();
  },
  teardown: function() {
    Ember.run(App, 'destroy');
  }
});

Time for another screencast! This one is shorter and shows how easy it is to wrap a jQuery plugin in an Ember.js component for reuse.

One of my more popular blog entries is on using Ember.js without Ember Data. Recently I’ve been going through my old entries and making sure they don’t have any glaring mistakes, and I realized this would be a good opportunity to convert my emberreddit project to ember-cli.

This screencast shows how you can build an Ember.js application without using Ember Data. It starts off simple and then shows how to build advanced stuff like an identity map yourself.

It assumes some knowledge of Ember. The guides and docs for Ember.js are available at: emberjs.com/guides/

To install NodeJS go to: nodejs.org/

The excellent ember-cli is available at: iamstef.net/ember-cli/

And finally the code created in this screencast can be found here: https://github.com/eviltrout/emberreddit

Thanks to Stefan Penner, Jo Liss and the Ember Core team for making this amazing framework and tools. Also thanks to Erik Bryn for showing me a few tips about the model hook!

I had a lot of fun making the screencast so I will likely do more in the future. Drop me a line and let me know what future screencasts you’d like to see.

Javascript is a fantastic example of how something, despite having visible warts and very poor design, can dominate the tech landscape. Nobody uses Javascript because it’s a beautiful language; they use it because it’s ubiquitous. Its warts are now well understood and most have workarounds.

An amazing omission in Javascript’s design is the lack of a built-in module system. As more projects used Javascript and shared more code, the need for a robust module system became necessary. Two contenders sprung up, Asynchronous Module Definition (AMD) and CommonJS (CJS). The former is much more popular with browser applications and the latter is much more popular with server applications written in node.js.

Having two major standards for defining modules led to a technological holy war in the Javascript community akin to the vim/emacs arguments of the editor world. It wasn’t pretty.

Fortunately, there is light at the end of the tunnel. TC39 has been hard at work on the next version of Javascript, called ES6 (short for EcmaScript 6). One of the major features of ES6 is a standard syntax for handling modules in Javascript.

A simple example of ES6 modules

By default anything you declare in a file in a ES6 project is not available outside that file. You have to use the export keyword to explicitly make it available. Here’s an example of how to export a user class:

// user.js

var localVariable = 123;  // not visible outside this file

export default function User(age) {
  this.age = age;
}; // can be imported by other files

And now if we wanted to use the User class in another file:

// user-details.js

import User from 'user';

var evilTrout = new User(35);

Pretty simple, isn’t it? There are many more examples of the syntax here if you are curious about other ways it can be used.

When will it be available in browsers?

In the past, it was very risky to use new Javascript features before they were standardized and widely available in browsers. You’d never know if someone was using an old or incompatible browser and it would cause your code to crash and burn.

These days, thanks to the Extensible Web movement, people are working hard at making it so that developers can try out advanced features before they’re compatible in all browsers.

The great news is you can use ES6 modules today! You just have to run your code through a transpiler. The transpiler will convert your ES6 modules into Javascript that browsers can understand today. In the future, when the browsers understand ES6 modules natively, you’ll be able to stop transpiling and it will just work.

The transpiler I’ve been using lately is es6-module-transpiler from Square. If you check out their build tools section you’ll see they’ve got integration stories for all the major Javascript build tools.

If you are using Rails on the server side, Dockyard has created an easy to use Gem version of it that you should be able to drop into your project.

ES6 Modules and Ember.js

The Ember community has bet big on ES6 modules. For example, if you are using Ember App Kit to structure your project, it includes ES6 module support via transpiling out of the box.

Recently, Robert Jackson converted the Ember source code to ES6 modules. This means that, if you have things set up properly in your development environment, you can import just the parts of Ember.js that you want to use and end up with a potentially smaller runtime.

ES6 modules integrate quite beautifully in an Ember project. If you’re not using ES6 modules, the standard way of making parts of your application available for discovery was by hanging them off your application’s global namespace. For example:

// app/controllers/user.js
App.UserController = Ember.ObjectController.extend({
  // ... controller code
});

Then if you transitioned to the user route, Ember would search for a UserController on your App object. This actually works quite well, but making everything available globally makes it too easy for developers to reach into components they have no business reaching into. If you make it easy for a developer to do the wrong thing, they will do it.

To contrast, if you are using Ember with an ES6 application you can define your user controller this way:

// app/controllers/user.js

export default Ember.ObjectController.extend({
  // ... controller code
});

Ember’s new resolver will then look for the module exported from the app/controllers/user path and will wire it up for you automatically.

Going Forward

I’ve found that since I started using ES6 modules in my projects that their code bases are a lot cleaner and more organized. It also just feels awesome to be using a standard before it’s widely available.

I’ve got a branch of Discourse that I am converting to ES6 modules one at a time. The bad news is that Discourse has hundreds of files to convert, so it will be some time before we are 100% on ES6. The good news is, with a little duct tape in our custom resolver, the application can run with some modules in the global Discourse namespace and some in ES6 format. I’m hoping to merge it into master shortly so our contributors can help with the converting efforts.

My advice is to not wait for browsers to implement these modules; start hacking today and put your project ahead of the curve. There are other ES6 features that can be transpiled too, and I’m excited to try some of those out too!

How many times has the following happened to you?

You go to a web site and it asks you to create an account. You fill out a form with all the obvious fields and hit submit. The page refreshes and shows you the form again.

Phone Number is required

Well, that’s annoying. There was no indication that the site needed your phone number. You prefer not to give out your phone number to every web site, but this one is run by a company you trust, so you scroll down and fill it out. You submit the form again.

Password is required

What the heck!? You already entered a password! You scroll down to the form and see that the fields are now empty. It turns out that even though you filled out the password fields the first time, after you missed the phone number they were cleared. You fill them out and submit the form again.

Username is not available

At this point you’re ready to throw your computer out the window. There are fewer things more frustrating than trying to get your data into the exact shape a web site wants, especially if it clears fields every time you fail.

Another annoyance: just about every time I order something online, right after I input my credit card information, I am presented with a spinner animation and the text “Do not hit the back button or refresh your browser!” It’s terrifying that a company that is taking my money over the Internet can’t handle me refreshing the page without charging my card twice.

The sad thing is that both of these problems are totally solvable. In fact, they’ve been easy to solve for over a decade. Yet you still see them all the time.

Lessons from Live Reloading

Recently I spent some time playing with Ember App Kit. Ember App Kit is a suggested project structure for your applications built by Stefan Penner (and a bunch of other awesome developers). It’s really great and if you’ve never tried it out you should!^[1]

One feature Ember App Kit includes out of the box is support for connect-livereload, which automatically reloads your changes in your browser whenever you save a file.

It sounds like a minor thing, but after using it for a few hours having to manually hit Cmd-R feels like a chore. It’s a great little productivity booster.

Live Reload also has a side effect: it encourages you to make your application refresh resistant.

In my application I had a multi-step wizard where you had to enter many form fields at once, and I found it so frustrating to have all my form data pulled out from underneath me every time I hit save.

The frustration led me to persist the form data temporarily in localStorage. Once I did that, every time my application refreshed it looked exactly the same as it did before. It was probably a grand total of 10 minutes of work, and my application was much more resilient for it.

The Refresh Test

A great experiment to perform on a Javascript heavy application is to simply refresh the page and see where you end up. Does it look the same as before? Did you lose any work?

Users will put up with losing a small amount of state on refresh, for example if they’d expanded a menu and it’s suddenly collapsed, but you should never throw away what they were working on.

One thing I love about Ember.js is that its router makes you think in terms of URLs, which gives you a great head start for handling refreshes and the dreaded back button.

Many people think of URLs as files, because in the past requesting a path like profile.php?id=eviltrout meant you really wanted a file on the server called profile.php with the parameter eviltrout.

I find it’s better to think of a URL as the serialized state of your application. A path of /profiles/eviltrout should mean “I’m viewing eviltrout’s profile.”

If you’re building a Javascript application and the URL is not changing as your users navigate around, that is practically begging to give them a bad experience at some point. Not only can they be easily frustrated if they hit the back or refresh buttons, but they won’t be able to bookmark or share links with others.

I’m not suggesting that the URL contain every possible interaction a user can make; if you do that you will end up with a huge headache and a meaningless URL. Instead, you should focus on the most important things a user will want to see when the page is refreshed. For example, on Discourse we maintain a user’s scroll position in a topic by changing the URL as they scroll.

Going forward, I’m going to make sure that all my applications handle refreshing elegantly and I recommend you do too! If you’re interested in more on this topic, Tom Dale has a great talk on this called Stop Breaking the Web.

1. If you’re a fan of using bleeding edge stuff, check out Stefan Penner’s ember-cli and Jo Liss' broccoli. Ember App Kit works well today but those projects are a glimpse of the future of where Ember development is headed.

It’s been possible to embed comments from a Discourse forum on to other sites for a while now, but the only publishing software we supported was Wordpress.

Many people, myself included, don’t use Wordpress to run their sites. This very blog is a series of static HTML documents that are generated by Middleman. (Like Discourse itself, my blog is open source; you can find the code here so feel free to submit pull requests!).

Well… Good News Everyone! It’s now possible to embed Discourse content into any HTML document. In fact, as you can see below, the comments on this very site are now handled by a Discourse instance I set up.

How it Works

You can now configure Discourse to support embedding. Once you’ve done that, you paste a small snippet of Javascript into your site template and you’re off to the races.

Discourse will create topics for your blog entries if you provide it with an RSS or Atom feed. It’s also smart enough to update them if the content changes, say for corrections or typos. When any users of your forum post comments, they will automatically be inserted into your blog.

If your site does not have an XML feed, Discourse will create topics based on your page contents using a readability algorithm. It’s remarkably accurate for those cases when your site doesn’t have a feed.

How to Set up Embedding

1. Install a Discourse Forum. We currently recommend using discourse_docker to do this. If you have ruby on your computer, you can even do it via a command line wizard.

2. Navigate to the Embedding Site Settings. The URL will be something like:

   discourse.yourdomain.com/admin/site_settings/category/embedding

3. Configure the settings appropriately. Here’s how it looks for my blog:

   

4. In your blog template, paste in the following snippet of Javascript and a <div> tag for where you’d like the comments to appear:

   <div id="discourse-comments"></div>
   
   <script type="text/javascript">
     var discourseUrl = "fishtank.eviltrout.com/",
         discourseEmbedUrl = 'eviltrout.com/link-to-blog-entry.html';
   
     (function() {
       var d = document.createElement('script'); d.type = 'text/javascript'; d.async = true;
         d.src = discourseUrl + 'javascripts/embed.js';
       (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(d);
     })();
   </script>
   

   Configuration

   1. You will have to change the discourseUrl variable so it points to the root URL of your Discourse forum.
   2. Make sure discourseEmbedUrl is the canonical URL to the page where you are embedding Discourse. You will want to make sure this uniquely identifies the page. For my blog template, I use the following code to link to the article: discourseEmbedUrl = 'eviltrout.com<%= current_page.url %>'
      
      

5. Reload your blog. Your blog posts should start syncing over to your forum and any comments should be imported into your blog. It’s that easy!

Going Forward

eviltrout.com has been running this set up for a few weeks now and it’s been great! Of course, like with any new feature of Discourse we’d love to hear your feedback so we can iterate on it and make it better. Let us know how it works for your use case.

Everything you render in a browser, whether it’s a blog post or a tweet or a video, has a performance cost.

At the very least, you will be asking the browser to render a handful of tags and text elements that make up your user interface. That structure, a subtree in the browser’s DOM, can be quite complicated and memory intensive.

The more tags and elements you render, the slower the browser is going to perform, and the more memory it is going to use to do it. It follows that if you give the browser less work to do, it will do it faster.

This principle holds true even if you are using a browser application framework like Ember.js. Every data binding you make between an element and an object has a cost. If you reduce your bindings and views, your interface will feel snappier.

Long Lived Applications

Discourse makes heavy use of infinite scrolling. If a user is reading a long topic with many posts, new posts will stream in asynchronously from the server as the scroll position approaches the end of the browser’s viewport.

For shorter topics, adding all that extra content to the DOM was not a performance issue. Modern browsers, even on mobile devices, could handle rendering of hundreds of posts of formatted text without breaking a sweat.

However, as Discourse installs began to see heavy use, we found some topics had thousands of posts, and some users would read many of them in one sitting. All of those inserted posts started having a negative effect; browsers would often start to feel “choppy” and could even crash, leaving users frustrated.

Cloaking Offscreen Content

The obvious solution to this problem was to unload content from the DOM as it scrolled offscreen and render it again if it came back onscreen.

The issue with this is that if you remove an element from the DOM, the browser will reflow, and all the other content underneath it will jump back upwards. In order to prevent the browser viewport from moving, you have to replace the element with a simpler one that has the exact same height as the element it’s replacing.

One of Ember’s strengths is how it breaks down your UI into a hierarchy of views. You have your ApplicationView which contains your TopicView and a collection of PostViews and so forth.

I took advantage of this structure and implemented a CloakedView class.

The idea is any of your views can be contained in a cloak. When onscreen, the cloak renders the contained view. When offscreen, the cloak will copy the height of its rendered content and unload it.

A PostView doesn’t care if it’s cloaked or not. It should only be concerned with how to render a post. We can choose to cloak when we display a list of posts with a special helper. So instead of rendering a collection of Posts like so:

{{collection content=topic.posts itemViewClass="postView"}}

We can drop in a replacement like so:

{{cloaked-collection content=topic.posts cloakView="post"}}

ember-cloaking

After implementing cloaking, there was an immediate drop of 30% RAM usage in long Discourse topics. Scrolling also remains smooth even if many posts are browser in one sitting. We’ve been running it in production for about a month and it’s been a huge win!

I’ve extracted the cloaking logic into a library called ember-cloaking.

For now it only works with vertical scrolling, but if you are doing a large amount of horizontal scrolling I’m sure it could be adjusted to work without too much effort.

The fact that I was able to implement this functionality in a generic way without much code is a testament to Ember’s excellent design.

If your application is rendering many items in the browser at once, especially if you are implementing infinite scrolling, you should give it a shot and let me know how it works for you!

One thing I’m really proud of is that when we launched Discourse, we had first class Internationalization (i18n) support ready to be used. Our first release only English, but thanks to our community we have 18 localizations of our software in progress! Here’s what Discourse looks like in Simplified Chinese:

On the server side, Discourse uses Rails' built in i18n support. It has been around for a long time and works easily so I won’t go into that. Check out the documentation for your server side framework of choice for more.

I18n in Ember.js

Our client side application is written in Ember.js, which doesn’t have built in support for i18n. However, it’s not difficult to add it in.

We use i18n-js, a project whose goal is to bring Rails translation support to Javascript. Don’t worry if you don’t use Rails on the server side. You can use all of the code in this post outside of Rails if you like. The Javascript code in 1i8n-js is all you’ll need.

Once you’ve included i18n-js in your project, you will have access to an I18n object in your javascript code to perform translations with. The first thing you’ll need to do is include a translations.js file that includes all your translations. Here’s how a simple one could look:

I18n.translations = {
  en: {
    hello: 'hello',
    cookieCount: {
      one: 'You have {{count}} cookie.',
      other: 'You have {{count}} cookies. Yum!'
    }
  },

  fr: {
    hello: 'bonjour'
    cookieCount: {
      one: 'Vous avez {{count}} biscuit.',
      other: 'Vous avez {{count}} biscuits. Le Yum!'
    }
  }
};

And then if you wanted to output a translation you can use the i18n.t function:

console.log(I18n.t('hello'));   // outputs hello because the default locale is `en`

I18n.locale = 'fr';
console.log(I18n.t('hello'));   // outputs bonjour

In an Ember app though, you’ll want to be able to access those translations in your handlebars templates. To do this, you’ll need to define a helper. You can just copy and paste this code into your app:

Ember.Handlebars.registerHelper('i18n', function(property, options) {
  var params = options.hash,
      self = this;

  // Support variable interpolation for our string
  Object.keys(params).forEach(function (key) {
    params[key] = Em.Handlebars.get(self, params[key], options);
  });

  return I18n.t(property, params);
});

Now your templates are ready to be translated:

<h1>{{i18n hello}}</h1>

<p>{{i18n cookieCount count=user.cookies.length}}</p>

Note that the I18n library is smart enough to notice when you supply a parameter named count to select the correct pluralization for a key. If the user has one cookie it won’t add that pesky “s.”

I18n support is so easy to add that I recommend it for just about every web project unless you’re absolutely sure you’ll never need it in another language. The Internet is a lot bigger than your home country, go forth and make it easier to translate!

I learned very quickly while working on a large open source project is that it is important to make my code hard to break. The primary line of defense for this is a comprehensive test suite, but I think it’s also very important to create functions that are easy to use and difficult to damage.

I find I even code this way on personal projects that will never be released. Even if you never work on a team with other developers, there is a good chance you will forget a lot of implementation details of the code that you aren’t actively working on. You need to protect your code from yourself!

I think a lot about state these days. How much data should an object have, and how should it expose that to other objects? I find many bugs are related to the state and scope of data not being what you’d expect.

An Example: ActiveRecord

ActiveRecord makes it easy to retrieve all rows from database and represent them as objects:

Product.all.each do |p|
  puts p.name
end

We didn’t have to specify that we wanted the name column from the database before we outputted it; by default ActiveRecord includes all the columns in the table.

Over time, many frequently used tables in databases such as Product tend to get more columns added to them to support new features. You might find that your table that started off with 4 columns is eventually over 50!

There is overhead involved in returning all those extra columns from the database. At the very least, the database has to send more data across the wire to your application. On top of that, Rails has to deserialize all the columns into their appropriate types in the object.

When returning a single Product you will probably not notice much of a difference. However, when returning hundreds of rows at once, the overhead can add up quite a bit.

Selecting only what you need

ActiveRecord provides a method called select that can be used choose the columns returned from the database. We could write something like this:

Product.select([:id, :name]).each do |p|
  puts p.name
end

This will certainly execute faster than the Products.all query above. However, if you do this, you are exposing yourself to many bugs due to inconsistent state.

The danger here is ActiveRecord returns a mixed state instance of Product. The returned object looks like a Product. It has all of the instance methods you defined on Product, however, it is missing some of the data that is normally there.

To illustrate this, imagine you have a function that returns a product’s name, but adds an asterisk if it’s on sale:

def fancy_product_title(product)
  if product.on_sale?
    return product.name + "*"
  else
    return product.name
  end
end

In this case, our method checks the on_sale column in the database to determine whether to append the asterisk. However, if you retrieved the Product using select([:id, :name]) you would not have this column present, and even if the product was on sale your users wouldn’t know about it.

Now this might seem like a pretty easy bug to squash. Any competant programmer could adjust this code to return on_sale in the select clause if if they saw it wasn’t ever being displayed.

That is demanding a much broader knowledge of the application and the flow of data than is necessary. It takes more development time, and doesn’t scale well when your codebase grows. Also, who wants to constantly think “hey, do I have all the data I need in this object to do my work?”

Keep it Consistent

You can eliminate any entire class of bugs by never using select. You should insist that your object instances always include all their data members.

What about the performance issues? I suggest instead that you design your data structures in a different way. Rather than returning inconsistent Product models, why not create a method that returns BasicProduct objects?

All Product instances have enough data to transform into BasicProduct instances if they need to. If you like inheritance you could make a Product extend a BasicProduct. If you’re not a fan of inheritance you could create a to_basic method.

This is just one example of how easy it is to leave things in an inconsistent state, especially when considering performance. I suggest that you make an effort to keep your data in sync as much as possible, even if it involves a little data modelling. You’ll have fewer bugs, and your code will be better to use in the