tag:www.holychao.com,2005:/articles Holy Chao : Holy Chao | Being a collection of deranged ramblings regarding Discordian tainted independence, and other such rubbish Being a collection of deranged ramblings regarding Discordian tainted independence, and other such rubbish 2012-03-24T14:32:25Z Typo urn:uuid:119db71c-8357-4fbc-8692-3e8eebcc8203 2012-03-24T13:43:26Z 2012-03-24T14:32:25Z Venerable High Pope Swanage I, Cogent Animal of Our Lady of Discord "What the hell is going on with everyone?" - Our increasing awareness of misogyny <p>My CEO/friend/person who's fired me twice Justin Gehtland exclaimed on the twitters yesterday: "What the hell is going on with everyone? Boston API Jam, Belvedere Vodka, Santorum. Is it National Misogyny Month and I missed the PSA???" I glibly replied: "People started paying attention to misogyny and calling it out. The misogyny didn't increase, your visibility into it did."</p> <p>I find this basically true and explanatory, but after a few hours of reflection and home improvement I'd like to discuss it in further length than 140 characters allow.</p> <p>I think we are seeing a consequence of a few important factors, and I'd like to describe a couple. First I'd note that our increased social interaction and eased communication have resulted in more people having more empathy with others. Secondly, in the U.S. we are in the middle of a Republican primary election where the defining attributes of the debate for many outside observers, including me, includes who can behave the most misogynistic.</p> <p>I find the first of these particularly encouraging; it suggests our movement toward a more egalitarian and sympathetic world. This movement has used technology as a lever, which I find pretty cool considering how I spend most of my vocational efforts. People can more easily object to, for instance, including women as an item on a bulleted list of highlights to voice their objections and to have those objections reinforced in a meaningful way today. This ad hoc organization of protest demonstrated material consequences in the real world: the sponsors of the Boston API Jam started pulling their financial support. The organizers accepted the fact they upset a significant number of people they tried to make money off of (they mismanaged it pitifully, but that's a different concern). Everyone else who organizes events now has to wonder if they will cross the same line. The organizers, Sqoot, do not appear to directly engage with the public, but with businesses. It appears they still feel consumer backlash; in 2012, every company is consumer facing. These material consequences amplified the visibility of the issue as well.</p> <p>Right now people understand that there are financial consequences for failing to conduct themselves in a way considerate of women. This is a consequence of a social reality, the financial consequences only exist as long as a sufficient number of people care to enforce them, but right now, it does exist. This is great. It might even form the foundation of lasting, meaningful social reforms that further undermine the disequality many women in the U.S. experience today.</p> <p>However, I find the second point cuts at how much confidence I can have in this prospect for meaningful progress in our social consciousness. We have a caustic political environment in the U.S., one that precipitates far more negativity and vitriol than it does pride and inspiration. Many people's political identities are defined by what they oppose, rather than what they promote. Worse still, these identities do not always derive from opposition to repugnant principals, but by opposing principals held by a repugnant political opposition.</p> <p>Some hard questions I do not think we can find the answers to today, but that do concern me: Would there be these same outcries against misogyny if they were not the sword with which Rush Limbaugh was attacked? Would people find the Boston API Jam ad offensive if we were not in the midst of a Republican primary advocating vociferously for stripping women of self-determination over what happens in their own bodies? Will the social reality that casual misogyny will precipitate calamitous financial ruin persist past November?</p> <p>These are hard questions because they essentially ask how we as people will behave in the future, or would behave in counterfactual scenarios. Only our future conduct, collectively, will determine the answers.</p> urn:uuid:697253e7-e44b-4625-9bbe-e87f7ab640dc 2012-02-06T21:55:01Z 2012-02-06T22:07:52Z Venerable High Pope Swanage I, Cogent Animal of Our Lady of Discord Typo needs to work on their migrations... <p>I use Typo, because I like Rails, and it's free, and the feature set is pretty nice. But often I find that migrating between major versions sucks. </p> <p>I just upgraded to 6 from 5, and experienced a problem every time I tried to read an article imported from 5. The failures looked something like: </p> <div><pre><notextile>ActionView::MissingTemplate (Missing template /articles with {:handlers=&gt;[:erb, :rjs, :builder, :rhtml, :rxml], :formats=&gt;[:html], :locale=&gt;[:en, :en]} in view paths &quot;/blog/themes/scribbish/views&quot;, ...)</notextile></pre></div> <p>After digging around a bit I found the culprit.</p> <p>New Typo apparently supports multiple presentations for content, and now has a 'post_type' column in the contents table. If you write a new article in Typo 6, it will get the value 'read'. I went to my database and updated all the old contents to have post_type of read. e.g.</p> <div><pre><notextile>UPDATE contents SET post_type=&apos;read&apos; WHERE post_type IS NULL;</notextile></pre></div> <p>After this, all my blog posts worked.</p> urn:uuid:5be7c2ed-1f8e-4104-864d-537a5244ecda 2012-01-16T15:22:31Z 2012-02-04T02:26:16Z Venerable High Pope Swanage I, Cogent Animal of Our Lady of Discord REE has a special trick... <p>If you're trying to install REE with RVM, and you just, for example plug in the following</p> <div><pre><notextile>rvm install ree-2011.03</notextile></pre></div> <p>It doesn't work terrifically under OS X</p> <p>It also serves as a local minima of functionality, in that it will get caught with the failure condition even if you remember to add something like:</p> <div><pre><notextile>rvm install ree-2011.03 -C --with-readline-dir=/usr/local</notextile></pre></div> <p>after having done the wrong thing.</p> <p>You need to do the following: <div><pre><notextile>rvm cleanup all rvm install ree-2011.03 -C --with-readline-dir=/usr/local</notextile></pre></div> urn:uuid:f982cfcd-d5c7-4a05-bf88-84903b1e424a 2011-08-13T10:14:42Z 2012-02-04T02:26:16Z Venerable High Pope Swanage I, Cogent Animal of Our Lady of Discord Try not to BCrypt so much. <p>At <a class="thinkrelevance.com">Relevance</a>, one of our favorite tools to use is <a class="https://github.com/plataformatec/devise">Devise</a>. It has become the de facto standard for authorization when we need a solution, and with good reason. It solves a specific problems and plays nicely with others.</p> <p>However, it can totally ruin your test suite's performance.</p> <p>By default, Devise ships configured to use bcrypt. If you investigate your config/initializers/devise.rb file, on or about line 64 you should see something like the following: <div><pre><notextile>config.stretches = 10</notextile></pre></div></p> <p>Shortly put, this is configuring the computational complexity bcrypt will calculate encrypted passwords with. This is a pretty cool feature of bcrypt, and it is a valuable security feature in its default configuration. If for some horrible reason, someone gets a hold of all the encrypted passwords in your db, the amount of time it will take to brute force crack the passwords with this default strength is sufficiently long as to allow you plenty of time to react and change passwords. (Assuming you are aware of the breach...)</p> <p>However, if you're using factory_girl and rspec (as we do), and generate a bunch of devise users who happen to have passwords (as we do), then you'll spend a lot of time in your tests calculating encrypted passwords with this config. There are some other solutions out there (mentioned in this <a class="https://github.com/plataformatec/devise/pull/969">github issue</a>) for getting around the bcrypt bottleneck when testing. <a class="jasonrudolph.com/">Jason Rudolph</a> and I have a better one though.</p> <p>First, change the line in your config/initializers/devise.rb from <div><pre><notextile>config.stretches = 10</notextile></pre></div> to <div><pre><notextile>config.stretches ||= 10</notextile></pre></div> Then in your config/environments/test.rb add <div><pre><notextile>Devise.setup do |config| config.stretches = 1 end</notextile></pre></div> </p> This configures your app to <em>by default</em> use an aggressive bcrypt strength, but allows you to override that configuration on a per environment basis. urn:uuid:35aa5361-ca35-447c-b119-1785e7c06da8 2011-05-24T15:20:14Z 2012-02-04T02:26:16Z Venerable High Pope Swanage I, Cogent Animal of Our Lady of Discord Kicking butt with CoffeeScript, Underscore, and Backbone <p>I've recently had the opportunity to work with some really exciting JavaScript tools that make developing apps that run in the browser much more powerful and managable than I've been accustomed to working with in the past. The trio of tools I am specifically interested in is <a class="jashkenas.github.com/coffee-script/">CoffeeScript</a>, <a class="documentcloud.github.com/underscore/">Underscore.js</a> and <a class="documentcloud.github.com/backbone/">Backbone.js</a>. I am specifically targeting technical audiences for this work, I will be assuming a good deal of comfort working with JavaScript, the DOM, and jQuery already. Additionally I will assume you understand Rails well enough to build a trivial app in it.</p> <hr/> <p>I'll start with a high level overview of the purposes of each of these tools. <ul> <li>CoffeeScript is a language built on top of JavaScript which puts a large deal of syntactic sugar in place to ease its crafting. It additionally emits JavaScript with several best practice idioms, and its output is generally built in order to pass <a class="www.jslint.com/">Lint</a> checks. The syntax is highly reminiscent of Ruby. When integrating CoffeeScript to a Rails app with Sprockets, source files will be compiled down to JavaScript on the fly providing a pleasantly responsive development cycle.</li> <li>Underscore.js is a library of clever utility functions. It includes mechanisms for templating, functional programming, and compensating for some of the curious interactions of JavaScript operating against a DOM in a browser. Many of its offerings seem redundant when compared with jQuery, and often in fact are. jQuery strongly focuses on DOM elements however, while Underscore plays very nicely with arbitrary javascript objects. This makes it a valuable tool when you start to embed non-UI logic in JavaScript applications.</li> <li>Backbone.js is a MVC library for JavaScript. Compared with <a class="www.sproutcore.com/">SproutCore</a> or <a class="cappuccino.org/">Cappuccino</a>, it is lighter weight and significantly decoupled. (To be fair, SproutCore is currently undergoing modularization efforts to reduce it's footprint.) Backbone.js features Models and Collections for modeling your data, Controllers for routing to functions, and Views for binding DOM fragments to functions and model data.</li> </ul> </p> <hr/> <h2>CoffeeScript</h2> <p>CoffeeScript is a language built on top of JavaScript to provide tools for easily creating code which adheres to best practices. It features: <ul> <li>Syntactic whitespace. Indenting a level creates a new JavaScript block.</li> <li>Terse syntax. Common tasks like defining functions and referring to <i>this</i> are conveniently abbreviated.</li> <li>Stricter scoping. JavaScript eagerly places things in global scope, but CoffeeScript generally creates immediate functions. By placing code in immediate functions, CoffeeScript constrains your work to local scopes by default.</li> <li>Controlling binding. When binding functions of an object to events on some DOM element in JavaScript, <i>this</i> will, without some intervention, be bound to the DOM element instead of the object the function is defined for.</li> <li>Splats. You can define functions of variable optional arguments using ellipsis.</li> <li>Default returns. Much like Ruby, the last expression evaluated in a function is automatically returned to the caller.</li> <li>Destructuring assignment. CoffeeScript makes it easy to pull apart maps and arrays</li> <li>String interpolation. CoffeeScript lifts Ruby's string interpolation mechanics out wholesale.</li> </ul> This is a high level manifest of what you can get by using CoffeeScript. For more details, visit <a class="jashkenas.github.com/coffee-script/">the website</a>. </p> <hr/> <h2>Backbone.js</h2> <p>Backbone is an MVC library which provides strong, decoupled foundations for building UIs. Its chief components are: <ul> <li>An event system which can be mixed into arbitrary objects using _.extend()</li> <li>Controllers, which route from location hashes to controller methods.</li> <li>Models and Collections, both designed to proxy remote objects communicated with JSON. If you subscribe to the contract of accessing all model attributes using the get and set methods, then event propagation will occur to queue changes in your views.</li> <li>Views, designed to bind together a DOM element and an object, to catch events in the relevant fragment of the DOM, and to invoke changes to the model.</li> <li>History, creating a chain of discrete UI states that can be reloaded and navigated using browser history functions.</li> </ul> </p> <hr/> <p>We'll cover relevant tools in Underscore as we cross them. It is such a mixed bag that I do not see much value in providing an overview.</p> <hr/> <h2>Let's build something!</h2> <p>I'll build a super trivial kanban board for this app. Our domain will consist of cards and lanes. Cards will have a name and a description. They will be placed in one of four lanes: Open, Development, Test, Approved. New cards will arrive in Open, and then transitions will move them forward and backward in the path until they reach Approved. My architecture will be built out of a Rails server which will serve up static resources to form the client app, and provide JSON endpoints for managing a shared state of the board. All user facing work will be done in CoffeeScript, or in HAML to produce templates that will end up as the content of Backbone views. Rails will not render any HTML as a result of actions in this example, an artificial restriction put in place to more put more weight on JavaScript. My work will be published as a <a class="https://github.com/aredington/Koffeeban">github project</a> for reference so that you can compare notes.</p> <p>I started by making a new Rails 3.1 app. I used rspec and haml for testing and templating respectively. I set up Card and Lane models and controllers. Lanes have many cards, and optionally previous and next lanes. Seed data creates an <i>Open</i> lane by default, and all cards are by default placed in the <i>Open</i> lane. I set up a default application layout and a Templates controller for serving up HTML fragments that will make up the UI. If you'd like to play along at home you can look at the state of the project at this point <a class="https://github.com/aredington/Koffeeban/tree/70274aacd89e9282f16ef1be9b5fbea9d15d9d50">here</a>. I have: <ul> <li>Seed data for 4 lanes.</li> <li>A JSON endpoint that lists all the lanes</li> <li>A default route going to the templates controller, and a default layout that includes our application.js</li> </ul></p> <hr/> <h3>Getting down to brass tacks</h3> <p>I'll start off with getting my dependencies in place. Rails 3.1 ships with jQuery and CoffeeScript as the defaults, this part is already taken care of for me! I'll need to add Backbone and Underscore still. I pull down underscore and backbone to the app/assets/javascript directory: <div><pre><notextile>wget -O app/assets/javascripts/underscore.js documentcloud.github.com/underscore/underscore.js wget -O app/assets/javascripts/backbone.js documentcloud.github.com/backbone/backbone.js</notextile></pre></div> Next I'll need to add require statements to app/assets/javascripts/application.js: <div><pre><notextile>//= require underscore //= require backbone</notextile></pre></div> <p>As a first cut of getting a handle on backbone goodies, I'll set up a trivial first action for our backbone app. I want to do the following: <ul> <li>Trigger a default controller action.</li> <li>Pull down all the lanes into a Backbone collection.</li> <li>Create a view on that collection that logs it's contents to the browser console.</li> </ul> This won't be much of a view, but it demonstrates a lot of the basic functionality of Backbone. It will not demonstrate the binding of model data to views a user can see.</p> <p>The controller is pretty simple, let's take a quick look: <div><pre><notextile>#Define LanesController on this to put it in the global scope. this.LanesController = Backbone.Controller.extend # Set up the routing map. The default route will go to the # LanesController.index() function. routes: &quot;&quot;: &quot;index&quot; # Fetch the data for the initial page, and display a view with it. index: -&gt; lanes = new Lanes() lanes.fetch() board = new LanesIndex(model: lanes)</notextile></pre></div> Some pieces to note here: <ul> <li>When CoffeeScript compiles to JavaScript, it wraps everything in an Immediate Function to keep it from polluting the global namespace. It also takes the binding of "this" from the scope where that Immediate Function is called, and propagates it into the body of the function, so that this is going to be bound either to window in the case of a browser, or globals in node.js. By defining things on this, you explicitly put them in the global namespace.</li> <li>The default route in Backbone is the empty string. To initialize CoffeeScript's controller logic, you initialize one or more controllers, and then call Backbone.history.start(). It will then route to the default route if there is one, and start monitoring the location hash in order to trigger changes in the controller.</li> <li>The index function is pretty brief, it just reads some data and then hands it to a view.</li> </ul> </p> <p>Let's take a look at the models now. <div><pre><notextile>this.Lane = Backbone.Model.extend # When a backbone model receives a JSON blob in # its constructor, it shoves all of the JSON properties # into an internal attributes hash. These are exposed by # get(name) and set(name, value) methods on the model. # You can directly access the attributes hash, but if # you shove values directly into it event propagation won&apos;t # happen. SO DON&apos;T DO IT! name: -&gt; @get(&quot;name&quot;) # Lanes is a collection of multiple Lane instances this.Lanes = Backbone.Collection.extend # Request URL with XHR to pull down, returns # a JSON blob. url: &quot;/lanes.json&quot; # Specify what this collection is a model of. # With this set, you can hand JSON blobs to the collection # and it will add new Backbone Models instantiated # from those JSON blobs. model: Lane # Parse accepts a JSON blob representing a collection of multiple # model objects. It returns a list of instantiated models. We get # an Array of JSON blobs back from Rails, so we can just shove # them in objects and we are off to the races. parse: (response)-&gt; _.map(response, (laneJson) -&gt; new Lane(laneJson))</notextile></pre></div> This is pretty trivial. I just want to expose the name of a lane with a convenience method, fetch a list of all the lanes from the server as a JSON result, and create lanes from the JSON result.</p> <p>Backbone is smart enough to do the right thing with the url property regardless of whether it is a string or a function. If you define url as a function, you can issue dynamic queries for filtered collections.</p> <p>Finally I'll implement half of a view. Why half? Views do two things: they bind to data, and they bind to HTML in the page. They are not responsible for specifying a presentation (unless you implement it yourself). They do provide convenience methods for declaratively binding to events in the DOM element which they own, and they will also conveniently bind this to the view, instead of the element which fires the event. First I'd like to demonstrate the binding to data, and then we can create some HTML and present it to the user in a later step.</p> <div><pre><notextile>this.LanesIndex = Backbone.View.extend # Initialize is called at the end of the view # constructor to set up the new view. You can bind to # events on the model this view was constructed for, # and do other housekeeping tasks here. initialize: -&gt; # bindAll accepts an object and a list of function names. # It then binds this in the body of each of those functions # to its first argument. In the example below, without using # bindAll this would evaluate to the collection firing the event # instead of the view to be rendered. _.bindAll(@,&quot;render&quot;) # When a collection fetches its contents from the server, it # fires a &quot;refresh&quot; event upon successfully parsing the contents # of the response. We re-render this view when the contents change # to keep the presentation to the user up to date. @model.bind(&quot;refresh&quot;, @render) # Render is a no-op by default, but the intent is to stuff data # from the view&apos;s model into the HTML element the view is holding onto. # At this stage in the example app, we are simply logging all of the Lane # objects to the browser console. render: -&gt; _.each(@model.models, (model) -&gt; console.log(model.name())) @</notextile></pre></div> <p>Now I have all the code I need. I put my backbone files in controllers, models, and views directories under the javascript directory, and then embed them in application.js like so: <div><pre><notextile>//= require models/lanes //= require views/lanes-index //= require controllers/lanes-controller</notextile></pre></div> I'm almost done. Now that all the backbone work exists, I need to invoke it. It's pretty easy! Just initialize the controllers that need to be active in the application (no need to hold onto the instances though!), and then call Backbone.history.start() <div><pre><notextile>new LanesController() Backbone.history.start()</notextile></pre></div> The default route on LanesController will fire when the page is first visited, and if you load up the app it will log the names of the seed lanes to the console. For those playing along at home, you can check out <a class="https://github.com/aredington/Koffeeban/tree/937269c253e6c152f0621c8d2ba638a903e23ffe">the tree</a> at this point. If you load it up, you should see the lane names logged to the console. <hr/> <p>Now let's start working on presenting information. Your options with generating content to stick in the page are enormous, there are numerous templating systems available for JavaScript. If you'd like you could build dom elements dynamically and insert content as appropriate. I'm going to use underscore's templating system, for two simple reasons. First, we need underscore for Backbone, so there's no extra effort to incorporate it. Second, underscore's templating performs reasonably well in the spectrum of Javascript templating tools. I have experienced some issues with the stack depth when using underscore templating though, so if you have nested templates, contemplate the appropriateness of using underscore before you commit to it.</p> <p>I like to take the approach of putting templates on the server and pulling them down when the app loads. I added a "lane" action to the templates controller, it returns the following HTML fragment: <div><pre><notextile>&lt;div class=&apos;lane&apos;&gt; &lt;div class=&apos;lane-name&apos;&gt; &lt;%= lane.name() %&gt; &lt;/div&gt; &lt;div class=&apos;lane-cards&apos;&gt; &lt;/div&gt; &lt;/div&gt;</notextile></pre></div> That ERBish looking bit is recognized by underscore as an escape. When you turn this into an underscore template, it will call the name method on lane, and fill in the content with the return value of name. </p> <p> Next, here's a little bit of code to prefetch all of the templates before starting up the app. <div><pre><notextile>this.KoffeeTemplates = { templatesUrls: lane: &quot;/templates/lane&quot; triggerReady: -&gt; @trigger(&quot;ready&quot;) init: -&gt; _.after(_.keys(@templatesUrls).length, @triggerReady) _.each(@templatesUrls, (path, name) -&gt; $.get path, (data) -&gt; KoffeeTemplates[name] = data KoffeeTemplates.triggerReady() ) } _.extend(KoffeeTemplates, Backbone.Events)</notextile></pre></div> This little bit of utility lets me specify templates on templateUrls. It will fetch the contents of those urls and add them to the KoffeeTemplates object as strings. I can add more templates by defining them on templatesUrls. E.g. if I wanted a card template, I would change <div><pre><notextile> templatesUrls: lane: &quot;/templates/lane&quot;</notextile></pre></div> to <div><pre><notextile> templatesUrls: lane: &quot;/templates/lane&quot; card: &quot;/templates/card&quot;</notextile></pre></div> With this change, when the "ready" event fires KoffeeTemplates.lane will return the lane template, and KoffeeTemplates.card will return the card template. I'll need to change our initialization in application.js to fetch the templates and then initialize the application. After the appropriate changes, application.js looks as follows: <div><pre><notextile>//= require jquery //= require jquery_ujs //= require underscore //= require backbone //= require koffee-templates //= require models/lanes //= require views/lanes-index //= require controllers/lanes-controller KoffeeTemplates.bind(&quot;ready&quot;, function() { new LanesController(); Backbone.history.start(); }) KoffeeTemplates.init();</notextile></pre></div> </p> <p> As I'm no longer logging things to the console, it'd be good to have my content actually affecting the DOM. I'm going to hook up the view's element to be the contents of the body tags by changing the LanesController.index method as follows: <div><pre><notextile> index: -&gt; lanes = new Lanes() board = new LanesIndex(model: lanes) $(&apos;body&apos;).append(board.el) lanes.fetch()</notextile></pre></div> Finally, I rewrote the view to render as a function of the lanes it holds. The new LaneIndex view looks like this: <div><pre><notextile>this.LanesIndex = Backbone.View.extend tagName: &quot;div&quot; id: &quot;board&quot; # Initialize is called at the end of the view # constructor to set up the new view. You can bind to # events on the model this view was constructed for, # and do other housekeeping tasks here. initialize: -&gt; # bindAll accepts an object and a list of function names. # It then binds this in the body of each of those functions # to its first argument. In the example below, without using # bindAll this would evaluate to the collection firing the event # instead of the view to be rendered. _.bindAll(@,&quot;render&quot;) # When a collection fetches its contents from the server, it # fires a &quot;refresh&quot; event upon successfully parsing the contents # of the response. We re-render this view when the contents change # to keep the presentation to the user up to date. @model.bind(&quot;refresh&quot;, @render) # Render is a no-op by default, but the intent is to stuff data # from the view&apos;s model into the HTML element the view is holding onto. # At this stage in the example app, we are simply logging all of the Lane # objects to the browser console. render: -&gt; laneTemplate = _.template(KoffeeTemplates.lane) _.each(@model.models, (model) =&gt; $(@el).append(laneTemplate(lane: model))) @</notextile></pre></div> With this code in place, it renders full page columns for our cards to live in, one column for each lane. As usual, those following the examples can view the tree <a class="https://github.com/aredington/Koffeeban/tree/be1d59183c8be911fd1963b2e8a68c2be83d091b">here</a>. urn:uuid:a2256c87-2048-4a18-ae96-b42bd3579ad1 2011-05-06T09:36:38Z 2012-02-04T02:26:16Z Venerable High Pope Swanage I, Cogent Animal of Our Lady of Discord BAMFCSV - BAMF, your data's here! <p>Rivaling the amazing transitive powers of Nightcrawler, <a class="twitter.com/#!/jondistad">Jon Distad</a> and I decided to tackle the problem of <a class="https://github.com/jondistad/bamfcsv">parsing CSV rapidly</a> under Ruby 1.9. "Aha!" you might be saying, "FasterCSV was already rolled into the stdlib in Ruby 1.9! Why would I need a gem to handle it?" Well you have a very good point there, FasterCSV was a good response to the performance of the old 1.8 stdlib CSV parser. However there are still cases where it doesn't quite go fast enough.</p> <p>In our specific use case for a client project, we needed to parse large result sets coming in a csv format. How large? About 25 megs large, 200,000 records large. Decently, but not outrageously large. When we used the FasterCSV route, we could consume the data in about 8 seconds or so on our staging environment. I groused about how it could be better with a C extension. Jon rose to the challenge and started working on one, and I dutifully obliged by contributing patches and we spent a couple of Fridays pairing on it. I liked our results.</p> <p>It's fast. How fast? Well we took our problem CSV input and started benchmarking with it. We then tuned it the only way that makes any sense: profiling, identifying hotspots, and circumventing them. Here's a quick benchmarking session I just ran on my MBP: <div><pre><notextile>alexs-MacBook-Pro:bamfcsv alex$ irb ruby-1.9.2-p136 :001 &gt; require &apos;benchmark&apos; =&gt; true ruby-1.9.2-p136 :002 &gt; require &apos;bamfcsv&apos; =&gt; true ruby-1.9.2-p136 :003 &gt; require &apos;csv&apos; =&gt; true ruby-1.9.2-p136 :004 &gt; Benchmark.measure { CSV.read &quot;observations.csv&quot; } =&gt; 2.050000 0.040000 2.090000 ( 2.085173) ruby-1.9.2-p136 :005 &gt; Benchmark.measure { CSV.read &quot;observations.csv&quot; } =&gt; 2.190000 0.050000 2.240000 ( 2.230679) ruby-1.9.2-p136 :006 &gt; Benchmark.measure { CSV.read &quot;observations.csv&quot; } =&gt; 2.190000 0.020000 2.210000 ( 2.215040) ruby-1.9.2-p136 :007 &gt; Benchmark.measure { CSV.read &quot;observations.csv&quot; } =&gt; 2.140000 0.050000 2.190000 ( 2.180277) ruby-1.9.2-p136 :008 &gt; Benchmark.measure { CSV.read &quot;observations.csv&quot; } =&gt; 2.170000 0.040000 2.210000 ( 2.208252) ruby-1.9.2-p136 :009 &gt; Benchmark.measure { BAMFCSV.read &quot;observations.csv&quot; } =&gt; 0.270000 0.040000 0.310000 ( 0.301174) ruby-1.9.2-p136 :010 &gt; Benchmark.measure { BAMFCSV.read &quot;observations.csv&quot; } =&gt; 0.210000 0.020000 0.230000 ( 0.233012) ruby-1.9.2-p136 :011 &gt; Benchmark.measure { BAMFCSV.read &quot;observations.csv&quot; } =&gt; 0.220000 0.020000 0.240000 ( 0.239818) ruby-1.9.2-p136 :012 &gt; Benchmark.measure { BAMFCSV.read &quot;observations.csv&quot; } =&gt; 0.210000 0.020000 0.230000 ( 0.224568) ruby-1.9.2-p136 :013 &gt; Benchmark.measure { BAMFCSV.read &quot;observations.csv&quot; } =&gt; 0.220000 0.020000 0.240000 ( 0.240832) </notextile></pre></div> That's about a 10x increase over the FasterCSV you'll get out of the box with 1.9.</p> <p>We haven't tried to match FasterCSV feature for feature. We haven't tried to implement good Windows support. But it's really fast, and if you're hurting on CSV parsing performance, it can help you out in 1.9.</p> <p>There's many 1.8 CSV libraries built as native extensions out there. Some report ridiculously fast performance, orders of magnitude faster than BAMFCSV. And in our attempts to use them under 1.9 they all blew up. So here's an effort to solve the same problem for Ruby 1.9.</p> urn:uuid:e780a904-e7f1-47de-906c-fa4908e37f21 2011-03-04T11:24:53Z 2012-02-04T02:26:16Z Venerable High Pope Swanage I, Cogent Animal of Our Lady of Discord First Cut of Cleavage <p>I've been playing with the last few days of my 20% time at <a class="www.thinkrelevance.com">Relevance</a> working with Michael Feathers' ideas about <a class="www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=16679&tth=DYN&tt=siteemail&iDyn=2">Code Turbulence</a>, specifically with a mind for being able to examine it through time. </p> <p> There's two basic approaches one can take to doing this, in my mind, which is taking a sequence of scatter plots and animating them (projecting the time dimension through time), or taking the sequence of scatter plots and layering them (projecting the time dimension through space). I chose to go with the latter. </p> <p> My implementation was done in Clojure, mostly because I was interested in the idea, it involved crunching through large batches of data (mining every commit of a git repository!) and I'd yet to take a project from inception to implementation in Clojure. </p> <p> As of right now it works (for a very generous definition of work), in so far that you can point it at an arbitrary git repo full of java source code and get out a neat three dimensional graph. It looks like this: <img src="/img/spacer.gif"> urn:uuid:531218a7-6460-4920-96e5-3e74952d0a05 2010-09-21T13:30:08Z 2012-02-04T02:26:16Z Venerable High Pope Swanage I, Cogent Animal of Our Lady of Discord Kymera Magic Wand Remote and Windows 7 <P>I recently was presented with the opportunity to purchase a Kymera Magic Wand Remote. "I get to play wizard <i>while</i> controlling my television?" I thought, "SIGN ME RIGHT UP!"</P> <P>It's a very neat toy and considering the technology problem it is trying to solve, very well done. (I find there's some crosstalk between "flick down" and "tap top").</P> <P>However it's a learning universal remote, and the problem with these is that if you use it with Windows via a Windows Media Center receiver device, the operating system will "Debounce" all inputs. Essentially you will end up able to do any input once, but not multiple times in a row without some other input intervening.</P> <P>This is a stupid, compatibility breaking feature. Fortunately you can disable it. Point REGEDIT to HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\HidIr\Remotes\745a17a0-74d3-11d0-b6fe-00a0c90f57da. Find the EnableDebounce key, and change it's value from 1 to 0. Poof! You've just done a little more magic to make Windows compatible with something Microsoft didn't build.</P> urn:uuid:d16adae3-cd22-4eef-a2ef-5e122b537132 2010-09-12T09:18:59Z 2012-02-04T02:26:16Z Venerable High Pope Swanage I, Cogent Animal of Our Lady of Discord Mac OS X 10.6.*, ruby 1.9.2, and rvm <P>I've been running into some complications trying to get <a class="www.ruby-lang.org">ruby</a> 1.9.2 installed by <a class="rvm.beginrescueend">RVM</a> on OS X 10.6.3. This is likely not an original problem, but I got it to work. My environment is pretty crufty, I have an oldish Macbook Pro that came installed with 10.4, and has had an upgrade path including 10.5. Macports, and it's ruby interpreter were on my system, as well as the default Mac ruby, and a ruby source tarball installation.</P> <P>I had since uninstalled all rubies except for the 1.9.2 source tarball, and uninstalled Macports in favor of migrating to <a class="mxcl.github.com/homebrew/">Homebrew</a>.</P> <P>My specific problem is working on a project in which the .rvmrc specified ruby-1.9.2-p0, and <code>rvm install ruby-1.9.2-p0</code> did not act as expected. I received the error</P> <div><pre><notextile>readline.c:1292: error: &apos;username_completion_function&apos; undeclared (first use in this function)</notextile></pre></div> <P>Starting with brew and rvm installed I then needed to do some more:<P> <P>Install readline:</P> <div><pre><notextile>brew install readline</notextile></pre></div> <P>Cleanup to only one version of readline:</P> <div><pre><notextile>brew cleanup readline</notextile></pre></div> <P>Link the brew install to /usr/local:</P> <div><pre><notextile>brew link readline</notextile></pre></div> <P>Install with rvm passing args to autoconf:</P> <div><pre><notextile>rvm install 1.9.2-p0 -C --enable-shared,--with-readline-dir=/usr/local</notextile></pre></div> <P>No compile errors, and proper readline support! Yay. My thanks to <a class="blog.plataformatec.com.br/tag/rvm/?author=2">George</a> at <a class="blog.plataformatec.com.br">Plataforma Tecnologia</a> for this solution.</P> urn:uuid:aeacc566-775c-4456-a08d-999b1a6c5bf0 2010-08-03T19:56:03Z 2012-02-04T02:26:16Z Venerable High Pope Swanage I, Cogent Animal of Our Lady of Discord Beating a nuisance I was having a problem with trying to get rails to play nicely with my development machine. I was really frustrated with trying all kinds of various solutions, but none of them would actually work. My problem was consistent: <div><pre><notextile><span><span><span>/</span><span>usr</span><span>/</span></span>lib/ruby/gems/<span>1.9</span>.<span>1</span>/gems/activerecord-<span>2.3</span>.<span>8</span>/lib/active_record/connection_adapters/abstract/connection_specification.rb:<span>76</span><span>:in</span> <span><span>`</span><span>rescue in rescue in establish_connection': Please install the postgresql adapter: </span><span>`</span></span>gem install activerecord-postgresql-adapter<span><span>`</span><span> (no such file to load -- pg) (RuntimeError)</span></span></span></notextile></pre></div> I beat my brains around it for about an hour or more, trying to install various versions of postgres adapters, rails stacks, and specifications in the config file. It turns out the problem was trivially simple. I started with <div><pre><notextile><span>apt-get install ruby1.9.1-full </span></notextile></pre></div> On a Ubuntu 10 installation. I symlinked ruby1.9.1 and irb1.9.1 to ruby and irb respectively. Then I downloaded the latest gem tarball and installed it with <div><pre><notextile><span>ruby setup.rb</span></notextile></pre></div> in the appropriate directory. I installed the "pg" postgres adapter, as well as rails through gems. I d
gipoco.com is neither affiliated with the authors of this page nor responsible for its contents. This is a safe-cache copy of the original web site.