Skip to content

Parallel Each

Parallel each allows you to iterated through any Enumerable handling the items in parallel.

results = []
[1,2,3,4].p_each do |i|
  results << i
end
results   # => [3,2,4,1]

The parallel processing happens in threads The given block is executed for each item in the array in a separate thread. Obviously, this could get dodgy for large Enumerables. To handle that #p_each will never have more than a set number of threads running simultaneously. The default limit is 20 but you can change that by passing a number to #p_each.

results = []
[1,2,3,4].p_each(2) do |i|
  sleep 1
  results << i
end
results   # => [2,1,3,4]

Parallel iterates through the items in order getting ahead of what can be processed immediately. This mean that you can use #p_each on very large lazy loaded lists (such as the results of ActiveRecord#find) without instantiating every item in the list at once.

Installation

Install the gem with this command

sudo gem install parallel-each

Performance

The performance characteristics of parallel iteration varies greatly depending on the platform and work being performed.

I/O bound work is where #p_each will really shine. In this situation the overall processing time will, generally, be improved regardless of the version/flavor of Ruby that is being used or the number of CPUs.

If the work is CPU bound the impacts of parallelization will depend on version/flavor of Ruby and the number of CPUs. With more than one CPU running Ruby 1.9 or JRuby should result in an improvement of the overall processing time. With one CPU or when running Ruby 1.8 there will be no improvement (and possibly a slight degradation) of the overall processing time.

API docs
parallel-each.rubyforge.com
Canonical repo
git://github.com/pezra/parallel-each.git
Github page
github.com/pezra/parallel-each
gipoco.com is neither affiliated with the authors of this page nor responsible for its contents. This is a safe-cache copy of the original web site.