Segment7 - The Blog

edit

Robot Co-op Software

Posted by Eric Hodel Mon, 20 Mar 2006 19:28:00 GMT

I’ve seen a lot of comments asking for information on our software setup, so here it is. If you’d like more detail just ask, I’ll fill you in as best I can either in a comment or in a future post.

UPDATE: Added link to Wikipedia’s MySQL configuration

All our online machines run FreeBSD 6.0-RELEASE. We use Amanda for backups which gets rsynced off-site. Subversion is used for revision control of all our configuration data.

We have our own CVSup mirror and package build machine which builds packages for all our boxes to keep things in-sync and to decrease the load on FreeBSD’s CVSup mirrors. We also NFS-mount the src and ports directories to reduce wasted disk space. portaudit is used for vulnerability monitoring and portupgrade for performing package upgrades.

Critical processes are watched by dwatch. Clocks are synchronized with ntpd.

Webservers (x4)

The webservers all run PAE kernels with 6GB of ram. 2GB is allocated to memcached and the rest gets chewed up by Rails processes. Each of our sites gets its own set of processes and we have 25 total Rails processes running per host.

For pages our webservers are all Apache 1.3 with mod_fastcgi. Apache logs are rotated by cronolog and processed by AWStats (but I might switch to Visitors).

We’re using Rails (of course) to generate our pages. We use a library very similar to CachedModel to fetch ActiveRecord objects out of memcached. We don’t use any page or fragment caching.

For images our webservers are WEBrick using a Ruby MogileFS library I wrote (but have not yet released). Image resizing is performed by RMagick, occasionally on-the-fly.

We use the lang/ruby18-nopthreads port of Ruby because we experienced an incredible load increase with the default pthread version.

Each machine runs memcached with 2GB of cache (so be sure to set kern.maxdsiz and kern.maxssiz in /boot/loader.conf appropriately). We store sessions, ActiveRecords and random other expensive-to-compute data in memcache.

Each machine is a MogileFS tracker node and file store. All our images get stored in MogileFS. We use NFS mode due to problems with components of Perlbal on FreeBSD. (With the way we’ve implemented image serving and FreeBSD’s NFS implementation it works out fine.)

Our webservers all run sendmail set up as a smart host pointing to a machine running postfix. This keeps outbound mail sending fast and reduces the amount of maintenance.

Database

We have one database server, it runs MySQL 4.1.x and isn’t all that special. Bob tuned it based on Wikipedia’s MySQL configuration.

Miscellaneous

We have one machine that does miscellaneous jobs. It runs postfix for inbound and outbound mail, runs crons that update memcached, analyzes log files and a small handful of other unimportant things.

Posted in Robot Co-op | 14 comments

Comments

  1. Chadwick said about 2 hours later:

    Thanks for posting this info Eric. Also of note, didn’t you once mention that you’re using Trac for internal PM?

  2. Eric Hodel said about 2 hours later:

    Chadwick: Yes, we use trac internally.

  3. Todd Huss said about 2 hours later:

    In my experience AwStats tends to really suffer from crawler inflation by overcount page views, unique visitors, and grossly overcounting referrals. Most log based analyzers overcount real user traffic. It’s fine for ballpark stuff (which is good enough for most sites) but once a site gets big enough there’s really no substitute for a good client based analytics solution like Google Analytics, Omniture, or Fireclick when it comes to accuracy.

  4. Alex said about 2 hours later:

    Have you looked at imlib2-ruby at all? [www.pablotron.org/software/imlib2-ruby/doc/]. imlib2 is supposed to be an incredibly fast library. Perhaps you don’t do enough image work for it to be worth it, but if it’s less of a memory hog than rmagick then it might be very useful for any app that does a lot of image resizing or basic manipulation.

  5. Eric Hodel said about 3 hours later:

    Alex: No, but I did just see MiniMagick. I’ll check both of them out.

  6. Eric Hodel said about 3 hours later:

    Todd: So far we’re using AWStats just for ballpark figures.

  7. Cesar said about 4 hours later:

    Eric, do you use FreeBSD for MySQL? Have you tested Linux vs FreeBSD for database performance?, I’ve heard that MySQL has performance problems on FreeBSD.

  8. Eric Hodel said about 5 hours later:

    Cesar: I use FreeBSD for everything. I’ll adhere to Don’t Repeat Yourself and Ya’ Ain’t Gonna Need It over switching to Linux for the DB. It would just be too much work to need to upgrade the software twice.

  9. john said about 6 hours later:

    FWIW: we use GraphicsMagick where I work, and it’s much faster and more stable than ImageMagick. it doesn’t have all of the features as IM, but has all that we care about.

    also, when compiled with gcc4, it gives us almost a 20% performance boost with running ‘convert’ on large images.

    you might not have a requirement to process images very fast, tho, since you mentioned that sometimes you’re doing them on-the-fly. :)

  10. Eric Hodel said about 6 hours later:

    John: So far there isn’t a port for GraphicsMagick in FreeBSD’s ports tree, so I haven’t even considered it. We very rarely generate extra images and keep the generated images forever so I haven’t worried about it.

  11. James said about 7 hours later:

    www.freebsd.org/cgi/cvsweb.cgi/ports/graphics/GraphicsMagick/

  12. blog.inlet-media.de said about 21 hours later:

    Can you tell me how you use Subversion for version control your configuration data? Did you commit the whole /etc directory?

  13. Eric Hodel said about 23 hours later:

    I just added a new post about Version Control and Sysadmin.

  14. Greg said 2 days later:

    Why are you using WEBrick to serve your images (why not something else)? I’d be really interested in your feedback on that because I’ll be needing to do something similar.

RSS feed for this post

Comments are disabled

Related searches:
freebsd software memcached machine webrick
gipoco.com is neither affiliated with the authors of this page nor responsible for its contents. This is a safe-cache copy of the original web site.