Eventlet A better way to write network applications

Congratulations to Eric Florenzano and the entire Convore team for their launch last week and the buzz they’ve been able to build about their product already.  That’s awesome!

I couldn’t help but note that Eric’s posted about the technology powering Convore, and guess what little library is listed there?  That’s right, Django.

…

Seriously though, it’s nice to see someone using such a seemingly-oldschool stack to build a cutting-edge application and not being apologetic about it at all.  And this is a great illustration of the sweet spot for Eventlet: quickly creating a novel scalable network application.

A good way to start off the week! This release contains numerous
bugfixes that have been contributed by excellent people since the last
release. It’s the best version of Eventlet ever!

Install it by one of these methods:

• easy_install eventlet
• pip install eventlet
• pypi.python.org/packages/source/e/eventlet/eventlet-0.9.14.tar.gz
• hg clone bitbucket.org/which_linden/eventlet/

To file issues/bugs:
bitbucket.org/which_linden/eventlet/issues/new/

Here’s the changes in this version:

• Many fixes to the ZeroMQ hub, which now requires version 2.0.10 or later. Thanks to Ben Ford.
• ZeroMQ hub no longer depends on pollhub, and thus works on Windows (thanks, Alexey Borzenkov)
• Better handling of connect errors on Windows, thanks again to Alexey Borzenkov and Nat Linden (#67, #69).
• More-robust Event delivery, thanks to Malcolm Cleaton
• wsgi.py now distinguishes between an empty query string (“”) and a non-existent query string (no entry in environ).
• wsgi.py handles ipv6 correctly (thanks, redbo)
• Better behavior in tpool when you give it nonsensical numbers, thanks to R. Tyler for the nonsense. :)
• Fixed importing on 2.5 (#73, thanks to Ruijun Luo)
• Hub doesn’t hold on to invalid fds (#74, thanks to Edward George)
• Documentation for eventlet.green.zmq, courtesy of Ben Ford

The md5sum for the tarball is dfc96ed14b27392fdc529abcafeed880
The sha256sum is
219207d165176b12565e7f8f40aaadb2965814ce5935dba1097083d0f183bf28
(that’s right, I totally went there!)

It’s been too long, hasn’t it? The major new things of interest in this release are the ZeroMQ support, eventlet.green.MySQLdb, and greenthread affinity in tpool. Please try out all three of these new features and file issues if you find bugs!

How get it:

• easy_install eventlet
• pip install eventlet
• Download directly from pypi
• hg clone bitbucket.org/which_linden/eventlet/

File issues/bugs here.

Full changelog:

• ZeroMQ hub, and eventlet.green.zmq make supersockets green. Thanks to Ben Ford! Check out the examples zmq_chat.py and
  distributed_websocket_chat.py.
• eventlet.green.MySQLdb added. It’s an interface to MySQLdb that uses tpool to make it appear nonblocking
• Greenthread affinity in tpool. Each greenthread is assigned to the same thread when using tpool, making it easier to work with
  non-thread-safe libraries.
• Eventlet now depends on greenlet 0.3 or later.
• Fixed a hang when using tpool during an import causes another import. Thanks to mikepk for tracking that down.
• Improved websocket draft 76 compliance, thanks to Nick V.
• Rare greenthread.kill() bug fixed, which was probably brought about by a bugfix in greenlet 0.3.
• Easy_installing eventlet should no longer print an ImportError about greenlet
• Support for serving up SSL websockets, thanks to chwagssd for reporting #62
• eventlet.wsgi properly sets ‘wsgi.url_scheme’ environment variable to ‘https’, and ‘HTTPS’ to ‘on’ if serving over ssl
• Blocking detector uses setitimer on 2.6 or later, allowing for sub-second block detection, thanks to rtyler.
• Blocking detector is documented now, too
• socket.create_connection properly uses dnspython for nonblocking dns. Thanks to rtyler.
• Removed EVENTLET_TPOOL_DNS, nobody liked that. But if you were using it, install dnspython instead. Thanks to pigmej and gholt.
• Removed _main_wrapper from greenthread, thanks to Ambroff adding keyword arguments to switch() in 0.3!

The md5sum of the tarball is 2273527ca1715bad018090fdd2d480cf.

Check out R. Tyler’s post on blocking detection in Eventlet.  It’s good stuff — if you’re concerned about performance under load, detecting places where your code blocks should be your first order of business.  Here’s the official documentation on enabling blocking detection.

Using signals is kind of appropriate for this sort of functionality, because signals are part of a sorta old-school way of implementing concurrency.  The nice thing about them versus, say, a watchdog thread, is that the Python interpreter really interrupts itself in the middle of whatever it was doing when the signal comes in, so the handler can print the lowest-level line of code that was blocking.

Hey there, it’s another release of Eventlet! This is a pretty short changelog, but if you’re using Eventlet with the Twisted hub, you may wish to look this over (and let me know who you are, by the way, I’d like to know who’s out there using Twisted).

* Eventlet no longer uses the Twisted hub if Twisted is imported — you must call eventlet.hubs.use_hub(‘twistedr’) if you want to use it. This prevents strange race conditions for those who want to use both Twisted and Eventlet separately. * Removed circular import in twistedr.py
* Added websocket multi-user chat example
* Not using exec() in green modules anymore.
* eventlet.green.socket now contains all attributes of the stdlib socket module, even those that were left out by bugs.
* Eventlet.wsgi doesn’t call print anymore, instead uses the logfiles for everything (it used to print exceptions in one place). * Eventlet.wsgi properly closes the connection when an error is raised * Better documentation on eventlet.event.Event.send_exception
* Adding websocket.html to tarball so that you can run the examples without checking out the source

Enjoy!

The MD5 of the tarball is: 96edb9eb4f73863f3d19afb5d5c85854

This is a stability release. There’s naught new except for greendns, which is probably not going to affect most people because most people don’t have dnspython installed already. But you should install it! Because it makes your dns queries non-blocking and snappy. Here’s the changelog:

0.9.10
======
* Greendns: if dnspython is installed, Eventlet will automatically use it to provide non-blocking DNS queries. Set the environment variable ‘EVENTLET_NO_GREENDNS’ if you don’t want greendns but have dnspython installed. * Full test suite passes on Python 2.7.
* Tests no longer depend on simplejson for >2.6.
* Potential-bug fixes in patcher (thanks to Schmir, and thanks to Hudson) * Websockets work with query strings (thanks to mcarter)
* WSGI posthooks that get called after the request completed (thanks to gholt, nice docs, too)
* Blocking detector merged — use it to detect places where your code is not yielding to the hub for > 1 second (this could use documentation) * tpool.Proxy can wrap callables
* Tweaked Timeout class to do something sensible when True is passed to the constructor

I hope you are having a sensational week!

A little more than a month ago, Ian Hickson released draft 76 of the WebSocket protocol, which was unusually incompatible with previous drafts.  Headers in the client upgrade request grew a Sec-Websocket prefix, and there is now an interesting challenge-response computation that needs to be performed on the headers and body.  Caveat Emptor to those who implement draft specifications, I guess!

Thus far, Chrome seems to be the major test browser for WebSockets (though Safari implements them as well), and they switched over to draft 76 a few weeks ago.  So the Chrome betas speak draft 75, and the nightlies speak 76.

Fortunately, it’s possible to write WebSocket servers that interoperate seamlessly with both versions, by detecting those new Sec-Websocket headers and doing the appropriate handshake for the appropriate version.  Andrew Godwin implemented this logic for Eventlet’s websocket module, so you can use it to serve sockets to any browser that’s out there today.  Find it in the development branch, which you can install by running easy_install eventlet==dev.

We’ll be keeping a weather eye on the spec as it proceeds.

Well, that was a quick turnaround! This release was occasioned by a few patches that came in right after 0.9.8 went out, and also the fact that I raced to upgrade the build machines to psycopg 2.2 and didn’t notice that it wasn’t working with 2.0.14. Come on, people, I expected you to keep up with the times here. :P

So here’s the full changelist:

• A fix for monkeypatching with psycopg versions < 2.2.
• Improved support for chunked transfers in wsgi, plus a bunch of tests from schmir (ported from gevent by redbo)
• A fix for the twisted hub from Favo Yang

Enjoy!

All right! It’s been a while since our last release, so it’s a relief to finally get this out there. This is primarily a bugfix release, but there are a few new things in it that might interest you.

• Support for psycopg v2.2′s async support, courtesy of Daniele Varrazzo’s great work. You use it by calling monkey_patch(psycopg=True).
• Websocket support moves to core. This means it’s easier than ever to use websockets with eventlet.wsgi. It’s, like, two lines now. Mad props to Ben Ford for providing 100% unit test coverage. Check out the example.
• eventlet.wrap_ssl exists! We talked about it a while back but never got around to implementing it until now.

Give it a go, see how you like it!

Here’s the full changelog:

• Support for psycopg2′s asynchronous mode, from Daniele Varrazzo
• websocket module is now part of core Eventlet with 100% unit test coverage thanks to Ben Ford. See its documentation at eventlet.net/doc/modules/websocket.html
• Added wrap_ssl convenience method, meaning that we truly no longer need api or util modules.
• Multiple-reader detection code protects against the common mistake of having multiple greenthreads read from the same socket at the same time, which can be overridden if you know what you’re doing.
• Cleaner monkey_patch API: the “all” keyword is no longer necessary.
• Pool objects have a more convenient constructor — no more need to subclass
• amajorek’s reimplementation of GreenPipe
• Many bug fixes, major and minor.

This is a well-written post from R. Tyler Ballance:

Eventlet meets Avro RPC in an unlikely (in my opinion) place: WSGI. Instead of building their own transport layer for RPC calls, Avro sits on top of HTTP for its transport layer, POST’ing binary data to the server and processing the response. Since Avro can sit on top of HTTP, we can use eventlet.wsgi for building a fast, simple RPC server.

I’ve never used Avro before, but its supporters seem to really like it.  It seems to have improved upon Protocol Buffers (while preceding them chronologically), and has a very efficient wire encoding.

• Main Site
• Bug Report Form
• Automated Builds
• Mailing List Archives
• IRC Channel

Released Versions:

• easy_install eventlet
• Download from PyPi

Development Versions:

• easy_install eventlet==dev
• hg clone bitbucket.org/which_linden/eventlet/
• download tip