ws4py – WebSocket client and server library for Python
Recently I released ws4py, a package that provides client and server WebSocket support for Python 2.6 and 2.7.
Let’s first have a quick overview of what ws4py offers for now:
- WebSocket specification draft-10 of the current specification.
- A threaded client. This gives a simple client that doesn’t require an external dependency.
- A Tornado client. This client is based on Tornado 2.0 which is quite a popular way of running asynchronous networking code these days. Tornado provides its own server implementation so I didn’t include mine in ws4py.
- A CherryPy extension so that you can integrate WebSocket from within your CherryPy 3.2.1 server.
- A gevent server based on the popular gevent library. This is courtesy of Jeff Lindsay.
- Based on Jeff’s work, a pure WSGI middleware as well (available in the current master branch only until the next release).
- ws4py runs on Android devices thanks to the SL4A package
Hopefully more client and servers will be added along the way as well as Python 3.x support. The former should be rather simple to add due to the way I designed ws4py.
The main idea is to make a distinction between the bytes provider and the bytes processing. The former is essentially reading and writing bytes from the connected socket. The latter is the function of making something out of the received bytes based on the WebSocket specification. In most implementations I have seen so far, both are rather heavily intertwined making it difficult to use a different bytes provider.
ws4py tries a different path by relying on a great feature of Python: the possibility to send data back to a generator. For instance, the frame parsing yields the quantity of bytes each time it needs more and the caller feeds back the generator those bytes once they are received. In fact, the caller of a frame parser is a stream object which acts the same way. The caller of that stream object is in fact the bytes provider (a client or a server). The stream is in charge of aggregating frames into a WebSocket message. Thanks to that design, both the frame and stream objects are totally unaware of the bytes provider and can be easily adapted in various contexts (gevent, tornado, CherryPy, etc.).
On my TODO list for ws4py:
- Upgrade to a more recent version of the specification
- Python 3.x implementation
- Better documentation, read, write documentation.
- Better performances on very large WebSocket messages
Related Posts
- Acceptance testing a CherryPy application with Robot Framework
- Hosting a Django application on a CherryPy server
- WebSocket for CherryPy 3.2
30 Responses to ws4py – WebSocket client and server library for Python
Pingback: Sylvain Hellegouarch: ws4py – WebSocket client... | Python | Syngu
On what browser is the echo_cherrypy_server example supposed to work?
I tried it on 14.0.835.157 beta-m, on Windows 7, 64bit, and it does not work…
After entering some text and pressing the submit button, nothing happens.
No errors in the console on chrome, or of cherrypy.
Thanks.
Hi Mohamed,
I was able to run it on Chrome 15 and didn’t try with 14.
Anser: en.wikipedia.org/wiki/WebSockets#Browser_support
I’m now wondering if it’s not because draft-10 describes the deflate-stream extension which has been dropped from draft-11 and that ws4py doesn’t implement. In such case, it’d be more accurate to say that ws4py implements draft-11 only.
Many thanks for ws4py … I can confirm that it works great with cherrypy 3.2.0 and Chrome 14.0.835.159 beta on OSX. It also works great with Firefox 7.0 on OSX, with a small modification … Firefox sends
Connection:keep-alive, Upgrade
in the initial handshake, and the current logic at ws4py/server/cherrypyserver.py:125 will only accept
Connection: Upgrade
I don’t know enough about the relevant standards to say who should change, but it’s an easy tweak to ws4py if you’re interested.
Finally, a dumb question: how should I initialize a handler derived from WebSocketHandler? I want to pass arguments to its constructor, but since it’s instantiated by WebSocketTool I don’t see how to get to the new instance to hook it into the rest of my application.
Cheers!
Tim
Hey Tim,
1. I will look at the connection header issue. But don’t hesitate to open a bug ticket on github.
2. Indeed it seems you cannot pass an instance itself. I may have to change that so that both can be done. In the mean time, off the top of my head, I’d say you can perform operations on the created instance from your traditional CherryPy page handler.
Look at line 369 in https://github.com/Lawouach/WebSocket-for-Python/blob/master/ws4py/server/cherrypyserver.py#L369
You see you have access to the handler instance and therefore could do whatever you want with it. Note that the handler is also available from the WebSocketHandlerPlugin.
Sylvain:
Thanks for the quick response, I filed the ticket. In the meantime, I figured-out that I can access the handler using cherrypy.request.ws_handler, and it’s working well – I’ve got a simple application that delivers images to the client running on my localhost.
In the long run, I need to deploy my little app behind an Apache reverse proxy that provides SSL and authentication. Any issues you’re aware of? I have some vague idea that secure sockets are required to go through a proxy, are they supported via cherrypy + ws4py?
Many thanks,
Tim
Hey Tim,
Thanks for the ticket.
As for deployment, I haven’t tried but I’m not certain how reverse proxying and WebSocket will fare together. CherryPy does support SSL but I’ll admit I haven’t yet tried secure WebSocket connection.
Sylvain:
Well, the good news is that, for a CherryPy server configured to use SSL, secure WebSocket connections seem to work.
This is all running on the same host, next step is to try and make it work through my Apache proxy.
Cheers,
Tim
Hey Timothy,
maybe stunnel can be a help for you:
www.stunnel.org/
Basically, you can make stunnel accept https connections via e.g.
accept = 0.0.0.0:443and forward them decrypted via e.g.
connect = localhost:1234to your application. By doing so, the SSL encryption is transparent to your Python application which also applies to open websockets if I understand things correctly. Of course this requires a careful security considerations behind your load balancing/stunnel (at least that your Python app must not reachable from the outside).
stunnel’s use with respect to load balancing and websockets is discussed for example here:
www.mail-archive.com/haproxy@formilux.org/msg02799.html
www.mail-archive.com/haproxy@formilux.org/msg02679.html
Hope that somehow helps..
Jan-Philip
Btw: Thanks to Sylvain for this library. Currently, I am using gevent-websocket, but we all should stick to some standard and so I think I will be using ws4py, too
Jan-Philip:
Thanks for the recommendation, I’ll take a look!
Cheers,
Tim
Hello Sylvain,
Thanks so much for your great work on this library. I’m pretty new to Python but have learned a lot from studying your code.
I’ve having trouble getting the chat example to work using CherryPy 3.2.2 and Chrome 18.0.1025.11 beta-m on WindowsXP. My clients connect just fine and are able to send messages. Publish() is called from ChatWebSocketHandler, and eventually calls output.append(). However, the message is never received on the browser side, even though TCPView shows that the connections are established.
Any idea of what could be going wrong?
Thanks again,
Mark
That’s weird indeed. I’ve been refactoring ws4py quite extensively internally since yesterday so hopefully things will be more stable once I commit (in theory soonish this weekend).
I may have narrowed it down a bit.
isinstance(data, bytearray) is returning false. isinstance(data.data, bytearray) returns true, however.
Not sure what this means. BTW I’m running Python 2.6.
Just getting my head around Web Sockets and ws4py. I had a gevent-based test server working using a WSGI function to send data based on URL, but with your recent code changes it appears that everything has to happen inside some sub-class of WebSocket, with no apparent way of getting the environment. Am I missing something? What about having **kwds on the __call__ of WebSocketUpgradeMiddleware and passing them onto websocket_class ??
Thanks for the work.
Dave
Hi Dave,
My fault. I should’ve been more careful with those aspects. I’ll fix it asap.
Hi Sylvain,
Great work.
What would greatly help me would be a simple example of a host-based process periodically sending data to a browser once a websocket is established. I’m looking at your chat example, which is great, but I’m more interested in point-to-point communication to different clients, and the threading required not to block.
Thanks!!
Marty
I downloaded the latest zip file of websockets. In cherrypyserver.py start_handler has some embedded tabs that makes for incorrect formatting, but in such a way that the python interpreter doesn’t complain. As well, at the end of the upgrade routine, on line 197, is the following line:
addr = (request.remote.ip, request.remote.port)
It has an embedded tab, and addr is unreferenced for the rest of the method. This is the same text with the incorrect formatting in start_handler, making me think some erroneous cut/paste operations got included.
Thanks,
Marty
Hi Marty,
I apologise for the delay, I forgot to check in. It would be easier if you could open a ticket on github instead of commenting issues here.
https://github.com/Lawouach/WebSocket-for-Python
Cheers,
Hi – I am currently making a higher level of libraries on top of ws4py and gevent, the goal of this little project is to make the realtime web apps development easier. Before I start to make even more decisions into this project, I want a little help from you on the geventserver. I know that geventserver is made out of gevent.pywsgi, somehow, I am still not sure how the server closes a ‘thread’, thus everything including the websocket and other stuff get cleaned up when a client closes purposely or unpurposely an established connection.
Many thanks on this question.
Hi,
The WSGI server is only there to parse the initial Upgrade request. Once the handshake and the upgrade have been completed, ws4py spawns a new greenlet which runs the WebSocket.run method. That greenlet runs as long as the WebSocket stream is opened (and therefore the underlying socket connection). Whenever the run() method terminates, the greenlet terminates as well.
Hi Sylvain – Thanks very much for what’s explained. Here is the further thinking on this, put it simply, how does server detect if a connection is disconnected? Almost certain, in a realtime application, the established connections will somehow want to talk to each other, broadcasting events or the like, this is where we need a pool or something to manage a collection of previously established connections. The ws4py.websocket provided several handy entry-point to do this, for example, we could register one connection into the pool in websocket.opened, and pop out one in websocket.closed, but, what if the disconnection isn’t a natural one? Instead of being initiated from either server or client, the disconnection is caused by a poweroff. The question is, how do we effectively and reliably detect a disconnected connection?
Regards,
Well in that case, the socket will timeout after a while (usually determined by the underlying OS since ws4py is blocking on the socket) and provoke an exception which would trigger the closed handler.
With that said, you’re not the only one wishing for something more usable than a timeout (which can be quite long). For now, a solution would be to add some component that simply monitors each socket (like an event loop). Perhaps based on the select family…
For now, I haven’t found the proper way of doing this without external dependencies and in a portable fashion. I wish to keep ws4py as light as possible with a lib-spirit rather than a framework approach.
Hi Sylvain – Thanks. Well the good thing is that after all there would be a timeout being propagated providing a chance to clean the zombies, though it might be quite long. (I guess it is also possible to globally set a reasonably long timeout? though this decision is by no means to be made inside a library such ws4py)
The monitor-loop you mentioned, is