spacer
spacer

Planet Python

Last update: November 16, 2012 12:46 PM

November 16, 2012


Jaime Buelta

Magical thinking in Software Development

I guess we all Python developers heard this kind of argument from time to time:

Python is slower than C++/Java/C# because is not compiled.

Other than the usual “blame the others” when working with other companies (usually big corporations than thinks than using anything except C# or Java is laughable), you can also see a lot of comments in technical blogs or places like Hacker News or Reddit with similar, simplistic arguments. You can recognise them on the usual rants about how technology X is The Worst Thing That Ever Happened™ and Should Never Be Used™

That’s a form of Software Development Magical Thinking. This can be really harmful for software development, specially when the opposite, positive form is used. Let me define Software Development Magical Thinking in this context:

Software Development Magical Thinking noun Assuming that a technology will magically avoid a complex problem just by itself.

Probably that will become clearer after a couple of examples:

Java is a static type language and it is safer than dynamic type languages like Ruby.

We program in C++ so our code is very fast.

MongoDB / NodeJS / Riak is web-scale.

Please note that those are not completely, utterly wrong statements. C++ can be very fast. Static typed languages can avoid some bugs related with input parameters type. But there is no guarantee that creating a system in C++ is going to act like a magic wand against slow code. Or that Erlang will avoid having a single point of failure. And you’ll get as sick of bugs and security issues both on static type language and dynamic type languages. *

Those are all complex problems that needs careful design and measurement to deal with them. Deep analysis of the problem, which is usually more complicated that appears on the first place. Or worst, the problem is not as bad as it looked and the designed system is more complex that it should, trying to catch a problem that never arises. Not to exclude previous experience to avoid subtle errors.

Let me say it again. There are problems that are HARD. In software systems they are confronted almost daily. And no single thing will make you forget them. Even if you use a very good tool for what you’re doing (like Erlang for concurrency), which usually implies paying a price (in development time, etc), doesn’t replace vigilance and issues could eventually appear. Unfortunately, making software is tough.

The problem with Software Development Magical Thinking is that it is very easy and it is also very natural. Seductive. We know that “general Magical Thinking”, simple solutions to very complex problems, is quite common. Hey, a lot of times, it even seems to work, because the Feared Problem will only present after certain size that is never attained, or after the designer leave the company and left a latent problem behind. Most of the time, making a totally informed decision is unrealistic, or simply not possible, and some risks must be taken.

But as software developers we should know that things are not that easy, even if we have to compromise. Each bug that takes time methodically eliminating causes. Every measurement that makes you wonder what is the best metric to reflect a value. Every time you realise that there was a back-of-the-envelope calculation that shows something that will have an impact on some design aspects. Those are all reminders that should makes us think that there are no silver bullets and we shouldn’t take lightly all those difficult problems.

Make decisions. Design systems. Choose a tool over others. Take risks. But don’t be delusional and careless. Be conscious that software can bite you back. Be vigilant. Be skeptic. Avoid Magical Thinking.

PD: And please, don’t say “Python is slow”. Just don’t. It is not for most of the jobs. It is not going to make you win a discussion unless you carefully measure and proof it. And, perhaps most importantly, raises my urge to kill.

* No, I am not going to comment anything the Mythical Web Scale property.


Tagged: development, english, erlang, mongo, mongodb, python, ruby spacer spacer

November 16, 2012 06:03 AM


Vasudev Ram

Mongoose web server does have a Python binding

In my previous post, about the Mongoose web server, I said I wished it had a Python binding.

It does have one. I must have missed seeing it due to the small screen of my mobile.

Many readers commented, telling me of my mistake. Thanks to all of them.

Though I saw the comments via Blogger email notifications, I could not publish any comments because I only have my mobile right now, no laptop, being on vacation in a remote area, and Blogger Android app not supporting comments at all, and Blogger web UI not working on my phone at all. I must get a better phone or a tablet soon  ...

Wikipedia article on mongooses is interesting:

en.m.wikipedia.org/wiki/Mongoose

I had seen a few mongooses when I used to roam around in the forests in Central India when I was a kid.

en.m.wikipedia.org/wiki/Indian_gray_mongoose

- Vasudev Ram
www.dancingbison.com

Vasudev Ram

November 16, 2012 12:44 AM

November 15, 2012


Andriy Kornatskyy

Lazy Attribute in Python

A lazy attribute is an attribute that is calculated on demand and only once. Here we will see how you can use lazy attribute in your Python class. Setup environment before you proceed: $ virtualenv env $ env/bin/pip install wheezy.core Let assume we need an attribute that is display name of some person Place the following code snippet into some file and run it: from wheezy.core.descriptors

November 15, 2012 09:24 PM


Jaime Buelta

Talks on PyCon Ireland 2012

Well, as usual, this year’s PyCon Ireland has been amazing. I always get impressed by the high quality of the talks and, in general, how much the attendants know. It is always a pleasure to share some thoughts about technology with incredibly talented people. Python Ireland is doing a great job.

This year I didn’t give one talk, but TWO! It was very exhausting, but fun. I am posting the slides here, in case someone find them interesting…

You can also download the source Keynote file, which includes notes.

EDIT: Videos added


Tagged: english, pycon, python, python ireland spacer spacer

November 15, 2012 11:18 AM


Davy Wybiral

Numeric Javascript

I just recently found this javascript library. So far it's looking pretty good. I've always wanted some sort of Javascript equivalent to NumPy and while this isn't anywhere near that extreme, it does offer some pretty handy features.

Here's a jsFiddle I put together to show it performing a simple implementation of PCA (Principle Component Analysis) on a fictional dataset. The first component should have a slope of approximately 0.357

Here's a quick snippet, the actual PCA function using numericjs...

function pca(X{
    /*
        Return matrix of all principle components as column vectors
    */        
    var X.length;
    var sigma numeric.div(numeric.dot(numeric.transpose(X)X)m);
    return numeric.svd(sigma).U;
}

November 15, 2012 11:15 AM


Vasudev Ram

Mongoose, tiny C web server and library

mongoose - Mongoose - easy to use web server - Google Project Hosting

Just saw this. The site says Mongoose is very small, written in C, and can be used both standalone and  as a library, to embed a web server in your applications. Seems potentially useful.

I had blogged about an IBM developerWorks article by Cameron Laird about lightweight web servers earlier, on this blog, IIRC.

The area interests me because it seems to have a lot of possibilities. And much earlier, I had blogged about web servers running on mobile phones, on my earlier blog,  jugad's Journal - jugad.livejournal.com.  The point of that is that it enables phones to act as web servers, not just as clients. That could open up a lot of new applications and spur innovation.

Wonder what would happen if Mongoose met  Python?

Sorry, couldn't resist :-)

I actually meant it in a good sense, as in, if someone develops a Python binding for Mongoose, since it is a library. That could lead to a lot more usage of it.

- Vasudev Ram
www.dancingbison.com

Vasudev Ram

November 15, 2012 02:27 AM


Greg Taylor

python-route53 released!

After some more time in the cooker, python-route53 1.0 has landed on PyPi. This is a stand-alone Route 53 package, independent from the one in boto. The major hilights are:

Read the documentation, see the source, and yell at me on the issue tracker or Twitter.

November 15, 2012 01:19 AM

November 14, 2012


Montreal Python User Group

Python Workshop: Introduction to Django

Have you ever wanted to write a dynamic web site?

Pythonistas, let’s continue our magnificent journey on the Pythonic Way. It’s now time to dive head first in web development using Django, a Python web framework. Our last workshop [1] of the Winter 2012 series  will show how: “Beautiful is better than ugly” and “Simple is  better than complex” apply also to the web. — Zen of Python.

The Montréal-Python fictional Rugby League (MPRL) desperatly needs  a web app to manage its teams and its season. In only 3 hours, we’ll make this happen.

Let’s code this together:

The workshop will be in English but all nations are welcome. No need  to know rugby nor having attended previous workshop. Our workshops are free and open to all.

The prerequisites, the technical setup and the content of this workshop can be found on the workshop’s wiki page.

Please take time to install the softwares (Python, Django [, South, IPython]) before the workshop. If you run into any issues during the installation, don’t  panic, Montréal-Python team will there at 5:30 PM to help you setup your laptop for the workshop, whatever OS you are running (The installation procedures are on the wiki page).

November 14, 2012 10:12 PM


Sandro Tosi

Project Euler - Problem 4

Here's my solution to Project Euler problem 4:

# handy function to check if a number is palindrome
def is_palindrome(i):
return str(i) == str(i)[::-1]

# what's the max?
max_p = 0

# multiply all numbers `i` between 100 and 998, with all between i+1 and 999
for i in xrange(100, 999):
for j in xrange(i+1, 1000):
p = i * j
if is_palindrome(p) and p > max_p:
max_p = p

print max_p

Comments are welcome.

Update: fixed as per Alex comment.

November 14, 2012 09:31 PM


Carl Trachte

The Ada Initiative - why I donated

This post has little to do with Python, but a fair bit to do with the programming community.

I donated to the Ada Initiative recently.  The option I went with was the $1/day for a year, or $365.  In the receipt one of the officers of the organization left the note, "If you have a minute, can you let your friends know why you donated?  People usually donate because someone they trust supports the Ada Initiative."  Hopefully this post does something postitive to promote the organization which promotes women in computing.

First the disclaimers.  Do I agree with all the members of the Ada Initiative on everything including their approaches to promoting women in computing?  Have I been a model citizen and will I behave angelically and perfectly with regard to the promotion of women in computing?  Not entirely and probably not, respectively.

That said, I think the Ada Initiative has the best chance to make a difference in this realm.  I saw that Mary Gardiner is involved.  I don't know her personally, but I do know that she has done some quality work reviewing Pycon papers for the Python community.  Her online communication always impressed me as measured and intelligent.  Smart and not a jerk go a long way in my book.  Her participation in the project gives me a warm and fuzzy that my money will be put to good use.  Someone with a track record of working hard and producing results is involved.  Further, I believe from tweets that Leigh Honeywell is a supporter of the organization.  I met her at Pycon a few years back and have followed some of her online activity.  Her promotion of the cause lends even more credibility in my eyes.  The woman who dealt with my donation and setting up the matching grant from my employer, Valerie Aurora, was very helpful and professional.  All in all, I have very strong confidence that these dedicated, hard working women will affect good in the world, or at least give it a damn good effort.

Lastly, there are personal reasons.  As some of you may know, I lost my wife earlier this year.  My wife was a Unix shell scripter back in the early 90's.  She put up with a lot of nonsense at work.  An incident that sticks in my head is a Christmas party where a coworker came up to her and arrogantly and dismissively said, "Nice t*ts."  That sort of thing just isn't right and did contribute to her leaving the field.  It's hard to explain, but when you are faced with mortality issues, you end up rehashing a lot of things in your head.  In the end you're generally a lot less timid about doing things you always wanted to or thought you should do.

Well,  that's the story.

November 14, 2012 06:33 PM


Ed Crewe

Cookie law, Cookieless and django tips.

django-cookieless

Last week I released a new add on for django, django-cookieless, it was a relatively small feature that was required for a current project, and since it was such a generic package seemed ideal for open sourcing as a separate egg. It made me realise that I hadn't released a new open source package for well over a year, and so this one is certainly long over due in that sense.

Cookie Law

It also over due in another sense, EU Cookie law has been in force since May 2011, so legally any sites that are used in Europe, and set cookies which are not strictly necessary for the functioning of the site, must now request permission of the user before doing so. Of course it remains to be seen if any practical enforcement measures will happen, although they were due to this summer in the UK, for example. Hence many of the first rush of JavaScript pop up style solutions, have come and gone, as a result of user confusion. But for public sector clients particularly, it is certainly simpler to just not use cookies, if they are not technically required. Also it may at least, make developers rather less blasé about setting cookies.

Certainly most people would prefer not to see their browsers filled with deliberate user tracking and privacy invasive cookies that are entirely unrelated to the sites functionality. In the same way most of us don't like being tracked by CCTV everywhere we go ... unfortunately, the current Law doesn't have a good technical solution behind it, hence it may well founder over time. This is because cookie control is too esoteric for ordinary users, and even with easy browser based privacy levels configuration, any technical solutions are problematic, because a single cookie can be used to both protect privacy (in terms of security - e.g. a CSRF token) and invade it.  It is entirely down to the specific applications usage of it, where these distinctions lie. Invasive methods can also be implemented via other session maintenance tools, such as URL rewriting, yet because no data is written to the users browser, it is outside the remit of this Law, so the Law makes little sense currently, and may well be unenforceable.

Perhaps it would of been better to aim to set laws related to encouraging adherence to set standards of user tracking, starting with compliance with the browser added 'Do Not Track' header, perhaps adding some more subtle gradations over time. With the targets of the Law, being companies whose core business is user tracking for advertising sales etc., starting with Google and working down. Rather than pushing the least transgressive public service sector, as the most likely to comply, to add a bunch of annoying 'Will you accept our cookies?' pop ups.

However even if  this law dries up and blows away, for our particular purposes, we needed django to cater for any number of sessions per browser (as well as not using cookies for anonymous users).
Django's default session machinery requires cookies, so ties a browser to a single session - request.session set against a cookie. But because django-cookieless provides sessions maintainable by form posts, it automatically delivers multiple sessions per browser.

There are a number of security implications with not using cookies, which revolve around the difficulty of preventing session stealing without them. Given this is the case, django-cookieless has a range of settings to reduce that risk, but even so I wouldn't recommend using it for sessions that are tied to authenticated users, and hence could lead to privilege escalation, if the session were stolen.

Django Tips

I thought the egg would be done in a day, but in reality it took a few days, due to a number of iterations that were necessary as I discovered a series of features around the lesser known (well to me) parts of  django. So I thought I would share these below, in case, any of the tips I gained are useful ...

  1. The request object life cycle goes through three main states in django:
    unpopulated - the request that is around at the time of process_request type middleware hooks - before it gets passed by the URL handler to decorators and then views.
    partly populated - the request that has session, user and other data added to it (mainly by decorators) and gets passed to a view
    fully populated - the request that has been passed through the view to add its data, and is used to generate a response - this is the one that process_response sees.
  2. I needed to identify requests that were decorated with my no_cookies decorator at the time of process_request. But the flag it sets has not be set yet. However there is a useful utility to work around this, django.core.urlresolvers.resolve, which when passed a path, gives a match object containing the view function to be used, and hence its decorators, if any.
  3. Template Tags that use a request get the unpopulated one by default.  I needed request to have the session populated for the option of adding manual session tags - see the tags code, to have the partly populated request in tags django.core.context_processors.request must be added to the TEMPLATE_CONTEXT_PROCESSORS in settings.

  4. The django test framework's test browser is in effect a complex mocking tool to mock up the action of a real browser, however like any mock objects - it may not exactly replicate the behaviour one desires. In my case it only turns on session mocking if it finds the standard django session middleware in settings. In the case of cookieless it isn't there, because cookieless acts as a replacement for it, and a wrapper to use it for views undecorated with no_cookies. Hence I needed to use a trick to set a TESTING flag in settings - to allow for flipping cookieless on and off.

November 14, 2012 05:14 PM


Michael Bayer

Pycon Canada - the SQLAlchemy Session In Depth

Video is up for my Pycon.ca talk, The SQLAlchemy Session - In Depth. In this talk, I delve into the key philosophies behind the design of SQLAlchemy's Session system. Starting with a brief review of the ACID model, I contrast the approach of the so-called "active record" pattern to that of the more explicit Session pattern, and how the two approaches integrate with the ACID model at work within a relational database. Afterwards, I present an HTML animation of a Session object at work.

  • Video (setting the quality to 480p is recommended)
  • Slides (pdf format)
  • HTML Demo (tar.gz)

November 14, 2012 01:31 PM


Tarek Ziade

Mozilla Services Hangout #1

We wanted to do this for a while: make the weekly show and tell we're having at Mozilla Services Europe every week a public event anyone can attend to.

Mozilla Services Europe is composed of Alexis, Hanno and myself.

I've tried to set up a BigBlueButton box in the past but I never found the time to finish setting everything up. Maybe one day...

In the meantine we used Google Plus Hangout, which gives us the ability to broadcast live the event on YouTube, and even record it for us. It's hassle-free, so yay!

The first hangout is composed of two parts:

tools we discuss during the hangout:

The recorded hangout on G+:

spacer

Please let us know if you liked it, what was boring or any issue or thing we can improve.

I will announce the next one as soon as we define a date - probably sometimes next week on Paris/Berlin time around 10 am.

November 14, 2012 10:54 AM


Lennart Regebro

Hello again Planet Python! Here’s what I have been up to!

I noticed last week that I wasn’t on Planet Python anymore. I have no idea why, or when, I got removed. Possibly I had the planet pointing to a feed that disappeared in a wordpress.com update or something. I’m sure nobody knows.

But, now I’m supposed to be back (thanks Michael), so this post is both a test, and an update of what I have been doing. Because as far as I know, I could have been gone for years. That sure taught me to check from time to time…

So what have I been doing the last few years?

I am also looking for work.


Filed under: python spacer spacer

November 14, 2012 08:56 AM


PyCon

PyCon US 2013: Big data? How about PyData!

spacer

You know what's better than one huge conference? One huge conference, and another on huge data!

(Sorry, I couldn't resist)

Today, I am proud to publicly announce a new development for PyCon / PyCon 2013 - we're partnering and working with the awesome team behind the PyData Conference to bring PyData Silicon Valley to the Santa Clara Convention Center during the sprints - that's right, come for PyCon, stay for the sprints and PyData!

PyData Silicon Valley will happen in the same building as the PyCon 2013 Sprints:

A bit more about PyData:

The PyData conference and workshop is a semi-annual event for scientists, engineers, and data analysts in the Python community. The conference focuses on techniques and tools for management, analytics, and visualization of data of different types and sizes with particular emphasis on big data.

Python is already the de-facto language for many areas of science and technology. Come join the conversation about its future in data science and analytics, and learn about all the great tools Python offers!

This is great news for both conferences - we are even looking at having our talk selection committee recommend talks submitted to PyCon 2013 for the PyData event if we run out of room, or see a better audience fit.

The PyData event is excellent - this years New York City event completely sold out - videos from that event to give you a taste are at vimeo.com/channels/pydata.

The PyData team really wants to hear from startups and teams within startups and other companies that are solving cool data problems with Python, so submit a talk proposal to pydataconf@continuum.io. They also to work with the community to get more talks & tutorials from women in the field - definitely reach out to them!

PyData is put on by the excellent team at Continuum Analytics (a PyCon 2013 sponsor) and the non profit NumFocus foundation. You can follow @PyDataConf on twitter for more info as it comes available.

Tickets to PyData Silicon Valley are on sale now through Eventbrite. Note that PyCon registration(s) and PyData registrations are separate! However, you can book your room for the extended stay if you're coming to PyCon through the PyCon registration system.

And of course, as always - we're still looking for PyCon 2013 sponsors!

- Jesse Noller, Chair - PyCon 2013

November 14, 2012 08:46 AM


Lennart Regebro

Python web expert looking for work!

Yes, I’m looking for work, either is the form of projects or a if you have an interesting offer, even a job.

My expertise is in Python and in web, especially together. I’ve been making web applications with Python for more than ten years now.

I’m not currently looking to relocate, so the job has to be remote or in Krakow, Poland.

My CV: Lennart Regebro


Filed under: django, plone, python, zope Tagged: job, projects, work spacer spacer

November 14, 2012 08:15 AM


Luke Plant

Dynamic typing in a statically typed language

A recent question on programmers.stackexchange.com asked What functionality does dynamic typing allow?

I thought one of the best short answers to this was from Mark Ransom:

Theoretically there's nothing you can't do in either, as long as the languages are Turing Complete. The more interesting question to me is what's easy or natural in one vs. the other.

This post is about providing an example to back that up, and to respond to people who claim that, since you can implement dynamic types in a statically typed language, statically typed languages give you all the benefits of dynamically typed languages.

[Edit: to those who think I'm being a language or dynamic typing advocate or engaging in any kind of bashing, please read that last paragraph again, and note especially the use of word 'all'.]

Let's set up a problem. It's made up, but it illustrates the point I want to make:

Given a file, 'invoices.yaml', take the first document in it, extract the 'bill-to' field, and save the data in it as JSON in an output file 'address.json'. You can take it for granted assured that the contents of that field can be serialised as JSON (e.g. doesn't contain dates), although that might not be true for the rest of the document. To keep the example focussed and simple, everything will be ASCII.

The particular YAML file I used was taken from an example YAML document I found on the web, and then expanded for the sake of illustration:

---
invoice: 34843
date   : 2001-01-23
bill-to:
    given  : Chris
    family : Dumars
    address:
        lines: |
            458 Walkman Dr.
            Suite #292
        city    : Royal Oak
        state   : MI
        postal  : 48046
---
invoice: 34844
date   : 2001-01-24
bill-to:
    given  : Pete
    family : Smith
    address:
        lines: |
            3 Amian Rd
        city    : Royal Oak
        state   : MI
        postal  : 48047

I'll use Python and Haskell as representatives of dynamic typing and static typing, because I know them and many would consider them to be very good representatives of their camps, and I'm a big fan of both languages.

I also think that examining any programming problem in the abstract, or with respect to ideas like ‘dynamic typing’ or ‘static typing’, is not very relevant, because in the real world we have to use real, concrete languages, and they come with a whole set of properties (in terms of the language definition, tool sets, communities and libraries) that make a massive impact on how you actually use them.

So I'm going to try to use real libraries that actually exist, ignore solutions that could theoretically exist but don't, and ignore problems that could theoretically exist but don't.

Python

Here is my Python solution:

import yaml
import json
json.dump(list(yaml.load_all(open('invoices.yaml')))[0]['bill-to'],
          open('address.json', 'w'))

Notes: I didn't have to consult docs once. This isn't j

gipoco.com is neither affiliated with the authors of this page nor responsible for its contents. This is a safe-cache copy of the original web site.