Planet TurboGears

September 08, 2012

Alessandro Molina

It’s a Pluggable World

One of the new additions in TG2.1.4 has been the support for the so called pluggable applications, this is a really powerful and convenient feature that probably not enough TurboGears users started embracing.

For people that never used them, pluggable applications provide a python package that can be installed and “plugged” inside any existing TurboGears application to add new features. Django has been probably the first framework to bring this feature to Python world and TurboGears implementation tries to be as convenient by making pluggable applications identical to plain TurboGears applications and providing a “quickstart-pluggable” command that creates the application skeleton for you. Pluggable applications can be installed using easy_install or pip and they can off course depend on any other pluggable application they need.

This year, at EuroPython 2012, I have been pleased to present a talk about using TurboGears for rapid prototyping (both in Italian and English, you should be able to find the videos on EuroPython youtube channel), so I decided to dedicate a part of it to pluggable applications as they are actually the fastest way to rapidly prototype a project. With my surprise most the questions I received were about the EasyCrudRestController and not about pluggable applications.

While the EasyCrudRestController is definitively a powerful tool, it’s far from being the answer to all the web developments needs. In most of the applications you are going to develop, users will probably prefer consulting content from something more engaging than an administration table of database entries.

This month, to create a set of utilities that can help people with their everyday needs, I decided to ask guys that work with me to make every part of the web sites that they were writing as pluggable applications. The result of this experiment has been that most of the pluggable apps that I did in my spare time (tgapp-smallpress, tgapp-photos, tgapp-tgcomments, tgext.tagging and so on) ended being used in real world projects and started to improve exposing hooks and ways to customize their behavior for the project they were going to be used.

After a few weeks, new pluggables like tgapp-fbauth, tgapp-userprofile, tgapp-calendarevents, tgapp-fbcontest, tgapp-youtubevideo has seen light and developing the target application started becoming blazing fast: Just plug what you need and customize it.

Embracing this philosophy the last project I’m working on has an app_cfg.py file that looks like:

plug(base_config, 'tgext.debugbar', inventing=True)
plug(base_config, 'tgext.scss')
plug(base_config, 'tgext.browserlimit')
plug(base_config, 'registration')
plug(base_config, 'photos')
plug(base_config, 'smallpress', 'press', form='XXX.lib.forms.ArticleForm')
plug(base_config, 'tgcomments', allow_anonymous=False)
from XXX.lib.matches import MyKindOfEvent
plug(base_config, 'calendarevents', 'eventi', event_types=[MyKindOfEvent()])
replace_template(base_config, 'smallpress.templates.article', 
                              'XXX.templates.press.article')
replace_template(base_config, 'smallpress.templates.excerpt', 
                              'XXX.templates.press.excerpt')

Thanks to this our development process has really improved: whenever a developer finds a bug he just has to propose a patch for the target pluggable, whenever someone notices a missing index on a query he has just to add it to the given pluggable. All the websites under development improved like people were working on the same project.

While existing pluggables might be limited, buggy or slow I’m getting confident that they will continue to improve, and some day they will surpass whatever custom implementation I can think of. I think I’m going to heavily rely on pluggable applications for any future project sticking to only one rule: “make it opensource”. This way, apart from probably helping other people, I’m also improving my own projects through other people feedbacks, bug reports and patches to the pluggables I used.

So, next time you have to start a new project give a look at the TurboGears CogBin and check if there is a pluggable application that looks like what you need. If you find any issue or find space for improvements just fork it and send a pull request, or send an email on the TurboGears Mailing List I’ll do my best to address any reported issue thanking you for your feedbacks as I’m aware that you are actually improving any past and future project that relies on that pluggable.

by amol at September 08, 2012 09:30 PM

August 23, 2012

Michael Pedersen

Announcing TurboGears 2.2.0!

That's right, we finally did it! It's up right now, and if you follow the directions on www.turbogears.org/ you'll get 2.2.0.

I'd like to take a minute to thank Alessandro Molina in particular. I did what support I could, but this release really belongs to him. Please pass along congratulations and thanks to him. And if there's bugs, I'll take the blame myself for not doing more to help.

The full changelog is below.



Important Commit Messages:


  • Fix template engine extension collision in tw2.core >= 2.1
  • Permit TGController subclasses to keep the decorations made on the parent class methods around
  • Support repoze.who v2
  • New auth configuration layer that doesn't require repoze.what anymore while still being compatible
  • Support validation dict using TW2 validators and make validation less dependant from FormEncode
  • Always prefer default renderer when multiple engines are registered for the same content_type
  • Fix empty documentation for some decorators
  • Remove url kwargs which were marked for removal in 2.2
  • Remove legacy renderers support
  • Disable AuthMetadataProvider when is None
  • Improved kajiki loader to solve some issues with py:extends
  • use .jinja extension for jinja templates by default
  • On Mako when using file loader templates where reloaded using the auto_reload_templates option, while when using dotted loader the mako.reloadfromdisk option was used. Merge the two options in auto_reload_templates as all the other templates engines use this one
  • __before__ and __after__ were renamed to _before and _after
  • Expose render call as a public api with render_template name
  • Create sqla module for sqlalchemy related things and add BalancedSession for master/slave load balancing
  • Change prefix for tw2 to make resources archiving work correctly
  • prefer_toscawidgets2 option in app_cfg enables tw2 and disables tw2. This should be the default in 2.2 newly quickstarted projects
  • Add option for setting the template rendering parameters.
  • Permit to avoid doctype injection when manually rendering templates with tg.render.render
  • Object dispatch has been delegated to the crank package, improving performance and test coverage
  • Quickstart new applications using repoze.who v2
  • Add dynamic response.charset to meta in mako templates
  • New quickstart template
  • Quickstart projects without repoze.what requirement using new auth layer
  • Fix issue with kajiki quickstart
  • Adapt the bootstrap based template to jinja changes, always provide master.html for extensions and remove the minimal template as the new one is minimal enough
  • Make genshi always available to have a lingua franca for pluggable apps and extensions
  • Storing sessions in encrypted cookies is easier to manage then the file based solution for newly quickstarted projects
  • tgext.admin now works with ToscaWidgets2
  • Paste quickstart command will now ask if you want to use an alternative templating system instead of asking for each one
  • New Upgrading TG2 page
  • Document new inherit option of @expose with examples
  • Fix missing kajiki setup_renderer reference and remove reference to default_renderer
  • Update Authentication documentation
  • Document flash and caching
  • Document lurl usage
  • Document WebFlash
  • Add more docs and example about app_globals and tmpl_context
  • Document ugettext and lazy_ugettext
  • Document DBsession().using_engine switch to a context manager
  • Minor fixes in MultipleDtabases documentation
  • Added documentation about master slave load balancing
  • Removing some old and now invalid parts of the documentation and add ToscaWidgets2 documentation
  • Remove some references to tw1 in core parts of the doc


Tickets Closed:
1 Improve documentation on authentication, identification and authorization
2 Include lazy translations in the documentation
11 Document SecureFormMixin from TW
19 DataGrid and pagination tutorial/​reference missing for TG2
21 sqlite db creation error in wiki20 tutorial
26 Documentation for app_globals unclear
27 Document webflash aka tg.flash
51 Turbogears 2.1b1 tgext.admin Doesn't Handle Boolean Options
57 add a --clean option to quickstart
68 Simplify database migration in TG2
70 Migrating turbogears to pip
71 Create some kind of "component" architecture for TG2
75 quickstart graphics aren't easy to modify
137 Proper use of url() for widgets in the documentation
138 Backport repoze.what-quickstart and repoze.what-pylonshq removal
139 Move dispatch to Crank
140 Verify quickstart and devtools dependency
149 Expose the tg.render.render function as render_template
150 Full ToscaWidgets2 support and make it default when quickstarting app
151 Minimal Search and Sorting functions in admin
152 Jinja Support in quickstart
153 Master /​ Slave database support
154 Merge autoreload options for template engines
156 Provide a way to quickstart with Kajiki
157 repoze.who-1.0 is bugged when using auth_tkt
158 Permit to inherit decorations on TGController subclasses
160 allow_only doesn't get checked in dynamically dispatched RestController

by Michael Pedersen (noreply@blogger.com) at August 23, 2012 11:15 PM

August 16, 2012

Alessandro Molina

What’s new about Sprox 0.8

Today Sprox 0.8 got released, it is the first release to add ToscaWidgets2 support. Depending on which version of ToscaWidgets is available inside your environment Sprox will either use TW1 or TW2 to generate its forms.

Being mostly a TW2 oriented release it might seem that not a lot changed since the previous version, but a little gem is hidden between all the TW2 changes as Sprox now supports setting default behavior for models themselves using the __sprox__ attribute inside model declaration.

class Parent(DeclarativeBase):
    __tablename__ = 'parents'
 
    uid = Column(Integer, primary_key=True)
    data = Column(String(100))
 
class Child(DeclarativeBase):
    __tablename__ = 'children'
 
    class __sprox__(object):
        dropdown_field_names = {'owner': ['data']}
 
    uid = Column(Integer, primary_key=True)
    data = Column(String(100))
 
    owner_id = Column(Integer, ForeignKey(Parent.uid))
    owner = relation('Parent')

The previous code example makes Sprox use the Parent data field for selection fields when choosing the parent of Child entities.

Apart from making easier to share options between your AddRecordForm and EditableForm __sprox__ attribute opens a great way to customize the TurboGears admin.

By adding a __sprox__ attribute inside your models you will be able to change the TurboGears admin behavior without having to create a custom admin configuration. Setting __sprox__ attribute makes possible to change most sprox properties changing CrudRestController behavior, the same properties that are documented on sprox.org can be specified inside the __sprox__ attribute by simply removing the underscores.

by amol at August 16, 2012 07:53 PM

July 21, 2012

Carlos Daniel Ruvacalba

Jinja2 Bytecode Cache

I have been a bit busy and thus off from turbogears development for a while, but I see 2.2 release shaping up really good, props for Michael and Alessandro for their work.

Now, looking at the stuff that I’m most familiar with (jinja2 support code), I have been thinking for a while that jinja2 does not have bytecode caching enable by default, in fact, there is no support at all in tg2 code for that, while Genshi does have caching available jinja2 is lacking in this area, where enabling the cache is actually very simple.

I created a few tests with the latest 2.2rc2 turbogears with jinja bytecode caching enabled, I used the included filesystem based cache and a custom (simple) in-memory cache, as we can see on the graphics from the benchmark the gains from enabling the cache a bit small, around 4 request per second more, but if you are looking to squeeze performance out of tg2 this could be a good option as its practically free, the cache is not really intrusive, if I make a change to the template it is reloaded automatically thus it doesn’t affect the normal development workflow.

spacer

Jinja2 Bytecode Benchmark: Cached vs Uncached

I should, however, do some additional test under gunicorn or uwsgi to see if there are even more substantial benefits to enabling the bytecode cache.

by Carlos Daniel at July 21, 2012 04:43 PM

July 16, 2012

Michael Pedersen

Are we ready for +TurboGears 2.2.0?



It's time to ask that most dreaded and anticipated of questions: Are we ready to release 2.2.0?

We've pushed out out second release candidate. We've had it available for testing. Only one flaw has been found, and that can be integrated into the release easily (and since it's a security flaw, it will be).

What say you, the community? Are you satisfied with what will be released if we were to release right now? I'd like to make the release official, and happen on Thursday, so speak before then!

Note that if you reshare this, please block comments. Make people reply to this post, so that we can be sure we'll see any replies in a timely fashion.

Thank you!

by Michael Pedersen (noreply@blogger.com) at July 16, 2012 09:32 PM

July 14, 2012

Michael Pedersen

Custom Renderers with Dynamic Content on TurboGears

One of the most common questions we get on the TurboGears mailing lists is actually a group of questions:
  • How do I make a controller return a customized PDF?
  • How do I display a custom graphic to a user?
  • How do I return an Excel spreadsheet?
This all boils down to the same question: How do I generate dynamic content that is tailored for the user and is returned in a non-HTML format? While I won't answer for each format (because of the fact that every format gets built differently), I will show you how to tell TurboGears about the format you want to send to the user.

The actual process of generating the output can be quite complex. For instance, if you are generating a PDF, you could be using ReportLab. If you're generating an Excel 2003 compatible file, you could be using Python-Excel. The list goes on and on. Because of this, I'm going to focus on a simple example: The vCard format. If you want to have your application send vCards (.vcf files), this example will show you how to do it, from beginning to end.

Surprisingly enough, this cen be almost entirely done by modifying the configuration of your TurboGears application. In my case, I added a class to find (and load) a template file, a class to manage the actual rendering, and then changed the base configuration class to register those two classes. I put everything except the template into my app_cfg.py file.

I'll show the classes, and then break down what each piece of the class does.

The Template Loader

from genshi.template import TemplateLoader, NewTextTemplate

class VCardTemplateLoader(TemplateLoader):
template_extension = '.vcf'

def get_dotted_filename(self, filename):
if not filename.endswith(self.template_extension):
finder = config['pylons.app_globals'].dotted_filename_finder
filename = finder.get_dotted_filename(
template_name=filename,
template_extension=self.template_extension)
return filename

def load(self, filename, relative_to=None, cls=None, encoding=None):
"""Actual loader function."""
return TemplateLoader.load(
self, self.get_dotted_filename(filename),
relative_to=relative_to, cls=NewTextTemplate, encoding=encoding)

First, we specify the file extension for the template. The normal file extension is .vcf, so I'm going to use that.

Next, the method get_dotted_filename is provided. This function attempts to find a real operating system filename for a given dotted filename that was passed in. A dotted filename will have dots as the path separators. So, for instance, a file name of "/my/path/file.vcf", in dotted form, will be "my.path.file". This function will be given the name that is provided to the @expose decorator. Since we don't have any need for special behavior here, we just go with the default dotted_filename_finder. If there is a need to change the normal search method for finding the dotted filename, we can do so here.

Finally, the load method: This method's job is to return an object that can be rendered. In this case, it will return an object provided by Genshi that represents a loaded template, ready to be rendered. The return value of this method will be passed into the __init__ method of the Renderer class, up next.

The Renderer

from pylons import templating

class RenderVCard(object):
"""Singleton that can be called as the vcard render function."""

genshi_functions = {} # auxiliary Genshi functions loaded on demand

def __init__(self, loader):
if not self.genshi_functions:
from genshi import HTML, XML
self.genshi_functions.update(HTML=HTML, XML=XML)
self.load_template = loader.load

def __call__(self, template_name, template_vars, **kwargs):
"""Render the template_vars with the Genshi template."""
template_vars.update(self.genshi_functions)

doctype = 'text/x-vcard'
method='vcf'
kwargs['doctype'] = doctype
kwargs['method'] = method

def render_template():
template = self.load_template(template_name)
return template.generate(**template_vars).render(encoding=None)

return templating.cached_template(
template_name, render_template,
**kwargs)


This class is set up as a singleton, allowing any thread to use it without having to reinitialize it all the time. genshi_functions are put in place as a dictionary for the class, allowing us to have only one reference to the HTML and XML functions from Genshi itself.

The __init__ method receives an instance of the loader class (from above). Since this rendered is aware that we are using Genshi, it gets a reference to the Genshi HTML and XML functions, setting them up in the genshi_functions dictionary (if this was not already done). It then saves a reference to the loader.

The __call__ method allows the renderer to be called by TurboGears. Failure to include this method will result in TurboGears being unable to render your custom content type, so make sure to provide it! It will be given a name (the name provided to the @expose decorator in your controller), a dictionary of variables, can receive other (optional) keyword arguments.

In our case, we are only going to care about the method and doctype keywords, and even then only to set them to their correct values.  The template_vars dictionary will be updated to include the Genshi functions, allowing HTML and XML to be referenced in the template.

Finally, we declare a function inside of the __call__ method. This allows us to use variables defined in the main function optionally. The final step, the return statement, will attempt to cache the result. If it cannot, or if it cannot use the cached result, then it will call the nested function (render_template), which can use all the variables defined in this scope. We could, alternately, just return the result of render_template, instead of the cached_template.

The Config

from tg.configuration import AppConfig, config

class MyProjectConfig(AppConfig):
def setup_vcard_renderer(self):
loader = VCardTemplateLoader(search_path=self.paths.templates,
auto_reload=self.auto_reload_templates)

self.render_functions.vcard = RenderVCard(loader)


base_config = MyProjectConfig()
base_config.renderers = []
#Set the default renderer
base_config.default_renderer = 'genshi'
# Add vcard rendering (genshi plain text renderer)
base_config.renderers.append('vcard')

Here, we define a function and we must be careful about the name: setup_vcard_renderer will set up a rendered named "vcard", and that will be put at the front of the @expose decorator. So, if we want to write this:

@expose('vcard:myvcard')

We're okay. If we want to write this:

@expose('vcf:myvcard')

Then the setup function must be named "setup_vcf_renderer". We also would have to change the last line of the function to say "self.render_functions.vcf" instead of "self.render_functions.vcard".

Note that the body of the setup_renderer function simply creates a loader, and then sets the value for the renderer function to be a new instance of the renderer class from above.

The Template

That's it for code changes. Since I chose to make this into a Genshi rendered template, I also had to write the template. Here it is:

{% python from datetime import datetime; now = datetime.now(); urls = user.links_to_dict(); phones=user.phones_to_dict() %}BEGIN:VCARD
VERSION:2.1
FN:${user.display_name}
{% if user.title %}TITLE:${user.title}{% end %}
{% for phone in phones %}TEL;${phone.upper()};VOICE;${phones[phone]}{% end %}
ADR;HOME:;;${user.streetaddress.replace('\n','=0D=0A')};${user.city};${user.state_province if user.state_province else ""};${user.postal_code};${user.country if user.country else ""}
LABEL;HOME;ENCODING=QUOTED-PRINTABLE:${user.streetaddress.replace('\n','=0D=0A')}=0D=0A${user.city}, ${user.state_province if user.state_province else ""} ${user.postal_code}=0D=0A${user.country if
user.country else ""}
{% if user.email_address %}EMAIL;PREF;INTERNET:${user.email_address}{% end %}
{% if 'homepage' in urls %}URL:${urls['homepage']}{% end %}
REV:${'%04d%02d%02dT%02d%02d%02dZ' % (now.year, now.month, now.day, now.hour, now.minute, now.second)}
END:VCARD

And that's it. Very simple template, as you can see. It does rely on functions and attributes that are in the user model, but that's a little outside of what I want to address here.

Conclusion

With minor variations, you can take total control over what you want to do. For instance, you can change the renderer class to one that looks at data in the request (tg.request), and generates output graphs using ImageMagick with .png file extensions. You could build your own XML templating syntax (or, use ReportLab Plus) to make .pdf files (again, changing the renderer class).

The possibilities are endless. You have control, from the way the template is found, to the way it is loaded, to the way it is rendered. If you would like to see this code in one file, it's available in our tutorials repository.

by Michael Pedersen (noreply@blogger.com) at July 14, 2012 09:48 PM

spacer

Subscriptions

Last updated:
September 10, 2012 01:45 PM
All times are UTC.

Powered by:
spacer

Planetarium:

gipoco.com is neither affiliated with the authors of this page nor responsible for its contents. This is a safe-cache copy of the original web site.