A Piece of CherryPy for CGI Programmers

An anonymous reader writes "IBM developerWorks is running an article outlining the strengths and offering some helpful advice on the Python framework 'CherryPy'. CherryPy uses the same concepts as CGI to bind a web server to a web application, but it improves performance and gains persistence across requests by handling all its requests within a single process."

Umm..

[Anonymous Coward]

FastCGI anyone?

I have been doing that with python and perl for years. I can even run the CGI on multiple different hosts with one webserver.

Whoops - sorry, to head off any criticism:

[TDScott]

that's a referral link in the parent post. To be honest, I'd recommend them anyway, but it's probably best to disclose it.

What being in a single process really means

[MostlyHarmless]

In the first few posts, I've seen a lot of relatively lacking-in-clue replies asking how CherryPy is different from ASP.NET, mod_python, FastCGI, etc. With most Apache-based web platforms, one process will handle many requests, but you cannot guarantee that every request will be handled by the same process: by default, apache starts multiple (possibly multi-threaded) servers, and creates and destroys them as necessary.

CherryPy, on the other hand, runs every request from the same process by using a thread pool instead of a process pool. This means that any global variables you change will be visible to any request. In many cases (keeping in mind memory restraints), you can share items in memory that would otherwise have to go through the database, which can help performance and make keeping track of state easier. Of course, multithreaded data sharing places its own demand on the programmer: the Python core is inherently thread-safe, but no programming language can protect you from race conditions and the like.

I've played around a little bit with CherryPy, and writing in it definitely feels Pythonic. It may still need some more development before it is fully mature, but it's something to at least keep an eye on.

(On a side note: I don't know how the IIS/ASP.NET process model works. It does let you store data across an application, but you are limited to a single Application hashtable, probably to be orthogonal to the Session and Viewstate objects and to reduce the likelihood that a programmer not experienced with concurrency would shoot him/herself in the foot.)

Re:What being in a single process really means

[nxtw]

In the first few posts, I've seen a lot of relatively lacking-in-clue replies asking how CherryPy is different from ASP.NET, mod_python, FastCGI, etc. With most Apache-based web platforms, one process will handle many requests, but you cannot guarantee that every request will be handled by the same process: by default, apache starts multiple (possibly multi-threaded) servers, and creates and destroys them as necessary.

ASP.NET uses a single process for each application. Every request is processed by the same process, both in Mono's implementation with mod_mono and on the IIS implementation. This is, as I said, nothing special, uncommon, or unique. My post was definitely not lacking in clue.

CherryPy, on the other hand, runs every request from the same process by using a thread pool instead of a process pool. This means that any global variables you change will be visible to any request. In many cases (keeping in mind memory restraints), you can share items in memory that would otherwise have to go through the database, which can help performance and make keeping track of state easier. Of course, multithreaded data sharing places its own demand on the programmer: the Python core is inherently thread-safe, but no programming language can protect you from race conditions and the like.

ASP.NET runs every request for an application within the same worker process. Static classes can be accessed from all threads throughout the application's lifetime (until it is stopped).

(On a side note: I don't know how the IIS/ASP.NET process model works.

Proving that you are indeed "lacking-in-clue"...

It does let you store data across an application, but you are limited to a single Application hashtable

This is not the only way to store data across an application in ASP.NET. Static classes, variables, etc. stick around for the entire time the application is running. ASP.NET also has its Cache object, which works like Application except Cached objects will expire at a programmer-defined time or after a programmer-defined timespan. The Cache object allows for callbacks for when an object is removed from the Cache, and cache dependencies.

I believe that ASP.NET's page and partial page caching uses this.

Re:You mean...

[Anonymous Coward]

Wow, tough crowd. Maybe Django does the trick for you? It does some unique stuff, and is python: http://www.djangoproject.com/ [djangoproject.com]

Nevow

[kevin_conaway]

I'll got ahead and put in a plug for Nevow [nevow.com] here, another web framework that is based on the EXCELLENT Twisted [twistedmatrix.com] framework.

If you're doing any sort of network programming in Python, you need to look at Twisted.

Re:Seems only useful if you already do Python CGI.

[justrob]

Using SQLObject is very popular with CherryPy users. CherryPy works with just about any templating system out there. This also makes it very easy to port from other Python web frameworks because you can use your existing templates.
  Subway [python-hosting.com] was created to use CherryPy, SQLObject and Cheetah templates in a very Ruby on Rails-like way, so you don't have to go through the 10 zillion decisions of what to use with CherryPy and tells you "what to do and where to put it".

Gee, it sounds just like...

[mengel]

... FastCGI which has had a several [sourceforge.net] python [lysator.liu.se] modules [suxers.de] for about 10 years...

But of course, if IBM says it's new, well it must be ;-)

Okay, I checked, and I exagerrated a little bit, the earliest CVS version on mod_fastcgi.c is:

Revision 1.1 / (download) - annotate - [select for diffs] , Tue Sep 16 15:38:22 1997 UTC (7 years, 11 months ago) by stanleyg

Re:Gee, it sounds just like...

[Anonymous Coward]

CherryPy can use FastCGI or mod_python or anything WSGI supports. Its not a replacement for FastCGI.

Re:People are looking at this the wrong way

[civilizedINTENSITY]

"is lightyears ahead of anything python simply because it gets compiled into machine code"

Actually,

.NET introduced a new programming language environment that compiles all source code into an intermediate language. .NET languages are compiled into the Microsoft Intermediate Language (MSIL), which is executed by the Common Language Runtime (CLR) software in the Windows computer. The MSIL is similar to Java's bytecode, except that whereas Java is one language, .NET supports multiple programming languages...

Now that is also not quite true, because people have written various JVM bytecode compilers for their languages, or implemented their languages in Java. (See Jython [jython.org] for the latter, or The Tcl/Java Project [sourceforge.net], or The scheme package [koala.ilog.fr] for Java based scheme. lists some 200 languages that have been ported to the JVM.) [robert-tolksdorf.de]

Re:People are looking at this the wrong way

[Beek]

> Put a function called hello() in your CherryPy application, and the return value of
> that function is displayed in your web browser when you visit http://address/hello [address]

Rails and Struts already have this feature.

(Of course, for Struts you need a some XML for every class you want to act this way.)

Re:People are looking at this the wrong way

[ziplux]

Someone is, and it's called Subway: http://subway.python-hosting.com/ [python-hosting.com]

Quixote does XMLRPC easier...

[Anonymous Coward]

As an alternative, Quixote http://www.mems-exchange.org/software/ [mems-exchange.org] has a slightly different flavor to exposing Python objects via the url, probably a little more secure out of the box (including cross site scripting protection) and does XMLRPC in a straightforward manner.

Re:People are looking at this the wrong way

[Anonymous Coward]

as does perl's catalyst [perl.org].

Re:rm -Rf

[sean23007]

I'm assuming you didn't read the article, because it explains that CherryPy doesn't just call whatever function is passed in the URL. The programmer has to explicitly expose his functions to the CherryPy server instance. Otherwise, they won't be called. So unless for some reason you @cpg.expose rmdir or kill or any other system function (which you would obviously never do) this isn't an issue.

Re:Bah, Scripting languages

[masklinn]

Python is an Object Oriented language experiment. An interpreted language that can be compiled.

Based on the Java ideal without the "Everything is an Object", clearer syntax than Perl, and more consistency within the syntax (than Java).

Beeep, wrong

Python is not based on any "java ideal", and everything in python is indeed an object, a module is an object, a function is an object, and "1" is an object by itself.

And it can't be compiled, it's a purely interpreted language, it's merely loosely syntax-checked and translated to bytecode (not compiled mind you, that's several steps under java). Some utils write machine code at "compile time" (Psycho), but it's not built in the base language in any way.

Re:" All it does is..."

[Ian Bicking]

The Python WSGI spec (PEP 333 [python.org]) is trying to address exactly this issue. And I agree it's an issue -- I maintain quite a lot of Python applications (not in CherryPy, but same basic setup) and it can be really annoying. CherryPy supports WSGI, though its WSGI support isn't perfect (but they're aware and working on it).

Relatedly I just released Paste Deploy [pythonpaste.org], which is also intended to address these issues. It's just a small piece that will only be useful given more infrastructure support, but I'm optimistic. In the end I think it will mean some specific setup to support Python applications on a server, but once the basic support is configured adding an application should be a one-line affair.

Re:You mean...

[marcello_dl]

>I use fastcgi with perl all the time. I love it, makes my apps very fast. Apache 2.0's "fcgi" module (fastcgi server component, but better maintained) works nicely almost out of the box.

You may also want to check out lighttpd [lighttpd.org]. It's smaller and faster than apache on static content (dynamic content depends on cgi speed, I guess ;) )

Re:Bah, Scripting languages

[Anonymous Coward]

"original features like being able to pass closures"

Lisp, anyone?

Clue-stick

[Anonymous Coward]

Why is CherryPy good? It makes the simple cases REALLY simple, and it can get out of your way if you have more complex cases.

def index():
    return 'Some text'
index.exposed=1

How simple? Your form submissions get turned into variables passed to the action page, which is a normal function.

For instance, if your form has fields Name and Password with action="loginUser", you write this:

def loginUser(Name, Password):
    #... stuff goes here
 
loginUser.exposed=1

And the arguments will be automatically passed. With CherryPy, your web app embeds its own multi-threaded web server; one practice is to put it behind Apache, so you can integrate it with already existing setups. It supports the Python Web Server Gateway Interface (WSGI) so you can use mod_python, IIS/ASP, FastCGI, SCGI , etc.

The ability to use Python makes web programming bearable again. Want something cool? SQLObject [sqlobject.org] turns your SQL (MySQL, PostGres, others) into Python objects.

dude=Person.get(1) # SELECT * from TABLE where ID=1
 
dude.name='Fred' # UPDATE TABLE SET name='Fred' WHERE ID=1

What's more, you don't have to define your class in code. You can read it straight from the DB schema:

class Person(SQLObject):
    _fromDatabase=True

You can also combine CherryPy with your choice of templating system. And for version control, check out how Trac works with SVN at the CherryPy [cherrypy.org] site.

If you're sick of PHP, learn Python and enjoy programming again.

Quick fix for 2.1

[mav[LAG]]

If you're like me and downloaded the latest cherrypy to follow along with the article, there's a quick fix that will make version 2.1 work with it. Just change any lines that say:

from cherrypy import cpg

to:

import cherrypy as cpg

More info here [cherrypy.org].

Re:Seems only useful if you already do Python CGI.

[Ian Bicking]

Leonard Richardson, who wrote the article in question, also recently wrote articles on SQLObject [ibm.com] and Cheetah [ibm.com].

Re:Bah, Scripting languages

[Anonymous Coward]

Not particularly, no, OO predates Java by far and considering in Java everything is NOT an object, their claim to that phrase is ridiculous at best. If "everything is an object" ideal comes from somewhere, it's Smalltalk.

Nor would I be particularly surprised if Python were quite object oriented from the beginning,