Running the Quixote Demo

Quixote comes with a tiny demonstration application that you can install and run on your web server. In a few dozen lines of Python and PTL code, it demonstrates most of Quixote's basic capabilities. It's also an easy way to make sure that your Python installation and web server configuration are cooperating so that Quixote applications can work.

Installation

The demo is included in the quixote.demo package, which is installed along with the rest of Quixote when you run python setup.py install. The driver script (demo.cgi) and associated configuration file (demo.conf) are not installed automatically -- you'll have to copy them from the demo/ subdirectory to your web server's CGI directory. Eg., if you happen to use the same web server tree as we do:

cp -p demo/demo.cgi demo/demo.conf /www/cgi-bin

You'll almost certainly need to edit the #! line of demo.cgi to ensure that it points to the correct Python interpreter -- it should be the same interpreter that you used to run setup.py install.

Verifying the installation

Before we try to access the demo via your web server, let's make sure that the quixote and quixote.demo packages are installed on your system:

$ python
Python 2.1.1 (#2, Jul 30 2001, 12:04:51) 
[GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2
Type "copyright", "credits" or "license" for more information.
>>> import quixote
>>> quixote.enable_ptl()
>>> import quixote.demo

(Quixote requires Python 2.0 or greater; you might have to name an explicit Python interpreter, eg. /usr/local/bin/python2.1. Make sure that the Python interpreter you use here is the same as you put in the #! line of demo.cgi, and the same that you used to install Quixote.)

If this runs without errors, then Quixote (and its demo) are installed such that you can import them. It remains to be seen if the user that will run the driver script -- usually nobody -- can import them.

Running the demo directly

Assuming that

then you should now be able to run the Quixote demo by directly referring to the demo.cgi script.

Start a web browser and load

http://localhost/cgi-bin/demo.cgi/

You should see a page titled "Quixote Demo" with the headline "Hello, world!". Feel free to poke around; you can't break anything through the demo. (That's not to say you can't break things with Quixote in general; since Quixote gives you the full power of Python for your web applications, you have the power to create stupid security holes.)

If you don't get the "Quixote Demo" page, go look in your web server's error log. Some things that might go wrong:

Running the demo indirectly

One of the main tenets of Quixote's design is that, in a web application, the URL is part of the user interface. We consider it undesirable to expose implementation details -- such as "/cgi-bin/demo.cgi" -- to users. That sort of thing should be tucked away out of sight. Depending on your web server, this should be easy to do with a simple tweak to its configuration.

For example, say you want the "/qdemo" URL to be the location of the Quixote demo. If you're using Apache with the rewrite engine loaded and enabled, all you need to do is add this to your httpd.conf:

RewriteRule ^/qdemo(/.*) /www/cgi-bin/demo.cgi$1 [last]

With this rule in effect (don't forget to restart your server!), accesses to "/qdemo/" are the same as accesses to "/cgi-bin/demo.cgi/" -- except they're a lot easier for the user to understand and don't expose implementation details of your application.

Try it out. In your web browser, visit http://localhost/qdemo/.

You should get exactly the same page as you got visiting "/cgi-bin/demo.cgi/" earlier, and all the links should work exactly the same.

You can use any URL prefix you like -- there's nothing special about "/qdemo".

One small but important detail here is "/qdemo" versus "/qdemo/". In the above configuration, requests for "/qdemo" will fail, and requests for "/qdemo/" will succeed. See the "URL rewriting" section of web-server.txt for details and how to fix this.

Understanding the demo

Now that you've gotten the demo to run successfully, let's look under the hood and see how it works. Before we start following links in the demo (don't worry if you already have, you can't hurt anything), make sure you're watching all the relevant log files. As with any web application, log files are essential for debugging Quixote applications.

Assuming that your web server's error log is in /www/log/error_log, and that you haven't changed the DEBUG_LOG and ERROR_LOG settings in demo.conf:

$ tail -f /www/log/error_log & \
  tail -f /tmp/quixote-demo-debug.log & \
  tail -f /tmp/quixote-demo-error.log

(Note that recent versions of GNU tail let you tail multiple files with the same command. Cool!)

Lesson 1: the top page

Reload the top of the demo, presumably http://localhost/qdemo/. You should see "debug message from the index page" in the debug log file.

Where is this message coming from? To find out, we need to delve into the source code for the demo. Load up demo/__init__.py and let's take a look. In the process, we'll learn how to explore a Quixote application and find the source code that corresponds to a given URL.

First, why are we loading demo/__init__.py? Because that's where some of the names in the "quixote.demo" namespace are defined, and it's where the list of names that may be "exported" by Quixote from this namespace to the web is given. Recall that under Quixote, every URL boils down to a callable Python object -- usually a function or method. The root of this application is a Python package ("quixote.demo"), which is just a special kind of module. But modules aren't callable -- so what does the "/qdemo/" URL boil down to? That's what _q_index() is for -- you can define a special function that is called by default when Quixote resolves a URL to a namespace rather than a callable. That is, "/qdemo/" resolves to the "quixote.demo" package; a package is a namespace, so it can't be called; therefore Quixote looks for a function called _q_index() in that namespace and calls it.

In this case, _q_index() is not defined in demo/__init__.py -- but it is imported there from the quixote.demo.pages module. This is actually a PTL module -- demo/pages.py does not exist, but demo/pages.ptl does. So load it up and take a look:

template _q_index(request):
    print "debug message from the index page"
    """
    <html>
    <head><title>Quixote Demo</title></head>
    <body>
    <h1>Hello, world!</h1>
    [...]
    </body>
    </html>
    """

A-ha! There's the PTL code that generates the "Quixote Demo" page. This _q_index() template is quite simple PTL -- it's mostly an HTML document with a single debug print thrown in to demonstrate Quixote's debug logging facility.

Outcome of lesson 1:

Lesson 3: error-handling

The next link in the "Quixote Demo" page is to the "error" document, which is handled by the error() function in demo/__init__.py. All this function does is raise an exception:

def error (request):
    raise ValueError, "this is a Python exception"

Follow the link, and you should see a Python traceback followed by a dump of the CGI environment for this request (along with other request data, such as a list of cookies).

This is extremely useful when developing, testing, and debugging. In a production environment, though, it reveals way too much about your implementation to hapless users who should happen to hit an error, and it also reveals internal details to attackers who might use it to crack your site. (It's just as easy to write an insecure web application with Quixote as with any other tool.)

Thus, Quixote offers the DISPLAY_EXCEPTIONS config variable. This is false by default, but the demo.conf file enables it. To see what happens with DISPLAY_EXCEPTIONS off, edit demo.conf and reload the "error" page. You should see a bland, generic error message that reveals very little about your implementation. (This error page is deliberately very similar, but not identical, to Apache's "Internal Server Error" page.)

Unhandled exceptions raised by application code (aka "application bugs") are only one kind of error you're likely to encounter when developing a Quixote application. The other ones are:

Publishing errors result in a 4xx HTTP response code, and are entirely handled by Quixote -- that is, your web server just returns the HTTP response that Quixote prepares.

Application bugs result in a 5xx HTTP response code, and are similarly entirely handled by Quixote. Don't get confused by the fact that Quixote's and Apache's "Internal Server Error" pages are quite similar!

Driver script crashes and Quixote bugs (which are essentially the same thing; the main difference is who to blame) are handled by your web server. (In the first case, Quixote doesn't even enter into it; in the second case, Quixote dies horribly and is no longer in control.) Under Apache, the Python traceback resulting from the crash is written to Apache's error log, and a 5xx response is returned to the client with Apache's "Internal Server Error" error page.

Lesson 4: object publishing

Publishing Python callables on the web -- i.e., translating URLs to Python functions/methods/PTL templates and calling them to determine the HTTP response -- is a very powerful way of writing web applications. However, Quixote has one more trick up its sleeve: object publishing. You can translate arbitrary names to arbitrary objects which are then published on the web, and you can create URLs that call methods on those objects.

This is all accomplished with the _q_getname() function. Every namespace that Quixote encounters may have a _q_getname(), just like it may have a _q_index(). _q_index() is used to handle requests for the empty name -- as we saw in Lesson 1, a request for "/qdemo/", maps to the "quixote.demo" namespace; the empty string after the last slash means that Quixote will call _q_index() in this namespace to handle the request.

_q_getname() is for requests that aren't handled by a Python callable in the namespace. As seen in Lessons 2 and 3, requests for "/qdemo/simple" and "/qdemo/error" are handled by the simple() and error() functions in the "quixote.demo" namespace. What if someone requests "/qdemo/foo"? There's no function foo() in the "quixote.demo" namespace, so normally this would be an error. (Specifically, it would be a publishing error: Quixote would raise TraversalError, which is the error used for non-existent or non-exported names. Another part of Quixote then turns this into an HTTP 404 response.)

However, this particular namespace also defines a _q_getname() function. That means that the application wants a chance to handle unknown names before Quixote gives up entirely. Let's take a look at the implementation of _q_getname():

from quixote.demo.integer_ui import IntegerUI
[...]
def _q_getname(request, component):
    return IntegerUI(request, component)

Pretty simple: we just construct an IntegerUI object and return it. So what is IntegerUI? Take a look in the demo/integer_ui.py file to see; it's just a web interface to integers. (Normally, you would write a wrapper class that provides a web interface to something more interesting than integers. This just demonstrates how simple an object published by Quixote can be.)

So, what is an IntegerUI object? From Quixote's point of view, it's just another namespace to publish: like modules and packages, class instances have attributes, some of which (methods) are callable. In the case of IntegerUI, two of those attributes are _q_exports and _q_index -- every namespace published by Quixote must have an export list, and an index function is almost always advisable.

What this means is that any name that the IntegerUI constructor accepts is a valid name to tack onto the "/qdemo/" URL. Take a look at the IntegerUI constructor; you'll see that it works fine when passed something that can be converted to an integer (eg. "12" or 1.0), and raises Quixote's TraversalError if not. As it happens, Quixote always passes in a string -- URLs are just strings, after all -- so we only have to worry about things like "12" or "foo".

The error case is actually easier to understand, so try to access http://localhost/qdemo/foo/. You should get an error page that complains about an "invalid literal for int()".

Now let's build a real IntegerUI object and see the results. Follow the third link in the "Quixote Demo" page, or just go to http://localhost/qdemo/12/. You should see a web page titled "The Number 12".

This web page is generated by the _q_index() method of IntegerUI: after all, you've selected a namespace (the IntegerUI object corresponding to the number 12) with no explicit callable, so Quixote falls back on the _q_index() attribute of that namespace.

IntegerUI only exports one interesting method, factorial(). You can call this method by following the "factorial" link, or just by accessing http://localhost/qdemo/12/factorial.

Remember how I said the URL is part of the user interface? Here's a great example: edit the current URL to point to a different integer. A fun one to try is 2147483646. If you follow the "next" link, you'll get an OverflowError traceback (unless you're using a 64-bit Python!), because the web page for 2147483647 attempts to generate its own "next" link to the web page for 2147483648 -- but that fails because current versions of Python on 32-bit platforms can't handle regular integers larger than 2147483647.

Now go back to the page for 2147483646 and hit the "factorial" link. Run "top" on the web server. Get yourself a coffee. Await the heat death of the universe. (Actually, your browser will probably timeout first.) This doesn't overflow, because the factorial() function uses Python long integers, which can handle any integer -- they just take a while to get there. However, it illustrates another interesting vulnerability: an attacker could use this to launch a denial-of-service attack on the server running the Quixote demo. (Hey, it's just a demo!)

Rather than fix the DoS vulnerability, I decided to use it to illustrate another Quixote feature: if you write to stderr, the message winds up in the Quixote error log for this application (/tmp/quixote-demo-error.log by default). The IntegerUI.factorial() method uses this to log a warning of an apparent denial-of-service attack:

def factorial (self, request):
    if self.n > 10000:
        sys.stderr.write("warning: possible denial-of-service attack "
                         "(request for factorial(%d))\n" % self.n)
    return "%d! = %d" % (self.n, fact(self.n))

Since the Quixote error log is where application tracebacks are recorded, you should be watching this log file regularly, so you would presumably notice these messages.

In real life, you'd probably just deny such a ludicrous request. You could do this by raising a Quixote publishing error. For example:

def factorial (self, request):
    from quixote.errors import AccessError
    if self.n > 10000:
        raise AccessError("ridiculous request denied")
    return "%d! = %d" % (self.n, fact(self.n))

Lesson 5: widgets

You can't get very far writing web applications without writing forms, and the building blocks of web forms are generally called "form elements": string input, checkboxes, radiobuttons, select lists, and so forth. Quixote provides an abstraction for all of these form elements: the Widget class hierarchy. The widget classes are explained in detail in widget.txt; I'm going to give a brief description of the "Quixote Widget Demo" page and the code behind it here.

If you follow the "widgets" link from the main page, you'll see a fairly ordinary-looking web form -- the sort of thing you might have to fill out to order a pizza on-line, with the oddity that this pizza shop is asking for your eye colour. (Hey, I had to demonstrate radiobuttons somehow!) This form demonstrates most of HTML's basic form capabilities: a simple string, a password, a checkbox, a set of radiobuttons, a single-select list, and a multiple-select list.

Whenever you implement a web form, there are two things you have to worry about: generating the form elements and processing the client-submitted form data. There are as many ways of dividing up this work as there are web programmers. (Possibly more: every time I tackle this problem, I seem to come up with a different way of solving it.) The form in the Quixote widget demo is implemented in three parts, all of them in demo/widgets.ptl:

This division of labour works well with Quixote's widget classes, since you need a collection of Widget objects whether you are generating the form elements or processing form data. For generating the form, we (1) create all the widget objects, and (2) generate an HTML document that includes the output of each widget object's render() method. (Laying out the form is the responsibility of render_widgets(), which is why it's littered with table tags.) For processing the form, we (1) create all the widget objects, and (2) generate an HTML document incorporating the user-submitted form values. In both cases, step (1) is handled by the widgets() template, which then calls either render_widgets() or process_widgets() for step (2).

Thus, there are three things you have to understand about widget objects: how to create them, how to render them, and how to use them to parse form values. Widget creation is the only step that's very interesting, since each widget class has different constructor arguments. For example, here's how we create the "name" widget in the pizza shop form:

widgets['name'] = widget.StringWidget('name', size=20)

When rendered, this widget will produce the following HTML:

<input size="20" name="name" type="text">

A more complex example is the "pizza size" widget:

widgets['size'] = widget.SingleSelectWidget(
    'size', value='medium',
    allowed_values=['tiny', 'small', 'medium', 'large', 'enormous'],
    descriptions=['Tiny (4")', 'Small (6")', 'Medium (10")',
                  'Large (14")', 'Enormous (18")'],
    size=5)

which will generate the following HTML when rendered:

<select size="5" name="size">
  <option value="0">Tiny (4")
  <option value="1">Small (6")
  <option selected value="2">Medium (10")
  <option value="3">Large (14")
  <option value="4">Enormous (18")
</select>

Some things you might need to know about widget creation:

Rendering widgets is easy: just call the render() method, passing in the current HTTPRequest object. (It's currently not used by the standard widget classes, but could be used by derived or compound widget classes for context-sensitive widget rendering.)

Parsing form values is just as easy: call the parse() method, again passing in the current HTTPRequest object. The return value depends on the nature of the widget, eg.:

$Id: demo.txt,v 1.13 2002/10/01 21:27:31 gward Exp $