Installing the Firebird database on a 64-bit RHEL Linux server

I just finished the painful exercise of installing Firebird on an Intel E64MT server running RHEL 3. I first tried installing the RPM versions of Firebird from here -- tried both the 1.0 version and the 1.5 version of the database, in both Classic and SuperServer incarnations. In all cases, the RPMs installed without error, but when I tried to install the KInterbasDB Python module (and also the DBD::Interbase Perl module), the loader complained about the Firebird shared libraries, which were obviously not 64-bit. So I ended up installing Firebird from source. Here's what I did:

1. Downloaded and bunzip-ed firebird-1.5.2.4731.tar.bz2.
2. Ran ./configure, then make and make install in the firebird-1.5.2.4731 directory.

By default, this installs the Classic version of Firebird in /usr/local/firebird (the Classic version allows processes to connect directly to database files; the SuperServer version requires processes to open a socket to a server process, which then accesses the database files).

The 'make install' command also asks for a password for the SYSDBA user. If you happen to forget the password, you can always retrieve it by looking at the file /usr/local/firebird/SYSDBA.password as root. You change the password by running /usr/local/firebird/bin/changeDBAPassword.sh.

3. Downloaded the kinterbasdb-3.1.src.tar.gz Python module and installed it via python setup.py install.
4. Downloaded and installed the mxDateTime Python module from here (this module is required by kinterbasdb.)
5. Tested kinterbasdb against the example employee database shipped with Firebird:

>>> import kinterbasdb
>>> conn = kinterbasdb.connect(database='/usr/local/firebird/examples/employee.fdb')
>>> cursor = conn.cursor()
>>> cursor.execute('select * from country')
>>> print cursor.fetchall()
[('USA', 'Dollar'), ('England', 'Pound'), ('Canada', 'CdnDlr'), ('Switzerland','SFranc'), ('Japan', 'Yen'), ('Italy', 'Lira'), ('France', 'FFranc'), ('Germany', 'D-Mark'), ('Australia', 'ADollar'), ('Hong Kong', 'HKDollar'), ('Netherlands', 'Guilder'), ('Belgium', 'BFranc'), ('Austria', 'Schilling'), ('Fiji', 'FDollar')]

At this point, I created a Firebird database file called test.gdb, owned by user root and group root. When I connected to this database as root via the isql utility in /usr/local/firebird, everything looked good:

[root@localhost]# /usr/local/firebird/bin/isql
Use CONNECT or CREATE DATABASE to specify a database
SQL> connect test.gdb;
Database: test.gdb
SQL>quit;

However, when I tried to connect to the database as a non-root user, I got this error:

$ /usr/local/firebird/bin/isql
Use CONNECT or CREATE DATABASE to specify a database
SQL> connect test.gdb;
Statement failed, SQLCODE = -551

no permission for read-write access to database test.gdb

What I needed to do was to change ownership and permissions for the test.gdb file so that it was owned by user firebird and group firebird, and it was group-writeable:

chown firebird.firebird test.gdb
chmod 664 test.gdb

My goal was to have the Firebird database accessible from a CGI script running from Apache.
The httpd process was running as user apache, so I had to add this user to the firebird group in /etc/group. I then restarted httpd to take into account the new group membership.

At this point, from my CGI script, I was able to open a connection to test.gdb via:

conn = kinterbasdb.connect(database='test.gdb', user='SYSDBA',
password='password')

I found the firebird-devel mailing list very helpful in solving some of the issues I faced. In particular, this message allowed me to get past the non-root user problems.

Third SoCal Piggies meeting

The SoCal Piggies had their third meeting at USC on May 17th. A record number of 7 Piggies attended (Daniel, Howard, Mark, Grig, Brian, Titus, Diane), joined by a guest from Caltech. It was nice to meet Howard, Brian and Mark and to talk about Python while munching on a fairly decent Pizza Hut nourishment.

Titus Brown presented part 2 of his Quixote 2.0 tutorial. He showed a few introductory examples, then delved into more advanced stuff, showing how to manage sessions with Quixote. One thing I took away from it is that it takes remarkably little code. Generally speaking, Quixote seems very crisp and clean, and Titus' tutorials are very good in smoothing the learning curve, which can otherwise be steep.

Titus then spent some time talking about his Web app. testing tool twill. We couldn't get a live demo unfortunately, but it was an interesting discussion nevertheless, covering things such as the cmd module, pyparsing, and especially a great use of metaclasses. Titus ended his presentation talking about his Cucumber module, another magical metaclass-heavy module that provides a mapping between Python code and SQL statements, hiding the complexity of SQL away from the programmer.

Daniel Arbuckle was next, and he gave us an overview of iterators, generators and continuations. He highly recommended the little-used itertools module, which provides functions designed to work with iterators. The functions are similar to standard Python built-ins like map and range and allow you to write code that is faster memory-efficient. Daniel then covered generators, which can be seen as a way to express iterators using functions as opposed to classes (if you see a yield statement anywhere in a function, that function is a generator). However, there are many other and sometimes slightly brain-melting uses of generators, such as cooperative multitasking and state machine control.

Daniel ended with a mind-boggling discussion of continuations, which (luckily or unluckily?) are not and will not be offered as a feature of Python, due to GvR's dislike of them. Especially interesting was the parallel Daniel made between continuations and the HTTP GET mechanism.

Many thanks to Titus and Daniel for their presentations.

The next meeting will be at USC again, on a date to be announced. Topics for next meeting:

• CherryPy tutorial (if Greg McClure will be able to attend the meeting)
• SimpleTAL and Pyrex tutorials (Diane)
• Others?

Installing and using cx_Oracle on Unix

Here's a mini-HOWTO on installing the cx_Oracle Python module on Unix systems (tested on Linux and Solaris.) I always forget one of these steps every time I install this module on one of my systems, so I thought I'd write them down. Other people might find this useful, so here goes:

1. Download cx_Oracle-4.1.tar.gz from here.
2. Become root on the target system.
3. Make sure you have the ORACLE_HOME environment variable defined for user root and pointing to the correct Oracle installation directory on your server. This step is necessary because the cx_Oracle installation process looks for headers and libraries under ORACLE_HOME.
4. Install cx_Oracle by running 'python setup.py install'.
5. Become user oracle, since this is the user you'll most likely want to run python scripts as when interacting with Oracle databases. Make sure you have the following line in oracle user's .bash_profile (or similar for other shells):

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME/lib

If you don't add ORACLE_HOME/lib to your LD_LIBRARY_PATH, you'll get an error such as:

>>> import cx_Oracle
Traceback (most recent call last):
File "", line 1, in ?
ImportError: ld.so.1: python: fatal: libclntsh.so.9.0: open failed: No such file or directory

Now you're ready to use the cx_Oracle module. Here's a bit of code that shows how to connect to an Oracle database and run a SELECT query:

import cx_Oracle
from pprint import pprint

connection = cx_Oracle.Connection("%s/%s@%s" % (dbuser, dbpasswd, oracle_sid))
cursor = cx_Oracle.Cursor(connection)
sql = "SELECT * FROM your_table"
cursor.execute(sql)
data = cursor.fetchall()
print "(name, type_code, display_size, internal_size, precision, scale, null_ok)"
pprint(cursor.description)
pprint(data)
cursor.close()
connection.close()

New features in Selenium 0.3

A new version of the Selenium Web app test tool has just been released by Jason Huggins and the rest of the team at ThoughtWorks. I'll highlight some of the new and updated features.

New TestRunner look and feel

The most striking feature in version 0.3 is a new and improved look for the TestRunner. Click here for a screenshot.


Simplified directory structure

Another important improvement is a much simplified directory structure for the Selenium package. The main Selenium package contains the 'BrowserBot' JavaScript engine and nothing else. All the other language-specific drivers/bindings are available for download separately.

Here's all you need to do to get started with Selenium 0.3:

1. Download selenium-0.3.0.zip from the download page
2. Unzip the archive. This will create a directory called selenium-0.3.0, with 2 sub-directories: doc and selenium
3. Copy the selenium directory somewhere where it can be served by the Web server hosting your Web app
4. Open http://www.yourwebserver.com/selenium/TestRunner.html -- you will see a screen similar to the screenshot above
5. Start tinkering with the default test suite shipped with Selenium.

To add your own test suite, create an HTML file containing a table with links to your tests:

Custom Test Suite

Test1

Save this file as CustomTestSuite.html in the tests subdirectory of the selenium directory.

In the table above, Test1 is a link to the actual test file, which is another HTML file (called Test1.html) containing your custom test table. To have Selenium run your custom test suite, pass a ‘test’ parameter to TestRunner.html by opening an URL such as this:

http://www.yourwebserver.com/selenium/
TestRunner.html?test=./tests/CustomTestSuite.html
(note: this is a single line)

Improved browser support

In addition to Firefox/Mozilla on Windows/Linux/Mac OS X and IE on Windows, Selenium now supports Konqueror on Linux. There is also experimental support for Safari on Mac OS X.

Standalone server for win32 platforms

A standalone server is available for win32 platforms. This used to be known as the 'Twisted Server' implementation of Selenium, but now the fact that it is based on Twisted is hidden behind the scenes, so that new users don't have to jump through the various hoops of installing Python/Twisted/etc.

The main advantage of using the standalone version of Selenium is that it provides a workaround for the JavaScript 'same origin policy' security limitation. The same origin policy aims to prevent Cross-Site-Scripting attacks by preventing documents or scripts loaded from one origin from getting or setting properties of a document from a different origin. The net effect of all this is that in order to be able to use Selenium to test your Web application, you need to deploy the Selenium engine/framework on the server hosting the AUT (application under test). With the standalone Selenium server, you don't have this limitation and you can start doing what all new users to Selenium seem to want to do: test Google searches :-)

For users on non-Windows platforms, the Twisted Server version is still available via Subversion. See my previous blog entry for a tutorial on using the Twisted Server version. Many concepts discussed in there still apply to the standalone server version, in particular how to compose special URLs for testing 3rd party Web sites.

Plone Product version available for download

While Selenium has been available as a Plone Product version for quite some time, it was never part of the 'official' download page. Now you can find it among the Selenium 0.3 downloads as selenium-0.3.0-python-plone.zip.

For details on using Selenium with Plone, see 2 of my tutorials: part 1 and part 2.

Finally, here is a screen recording movie in AVI format that shows Selenium in action (note: choose 'Full screen' in your media player for better resolution.) The test run goes in 'Walk' mode through a new user registration test in a default, out-of-the-box installation of Plone. Enjoy!