Fun Stuff at OSCON 2005

OSCON 2005 was held in a convention center this year, instead of a hotel, because it just got too big (2000+ people). Too big, in fact, for pudge and myself to cover more than a fraction of the talks and the ideas flitting around the hallways. But here's some of what I found cool last week. And if you attended or presented at OSCON and want to tell us about all the neat stuff we missed, please, share your thoughts in the comments, or submit a fact-rich writeup and we'll maybe do a followup story later.

Mike Shaver's talk on writing Firefox extensions was packed to the walls. If you've been wanting to try it, Firefox 1.5 makes development easier, and should be out soon, so now's a good time. This talk and the tutorial on Ajax persuaded me to start using the DOM Inspector and debugging some JavaScript to get a better understanding of webpage manipulation.

Aaron Boodman's talk on his extension Greasemonkey was a walkthrough of writing a simple GM user script, a discussion of what's coming up, and some Q&A. Greasemonkey 0.5 ("Now With Security!") is in beta: there are multiple security changes that suggest someone really has sat down and thought the whole model through. GM works with Firefox, Seamonkey, Opera, and Windows MSIE (but not, oh please somebody correct this oversight, Safari).

Ruby on Rails is hot; if you want to develop a web app quickly you can't ignore it. It stresses "convention over configuration" with reasonable defaults. The tutorial went from installation to the "hello world" of the web, a blog (!), in a few hours. Anyone have a real-world example of Rails scaling to a large project and lots of traffic?

DarwinBuild is an open-source project from Apple that aids in building the open-source components of Darwin/Mac OS X. Given a build number of Mac OS X, it will fetch and build the software for that version, allowing you to modify the source as needed, making it easy for any developer to modify everything from the kernel to various utilities (just remember to reapply the modifications after running Software Update, if necessary). You can read more about it from, in addition to the web site, the presentation slides.

Google and O'Reilly gave out the 2005 open source awards, with $5000 attached to each. Congratulations to the winners.

Tony Baxter's Shtoom is a cross-platform VoIP client and software framework, written in Python, for writing your own phone applications.

Novell is still moving its employees from Windows to Linux, which we first heard at last year's OSCON. The migration from Microsoft Office to OpenOffice is complete, and the big step, from Windows to Linux, is 50% complete, projected to be 80% by November. Miguel de Icaza gave flashy demos of some Linux desktop applications that didn't impress this cynical observer very much.

PlaceSite is an open-source project looking to bring physical proximity awareness to Internet access at coffeeshops and other meetingplaces: think "local-only Friendster" and you're not far off. They got feedback from a monthlong trial earlier this year and are working on a new version that will be easy to deploy. Could be neat.

In a great 2-hour session on Wednesday, we got to hear from representatives of four leading open source databases about what they've been working on lately. Here are the summaries...

Ingres r3 has an impressive list of big features. Ingres was just open-sourced by Computer Associates this summer, and it's gotten a lot of attention for being a full-featured enterprise database. Ingres supports table partitioning that can be either range-based or hash-based, which can greatly improve performance in many cases. Its optimizer can now come up with parallel execution plans, which can be useful even on single-CPU machines and non-partitioned tables. There's also federated data storage (one can access data stored in another RDBMS through Ingres) and replication. And they're working on a concurrent access cluster, to allow data to be manipulated not just by multiple threads on one machine, but multiple machines.

A side note: Computer Associates was invited by O'Reilly to talk about its recently open-sourcing Ingres. Its representative, while confessing that introducing a new license was "probably the wrong thing to do," said that other licenses wouldn't have worked for them (the GPL "was seen as viral"). The one question that the audience had time to ask was "is Ingres a dump" -- is CA making it open-source to transfer the responsibility of support from the company to the community? The three-part "no" answer was that there are more CA developers working on Ingres now, that Ingres is at the core of their new releases, and that they've sponsored a "million-dollar challenge" to foster community interest. Time will tell I guess.

Firebird 2.0 has been in alpha since January and a beta is expected soon. Since 2000 much of their development has been aimed at making the product easy to install, and making the code easy for a distributed group of developers to work on. This year they're building features on that groundwork. Their design includes 2-phase commits (since the beginning), cooperative garbage collection (as a transaction encounters unneeded data, it removes it) and self-balancing indexes. Backup has been improved. When 2.0 gets to beta, I'm going to check this out, it sounds like very interesting technology (and apparently it will install with four clicks!).

MySQL 5.0 is in beta, and has been feature-frozen since April. Back in 4.1, its abstracted table-type has been put to advantage with odd engines like Archive (only insert, no update); Blackhole for fast replication; and an improvement to MyISAM for logging (allowing concurrent selects with inserts-at-table-end). Their Connector/MXJ lets you run a native MySQL server embedded inside a Java application. In 5.0 we're seeing stored procedures per the SQL:2003 standard, triggers, updatable views, XA (distribution transaction), SAP R/3 compatible server side cursors, fast precision math, a federated storage engine, a greedy optimizer for better handling of many-table joins, and an optional "strict mode" to turn some of MySQL's friendly nonstandard warnings into compliant errors. And they're working on partitioning, ODBC, and letting MySQL Cluster's non-indexed columns to be stored on disk.

PostgreSQL 8.1 is expected to be released in November or December, after a feature-freeze in July -- and it's an impressive list of new features. Their optimizer will make use of multiple indexes when appropriate, which is pretty darn exciting. The recommendation will be that in most cases it will be most efficient to have only single-column indexes and let the optimizer figure out which combination to use. They're implementing a 2-phase commit, they're bringing the automatic vacuum into the core code, and they removed a global shared buffer lock so they're now getting "almost linear" SMP performance scaling. I've never felt the need for Postgres, but I'm definitely going to look at 8.1.

Re:MySQL w/ XA

[Tony Hoyle]

MySQL is more expensive than SQL Server or Oracle once you start working on large systems though (MySql is $300 per client.. on a web farm you could easily be expecting 1000 clients simultaneously. SQLServer and Oracle are much cheaper at these volumes.).

They had a nice niche with the small developer, where they worked out cheaper (maybe 1 or 2 clients in-house) plus they had the hobbyist/GPL market of course but going for the bigger market is going to backfire on them unless they change their pricing structure.

Re:CA doesn't get it re: GPL and "viral"

[Phroggy]

There is always a third option: Don't take someone else's code without honoring its license terms in the first place. Write it yourself, if you want to release under a different license.

Um, correct me if I'm wrong, but isn't this exactly what CA has done?

Re:MySQL w/ XA

[JohanV]

I have to disagree with MySQL being the natural choice. With both MySQL and the drivers being GPL a vendor of POS applications would either have to GPL his POS application, or pay for a commercial license for each sold unit. Neither appear to be particularly attractive.

XA support has been committed into the upcomming PostgreSQL release and is already supported by Firebird. Considering their licensing, both are better choices.

Re:RoR large scale?

[Decaff]

There is no downside to using ruby on rails. Closest thing to a silver bullet since the web came out.

There are several significant downsides to using Ruby on Rails.

Firstly, the way that ActiveRecord works by default - generating classes at run-time based on database tables - is considered by many (well, me at least!) to be a very backward step, as it makes code vunerable to changes in those tables, and also makes portability of code between different databases non-trivial. There are far better ways to do this - the Python ORM Dejavu (in in which the data model is expressed as classes) is an example. Almost all modern development languages work this way - with the exception of RoR!

Secondly, Ruby is slow. There may be future JIT systems that help deal with this, but they are not there yet.

Thirdly, Ruby is changing, and it is likely (from what I read) that the next version will not be fully compatible, so any major project developed now in Rails will have upgrading issues.

So Ruby on Rails is very far from a silver bullet. It may a neat way to get small (in terms of code) websites up quickly.

Re:Ruby on Rails

[JLyle]

I've heard that Ruby is a bit of a complicated language to learn, but have thought about looking into it a little.

People will say a lot about Ruby's strengths and weaknesses, but I don't think I've ever heard someone claim that it's a complicated language to learn. On the contrary, most experienced programmers seem to pick up the basics of Ruby in a matter of hours.

Re:RoR large scale?

[Decaff]

Thanks for the FUD, Guido!

-1, Troll. Please. There are people running real, large-scale web sites on Ruby-on-Rails. Tobias just named several.

This has no relevance to whether or not Ruby on Rails is a fundamentally good design for long-term support and growth of any given software project. We all know that poor designs have been used in large projects!

The issue of database-dependence is a real and important one, and should not be trivially dismissed as 'trolling'. The idea of working dynamically from database tables is great for getting things up and running fast, but is not great for long term use, as at some future date the database may need to be modified or shared with other applications. The changes that result may break existing Ruby on Rails applications. More mature ORM systems have a mapping layer that can help isolate code from such changes. By abandoning that layer as the default setup, Ruby on Rails loses (in my view) an important feature.

Re:Videos?

[Anonymous Coward]

Here's a description - no video yet...

http://www.oreillynet.com/pub/wlg/7558 [oreillynet.com]

Miguel's talk was definitely a good choice for the closing timeslot. He talked about numerous happenings at Novell, from their efforts to move all employees off Windows / MS Office onto Linux / OpenOffice.org, to several new X technologies that impressed the heck out of me.

Novell currently has some 5500 employees, and has already managed to move all of them over to OO.o. They are also 50% finished with moving every PC to single-boot linux, and expect to reach 80% by November. In the mean time, it sounds like many employees still multi-boot.

Eating their own dog food is one thing, but Novell still needs to work on the Linux desktop to increase its mass appeal. To this end, they have considerably hardened the Mono VM (a clone of Microsoft's .NET VM), to the point that it can now withstand weeks of continuous heavy load without erroring or crashing. From this point, they are working in two directions: making all user hardware Just Work, and implementing all missing desktop applications in Mono languages, such as C# and Python (via IronPython). It sounded like they are also putting some effort into convincing Windows ISVs to migrate to Mono.

They've been working on useability issues in Gnome as well; Novell likes to video users with three simultaneous camera angles (face, hands, and monitor), show the videos to the developers, and watch their mental models get massively readjusted to match reality (well, more closely, at least).

Miguel also showed off some individual technologies, like two-way iPod sync, multi-hop directory sync, and so on; some of these were actually ports of old-world Novell technologies to pure Mono code. He bragged that with Beagle they managed to be the first out the door with desktop search, though only by a mere 6 hours.

Nearing the end of his talk, Miguel showed off some of the amazing changes happening to X these days. The Cairo compositing / rendering model (similar to the PDF rendering model) is now available for X, and can be accelerated in OpenGL using Glitz and XRENDER. Ugly hacks for window transparency and other such effects are no more.

X itself can run on top of OpenGL using the Xgl server, which is nearly complete. Xgl comes with a Composition Manager, which allows all sorts of composeable eye candy and special effects; Miguel showed off some silly-but-cool wobbling window effects based on this.

Finally, since Xgl has the full power of OpenGL behind it, all of the virtual desktops are wrapped around a cube that is just dragged to show the desired desktop. It's even possible to start a movie playing, drag the window halfway across the boundary between desktops, rotate the desktop cube through 45 degrees, and then watch the movie, now split across the border and in correct perspective across each face -- without any visible rendering issues.

All in all, a very cool talk, and a great way to close out this year's OSCON.