Do We Need Databases on the Desktop?

Recently Simon Morris posted a blog called In defence of the desktop where he asks :"If SE is truly the edition of Java aimed at the desktop, and most real desktop applications (browsers, players, word processors, video editors) are not database heavy, why is Java DB being included in the SE JDK?". I'd like to challenge the idea that real desktop applications don't need databases. They may not be database heavy (in that storing data is not their primary function) but I do think that there are a lot of desktop apps which use databases, or could be improved by doing so.

When it comes to desktop apps I don't think of a database as "table based storage for relational data accessed by SQL". I prefer think of it as "reliable and search able storage", since that's what it really means to desktop apps. When I wrote my two part series on using Java Persistence in desktop apps, that's what I was thinking.

So do desktop apps need a database? Rather than say yes and describe why this is so, I thought I'd simply go through the applications installed on my computer and speculate about which ones have or could have a real database inside.

Here is the contents of /Applications on my MacBook running Mac OS X 10.4:

• Address Book Well, duh. It's a big searchable list of data with well defined fields. Okay, next one.
• Adium and iChat and Skype These are instant messenging or audio chat applications. The actual storage of contacts is either done in a flat file, derived from the aforementioned Address Book, or stored on the server. Thus the contacts probably don't use a database. However, the chat logs certainly could. You've got lots of records, organized by date and participants, and fast searching is highly desirable. Certainly a good candidate for a local database.
• Adobe Photoshop CS, Adobe Reader 7.0.7. I can't see many direct uses here.
• Automator This is a Mac specific development tool for building scripts that control other programs. While the scripts themeselves are stored as flat files, the list of available bindings for every applications certainly need to be stored and searched quickly, even if the underlying storage is XML files in the relevant apps. Much like other development tools Automator (could) parse the application bindings into a local searchable storage cache. Yes on the DB.
• Bugster is a Sun app which searches the bug database. While it's normally searching a remote database, I can see how you would want to use a test database while developing the software. Still, it's not using a local database at deployment time so it doesn't count.
• Calculator. It's a calculator. It calculates. Doesn't really need a database.
• Camino, Firefox, Safari: webbrowsers. You might not think that they need databases but they do have two things that need to be stored and searched: history, and bookmarks. Both of those could greatly benefit from using a real database underneath.
• Brood War, Chess, Snes9x, Starcraft: games. No direct need for a database.
• Cyberduck, Fugu, LimeWire, Democracy: file transfer or media downloading applications. They could use databases for searchable records of what you've transfered and when. (none of them do currently, however)
• Dictionary : a dictionary is a database. 'Nuff said.
• Font Book : while it does search through a dataset, the data comes live from installed fonts so it probably wouldn't benefit from the database.
• LEGO Digital Designer, Google Earth: these are both interesting examples. They are both very non-traditional applications. They are both very graphics heavy. And they both store local searchable caches of networked data. I think they could both benefit from internal databases.
• DVD Player, HandBrake 0.7.0, MPlayer OS X 2, Image Capture, Image Tricks, ImageWell, Preview, Photo Booth, QuickTime Player, RealPlayer, VLC, Windows Media Player: These are all straight up media playing and manipulation applications. Some of them include a list of tasks they have done before (DVDs ripped, for example) but none of them really take advantage of databases
• Audio Hijack Pro, Audio Hijack: this is an interesting case. These are media manipulation apps but they are used constantly and can produce a lot files which need to be managed. A good case for a database.
• Mail: tons of searchable records going back years. Definitely has a database in it.
• MarsEdit, NetNewsWire: These are applications for posting and view weblogs and other rss feeds. They both need detailed records of transactions (what you've posted and read) and keep extensive caches of network stored data. Definitely a need for a local database.
• Microsoft Office 2004, Swift Publisher, TextEdit, Stickies, RapidWeaver. Word processors, page layout, and text editors. Nothing very databasey in these other than integration with external databases.
• NetBeans, jEdit 4.2: these are both text editors at their heart, but being development tools they need lots of cross references for managing classpaths and doing code completion. They typically scan the classpath at startup or install time and store the results in a structured cache. That cache could very easily be an embedded database (and would be a lot easier to maintain).
• Mori and OmniOutliner are both list managers that could very easily use a database for storage. Lots of entries to be searched, though it's not quite a structured as something like iTunes. Lucene might be a better choice than SQL, or perhaps the combination of the two.
• Parallels. OS Virtualization software. No real need for a database
• PostgreSQL and SQL-Ledger these are actual database tools so they don't really count.
• Quicken 2006 pretty much is just a database with a very fancy front end. It stores, searches, and reports on financial data. I'd say that's a yes.
• Quicksilver, iTerm.app, VPNClient, Internet Connect, System Preferences, iSync, Missing Sync for Palm OS, Sherlock : these are all system utilities. With the exception of the syncing programs which have large datasets to deal with most of these don't really need databases.
• iCal, OSX's system wide calendar database. The actual storage is part of the OS rather than confined to a single app, but this only makes the case for using a database even stronger because reliability is even important.
• the iApps: GarageBand, iDVD, iMovie HD, iWeb, iPhoto, iWork '06 These are the famous iApps that Apple sells. Most of these can use an internal database for managing the media they work with. iPhoto in particular has thousands of photos to search through.

And the granddaddy of all local database apps

• iTunes: this is the quintessential example of a local app with a database. Megs of structured data with high performance searching. I can't think of a better use for an embedded database. And many, many people use it.