File Formats Blog

W3C has been really busy. XPath 2.0, XSLT 2.0, and XQuery 1.0 are now proposed recommendations.

Labels: XML

# posted by Gary McGath @ 9:10 AM 0 comments

Not much to add today. I went to only one session, on OAI/PMH. Even though Dublin Core is the official lingua franca for OAI data providers, many people obviously consider it old hat, the the trend is toward MODS. The number of data providers on line to the public is growing rapidly.

This was a good conference.

# posted by Gary McGath @ 3:01 PM 0 comments

In a separate post, I've covered other events at the Digital Library Federation forum today. This is about the subject which I actually know something about: JHOVE. Since this was a public forum, it gives me a chance to talk about where we are without inadvertently saying anything that shouldn't be public yet. Stephen Abrams led the discussion. I wasn't officially on the panel, but jumped in a lot. You're allowed to do things like that when you wrote almost all the code.

JHOVE is being used quite a bit among digital libraries to identify and validate file formats. But having gotten this use, it's showing where it needs improvement. Once the project for the next version starts up -- it's still waiting for money -- there will be a lot of added features to make users happier. Here is a sampling:

• Making modules more consistent in design, and easier to subclass. This should make it much easier to create third-party modules.
• Plug-in support of non-JHOVE modules, and in particular, of the NLNZ Metadata Extraction Tool.
• Separation of identification from validation. Currently, if a TIFF file is defective, JHOVE throws up its hands and says it doesn't know what kind of file it is, even if the defect is minor.
• More persistent analysis. Most existing JHOVE modules give up at the first error.
• Greater customizability. It will be possible to make JHOVE disregard particular types of defects, and to control what gets reported as metadata.
• The ability to handle multiple formats within a file, and multiple files which form a single representation.
• A mechanism for open-source collaboration. JHOVE is currently open source, but there's no mechanism, other than sending us email, to submit code changes.
• Greater ease of integration into applications.
• Integration with the upcoming Global Digital Format Registry (which Stephen also discussed today).
• Use of Java 5, hopefully gaining efficiency by using the nio package.

It's mapped out as a two-year project. I hope it'll start up soon; I'd love to get back to being a "wizard of files and formats."

Labels: JHOVE

# posted by Gary McGath @ 9:43 PM 0 comments

As I expected, there were lots of interesting events at the DLF conference in Boston today. I'll split it into two posts: JHOVE2 and everything else. This is the "everything else" part.

I only caught the tail end of the keynote address by Anurag Acharya, the founder of Google Scholar. He's quite a good speaker, though the sound system wasn't cooperating very well.

The Developers' Forum Panel, on networked storage, discussed the preservation of resources on the Internet. This is an unpredictable phenomenon; some things may disappear forever, but more and more is being saved somewhere. One of the speakers, Clay Shirky, wryly noted that while it's very difficult to get hold of a book he published about ten years ago, all of his Usenet rants from that time are still preserved for anyone to read.

Session 7 included presentations of a couple of Harvard Library projects I've worked on, Virtual Collections and a project to harvest a large set of our own images from VIA for ARTstor. My name was on the latter, though I had nothing to do with the presentation. This session also included a discussion of a project called SUSHI, which deals with usage statistics. (I expected it to mean a wrapper for raw and somewhat fishy data.) The speakers were periodically upstaged by a mouse -- the carbon-based lifeform, that is -- which kept running around the front of the hall and occasionally into the audience.

Skipping over the JHOVE2 BOF session, which I'll give a post of its own, the last item I attended was a BOF session on "DLF Implementation Guidelines for Shareable MODS, DLF Aquifer Metadata Working Group." I went in knowing a pretty good amount about MODS, which is an XML schema for descriptive metadata, but nothing about Aquifer, which is a DLF project to promote distributed digital library content. MODS isn't quite as anarchic as PREMIS, but it does have lots of options, and the session was about Aquifer's work to recommend a set of best practices, specifically for shareable MODS records describing digitally archived objects. This is something I want to learn more about.

# posted by Gary McGath @ 9:24 PM 0 comments

The sessions which are really of interest to me at DLF are all happening today (sometimes two and three at a time), so my report on yesterday will be short.

There's a clear trend toward the development of large digital collections compiled from diverse sources. The Internet Archive presentation was talking about petabytes. There's starting to be talk, even within the digital library community, that perhaps more information is being archived than anyone could ever care about.

A presentation by Denise Troll Covey of Carnegie-Mellon on "Faculty Rights and Other Scholarly Communication Practices" was rather interesting. I was actually hoping to hear something about what the issues are in Open Access vs. traditional publishing, but what we got was the results of a survey of faculty members on their views, which was still pretty interesting. Quite a number of faculty who publish (at least among those surveyed, all at CMU) don't really care much about what rights they sign away, and aren't very clear on what Open Access is. (Here's a piece that I found on the subject with a very quick Web search.) A significant minority said that they'd just "self-archive" their papers (jargon for putting them up on their own or their department's websites) without caring what the publishing agreement said.

More tonight or tomorrow morning.

# posted by Gary McGath @ 6:17 AM 0 comments

The DLF conference hasn't opened yet, but today there was a tutorial in PREMIS at Harvard, continuing through tomorrow morning. This was the first public PREMIS tutorial in the Americas.

PREMIS, a somewhat strained acronym for "Preservation Metadata: Implementation Strategies," is the name of a working group and a data model. It's intended to go with digital archives, holding information about the archiving of objects in a digital repository. It's contrasted with, among other things, descriptive metadata; it's about the storage, legitimacy, and integrity of digital objects rather than their content. This is fairly new; the PREMIS final report was released in May of 2005.

The presenters made it clear that PREMIS is a data model rather than a particular implementation, though tomorrow we'll be hearing about an XML schema to represent it.

PREMIS is very solid on some things, such as events; it provides a rich data model for tracking the ingest, validation, and modification of archival objects. Other features need more work; the "rights" definitions currently cover only grants of permission and don't even include a way to express copyright status. (Work is continuing in this area.)

I was quite pleased to hear JHOVE come up repeatedly in the discussion. It was mentioned that file format metadata standards (see, there is a connection between this post and the nominal subject of the blog!) are still few and far between. There are NISO metadata for images and AES metadata for audio, but the gaps remain larger than the standards. Each file format class has different needs, so the general problem is messy.

PREMIS allows lots of different ways to do things; I'm still trying to get a sense of whether this will be a problem when communicating preservation metadata.

A couple of useful URLs:

• Implementation registry
• Premis Implementors' Group (PIG): Wiki, mailing list, and other stuff

# posted by Gary McGath @ 7:48 PM 0 comments

I'm planning on blogging the DLF conference, so there may be more than the usual blue-moonthly posting here this week.

Here's a link to a posting on one upcoming event.

# posted by Gary McGath @ 10:10 AM 0 comments

I'll be at the DLF Fall Forum in Boston, and in particular at the JHOVE 2 discussion on Thursday at 4:15.

# posted by Gary McGath @ 11:41 AM 0 comments