File Formats Blog

The DLF Fall Forum was in Providence, Rhode Island this year. It's ending just about now as I write this, but I attended only on Wednesday and Thursday, so here are some highlights.

There was a discussion by the Aquifer people of conformity to implementation guidelines (PDF) for MODS records. MODS is full of options, which results in people doing the same thing in different ways. (This is in contrast to Dublin Core, which has so few options that people invent new and different ways to do the same thing. Somehow you get to the same place either way.)

I got to meet several of the people who are working on JHOVE2 for the first time. The functional requirements are fairly stable by now, and there should be some pieces available by spring.

There was a presentation (not available online) comparing the performance of JHOVE and DROID. JHOVE won consistently when dealing with known file formats, but was slower than DROID when trying to identify VRML files, which it knows nothing about. I suggested disabling the HTML module, which can waste large amounts of time going through large ASCII files like VRML, if it isn't needed.

That study, by Quyen Nguyen of NARA, isn't available online, but he mentioned a study at the University of London Computer Centre which compares JHOVE, DROID, and Stanford's Empirical Walker (which I hadn't even heard of before). It discusses differences in functionality rather than speed.

Labels: conferences, JHOVE, metadata

# posted by Gary McGath @ 12:57 PM 0 comments

The Dublin Core Metadata Initiative (DCMI) has a couple of new working drafts:

• Interoperability Levels for Dublin Core Metadata. From its description: "This document discusses the design choices involved in designing applications for different types of interoperability. At Level 1, applications use data components with shared natural- language definitions. At Level 2, data is based on the formal-semantic model of the W3C Resource Description Framework. At Level 3, data is structured as Description Sets (records). At Level 4, data content is subject to a shared set of constraints (described in a Description Set Profile)."
• Guidelines for Dublin Core Application Profiles. From the description: "This document provides guidelines for the creation of Dublin Core Application Profiles. The document explains the key components of a Dublin Core Application Profile and walks through the process of developing a profile. The document is aimed at designers of application profiles -- people who will bring together metadata terms for use in a specific context."

Labels: metadata, XML

# posted by Gary McGath @ 10:00 AM 0 comments

As I posted earlier, Thomas Reuters Corporation is using litigation to prevent reverse engineering of the EndNote format. George Mason University, the target of the litigation, has responded by saying that it will continue to support Zotero and is dropping its license for EndNote.

Usually I try to maintain a journalistic tone in this blog, but I just have to say it: Good for George Mason University, and screw Reuters!

Labels: metadata

# posted by Gary McGath @ 10:55 AM 0 comments

Live-blogging this isn't. I didn't get a chance to post from London, and then I spent the next week having fun around Germany. But here are some notes on what happened, with emphasis on file format and metadata issues.

Participants got a nice thick book of proceedings to take home, providing more information than the twenty-minute presentations did. I think the presentations should have gotten half an hour each, even though that would have meant fewer of them.

Trends in archives and libraries are toward JPEG2000 as an image storage format, even though it has to be converted before browsers can use it. Oya Rieger, talking about digitization of books, cited the smaller file size compared to JPEG, and the support of archival features such as incorporation of metadata into the file.

PREMIS is gaining traction as a format for preservation metadata. For instance, the British Library is using PREMIS 1.1, along with METS and MODS, for an eJournal archiving project. The article is available on the D-Lib site.

Stephen Abrams gave a presentation on JHOVE2, which is still in the design and prototyping stages. It's getting obvious from various discussions that the problems in JHOVE are becoming more of a concern. JHOVE2 will only partially replace JHOVE, since budgetary considerations limit the number of format modules planned. It will separate identification from validation; this means that if a file is a slightly flawed TIFF, it will tell you rather than just saying it's a "bitstream." It will also add assessment, which I don't fully understand, but will incorporate locally defined acceptability criteria. I think this means that if you can deal with certain common but minor deviations from a format specification, you can tell JHOVE2 that such files are acceptable.

JHOVE2 will get away from the one file, one object idea. In its model, a file may contain more than one document, or a document may be spread across multiple files. It was also mentioned that there will be API-level support for the edited of objects, but third parties will have to supply any editors. If my notes are complete, JHOVE will support JPEG2000, TIFF, WAVE, SGML, UTF-8, XML, PDF, shapefile, and ICC profiles. AIFF, GIF, HTML, and JPEG won't be supported. (GIF surprises me a bit; it's so simple that I'm sure someone will write a JHOVE2 module for it.)

JHOVE2 will use SourceForge for distribution and issue tracking, as JHOVE now does (though JHOVE is just a spare-time project without support right now). There's a public information site on JHOVE2 available right now, so far without a lot of content. The paper on JHOVE2 presented in the conference proceedings is available online.

Labels: conferences, JHOVE, JPEG2000, metadata

# posted by Gary McGath @ 8:14 PM 0 comments

Dublin Core Metadata Initiative has published a DCMI recommendation on "Expressing Dublin Core metadata using HTML/XHTML meta and link elements". This provides a standardized way to embed metadata on a single work in a web page.

Labels: HTML, metadata, XML

# posted by Gary McGath @ 12:35 PM 0 comments

There are a couple of announcements from the Dublin Core Metadata Initiative:

A major maintenance update of the DCMI metadata terms has been issued. The revisions are described here.

There is a new DCMI recommendation on Expressing Dublin Core metadata in RDF, replacing earlier documents. There is discussion of the changes in "Notes on DCMI specifications for Dublin Core metadata in RDF."

See DCMI 2008 news for more details on these announcements and others.

Labels: metadata

# posted by Gary McGath @ 9:16 AM 0 comments

The Dublin Core Metadata Initiative has published DC-TEXT, "a syntax for serializing, or representing, a Dublin Core metadata description set in plain text" as a DCMI Recommended Resource. This provides a more human-readable alternative to formats such as XML.

Labels: metadata

# posted by Gary McGath @ 2:27 PM 0 comments

There's some preliminary information available on Dublin Core description set profiles, including a presentation-style PDF by Mikael Nilsson and a more detailed description. From what I'm getting out of it, it's a way for organizations to define their own metadata and formally specify constraints on it. Some of its features are similar to those of XML schemas. As explained in the article:

Labels: metadata

# posted by Gary McGath @ 12:40 PM 0 comments