All items from DuraSpace Blog » 2008 » March

Ithaca, NY The National Science Digital Library (NSDL) NCore team has announced the release of an open source generic version of the NSDL Search and NDR Interactions MediaWiki extensions and WordPressMU plugins that power the NSDL Expert Voices blogosphere and NSDL Wiki. Download the plugins here:
http://sourceforge.net/project/showfiles.php?group_id=204090
http://wiki.nsdl.org/index.php/NCore
NCore technologies and standards replace NSDL’s original metadata records-based data paradigm allowing for greater flexibility in collaborating and creating context around library resources. In the initiative’s first phase, completed in January 2007, NSDL Core Integration successfully implemented existing library services on top of NCore—a change that was largely invisible to users. Now, NSDL is leveraging what the Fedora-based NCore platform has to offer by introducing next-generation library services and collaborative tools.

San Francisco, CA The Topaz Project hosted developers from Mulgara, Fedora Commons, and Topaz on March 5-6, 2008 for a series of design meetings and discussions based on Mulgara’s XA2 Storage Layer.

Paul Gearon, Mulagara, presented an overview of how Mulgara’s storage currently works. It consists of a “string pool” (mapping numbers to URIs and literals) and a set of six indexes: GSPO, GPOS, GOSP, SPOG, POSG, OSPG. Each of the indexes contains a full copy of the (Subject, Predicate, Object, Graph) quads, just in a different sorted order.

Andrae Muys, Netymon, a specialist supplier of technology development and consultancy services, introduced his ideas on XA2, a redesign of Mulgara’s storage layer. Major goals are to support concurrent writes and up to 100 Billion triples in one instance (two orders of magnitude above what Mulgara can achieve today). Read Muys’ discussion paper:

Six Reasons Why Mulgara’s XA2 Storage Layer Should Matter to You

Muys proposes replacing the string pool with a Trie-based data structure capable of storing 100 billion+ strings. He also suggests a novel approach for the indexes using redundant binary numbers. This approach would reduce the cost of updates to the indexes.

A series of triplestore-related talks hosted by SDForum in Palo Alto. David Wood gave an overview of his company (Zepheira) and talked about Mulgara goals. Andy Palmer (co-founder of Vertica, which uses C-Store) and Sam Madden (co-developer of C-Store at MIT) gave an overview of how Vertica can be used as a read-optimized triplestore.

Ideas for incremental improvements to Mulgara that would improve performance were presented. One idea was to change the string pool so it doesn’t constantly bother with re-allocation of space. This would keep “dirty” data on disk for longer, deferring cleanup to

a later time. Another idea was to use BTrees for the indexes. Currently, Mulgara employs AVLTrees. AVLTrees have the advantage of always being balanced (so they’re always optimized for read performance), but they are relatively expensive to re-balance when

deletes occur. Paul also presented an interesting “out of the box” idea on storing all the indexes as pairs: (subjectID, statementID), (predicateID, statementID), (objectID, statementID), versus triples (subject, predicate, object). This would give every statement in the store a unique ID (think “reification”), but he made a case for why it would actually be more performant than what we have today.

James Leigh introduced the idea of supporting Sesame’s SAIL api in Mulgara. SAIL is similar in purpose to Trippi, but it supports transactions and uses the OpenRDF APIs (as opposed to JRDF, which is falling out of favor). The Mulgara team was generally receptive to the idea. From a technical perspective, it appears the challenge will be mapping the abstract syntax of iTQL to that of SAIL (which is currently the union of SERQL and SPARQL). From a social perspective, there is some concern that the use of SAIL (traditionally

seen as “the Sesame API”) may weaken Mulgara’s brand. However, there is general recognition of the need for a (transactional) JDBC-like API for working with triplestores, and if this API were supported by both Sesame and Mulgara, it would be a win for the community, and would attract more users to Mulgara.

Contributed by Chris Wilper, Lead Software Developer, Fedora Commons

Stockholm, Sweden The National Library of Sweden will host a Scandinavian Fedora Users Group Meeting on June 24-25, 2008. Meeting details and a call for proposals will follow. Contact Eva Müller for more information.

Ithaca, NY Neil Beagrie, a leading expert on digital preservation and curation with an international reputation across the archive, library, science and research sectors in the long-term management of digital assets, will join Sandy Payette, Executive Director, and Eddie Shin, Senior Software Developer, Fedora Commons, in delivering plenary lectures at “Fedora Day”—the Fedora User Group Meeting—at OR08 in Southampton on April 3, 2008.
Payette and Shin will present an overview of Fedora Commons organizational and technical
progress. Beagrie will reflect on current digital preservation trends, review Fedora’s history in support of long-term durability of digital content, and provide repository developers and managers with suggestions for effective digital preservation strategies.
Neil Beagrie has played a major role in digital preservation for several organizations including founding the Digital Preservation Coalition and overseeing development of the UK Digital Curation Centre: he was also the international consultant to the U.S. National Digital Information Infrastructure and Preservation Program (NDIIPP). His career has spanned a range of senior information management roles at the British Library, the JISC, the Arts and Humanities Data Service, and the Royal Commission on the Historical Monuments of England.