MOVING Content: Institutional Tools and Strategies for Fedora 3 to 4 Upgrations

Thu, 2015-06-11 11:46 -- carol

Winchester, MA  The Fedora team has made tools that simplify content migration from Fedora 3 to Fedora 4 available to assist institutions in establishing production repositories. Using the Hydra-based Fedora-Migrate tool — which was built in advance of Penn State’s deadline to have Fedora 4 in production, before the generic Fedora Migration Utilities was released —  Penn State’s ScholarSphere moved all data from production instances of Fedora 3 to Fedora 4 in about 20 hours. The new ScholarSphere repository now takes advantage of Fedora 4’s processing of large files. The ScholarSphere service was collaboratively developed by The University Libraries and Information Technology Services (ITS) to offer coordinated digital stewardship support for Penn State e-research and e-science initiatives.

Fedora-Migrate and Fedora Migration Utilities are now available for community testing to ensure that Fedora community members have the tools they need to begin work toward establishing Fedora 3 to 4 repository tests and production workflows.

Most pilot institutions tested Fedora-Migrate and Fedora Migration Utilities while taking slightly different approaches to using the migration tools to determine how to move towards establishing production Fedora 4 repositories.

Last year the National Library of Wales, The University of New South Wales, York University, and Columbia University developed pilot projects to test the migration of content from Fedora 3 to Fedora 4 under a variety of scenarios in preparation for the launch of production repositories with superior Fedora 4 features.  Each institution took a slightly different approach to scoping their test project and defining its successful implementation.

Like Penn State’s ScholarSphere, Columbia University’s Academic Commons took advantage of the Fedora-Migrate tool to determine if Fedora 3 data could be modeled in Fedora 4. The Fedora-Migrate tool, which is designed primarily for Hydra implementations with an emphasis on those based on Sufia, iterates over existing Fedora 3 objects using the Rubydora gem. For each object it finds, it creates a new object with the same id in Fedora 4 and proceeds to migrate each datastream, including versions if they are defined, and verifies the checksum of each. Permissions and relationships are migrated as well but using different procedures due to the changes to permissions and relationships in Fedora 4. Developers at Columbia noted that future migrations would require additional modifications to Fedora-Migrate because more Fedora 3 features are used as part of Columbia content models.

The National Library of Wales (NLW) tested the general Migration Utilities process your Fedora 3 FOXML documents and convert them to Fedora 4 resources. The NLW pilot project tested the migration of a large Newspaper title (187,331 objects) and a modern digitized Journal with complex rights issues. The NLW team found that the the Migration Utilities were easy to use. The team tested the utilities and were able to provide valuable feedback to the developers by logging issues in the JIRA bug tracker. This feedback allows for improvement of the migration utilities to support a broader range of use cases.

University of New South Wales (UNSW) directed their pilot efforts towards coming up with a strategy for upgrading the Library’s Fedora 3-based repositories. A key part of this strategy was the development of a Fedora 4 data model that would remain compatible with existing UNSW repositories but would also align with the Portland Common Data Model. The project has established a test Fedora 4 instance that implements the preliminary data model, and later in 2015 the first UNSW legacy repository will be migrated to Fedora 4.

York University Libraries has developed YUDL which is an institutional Islandora repository that runs on the latest version of all Islandora Foundation modules. The repository is as close to a stock/generic Islandora instance as possible. The aim of the project was to serve as a basis for a generic Islandora Fedora 3 to Fedora 4 upgration using Fedora Migration Utilities. To this end, the project contributed greatly to the development of the Fedora Migration Utilities, including detailed mappings from Fedora 3 to Fedora 4 data models.

These Fedora 4 migration tools, Fedora-Migrate and Fedora Migration Utilities, are now available thanks to community efforts. Please download, test and provide feedback to help improve them!