Kindura: A Pragmatic Open Source Approach to Institutional Cloud Storage with Fedora and DuraCloud

Mon, 2012-07-30 14:58 -- carol

Ithaca, NY Institutions with IT departments offer services and storage based on infrastructure that has been accumulated over time. Limitations of in-house services combined with evolving data curation, management and cost requirements, have made the flexibility of cloud computing and storage options appealing. The Kindura Project sought to test a hybrid cloud, shared service and in-house solution for providing repository-focused services to researchers across partner institutions to enable an institution to make use of its own unused internal archival storage along with elastic commercial or academic clouds. The project team looked at requirements for cloud storage in a repository as a proof of concept with the aim of producing a case study and documentation using ad hoc research data and related documents from multiple institutions not covered under existing preservation or archiving plans.

The Kindura project was led by Kings College London and funded by the JISC Flexible Service Delivery Program. The Science and Technologies Facilities Council (STFC) and DuraSpace were project partners. Authors Simon Waddington, Gareth Knight, Mark Hedges, Jens Jensen, and Roger Downing describe investigations and outcomes in the Journal of Digital Information, Vol 13, No 1 (2012) Kindura: Repository services for researchers based on hybrid clouds http://journals.tdl.org/jodi/article/view/5877>.

"We carried out our pilot using DuraCloud, which has been developed by DuraSpace. We used DuraCloud to broker between storage or compute resources supplied by external cloud services, shared services, or in-house services. The Fedora repository, which interoperates easily with DuraCloud, provided the researcher front-end for managing research outputs and information. The data management infrastructure was based on the iRODS grid-based storage system to leverage its facilities for automatic replication and server-side data processing workflows."

Kindura performed cost optimization analyses for storage  balanced with data transfers in and data transfers out. Semi-automatic migration workflows could send data to a lower tier storage based on data lifecycle stages.

The Kindura project provided researchers with technical proof of concept for a flexible institutional repository hybrid storage model based on cloud storage combined with existing IT resources. The project team welcomes feedback from researchers.

For a complete project overview plus related presentations,  published papers and information about how to download Kindura open source code please visit:

https://jiscinfonetcasestudies.pbworks.com/w/page/45197715/Kindura

Project web site:

http://kindura.cerch.kcl.ac.uk/