NEW RELEASE: Message-based Integrations for Fedora

Tue, 2016-08-30 14:40 -- carol

From Aaron Coburn, Programmer and Systems Administrator, Amherst College

Amherst, MA  I would like to announce the immediate availability of version 4.6.0 of the Fedora Messaging Toolbox.

The messaging toolbox is designed to support a variety of asynchronous integrations with external tools and services, such as a Solr search engine or an external Triplestore. Version 4.6.0 of the messaging toolbox is compatible with both the forthcoming 4.6.0 release of the Fedora Commons server and previous releases of Fedora.

This release has been tested to run on Karaf v4.0.5 and v4.0.6.

Please Note: the 4.7.x branch of the messaging toolbox will not be compatible with version 4.5.x of Fedora or any earlier release.

Release Highlights:

• HTTPS support for the Solr integration
• HTTPS support for the Triplestore integration
• New fcrepo-ldpath feature
• Shared ActiveMQ service
• Shared fcrepo-camel service

I would like to especially thank Mohamed Mohideen Abdul Rasheed and Peter Eichman from the University of Maryland for their contributions leading to this release.

More information about installing and running the individual components of this toolbox can be found on the github page: https://github.com/fcrepo4-exts/fcrepo-camel-toolbox

There are several changes to note for existing users of the messaging toolbox:

1. As in previous releases, each component can be configured separately, but now the configuration for the connection to the repository is shared among components. That is, if Fedora requires authentication, users will define that in only a single configuration file, rather than across all configuration files. That configuration file can be found in $KARAF_HOME/etc/org.fcrepo.camel.service.cfg

2. Rather than hard-coding an ActiveMQ-based listener in the camel components, the ActiveMQ-based code has been separated into its own service, so that once Fedora offers multiple message broker options, the toolbox code will be able to support them (RabbitMQ, ZeroMQ, etc). This means that users will need to install `fcrepo-service-activemq` in addition to the desired integration component. E.g.:

$> feature:install fcrepo-service-activemq
$> feature:install fcrepo-indexing-triplestore

3. In previous releases, various integrations supported the use of scheme-less URLs. For example, the triplestore baseURL might have been configured to be: localhost:8080/fuseki/test/update or the Fedora baseURL might have been: localhost:8080/fcrepo/rest. Now, all URLs *must* have a scheme: http://localhost:8080/fuseki/test/update. This also means that it is now possible to use HTTPS connections: https://repo.example.org/fuseki/repo/update.

4. The current (and forthcoming) Fedora release include a module called fcrepo-transform (part of fcrepo-webapp-plus), which allows using arbitrary LDPath queries on a given RDFSource resource in the repository. This is especially useful for querying resources with string literal-based metadata, but for anyone making extensive use of linked data best practices and using URIs to describe resources, the existing fcrepo-transform tooling is quite limited: it cannot dereference any of these URIs. The `fcrepo-ldpath` is a new feature for the fcrepo-camel-toolbox project: it allows users to run an arbitrary LDPath program, starting at any Fedora resource, but now it can follow links across multiple in-repository resources as well as dereferencing externally referenced URIs.

For example, if a Fedora resource contains a triple of the form: <> dcterms:subject <http://id.loc.gov/authorities/subjects/sh85020108>, an external (e.g. Solr) index may be significantly more useful to have the de-referenced string value of the LoC subject heading.

In this case, an LDPath program such as:

id = . :: xsd:string ;
subject = dcterms:subject / skos:prefLabel :: xsd:string ;

will produce a JSON structure such as:

[ { "subject" : [ "Carbon dioxide" ] , "id" : [ "http://localhost:8080/fcrepo/rest/resource" ] } ]

In this case, that string "Carbon dioxide" does not exist in the local repository and is only materialized by dereferencing the remote URI.

In this release, the fcrepo-indexing-solr component does not use this new feature -- it still relies on fcrepo-transform. In the next release of this toolbox, though, the solr indexing feature will be based on the fcrepo-ldpath component.

Installing the fcrepo-ldpath component involves also specifying a cache implementation such as:

$> feature:install fcrepo-ldcache fcrepo-service-ldcache-file

 

preserve