Dataverse Software 5.8

This release brings new features, enhancements, and bug fixes to the Dataverse Software. Thank you to all of the community members who contributed code, suggestions, bug reports, and other assistance across the project.

Release Highlights

Support for Data Embargoes

The Dataverse Software now supports file-level embargoes. The ability to set embargoes, up to a maximum duration (in months), can be configured by a Dataverse installation administrator. For more information, see the Embargoes section of the Dataverse Software Guides.

• Users can configure a specific embargo, defined by an end date and a short reason, on a set of selected files or an individual file, by selecting the 'Embargo' menu item and entering information in a popup dialog. Embargoes can only be set, changed, or removed before a file has been published. After publication, only Dataverse installation administrators can make changes, using an API.

• While embargoed, files cannot be previewed or downloaded (as if restricted, with no option to allow access requests). After the embargo expires, files become accessible. If the files were also restricted, they remain inaccessible and functionality is the same as for any restricted file.

• By default, the citation date reported for the dataset and the datafiles in version 1.0 reflects the longest embargo period on any file in version 1.0, which is consistent with recommended practice from DataCite. Administrators can still specify an alternate date field to be used in the citation date via the Set Citation Date Field Type for a Dataset API Call.

The work to add this functionality was initiated by Data Archiving and Networked Services (DANS-KNAW), the Netherlands. It was further developed by the Global Dataverse Community Consortium (GDCC) in cooperation with and with funding from DANS.

Major Use Cases and Infrastructure Enhancements

Newly-supported major use cases in this release include:

• Users can set file-level embargoes. (Issue #7743, #4052, #343, PR #8020)
• Improved accessibility of form labels on the advanced search page (Issue #8169, PR #8170)

Notes for Dataverse Installation Administrators

Mitigate Solr Schema Management Problems

With Release 5.5, the <copyField> definitions had been reincluded into schema.xml to fix searching for datasets.

This release includes a final update to schema.xml and a new script update-fields.sh to manage your custom metadata fields, and to provide opportunities for other future improvements. The broken script updateSchemaMDB.sh has been removed.

You will need to replace your schema.xml with the one provided in order to make sure that the new script can function. If you do not use any custom metadata blocks in your installation, this is the only change to be made. If you do use custom metadata blocks you will need to take a few extra steps, enumerated in the step-by-step instructions below.

New JVM Options and DB Settings

• :MaxEmbargoDurationInMonths controls whether embargoes are allowed in a Dataverse instance and can limit the maximum duration users are allowed to specify. A value of 0 months or non-existent setting indicates embargoes are not supported. A value of -1 allows embargoes of any length.

Complete List of Changes

For the complete list of code changes in this release, see the 5.8 Milestone in Github.

For help with upgrading, installing, or general questions please post to the Dataverse Community Google Group or email [email protected].

Installation

If this is a new installation, please see our Installation Guide. Please also contact us to get added to the Dataverse Project Map if you have not done so already.

Upgrade Instructions

0. These instructions assume that you've already successfully upgraded from Dataverse Software 4.x to Dataverse Software 5 following the instructions in the Dataverse Software 5 Release Notes. After upgrading from the 4.x series to 5.0, you should progress through the other 5.x releases before attempting the upgrade to 5.8.

If you are running Payara as a non-root user (and you should be!), remember not to execute the commands below as root. Use sudo to change to that user first. For example, sudo -i -u dataverse if dataverse is your dedicated application user.

In the following commands we assume that Payara 5 is installed in /usr/local/payara5. If not, adjust as needed.

export PAYARA=/usr/local/payara5

(or setenv PAYARA /usr/local/payara5 if you are using a csh-like shell)

1. Undeploy the previous version.

• $PAYARA/bin/asadmin list-applications
• $PAYARA/bin/asadmin undeploy dataverse<-version>

2. Stop Payara and remove the generated directory

• service payara stop
• rm -rf $PAYARA/glassfish/domains/domain1/generated

3. Start Payara

• service payara start

4. Deploy this version.

• $PAYARA/bin/asadmin deploy dataverse-5.8.war

5. Restart payara

• service payara stop
• service payara start

6. Update Solr schema.xml.

/usr/local/solr/solr-8.8.1/server/solr/collection1/conf is used in the examples below as the location of your Solr schema. Please adapt it to the correct location, if different in your installation. Use find / -name schema.xml if in doubt.

6a. Replace schema.xml with the base version included in this release.

   wget https://github.com/IQSS/dataverse/releases/download/v5.8/schema.xml
   cp schema.xml /usr/local/solr/solr-8.8.1/server/solr/collection1/conf

For installations that are not using any Custom Metadata Blocks, you can skip the next step.

6b. For installations with Custom Metadata Blocks

Use the script provided in the release to add the custom fields to the base schema.xml installed in the previous step.

   wget https://github.com/IQSS/dataverse/releases/download/v5.8/update-fields.sh
   chmod +x update-fields.sh
   curl "http://localhost:8080/api/admin/index/solr/schema" | ./update-fields.sh /usr/local/solr/solr-8.8.1/server/solr/collection1/conf/schema.xml

(Note that the curl command above calls the admin api on localhost to obtain the list of the custom fields. In the unlikely case that you are running the main Dataverse Application and Solr on different servers, generate the schema.xml on the application node, then copy it onto the Solr server)

7. Restart Solr

Usually service solr stop; service solr start, but may be different on your system. See the Installation Guide for more details.