Dataverse 6.7

Please note that there is a bug (#11698) in 6.7 where loading the create/edit dataset screen is very slow. We plan to release a 6.7.1 hotfix (#11701). Please wait to upgrade until 6.7.1 has been released.

Please note: To read these instructions in full, please go to https://github.com/IQSS/dataverse/releases/tag/v6.7 rather than the list of releases, which will cut them off.

This release brings new features, enhancements, and bug fixes to Dataverse. Thank you to all of the community members who contributed code, suggestions, bug reports, and other assistance across the project!

Release Highlights

Highlights for Dataverse 6.7 include:

• Keeping S3 storage working after December 2025
• Limiting files per dataset
• Curation status label enhancements
• Configurable search services
• Linking drafts
• Dataset metadata exports from drafts
• API for switching Datasets to DOIs, for example
• TK Labels
• Model Context Protocol (MCP) server
• A new AI Guide
• OJS 3 is now a supported integration
• Tagged Docker images
• Rate limiting statistics API
• Infrastructure: Payara upgraded to 6.2025.3
• Security fixes

Features Added

Keep S3 Storage Working After December 2025

To support S3 storage, Dataverse uses the AWS SDK. We have upgraded to v2 of this SDK because v1 reaches End Of Life (EOL) in December 2025.

As part of the upgrade, the payload-signing setting for S3 stores (dataverse.files.<id>.payload-signing) has been removed because it is no longer necessary. With the updated SDK, a payload signature will automatically be sent when required (and not sent when not required).

Dataverse developers should note that LocalStack is used to test S3 and older versions appear to be incompatible. The development environment has been upgraded to LocalStack v2.3.2 to v4.2.0, which seems to work fine.

See also #11073 and #11360.

Limiting Files Per Dataset

It's now possible to set a limit on the number of files that can be uploaded to a dataset. Limits can be set globally, per collection, or per dataset.

See also the guides, #11275, and #11359.

Curation Status Label Enhancements

The External/Curation Status Label mechanism has been enhanced:

• adding tracking of who creates the status label and when
• keeping a history of past statuses
• updating the CSV report to include the creation time and assigner of a status
• updating the getCurationStatus API call to return a JSON object for the status with label, assigner, and create time
• adding an includeHistory query param for these API calls to allow seeing prior statuses
• adding a facet to allow filtering by curation status (for users able to set them)
• adding the creation time to Solr as a pdate to support search by time period, e.g. current status set prior to a given date
• standardizing the language around "curation status" vs "external status"
• adding a "curation-status" class to displayed labels to allow styling
• adding a dataverse.ui.show-curation-status-to-all feature flag that allows users who can see a draft but not publish it to also view the curation status

Due to changes in the Solr schema (the addition of fields "curationStatus" and "curationStatusCreateTime"), updating the Solr schema and reindexing is required as described below in upgrade instructions. Background reindexing should be OK. See also #9247 and #11268.

Configurable Search Services

Dataverse now has an experimental capability to dynamically add and configure new search engines. The current Dataverse user interface can be configured to use a specified search engine instead of the built-in solr search. The Search API now supports an optional searchService query parameter that allows using any configured search engine. An additional /api/search/services endpoint allows discovery of the services installed.

In addition to two trivial example services designed for testing, Dataverse ships with two search engine classes that support calling an externally-hosted search service (via HTTP GET or POST). These classes rely on the internal solr search to perform access-control and to format the final results, simplifying development of such an external engine.

Details about the new functionality are described in the guides. See also #11281.

Linking Drafts

It is now possible to link draft datasets to other Dataverse collections. As usual, the datasets will only become publicly visible in the linked collection(s) after they have been published. To publish a linked dataset, your account must have the "Publish Dataset" permission for the Dataverse collection in which the dataset was originally created. Permissions in the linked Dataverse collections do not apply. See also #10134.

Dataset Metadata Can Be Exported From Draft Datasets (via API)

In previous versions of Dataverse, it was only possible to export metadata from published datasets. It is now possible to export metadata from draft datasets via API as long as you supply an API token that has access to the draft. As before, when exporting metadata from published datasets, only the latest published version is supported. Internal exporters have been updated to work with drafts but external exporters might need to be updated (Croissant definitely does). See "upgrade instructions" below for details. See the guides, #11305, and #11398.

API for Switching Datasets to DOIs, for Example

In some cases, you might want draft datasets to begin their life with a zero-cost PIDs such as Permalinks and later decide to give certain datasets a DOI. To support use cases like this, a new API for persistent identifier reconciliation has been added.

Here's how it works. An unpublished dataset can be updated with a new pidProvider. If a persistent identifier was already registered when the dataset was registered, this is undone and the new provider (if changed in the meantime) is used. Note that this change does not affect the storage repository where the old identifier is still used. See the guides, #10501, and #10567.

TK Labels

New API calls to find projects at https://localcontextshub.org associated with a dataset have been added. This supports integration via an external vocabulary script that allows users to associate such a project with their dataset and display the associated Notices and Tribal Knowledge Labels.

Connecting to LocalContexts requires a LocalContexts API Key. Using both the production and sandbox (test) LocalContexts servers are supported.

See also the guides and #11294.

Model Context Protocol (MCP) Server for Dataverse

Model Context Protocol (MCP) is a standard for AI Agents to communicate with tools and services, announced in November 2024.

An MCP server for Dataverse has been deployed to https://mcp.dataverse.org, powered by the code at https://github.com/gdcc/mcp-dataverse written by Vyacheslav Tykhonov.

All are welcome to experiment with the MCP Server and give feedback in the thread on Google Group and Zulip. See also #11474.

AI Guide

Information about various Dataverse-related AI efforts have been documented in a new AI Guide. See also #11540 and #11541.

OJS 3 is Supported

OJS 3 (version 3.3 and higher) is now supported as an integration with Dataverse. See the guides and #11518 for details.

Tagged Docker Images

Please note: We are aware that the "latest" tag does not (yet) appear on Docker Hub (#11661). In the meantime, you can use "6.7-noble" (see all available tags), which is equivalent.

Container image management has been enhanced to provide better support for multiple Dataverse releases and improved maintenance workflows.

Versioned Image Tags: Application ("dataverse") and Config Baker images on Docker Hub now have versioned tags, supporting the latest three Dataverse software releases. This enables users to pin to specific versions (e.g. 6.7), providing better stability for production deployments. Previously, the "alpha" tag could be used, but it was always overwritten by the latest release. Now, you can choose the 6.7 tag, for example, to stay on that version. Please note that the "alpha" tag should no longer be used and will likely be deleted. The equivalent is the new "latest" tag.

Backport Support: Application and Config Baker image builds now support including code backports for past releases, enabling the delivery of security fixes and critical updates to older (supported) versions.

Enhanced Documentation: Container image documentation has been updated to reflect the new versioning scheme and maintenance processes.

Config Baker Base Image Change: The Config Baker image has been migrated from Alpine to Ubuntu as its base operating system, aligning with other container images in the project for consistency and better compatibility. The past releases have not been migrated, only future releases (6.7+) will use Ubuntu.

Workflow Responsibility Split: GitHub Actions workflows for containers have been reorganized with a clear separation of concerns:

• container_maintenance.yml handles all release-time and maintenance activities
• Other workflows focus solely on preview images for development merges and pull requests

These improvements provide more robust container image lifecycle management, better security update delivery, and clearer operational procedures for both development and production environments. See also the Container Guide, #10618, and #11477.

Rate Limiting Statistics API

A new Rate Limiting Statistics API gives insight into the current state of rate limiting such as the number of users being limited and the number of available bucket tokens for a command.

See also the guides, #11413, and #11424.

Payara 6.2025.3

The recommended Payara version has been updated to Payara-6.2025.3. See the upgrade instructions below and #11357.

Rclone Support

Rclone ("rsync for cloud storage") is a command-line program to sync files and directories to and from different cloud storage providers. As of version 1.70 Rclone supports Dataverse. See the announcement, the guides, #11608, and #11609.

Unique Filenames for Zip Downloads

The Data Access APIs that generate multi-file zipped bundles will offer file name suggestions based on the persistent identifiers (for example, doi-10.70122-fk2-xxyyzz.zip), instead of the fixed filename dataverse_files.zip as in prior versions. This means you'll see unique names in your "downloads" folder. See the guides, #9620, and #11466.

New Metadata Field Type: String

The "string" type has been added as a new field type for metadata fields.

In contrast to "text" fields, "string" fields are stored and indexed exactly as provided, without any text analysis or transformations.

This field type is suitable for fields like IDs (e.g. ORCIDs) or enums, where exact matches are required when searching.

See also #11147 and #11321.

Tabular Tags Can Now Be Replaced

Previously the API POST /files/{id}/metadata/tabularTags could only add new tags to the tabular tags list. Now with the query parameter ?replace=true the list of tags will be replaced.

See also the guides, #11292, and #11359.

Make Data Count Improvements

Counter Processor, used to power Make Data Count metrics in Dataverse, is now maintained in the https://github.com/gdcc/counter-processor repository. Multiple improvements to efficiency and scalability have been made. The example counter_daily.sh and counter_weekly.sh scripts that automate using Counter Processor, available from the MDC section of the Dataverse Guides have been updated to work with the latest Counter Processor release and also have minor improvements. See also #11489.

Improved Navigation for Guides

Navigation across the guides has been improved. You can now click in the upper left to go "home". The navbar has been simplified with fewer links. The bottom of every page now has "Next" and "Previous" links. A "Source" link at the bottom has also been added. See also #10942.

Video Subtitles (vtt Files)

Video subtitles (vtt files) are now supported and indexed using full text indexing, if configured.

All new files uploaded with a .vtt extension will be assigned the context type "text/vtt" and shown as "Web Video Text Tracks". See upgrade instructions below to convert existing files.

The upgrade instructions below also explain how to upgrade to v1.5 of the Dataverse Previewers, which includes an updated video previewer that supports subtitles. The new previewer version presents vtt files as subtitles for videos, and the naming convention is <video-basename>.<language-tag>.vtt. The previewer does not rely on the content type. A proper content type may hint users to ask permission for the subtitles together with a video.

See also #11041.

Dataset Types Can Set Available Licenses

Licenses (e.g. "MIT") can now be linked to dataset types (e.g. "software") using new superuser APIs. The create Dataset Type APIs have been extended to allow you to set metadata blocks and/or licenses on the creation of a Dataset Type. (You can change both later.)

If a license is not available for a given Dataset Type then the Create Dataset API will prevent that license from being applied to the dataset. Also, the UI will only show those licenses that are available to the Dataset Type.

For more information, see the guides (overview, new APIs), #10520, and #11385.

Loading Metadata Blocks in Docker

The tutorial on running Dataverse in Docker has been updated to include how to load a metadata block and then update Solr to know about the new fields. See also #11004 and #11204.

Solr Indexing Speed Improved

The performance of Solr indexing has been significantly improved, particularly for datasets with many files.

A new dataverse.solr.min-files-to-use-proxy microprofile setting can be used to further improve performance/lower memory requirements for datasets with many files (e.g. 500+) (defaults to Integer.MAX, disabling use of the new functionality). See also #11374.

Improved Efficiency for Per-Request Filters

This release improves the performance of Dataverse's per-request handling of CORS Headers and API calls.

It adds new JVM options/Microprofile settings (starting with dataverse.cors and dataverse.api) replacing the now deprecated database settings (starting with :BlockedApi and :AllowCors). (See "new settings" and "deprecated settings" below for a full list.)

Additional changes:

• CORS headers can now be configured with a list of desired origins, methods, and allowed and exposed headers.
• An X-Dataverse-unblock-key header has been added that can be used instead of the less secure unblock-key query parameter when the :BlockedApiPolicy is set to unblock-key.
• Warnings have been added to the log if the Blocked API settings are misconfigured or if the key is weak (when the unblock-key policy is used).
• The new dataverse.api.blocked.key can be configured using Payara password aliases or other secure storage options.

See also the guides and #11454.

New OIDC Feature Flag

A new feature flag called API_BEARER_AUTH_USE_BUILTIN_USER_ON_ID_MATCH has been introduced, which allows the use of a built-in user account when an identity match is found during OIDC API bearer token authentication.

This feature enables automatic association of an incoming Identity Provider (IdP) identity with an existing built-in user account, bypassing the need for additional user registration steps.

See the guides, #11193, #11197, and #11314.

dataverse-metadata-crawler

The dataverse-metadata-crawler was added to the guides. See #11581.

Bugs Fixed

Reduced Chance of Losing Metadata Edits

Changes were made to the "edit dataset metadata" page to reduce the chance of losing metadata edits.

The remedy for the problem consists of two parts: - Do not show the "Host Dataverse" field when there is nothing to choose. This mimics the behaviour for templates. - When you accidentally start typing in the "Host Dataverse" field, undo the change with backspace, fill in the other metadata fields and save the draft, the page used to get blocked due to an exception. Reloading the page would erase all your input. The exception (caused by an invalid argument) is remedied returning the currently selected value.

See also #11301.

Improved "Role Has Already Been Granted" Message

A simple "role has already been granted" message is now given, fixing a bug where "dataset" was incorrectly indicated instead of "collection". See also #11191 and #11362.

NcML Previewer Bug Fix

Dataverse Previewers v1.4 contains a bug in the NcML previewer that prevented it from working with signed URLs. See #11252 for screenshots.

This has been fixed in the "betatest" and v1.5 versions of the previewer. See also #11252 and #11311. Upgrading to v1.5 of all previewers is recommended in the upgrade instructions below.

Search API Bug Fix

The Search API now returns all type totals (Dataverses, Dataset, and Files) regardless of the list of types requested. None requested types were returned with total count set to 0. &type=dataverse&type=dataset would result in "Files": 0 since type=file was not requested. Now all counts show the correct totals. See also #11280.

Other Bug Fixes

• Deeply nested compound fields are not (yet) supported by Dataverse but the Search API now properly avoids returning duplicate values for them. See #11172.
• An issue causing more than one edit of a versionNote to fail, when done without a page refresh, has been fixed. See #11394.
• The deaccessionedReason was missing in the fileDifferenceSummary json object returned by API GET "$SERVER_URL/api/files/{ID}/versionDifferences". See #11438.
• Memory usage has been reduced and potential memory leaks closed in the metadata exporters. See #11417.

API Updates

Update File Metadata API

A new API endpoint has been added to allow updating file metadata for one or more files in a dataset. See the Native API documentation for details on usage and #11271.

Extend Restrict API to Include New Attributes

The "restrict" API only allowed for a boolean to update the restricted attribute of a file. For backward compatibility, this is still supported, but now a richer JSON object can be passed instead. The JSON object allows for the required restrict flag as well as optional attributes: enableAccessRequest and termsOfAccess. If enableAccessRequest is false then the termsOfAccess text must also be included.

See the guides, #11299, and #11349.

Categories Can Now Be Replaced

Previously the API POST /files/{id}/metadata/categories could only add new categories to the categories list. Now with the query parameter ?replace=true the list of categories will be replaced.

See also the guides, #11401, and #11359.

Application Terms of Use Available via API

It's now possible to retrieve the Application Terms of Use (called General Terms of Use in the UI) via API. These are the terms users agree to when creating an account. See the guides, #11415 and #11422.

dvObject and type Fields Added to Featured Items

Dataverse Featured Items can now be linked to Dataverses, Datasets, or Datafiles.

Pre-existing featured items as well as new items without dvObjects will be defaulted to type=custom. See also #11414.

Edit Dataset Metadata: Removing Fields

The "edit dataset metadata" endpoint now allows removing fields (by sending empty values) as long as they are not required by the dataset. See also #11243.

Edit Dataset Metadata: Prevent Inconsistencies

A new sourceInternalVersionNumber optional query parameter, which prevents inconsistencies by managing updates that may occur from other users while a dataset is being edited. See also #11243.

api/roles/userSelectable

A new endpoint (api/roles/userSelectable) has been implemented, which returns the appropriate roles that the calling user can use as filters when searching within their data. See #11434.

Security Updates

This release contains important security updates. If you are not receiving security notices, please sign up by following the steps in the guides.

Updates for Documentation Writers

Sphinx Upgraded

Sphinx has been upgraded to 7.4.0 and new dependencies have been added, including semver. Please re-run the pip install -r requirements.txt setup step to upgrade your environment. Otherwise you might see an error like ModuleNotFoundError: No module named 'semver'.

Updates for Developers

Development of Dataverse on Windows

Development of Dataverse on Windows has been confirmed to work as long as you use WSL rather than cmd.exe. See the updated quickstart, the rewritten page on Windows, #10606, and #11583.

Keycloak SPI for Built-In Users

A Keycloak SPI, builtin-users-spi, has been implemented that allows the use of Keycloak on instances with built-in accounts for OIDC authentication, enabling the use of the SPA on those instances.

Looking ahead, this authenticator SPI could also support mapping Shibboleth users coming in through Keycloak to existing Shib users without changing the provider in the Dataverse database. However, this would require changes to the storage provider to support more than just built-in users.

The SPI code is available in the Dataverse code repository (conf/keycloak/builtin-users-spi).

File Previews Available in Dev Environment, More Docs

In Dataverse 6.5 File Previewers were enabled in the "demo or eval" containerized (Dockerized) environment (#11025). These previewers are now available in the development environment as well and documentation has been added explaining how to configure them. See also #10506 and #11181.

XML Parsers

The configuration of XML parsers used in Dataverse has been centralized and unused functionality has been turned off to enhance security. See #11619.

End-Of-Life (EOL) Announcements

Whole Tale EOL

Unfortunately, the Whole Tale project is no longer active and has been removed from the list of integrations in the Admin Guide. See #11497.

New Settings

The following settings have been added:

• dataverse.api.blocked.policy: Policy for blocking API endpoints
• dataverse.api.blocked.endpoints: List of API endpoints to be blocked (comma-separated)
• dataverse.api.blocked.key: Key for unblocking API endpoints
• dataverse.bagit.sourceorg.name
• dataverse.cors.origin: Allowed origins for CORS requests
• dataverse.cors.methods: Allowed HTTP methods for CORS requests
• dataverse.cors.headers.allow: Allowed headers for CORS requests
• dataverse.cors.headers.expose: Headers to expose in CORS responses
• dataverse.files.hide-schema-dot-org-download-urls: now configurable via MicroProfile Config, see #11482
• dataverse.localcontexts.url
• dataverse.localcontexts.api-key
• dataverse.search.services.directory
• dataverse.search.default-service
• dataverse.solr.min-files-to-use-proxy
• dataverse.ui.show-curation-status-to-all
• :GetExternalSearchUrl
• :GetExternalSearchName
• :PostExternalSearchUrl
• :PostExternalSearchName

Deprecated Settings

• bagit.SourceOrganization entry in Bundle.properties
• :AllowCors
• :BlockedApiPolicy
• :BlockedApiEndpoints
• :BlockedApiKey

Removed Settings

• dataverse.files.<id>.payload-signing: See #11360

Backward Incompatible Changes

Generally speaking, see the API Changelog for a list of backward-incompatible API changes.

show_my_data removed from Search API

An undocumented Search API parameter called "show_my_data" has been removed. It was never exercised by tests and is believed to be unused. API users should use the MyData API instead. See #11287 and #11375.

curationStatus API

/api/datasets/{id}/curationStatus API now includes a JSON object with curation label, createtime, and assigner rather than a string label and it supports a new boolean includeHistory parameter (default false) that returns a JSON array of statuses. See #11268.

listCurationStates API

/api/datasets/{id}/listCurationStates includes new columns "Status Set Time" and "Status Set By" columns listing the time the current status was applied and by whom. It also supports the boolean includeHistory parameter. See #11268.

XML serialization of empty elements

Due to updates in libraries used by Dataverse, XML serialization may have changed slightly with respect to whether self-closing tags are used for empty elements. This primarily affects XML-based metadata exports. The XML structure of the export itself has not changed, so this is only an incompatibility if you are not using an XML parser. See #11360.

Complete List of Changes

For the complete list of code changes in this release, see the 6.7 milestone in GitHub.

Getting Help

For help with upgrading, installing, or general questions please post to the Dataverse Community Google Group or email [email protected].

Installation

If this is a new installation, please follow our Installation Guide. Please don't be shy about asking for help if you need it!

Once you are in production, we would be delighted to update our map of Dataverse installations around the world to include yours! Please create an issue or email us at [email protected] to join the club!

You are also very welcome to join the Global Dataverse Community Consortium (GDCC).

Upgrade Instructions

Upgrading requires a maintenance window and downtime. Please plan accordingly, create backups of your database, etc.

These instructions assume that you've already upgraded through all the 5.x releases and are now running Dataverse 6.6.

0. These instructions assume that you are upgrading from the immediate previous version. If you are running an earlier version, the only supported way to upgrade is to progress through the upgrades to all the releases in between before attempting the upgrade to this version.

If you are running Payara as a non-root user (and you should be!), remember not to execute the commands below as root. By default, Payara runs as the dataverse user. In the commands below, we use sudo to run the commands as a non-root user.

Also, we assume that Payara 6 is installed in /usr/local/payara6. If not, adjust as needed.

export PAYARA=/usr/local/payara6

(or setenv PAYARA /usr/local/payara6 if you are using a csh-like shell)

1. List deployed applications

$PAYARA/bin/asadmin list-applications

2. Undeploy the previous version (should match "list-applications" above)

$PAYARA/bin/asadmin undeploy dataverse-6.6

3. Stop Payara

sudo service payara stop

4. Upgrade to Payara-6.2025.3

The steps below reuse your existing domain directory with the new distribution of Payara. You may also want to review the Payara upgrade instructions as it could be helpful during any troubleshooting: Payara Release Notes. We also recommend you ensure you followed all update instructions from the past releases regarding Payara. (The most recent Payara update was for v6.6.)

Move the current Payara directory out of the way:

mv $PAYARA $PAYARA.6.2025.2

Download the new Payara version 6.2025.3 (from https://www.payara.fish/downloads/payara-platform-community-edition/ or https://nexus.payara.fish/repository/payara-community/fish/payara/distributions/payara/6.2025.3/payara-6.2025.3.zip), and unzip it in its place:

cd /usr/local
unzip payara-6.2025.3.zip

Replace the brand new payara/glassfish/domains/domain1 with your old, preserved domain1:

mv payara6/glassfish/domains/domain1 payara6/glassfish/domains/domain1_DIST
mv payara6.6.2025.2/glassfish/domains/domain1 payara6/glassfish/domains/

Start the new version of Payara:

sudo service payara start

5. Download and deploy this version

wget https://github.com/IQSS/dataverse/releases/download/v6.7/dataverse-6.7.war
$PAYARA/bin/asadmin deploy dataverse-6.7.war

Note: if you have any trouble deploying, stop Payara, remove the following directories, start Payara, and try to deploy again.

sudo service payara stop
sudo rm -rf $PAYARA/glassfish/domains/domain1/generated
sudo rm -rf $PAYARA/glassfish/domains/domain1/osgi-cache
sudo rm -rf $PAYARA/glassfish/domains/domain1/lib/databases
sudo service payara start

6. For installations with internationalization or text customizations:

Please remember to update translations via Dataverse language packs.

If you have text customizations you can get the latest English files from https://github.com/IQSS/dataverse/tree/v6.7/src/main/java/propertyFiles.

7. Restart Payara

sudo service payara stop
sudo service payara start

8. If you have enabled the Croissant exporter, update it and run reExportAll to update dataset metadata exports

After Dataverse 6.6 was released on 2024-03-18, two versions of the Croissant exporter have been released. You are encouraged to upgrade to the latest version, which is 0.1.5.

Under "installation" at the README at https://github.com/gdcc/exporter-croissant you'll find instructions about upgrading the Croissant exporter. In the same repo you can find a changelog if you are curious about what has changed.

Afterwards, we recommend reexporting all dataset metadata. (Reexporting just a single export format, like Croissant, is not supported.) Below is the simple way to reexport all dataset metadata. For more advanced usage, please see the guides.

curl http://localhost:8080/api/admin/metadata/reExportAll

9. Archival bags

If you are using archival bags, be sure that the dataverse.bagit.sourceorg.name JVM option is set.

Archival Bags now use the JVM option dataverse.bagit.sourceorg.name in generating the bag-info.txt file's "Internal-Sender-Identifier" (in addition to its use for "Source-Organization") rather than pulling the value from a deprecated bagit.SourceOrganization entry in Bundle.properties ("Internal-Sender-Identifier" is generated by appending " Catalog" in both cases). Sites using archival bags would not see a change if these settings were already using the same value. See #10680 and #11416.

10. API Filters

Per-request filtering has been improved. Migrate to the new settings as explained below as the old settings have been deprecated.

The deprecated database settings will continue to work in this version. To use the new settings (which are more efficient),

If :AllowCors is not set or is true:

bin/asadmin create-jvm-options -Ddataverse.cors.origin=*

Optionally set origin to a list of hosts and/or set other CORS JvmSettings Your currently blocked API endpoints can be found at http://localhost:8080/api/admin/settings/:BlockedApiEndpoints

Copy them into the new setting with the following command. As with the deprecated setting, the endpoints should be comma-separated.

bin/asadmin create-jvm-options '-Ddataverse.api.blocked.endpoints=<current :BlockedApiEndpoints>'

If :BlockedApiPolicy is set and is not 'drop'

bin/asadmin create-jvm-options '-Ddataverse.api.blocked.policy=<current :BlockedApiPolicy>'

If :BlockedApiPolicy is 'unblock-key' and :BlockedApiKey is set

`echo "API_BLOCKED_KEY_ALIAS=<value of :BlockedApiKey>" > /tmp/dataverse.api.blocked.key.txt`

sudo -u dataverse /usr/local/payara6/bin/asadmin create-password-alias --passwordfile /tmp/dataverse.api.blocked.key.txt

When you are prompted "Enter the value for the aliasname operand", enter api_blocked_key_alias

You should see "Command create-password-alias executed successfully."

   bin/asadmin create-jvm-options '-Ddataverse.api.blocked.key=${ALIAS=api_blocked_key_alias}'

Restart Payara:

service payara restart

Check server.log to verify that your new settings are in effect.

Cleanup: delete deprecated settings:

curl -X DELETE http://localhost:8080/api/admin/settings/:AllowCors
curl -X DELETE http://localhost:8080/api/admin/settings/:BlockedApiEndpoints
curl -X DELETE http://localhost:8080/api/admin/settings/:BlockedApiPolicy
curl -X DELETE http://localhost:8080/api/admin/settings/:BlockedApiKey

11. Upgrade to Dataverse Previewers v1.5

Dataverse Previewers has been upgraded to v1.5. See the announcement for upgrade instructions.

12. Re-detect video subtitle (vtt) files

Existing files with extension ".vtt" will keep the content type application/octet-stream presented as "Unknown". The following query shows the number of files per extension with an "Unknown" content type:

SELECT substring(m.label from (length(label) - strpos(reverse(m.label), '.') + 2)) AS extension, COUNT(*) as count
FROM datafile f LEFT JOIN filemetadata m ON f.id = m.datafile_id
WHERE f.contenttype = 'application/octet-stream'
GROUP BY extension;

If vtt does not appear in the result, you are done. Otherwise, you may want to update the content type for existing files and reindex those datasets.

First figure out which datasets would need reindexing:

select distinct
  o.protocol, o.authority, o.identifier,
  v.versionnumber, v.minorversionnumber, v.versionstate
  from      datafile       f
  left join filemetadata   m on f.id = m.datafile_id
  left join datasetversion v on v.id = m.datasetversion_id
  left join dvobject       o on o.id = v.dataset_id
  WHERE contenttype = 'application/octet-stream'
  AND 'vtt' = substring(m.label from (length(label) - strpos(reverse(m.label), '.') + 2))
  ;

Then update the content type for the files:

UPDATE datafile SET contenttype = 'text/vtt' WHERE id IN (
  SELECT datafile_id FROM filemetadata m
  WHERE contenttype = 'application/octet-stream'
  AND 'vtt' = substring(m.label from (length(label) - strpos(reverse(m.label), '.') + 2))
);

The vtt files will be reindexed in a step below.

13. Update Solr schema and reindex

Due to changes in the Solr schema (the addition of fields "curationStatus" and"curationStatusCreateTime"), updating the Solr schema and reindexing is required.

Download the updated schema.xml file:

wget https://raw.githubusercontent.com/IQSS/dataverse/v6.7/conf/solr/schema.xml
cp schema.xml /usr/local/solr/solr-9.8.0/server/solr/collection1/conf

13a. For installations with additional metadata blocks or external controlled vocabulary scripts, update fields

• Stop Solr instance (usually service solr stop, depending on Solr installation/OS, see the Installation Guide).

• Run the update-fields.sh script that we supply, as in the example below (modify the command lines as needed to reflect the correct path of your Solr installation):

wget https://raw.githubusercontent.com/IQSS/dataverse/v6.7/conf/solr/update-fields.sh
chmod +x update-fields.sh
curl "http://localhost:8080/api/admin/index/solr/schema" | ./update-fields.sh /usr/local/solr/solr-9.8.0/server/solr/collection1/conf/schema.xml

Note that Docker-based installations use a different directory: solr/data/data/collection1/conf/schema.xml.

• Start Solr instance (usually service solr start depending on Solr/OS).

14. Reindex Solr

curl http://localhost:8080/api/admin/index