This pull request's diff hasn't been prepared yet. Log in with GitHub to have us prepare it for review.

[9.5] ESQL:DS: Bind external sources to registered datasets on data nodes (#162545) #162756

Merged
bpintea opened 7:52pm on October 20, 2026 wanted to merge 910 Ξ” into elastic/elasticsearch 9.5 from
backport/9.5/pr-162545

Pull Request Overview

  • Opened on October 20, 2026
  • Status Merged
  • Commit count 3 with first commit October 20, 2026

Total Delta

910 Total Diff Delta

Open Days

Open 10 weekdays

Test Delta

420 Diff Delta in Test Files
Breakdown by Phase

How long has this pull request spent in each phase of its lifecycle?

Data pending calculation for pull request

Author avatar

[9.5] ESQL:DS: Bind external sources to registered datasets on data nodes (#162545)

Backports the following commits to 9.5:
- ESQL:DS: Bind external sources to registered datasets on data nodes (#162545)

An ExternalSourceExec reaching a node carries its storage config, source path and splits from the plan. This change makes the executing node rebuild those from the dataset the exec names, using its own cluster state, before any local planning rule or operator reads them.



  • DatasetConfigResolver (new): looks the exec's dataset up in DatasetMetadata / DataSourceMetadata, replaces config and sourcePath with the registered definition, and refuses a null dataset name on release builds, an unregistered dataset or data source, and a split outside the dataset's resource (same scheme, host, port and user-info, no .. segment, within the object, directory or literal glob prefix).


  • PlannerUtils.localPlan applies it on the whole plan and inside the fragment walk after split stamping; ComputeService.runCompute and DataNodeComputeHandler.handleExternalSourceRequest pass it in. DatasetRewriter.datasetConfig is the shared config builder for the coordinator and the rebind.


  • LocalExecutionPlanner.planExternalSource refuses a null dataset name on release builds and a connector dispatch for a named dataset. The snapshot flag is a constructor argument.


  • CredentialPolicy (new SPI), carried by StorageProviderServices and applied by StorageProviderFactory.of(..., policy): federated and managed identity are checked on the parsed configuration before a provider is constructed, matching the PUT validator. DataSourceModule takes the federatedIdentityEnabled supplier; S3, GCS and Azure pass the policy, GCS switched to the services overload.

9.5-specific differences from the main change:
- There is no _definition_version consistency check on this branch (the definition version does not exist on 9.5): a dataset edited between planning and execution is read under the node's current definition.
- The localPlan failure handling in handleExternalSourceRequest (failWithoutStarting on a refusal) is brought along, since the rebind can now throw there; the rest of #162340 is not ported.
- Resource comma lists are decomposed the way 9.5 listing does (plain split on commas).

Rolling upgrade: a 9.4 coordinator does not serialize the dataset name, so a 9.5.6 data node refuses its external plans on release builds (fail-closed, as on main). 9.5.x to 9.5.6 is unaffected. No transport version or wire change.

PR was closed without comments.