Introduce tf.distribute.AutoStrategy #129224

Open
AshiteshSingh opened 6:08am on October 17, 2026 wants to merge 667 Ξ” into tensorflow/tensorflow master from
feat/auto-strategy
This work implements Introduce tf.distribute.AutoStrategy (closed)

Introduce tf.distribute.AutoStrategy

Description

This pull request introduces the "Zero-Code" Distributed Training Optimizer feature: tf.distribute.AutoStrategy.

Currently, developers have to manually probe their hardware environments or manage boilerplate conditionals in their training scripts to select the appropriate distribution strategy (MirroredStrategy, TPUStrategy, OneDeviceStrategy, etc.).

AutoStrategy is a dynamic factory API that automatically resolves and initializes the optimal strategy for the user based on the environment and available hardware.

Key Changes

  • New API: Added tf.distribute.AutoStrategy() exposed via @tf_export.
  • Heuristics Engine:
    • Detects TF_CONFIG to return MultiWorkerMirroredStrategy for distributed clusters.
    • Automatically initializes and routes to TPUStrategy if TPUs are present via tf.config.list_logical_devices.
    • Discovers multiple local GPUs and returns MirroredStrategy.
    • Safely falls back to OneDeviceStrategy (GPU or CPU) in resource-constrained environments.
  • Testing: Added robust unit tests in auto_strategy_test.py to ensure correct factory resolutions across mocked hardware environments.
  • Build integration: Updated the Python bazel BUILD targets to include the new strategy module seamlessly.

Motivation and Context

Simplifies the boilerplate MLOps setup required for distributed machine learning in TensorFlow, especially when moving model code from a single-GPU dev environment to a TPU or multi-worker production cluster. This strongly aligns with TensorFlow's goal of offering an enterprise-ready, out-of-the-box deployment experience.

How Has This Been Tested?

  • [x] Unit tests covering CPU fallback and multi-worker TF_CONFIG discovery (bazel test //tensorflow/python/distribute:auto_strategy_test)
  • [x] Linting and syntax checks verified against tf_py_strict_test.

Types of changes

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
AshiteshSingh authored
of work between September 2 and October 17
Diff Delta:
667
About 17 Diff Delta/hour
Classified as:  Bug Fix, Feature Add, General

AshiteshSingh's Description of Work #126327 Introduce tf.distribute.AutoStrategy


Note: This pull request replaces closed PR #126327 (which was closed due to an upstream rebase script pushing master to the branch; see PR #126327 comment). All 23 commits and changes are intact as requested by maintainer @dmiltr3.


Introduce tf.distribute.AutoStrategy

Description

This pull request introduces the "Zero-Code" Distributed Training Optimizer feature: tf.distribute.AutoStrategy.

Currently, developers have to manually probe their hardware environments or manage boilerplate conditionals in their training scripts to select the appropriate distribution strategy (MirroredStrategy, TPUStrategy, OneDeviceStrategy, etc.).

AutoStrategy is a dynamic factory API that automatically resolves and initializes the optimal strategy for the user based on the environment and available hardware.

Key Changes



  • New API: Added tf.distribute.AutoStrategy() exposed via @tf_export.


  • Heuristics Engine:


    • Detects TF_CONFIG to return MultiWorkerMirroredStrategy for distributed clusters.

    • Automatically initializes and routes to TPUStrategy if TPUs are present via tf.config.list_logical_devices.

    • Discovers multiple local GPUs and returns MirroredStrategy.

    • Safely falls back to OneDeviceStrategy (GPU or CPU) in resource-constrained environments.




  • Testing: Added robust unit tests in auto_strategy_test.py to ensure correct factory resolutions across mocked hardware environments.


  • Build integration: Updated the Python bazel BUILD targets to include the new strategy module seamlessly.

Motivation and Context

Simplifies the boilerplate MLOps setup required for distributed machine learning in TensorFlow, especially when moving model code from a single-GPU dev environment to a TPU or multi-worker production cluster. This strongly aligns with TensorFlow's goal of offering an enterprise-ready, out-of-the-box deployment experience.

How Has This Been Tested?


  • [x] Unit tests covering CPU fallback and multi-worker TF_CONFIG discovery (bazel test //tensorflow/python/distribute:auto_strategy_test)

  • [x] Linting and syntax checks verified against tf_py_strict_test.

Types of changes


  • [ ] Bug fix (non-breaking change which fixes an issue)

  • [x] New feature (non-breaking change which adds functionality)

  • [ ] Breaking change (fix or feature that would cause existing functionality to change)

5 total changed files
(1 file ignored)
Loading changes...
Loading changes...
Loading changes...
End of PR:
Like the credits, but with fewer stunts.