Introduce tf.distribute.AutoStrategy
Description
This pull request introduces the "Zero-Code" Distributed Training Optimizer feature: tf.distribute.AutoStrategy.
Currently, developers have to manually probe their hardware environments or manage boilerplate conditionals in their training scripts to select the appropriate distribution strategy (MirroredStrategy, TPUStrategy, OneDeviceStrategy, etc.).
AutoStrategy is a dynamic factory API that automatically resolves and initializes the optimal strategy for the user based on the environment and available hardware.
Key Changes
- New API: Added
tf.distribute.AutoStrategy()exposed via@tf_export. - Heuristics Engine:
- Detects
TF_CONFIGto returnMultiWorkerMirroredStrategyfor distributed clusters. - Automatically initializes and routes to
TPUStrategyif TPUs are present viatf.config.list_logical_devices. - Discovers multiple local GPUs and returns
MirroredStrategy. - Safely falls back to
OneDeviceStrategy(GPU or CPU) in resource-constrained environments.
- Detects
- Testing: Added robust unit tests in
auto_strategy_test.pyto ensure correct factory resolutions across mocked hardware environments. - Build integration: Updated the Python bazel
BUILDtargets to include the new strategy module seamlessly.
Motivation and Context
Simplifies the boilerplate MLOps setup required for distributed machine learning in TensorFlow, especially when moving model code from a single-GPU dev environment to a TPU or multi-worker production cluster. This strongly aligns with TensorFlow's goal of offering an enterprise-ready, out-of-the-box deployment experience.
How Has This Been Tested?
- [x] Unit tests covering CPU fallback and multi-worker
TF_CONFIGdiscovery (bazel test //tensorflow/python/distribute:auto_strategy_test) - [x] Linting and syntax checks verified against
tf_py_strict_test.
Types of changes
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)