Skip to content

feat(models): add MaxText pipeline parallelism configuration - #1753

Draft
sontungkieu wants to merge 3 commits into
google:mainfrom
sontungkieu:codex/maxtext-pipeline-parallelism
Draft

feat(models): add MaxText pipeline parallelism configuration#1753
sontungkieu wants to merge 3 commits into
google:mainfrom
sontungkieu:codex/maxtext-pipeline-parallelism

Conversation

@sontungkieu

Copy link
Copy Markdown

Summary

  • add a validated MaxTextPipelineConfig for hybrid pipeline/tensor layouts
  • route MaxText-specific mesh and runtime overrides through AutoModel
  • document PP2xTP4 and PP4xTP2 usage
  • cover mesh axes, batch divisibility, and invalid layouts with regression tests

Motivation

Tunix MaxText users currently need to hand-assemble the full MaxText mesh and pipeline override set. This adds a reusable typed configuration while leaving ordinary MaxText and multi-slice configuration paths unchanged.

Validation

  • 10 passed in tests/models/maxtext_parallelism_test.py on CPU JAX
  • pyink/isort checks on the touched MaxText integration files
  • runtime evidence from an 8-device TPU v5e benchmark exists downstream; this draft PR does not claim training or decode throughput

@google-cla

google-cla Bot commented Jul 24, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants