GPU Goodput Is the Whole Product: Cosmos 3 + SageMaker HyperPod Rewrites the Physical AI Factory
发布于 2026-09-09
GPU Goodput Is the Whole Product: Cosmos 3 + SageMaker HyperPod Rewrites the Physical AI Factory A robot or an autonomous vehicle that has to act in the physical world cannot be trained in a single jo...
GPU Goodput Is the Whole Product: Cosmos 3 + SageMaker HyperPod Rewrites the Physical AI Factory
A robot or an autonomous vehicle that has to act in the physical world cannot be trained in a single job. It needs a continuous loop: synthetic data generation, post-training of perception and policy models on the new data, and closed-loop evaluation in simulation before the policy ever touches a real wheel or gripper. AWS and NVIDIA published a reference architecture for that loop on 2026-09-04 in a joint Machine Learning Blog post: Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod1. The value is architectural rather than a claim of a production deployment: the post describes the model, cluster, storage path, and cost metric as one system.
One model, three modes, three sizes — the unification that makes the factory possible
Cosmos 3 is an open omnimodal world foundation model that treats video, image, action, and sound as a single token stream, and runs the same transformer trunk in three modes: a forward-dynamics world model for synthetic video generation, an inverse-dynamics action labeler, and a deployable action policy1. NVIDIA describes the architecture as a unified Mixture-of-Transformers (MoT), combining an autoregressive reasoner with a diffusion generator for multimodal generation2. NVIDIA releases the family under the Linux Foundation's OpenMDW-1.1 license, which places no restrictions on outputs but terminates the grant if the licensee sues anyone for patent or copyright infringement over the model materials.
The model ships in three tiers, mapped to the hardware teams actually buy: Cosmos3-Super at 64B parameters for H200 / B200 / GB200 data-center deployments; Cosmos3-Nano at 16B for RTX Pro 6000 / H100 / B200 workstations and data centers; and Cosmos3-Edge at 4B for Jetson AGX Orin, Thor, or RTX Pro 6000 on-device4. NVIDIA also lists 4-step distilled variants — Cosmos3-Super-Text2Image-4Step and Cosmos3-Super-Image2Video-4Step — that deliver a 17–25x speedup over their full-step counterparts4.
The reason that matters for infrastructure is that "one model covering all three stages" turns the pipeline into three workloads scheduled on one persistent GPU node pool, rather than three pools scheduled independently5. That move is what makes the "factory" framing worth saying out loud instead of being a marketing word.
The hardware pillar: HyperPod, EKS, FSx for Lustre, EFA, GPUDirect Storage
Cosmos 3 on AWS runs on a single persistent Amazon SageMaker HyperPod cluster managed through Amazon Elastic Kubernetes Service. HyperPod provisions a resilient GPU node pool; EKS gives one control plane across the whole loop. The shared filesystem is FSx for Lustre, sized in the multi-terabyte range — the example workspace in the AWS post tops out around 9.4 GB/s aggregate over a 9.6 TiB filesystem5.
The number that matters here is not the GPU count. It is the per-client bandwidth. Without Elastic Fabric Adapter (EFA), per-client file-system throughput is capped at 100 Gbps. With EFA enabled and GPUDirect Storage, an EFA-enabled file system reaches 700 Gbps per client, and on p5en instances it reaches up to 1,200 Gbps. AWS recommends enabling EFA for any file system above 10 GB/s for this reason6. Those figures make the data path a first-order design constraint: peak accelerator capacity is irrelevant if the filesystem and network cannot keep it fed.
The right parallelism: H200, p5en, and why scaling efficiency stays high
The reference architecture targets the Cosmos3-Nano policy checkpoint running on the public DROID dataset, a large-scale in-the-wild robot manipulation dataset collected by a multi-institution team that includes Stanford, Berkeley, CMU, Google DeepMind, Meta, and Toyota Research Institute7. The post-training runs end-to-end on p5en.48xlarge nodes, each equipped with 8x NVIDIA H200 GPUs. Each H200 carries 141 GB of high-bandwidth memory (HBM)8. The parallelism strategy on the demonstrated 2-node run uses 16 ranks across 2 nodes8.
The number worth quoting is that scaling efficiency stayed between 0.97 and 0.99 of linear as the run scaled from 1 to 4 nodes (8 to 32 GPUs), and Model FLOPs Utilization held within about 3% across the ladder9. That is the difference between a reference architecture that happens to run on H200 and one whose node shape, job layout, and parallelism strategy are designed together. It does not prove that every Cosmos 3 workload will scale identically, but it does show that the published walkthrough is a measured system rather than a diagram.
The accompanying repository at awslabs/awsome-distributed-ai/tree/main/3.test_cases/pytorch/cosmos3 contains the runnable infrastructure templates and job manifests7. This is reproducible infrastructure, not slideware.
The metric: GPU goodput, not peak throughput
The most counter-intuitive argument in the AWS post is the one about cost. Once capacity is committed to the whole loop, the buyer pays for GPU-hours whether or not the pipeline is making progress on them. The right metric, AWS argues, is therefore not the peak throughput of any one job but GPU goodput: useful pipeline progress per reserved GPU-hour, summed across the synthetic-data, post-training, and evaluation stages10.
The implication is that you stop optimizing for the throughput of any single job and start optimizing for the throughput of the loop. Two things follow. First, you stop saving checkpoints aggressively: the reference architecture saves a checkpoint every 2,300 s, which costs on the order of 1% of wall-clock10. That is the cost of being able to resume after any node failure, and it is worth paying. Second, you measure the loop, not the job. The post includes an illustrative example of checkpoints landing roughly every three hours on a 16,384-H100 GPU run — and is explicit that this is an order-of-magnitude illustration, not a Cosmos 3 throughput claim10. The point is not the number; the point is that a "factory" asks "how often does the whole loop land a new usable policy," not "how fast does any one job train."
What this means for builders
Three things change for engineering teams looking at this reference architecture.
First, one open model family is now a real option for the whole pipeline. Cosmos 3's three tiers cover data-center synthetic-data generation (Super 64B), workstation and cluster post-training (Nano 16B), and on-device real-time policy (Edge 4B)4. The 4-step distilled variants give you a 17–25x speedup path without retraining4. OpenMDW-1.1 does not restrict generated outputs, which matters to teams evaluating how they can ship results, although the license still deserves legal review rather than blind reliance on a summary3.
Second, the cluster is the product. The reference architecture's value is not the model; the model is open and you can download it. The value is the integration — HyperPod on EKS, FSx for Lustre with EFA, GPUDirect Storage, the parallelism strategy that holds 0.97–0.99 of linear on H2009. The inference is that "do we have a world model" is no longer the right question; "do we have a persistent, resilient GPU cluster that can keep improving it" is.
Third, the cost metric changes. Anyone running a Physical AI pipeline that they pay for by the hour should be tracking GPU goodput, not peak job throughput10. Anyone writing a budget proposal for a Physical AI factory should be quoting reserved-capacity cost per unit of pipeline progress, not cost per training run. The post saves 1% of wall-clock on checkpointing because that is the right tradeoff — it is the cost of resumability on a cluster that might lose a node at any time10.
This reference architecture joins three pieces in one public design: an open world foundation model that spans the loop1, the cluster substrate intended to run it continuously568, and GPU goodput as the metric for judging progress against committed capacity10. The Physical AI factory is not a slide anymore. It is a working repository on a public dataset, with runnable manifests, a published near-linear scaling result, and a license that does not restrict generated outputs. The interesting question is not whether the architecture is possible; it is who treats the cluster as the product first.