Prhub

#38770 [CI] Temporarily disable GB300 tests

原始 PR 作者 alisonshao 合并时间 2026-09-10 09:31 文件变更 3 提交数 1 评论 0 代码增减 +35 / -35

执行摘要

临时禁用所有 GB300 GPU 的 CI 测试与发布构建,解除流水线阻塞。

According to the PR body: "No GB300 runners are online, so every GB300 job sits queued indefinitely and holds up pr-test-finish on every PR." The cause was that both GB300 runners went down independently, causing all tests dispatched to the 4-gpu-gb300 label to be stuck in a pending state indefinitely. This blocked the completion of the pr-test-finish job, which is a prerequisite for all PRs, thereby halting the entire CI/CD pipeline for the repository.

Recommendation for Reading: This PR is a low-priority, operational change. It is worth a quick scan to understand the current state of the CI infrastructure and to note that GB300 tests are off. There is no deep code logic or design to review. The key takeaway for any maintainer is to remember that GB300 tests are disabled and to watch for a follow-up revert commit once the runners are restored.
Key Observation: The PR demonstrates a clean, reversible pattern for disabling CI stages: commenting out job definitions and their needs references while leaving the underlying test registrations and runner configs intact.

讨论亮点

There were no review comments or discussions. The PR was approved by mmangkad with an empty body, indicating a straightforward operational fix that was accepted without technical debate.

实现拆解

  1. Disable PR Test Stage: In .github/workflows/pr-test.yml, the job base-c-test-4-gpu-gb300 is commented out, and its reference in the needs list of the pr-test-finish job is also commented out. This stops the per-commit test stage from dispatching jobs to the dead runners.
  2. Disable Nightly Test Stage: In .github/workflows/nightly-test-nvidia.yml, the entire job definition for nightly-4-gpu-gb300 is commented out, along with its two references in the needs lists of the final aggregation jobs (nightly-test-finish and nightly-test-finish-extra). This prevents the nightly test suite from attempting to use the GB300 runners.
  3. Disable Wheel Build: In .github/workflows/release-whl-deepgemm.yml, the matrix entry for the sm100-aarch64 wheel build, which specifically targeted the 4-gpu-gb300 runner, is commented out. This stops the failed build from queuing.
  4. Preservation for Re-enablement: The PR explicitly states that tests remain registered in run_suite.py and the 4-gpu-gb300 entry is kept in runner_configs.yml. This means re-enabling is a simple revert. The rust-ext-build-aarch64 job is also left active to seed the aarch64 cache for future use.
文件 模块 状态 重要度
.github/workflows/pr-test.yml CI 流水线 modified 4.35
.github/workflows/nightly-test-nvidia.yml CI 流水线 modified 4.35
.github/workflows/release-whl-deepgemm.yml CI 流水线 modified 3.25

分析完成后,这里会展示 LLM 生成的相对完整源码片段和详细注释。

评论区精华

没有提炼出高价值讨论线程

当前评论区没有形成足够清晰的争议点或结论,后续有更多讨论时会体现在这里。

风险与影响

The primary risk is operational, not technical: if the GB300 runners come back online and the team forgets to revert this commit, the relevant tests will remain disabled, potentially allowing regressions on GB300-specific hardware to slip through. There is also a minor risk that the rust-ext-build-aarch64 job (left active) might fail if it had an implicit dependency on the now-disabled GB300 stage, though the PR author notes this is intentional.

Impact on CI Pipeline: This change unblocks the CI for all PRs and nightly runs. The pr-test-finish job will no longer be held hostage by the pending GB300 jobs, allowing merges to proceed.
Impact on Test Coverage: All tests specifically targeting the GB300 hardware are temporarily disabled. This reduces test coverage for that specific GPU platform until the runners are restored and the change is reverted.
Impact on Releases: The wheel build for the sm100-aarch64 target (likely for GB300 or related aarch64 hardware) is paused, meaning no new wheels for that architecture will be produced until re-enabled.

临时测试禁用 需要后续恢复操作

关联 Issue

未识别关联 Issue

当前没有检测到明确关联的 Issue 链接,后续同步到相关引用后会出现在这里。

完整报告

参与讨论