Prhub

#36248 [PP] Support prefill CUDA graph proxy tensors

原始 PR 作者 YAMY1234 合并时间 2026-08-31 14:11 文件变更 12 提交数 15 评论 8 代码增减 +353 / -64

执行摘要

实现拆解

  1. 核心源码路径
    - python/sglang/srt/model_executor/runner/prefill_cuda_graph_runner.py(data-contract):源码主路径;涉及符号 _build_layer_model_forward_kwargs, _static_pp_proxy_tensors, _capture_pp_proxy_tensors;包含 控制流调整、配置键调整、符号定义调整;+40/-10;关键符号:_build_layer_model_forward_kwargs, _static_pp_proxy_tensors, _capture_pp_proxy_tensors
    - python/sglang/srt/model_executor/runner_utils/buffers.py(data-contract):源码主路径;涉及符号 _allocate_pp_proxy_tensors;包含 控制流调整、配置键调整、符号定义调整;+59/-46;关键符号:_allocate_pp_proxy_tensors
    - python/sglang/srt/arg_groups/memory_hook.py(dependency-wiring):源码主路径;包含 导入关系调整、控制流调整、配置键调整;+15/-1

  2. 测试与验证
    - test/registered/unit/model_executor/test_prefill_cuda_graph_runner_helpers.py(test-coverage):测试配套;涉及符号 _make_pp_buffers_and_registry, test_pp_proxy_stable_buffers_accept_full_and_hidden_only_contracts, test_layer_model_kwargs_bind_optional_inputs_by_signature, proxy_before_embeds;包含 测试覆盖调整、导入关系调整、配置键调整;+127/-0;关键符号:_make_pp_buffers_and_registry, test_pp_proxy_stable_buffers_accept_full_and_hidden_only_contracts, test_layer_model_kwargs_bind_optional_inputs_by_signature, proxy_before_embeds
    - test/registered/unit/server_args/test_server_args.py(test-coverage):测试配套;涉及符号 TestPipelineParallelPrefillCudaGraphPolicy, test_pp_prefill_graph_is_opt_in, test_pp_prefill_capture_limit_policy;包含 测试覆盖调整、导入关系调整、控制流调整;+54/-0;关键符号:TestPipelineParallelPrefillCudaGraphPolicy, test_pp_prefill_graph_is_opt_in, test_pp_prefill_capture_limit_policy
    - test/registered/unit/model_executor/model_runner_components/test_layer_setup.py(test-coverage):测试配套;涉及符号 test_pipeline_placeholders_preserve_global_layer_ids;包含 测试覆盖调整、配置键调整、符号定义调整;+14/-0;关键符号:test_pipeline_placeholders_preserve_global_layer_ids

文件 模块 状态 重要度
python/sglang/srt/model_executor/runner/prefill_cuda_graph_runner.py sglang/srt modified 8.18
python/sglang/srt/model_executor/runner_utils/buffers.py sglang/srt modified 7.91
test/registered/unit/model_executor/test_prefill_cuda_graph_runner_helpers.py prefill/cuda/graph/runner/helpers modified 7.79
test/registered/unit/server_args/test_server_args.py server/args modified 6.71
python/sglang/srt/arg_groups/memory_hook.py sglang/srt modified 6.34

关键符号

_build_layer_model_forward_kwargs _static_pp_proxy_tensors _capture_pp_proxy_tensors _allocate_pp_proxy_tensors _make_pp_buffers_and_registry test_pp_proxy_stable_buffers_accept_full_and_hidden_only_contracts test_layer_model_kwargs_bind_optional_inputs_by_signature proxy_before_embeds test_finalize_pp_proxy_trims_padded_token_rows test_bcg_eager_tail_uses_live_multimodal_embeddings

分析完成后,这里会展示 LLM 生成的相对完整源码片段和详细注释。

评论区精华

没有提炼出高价值讨论线程

当前评论区没有形成足够清晰的争议点或结论,后续有更多讨论时会体现在这里。

风险与影响

关联 Issue

未识别关联 Issue

当前没有检测到明确关联的 Issue 链接,后续同步到相关引用后会出现在这里。

完整报告

参与讨论