Prhub

#27742 test(sgl-router): cover sticky scale-up no-redistribution e2e

原始 PR 作者 Kangyan-Zhou 合并时间 2026-06-10 23:54 文件变更 1 提交数 1 评论 1 代码增减 +82 / -0

执行摘要

新增 sticky scale-up 集成测试,覆盖 true-sticky 特性

sticky-session 策略的 true-sticky 属性(添加 worker 不重分布已有路由键)在单元测试层面已有覆盖(sticky.rs 中的 adding_a_worker_does_not_redistribute_existing_key),但缺少端到端的集成测试。已有的集成测试(tests/proxy/sticky_routing.rs)覆盖了同键固定、移除时重映射、无键回退和动态头名称,但未覆盖运行时 scale-up。如果 registry-add → candidate-set → policy 路径中出现回归(例如在注册表变更时重新播种固定关系),单元测试无法捕获。

该 PR 值得查看,尤其是负责 SGLang Router 或 sticky-session 策略的工程师。它展示了如何将单元测试级别的核心属性扩展到端到端集成测试,并包含对测试前提的守卫断言(如 healthy_workers_for 数量),避免测试因未来变更而流于形式。

讨论亮点

该 PR 没有 review 评论。

实现拆解

  1. experimental/sgl-router/tests/proxy/sticky_routing.rs 中新增测试函数 adding_a_worker_does_not_redistribute_existing_key
  2. 测试步骤:
    • 启动两个 MockWorker,构建 sticky 路由上下文和路由器。
    • 发送一个带有路由键 "alice" 的请求,将其固定到某个 worker,记录该 worker URL。
    • 通过 WorkerRegistry::add 动态添加第三个 MockWorker,并断言该 worker 确实成为健康候选者(healthy_workers_for 返回 3 个)。
    • 连续发送 N 次(N=5)相同路由键的请求,验证所有请求均路由到最初固定的 worker,且 sticky 指标中 assigned=1、remap=0、hit=N。
  3. 测试不依赖定时器或睡眠,通过现有 MockWorker 和确定性控制保证稳定性。
  4. 所有改动均为测试代码,生产代码无变化。
文件 模块 状态 重要度
experimental/sgl-router/tests/proxy/sticky_routing.rs 路由测试 modified 5.89

关键符号

adding_a_worker_does_not_redistribute_existing_key

关键源码片段

experimental/sgl-router/tests/proxy/sticky_routing.rs test-coverage

这是唯一的变更文件,新增了一个集成测试 `adding_a_worker_does_not_redistribute_existing_key`,覆盖 sticky-session 策略在运行时 worker 扩容时不重新分配已有路由键的核心属性。

/// True-sticky scale-up: a worker that joins the registry at runtime must
/// NOT redistribute an already-pinned key — the defining difference from
/// consistent hashing, where adding a node remaps a fraction of keys. The
/// policy unit tests assert this over a bare worker slice; this drives it
/// end-to-end through the HTTP stack and a live `WorkerRegistry::add`, so
/// the freshly-added worker is a genuine healthy candidate the policy could
/// pick — and provably doesn't.
#[tokio::test]
async fn adding_a_worker_does_not_redistribute_existing_key() {
    // Arrange: start 2 workers with sticky policy
    let w0 = MockWorker::start(vec![]).await;
    let w1 = MockWorker::start(vec![]).await;
    let ctx = build_sticky_ctx("x-sgl-routing-key", &[w0.url.clone(), w1.url.clone()]);
    let app = build_router(ctx.clone());    // Act: pin key "alice" to whichever worker is initially selected
    let res = app
        .clone()
        .oneshot(chat_request(Some(("x-sgl-routing-key", "alice"))))
        .await
        .unwrap();
    assert_eq!(res.status(), StatusCode::OK);
    let pinned_url = success_counts(&ctx.metrics.render())
        .into_iter()
        .find(|(_, c)| *c > 0)
        .map(|(url, _)| url)
        .expect("first request should have been served by some worker");    // Scale up: add a 3rd worker at runtime via WorkerRegistry::add (full HTTP stack)
    let w2 = MockWorker::start(vec![]).await;
    ctx.registry
        .add(WorkerSpec {
            id: WorkerId("w2".into()),
            url: w2.url.clone(),
            mode: WorkerMode::Plain,
            model_ids: vec![ModelId("tiny".into())],
            bootstrap_port: None,
        })
        .unwrap();    // Guard: ensure w2 is actually eligible (premise check)
    assert_eq!(
        ctx.registry
            .healthy_workers_for(&ModelId("tiny".into()))
            .len(),
        3,
        "added worker must be an eligible candidate"
    );    // Same-key requests after scale-up; all must stay on original pin
    const N: usize = 5;
    for _ in 0..N {
        let res = app
            .clone()
            .oneshot(chat_request(Some(("x-sgl-routing-key", "alice"))))
            .await
            .unwrap();
        assert_eq!(res.status(), StatusCode::OK);
    }    // Assert metrics: one assignment, zero remaps, N hits
    let metrics = ctx.metrics.render();
    assert_eq!(sticky_count(&metrics, "assigned"), 1, "{metrics}");
    assert_eq!(sticky_count(&metrics, "remap"), 0, "{metrics}");
    assert_eq!(sticky_count(&metrics, "hit"), N as u64, "{metrics}");    // Assert all requests landed on the original pin worker
    let counts = success_counts(&metrics);
    assert_eq!(
        counts.get(&pinned_url).copied().unwrap_or(0),
        (N + 1) as u64,
        "all same-key requests must stay on the original pin: {counts:?}"
    );
}

评论区精华

没有提炼出高价值讨论线程

当前评论区没有形成足够清晰的争议点或结论,后续有更多讨论时会体现在这里。

风险与影响

该 PR 仅包含测试代码,无生产代码变更,因此回归风险极低。新增的测试断言了 sticky 策略的核心行为,可帮助早期发现该功能的回归。

对用户无直接功能影响。对团队而言,该测试增强了对 sticky-session 策略 scale-up 场景的覆盖,提升了 CI 中对该行为回归的检测能力。

关联 Issue

未识别关联 Issue

当前没有检测到明确关联的 Issue 链接,后续同步到相关引用后会出现在这里。

完整报告

参与讨论