vllm.models.deepseek_v32.nvidia.glm52_low_latency_gemm

GLM-5.2 decode GEMM selection for unquantized BF16 on SM103/SM107.

Dispatch is by local (N, K) weight shape and token count M. :data:GLM52_PROJECTIONS was measured on B300 (SM103) and :data:GLM52_PROJECTIONS_SM107 on VRNVL72 (SM107). The SM107 table also covers the TP4 projections (q_b, o_proj, shared expert) and the replicated indexer projections, which run as BF16 GEMMs at decode batch sizes.

Functions:

• build_glm52_plan –

  Plan for a weight the walk below cannot reach (a plain nn.Linear).

_low_latency_table()

Measured dispatch table for the current device, or None if unsupported.

Source code in vllm/models/deepseek_v32/nvidia/glm52_low_latency_gemm.py

def _low_latency_table() -> dict[tuple[int, int], GLM52ProjectionSpec] | None:
    """Measured dispatch table for the current device, or None if unsupported."""
    if current_platform.is_device_capability((10, 7)):
        return GLM52_PROJECTIONS_SM107
    if current_platform.is_device_capability((10, 3)):
        return GLM52_PROJECTIONS
    return None

build_glm52_plan(weight, dtype)

Plan for a weight the walk below cannot reach (a plain nn.Linear).

Source code in vllm/models/deepseek_v32/nvidia/glm52_low_latency_gemm.py

def build_glm52_plan(
    weight: torch.Tensor | None, dtype: torch.dtype
) -> dict[int, ResolvedCall] | None:
    """Plan for a weight the walk below cannot reach (a plain ``nn.Linear``)."""
    if dtype != torch.bfloat16:
        return None
    table = _low_latency_table()
    if table is None:
        return None
    if weight is None or weight.dim() != 2 or weight.dtype != torch.bfloat16:
        return None
    spec = table.get(tuple(weight.shape))
    if spec is None:
        return None
    _request_warmup(dtype, {config for _, config in spec.cute_configs})
    return spec.build_plan()