vllm.models.common.ops.sequence_parallel

Functions:

• is_sequence_parallel_active –

  Whether a batch of num_tokens shards the stream along the sequence.

is_sequence_parallel_active(is_enabled, num_tokens) cached

Whether a batch of num_tokens shards the stream along the sequence.

Below one token per rank the shards are mostly padding, so the batch stays replicated.

Source code in vllm/models/common/ops/sequence_parallel.py

@lru_cache
def is_sequence_parallel_active(is_enabled: bool, num_tokens: int) -> bool:
    """Whether a batch of ``num_tokens`` shards the stream along the sequence.

    Below one token per rank the shards are mostly padding, so the batch
    stays replicated.
    """
    return is_enabled and num_tokens >= get_tensor_model_parallel_world_size()