vllm.models.step5.nvidia.ops

Modules:

• add –

  Add low-precision tensors without materializing their FP32 conversions.

• attention_gate –

  Headwise sigmoid output gating with the original low-precision rounding.

• indexer_norm_rope –

  Indexer normalization, rotary embedding and FP8 query preparation.

• mtp_norm –

  Normalize MTP embedding and hidden states directly into the projection input.

• prefill_indexer_metadata –

  Step-local indexer metadata shared across sparse attention layers.

• qkv_indexer_norm_rope –

  Joint attention QKV and sparse-indexer preparation for decode.

• rms_norm –

  RMSNorm with independent input, weight and output dtypes.

• sparse_decode_attention –

  DSA decode attention read directly from paged KV by selected regions.

• sparse_indexer –

  Packing and stable region selection for the sparse indexer.

• sparse_indexer_scores –

  FP8 indexer scores directly from paged summary keys.

• sparse_prefill –

  Compact sparse-prefill KV gathering and reusable host planning metadata.

• sparse_prefill_pages –

  Compact eight-token pages for sparse prefill attention.

• summary_update –

  CSA region summaries, compressed once when a region's eighth row arrives.

• swa_decode –

  Sliding-window GQA decode with split KV and a fused headwise gate.