com.microsoft.FusedMatMul

com.microsoft · ONNX Runtime contrib operator · contrib since_version 1

Description

Matrix product of two N-dimensional tensors A and B, following NumPy-style matrix-multiplication broadcasting. Supports optional transposition of either operand's last two dimensions, optional batch-dimension transposition, and a scalar alpha multiplier. Float32 and float16 are supported; double and bfloat16 are not.

See the ONNX Runtime FusedMatMul contrib-operator spec for the reference semantics.

Inputs

Outputs

Attributes

Default values (overridable per request):

Type constraints

Implementation variants

One implementation is selected per call from the device capabilities, the request shapes and the dtypes; these notes say what each one covers.

• subgroup_matrix_transbatch_b_f16 — Aligned rank-3 transBatchB product with sufficient reduction depth and output tiles to amortize subgroup-matrix staging.
• subgroup_matrix_transbatch_b_f32 — Aligned rank-3 transBatchB product with sufficient reduction depth and output tiles to amortize subgroup-matrix staging.
• rank2_band_vec4_splitk — Splits the vec4 band's K axis across up to sixteen workgroups. Each range writes an f32 partial band with alpha applied, and a combine pass sums the partials.
• rank2_band_vec4 — Few-row band for a rank-2 product without transposes. Each lane owns one vec4 column group and one accumulator per row, so every B word feeds all 2 to 16 rows; alpha is applied at the store.
• rank2_band_vec4_f32_preferred — Few-row band for a rank-2 product without transposes. Each lane owns one vec4 column group and one accumulator per row, so every B word feeds all 2 to 16 rows; alpha is applied at the store.
• subgroup_matrix_splitk — Partitions the K reduction of small-M rank-2 products across subgroup-matrix workgroups, then combines float32 partials that already include alpha.
• plain_rank2_tiled_reg — Register-blocked rank-2 Y = alpha * A @ B specialization for non-transposed inputs on tiers without subgroup-matrix support.
• transbatch_b_tiled_reg — Register-blocked logical rank3 product with an interleaved physical B batch axis.

Device requirements

Some implementation variants require subgroup-matrix and subgroups. These are route-specific capabilities, not package-wide requirements; availability also depends on the request shape and dtype.

Files

• metadata.json — kernel metadata (id, digests, per-variant templates, provenance)
• manifest.json — the op contract (source of truth)
• test.json — correctness cases
• bench.json — benchmark + tuning cases
• fused-matmul-subgroup-matrix.wgsl.jinja
• matmul-band-vec4.wgsl.jinja
• matmul-subgroup-matrix-ext.wgsl.jinja
• matmul-tiled-general-reg.wgsl.jinja
• matmul-tiled-general.wgsl.jinja
• matmul-vector-matrix-vec4.wgsl.jinja
• reduce-axis0-splitk-combine.wgsl.jinja

Use with @huggingface/kernels

npm install --save-exact @huggingface/kernels@0.0.1-preview.2

Required output shapes and logical data types are inferred from the supplied inputs and attributes; result tensors are allocated automatically.

The version: 1 option selects the published kernel contract; it is independent of any operator opset, contrib since_version, or model version. It follows the v1 branch as fixes land. To pin exact artifact bytes, pass a 40-character commit revision instead of version.

Replace each *Data placeholder with a typed array containing the corresponding input data.

import { getKernel } from "@huggingface/kernels";

const kernel = await getKernel("webgpu-kernels/com.microsoft.FusedMatMul", { version: 1 });
const { Y } = await kernel({ A: { data: AData, shape: [3] }, B: { data: BData, shape: [3] } });