Skip to content

docs: explain FP8 format suffixes (e4m3fn, e5m2, e8m0fnu) in startup flags page #1313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lin-bot23 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs: explain FP8 format suffixes (e4m3fn, e5m2, e8m0fnu) in startup flags page #1313

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions development/comfyui-server/startup-flags.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,11 @@ python main.py --disable-auto-launch
| `--fp8_e5m2-unet` | Store UNet weights in fp8 (e5m2). |
| `--fp8_e8m0fnu-unet` | Store UNet weights in fp8 (e8m0fnu). |

The suffix in each FP8 flag identifies the IEEE-compatible format:
- **e4m3fn** — 4 exponent bits, 3 mantissa bits. Good balance of precision and range for model weights on NVIDIA Ada Lovelace and newer GPUs.
- **e5m2** — 5 exponent bits, 2 mantissa bits. Wider dynamic range but lower precision than e4m3fn. Useful when weight magnitudes vary widely.
- **e8m0fnu** — 8 exponent bits, 0 mantissa bits. Maximum dynamic range with no fractional precision (acts as a per-tensor scale factor).
Comment on lines +109 to +112

@coderabbitai coderabbitai Bot Jul 22, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== Locate file =="
git ls-files | grep -F 'development/comfyui-server/startup-flags.mdx' || true

echo "== Relevant file excerpt =="
if [ -f development/comfyui-server/startup-flags.mdx ]; then
  sed -n '90,145p' development/comfyui-server/startup-flags.mdx | nl -ba -v90
fi

echo "== Search for FP8/e8m0fnu references in repo =="
rg -n "e8m0fnu|e4m3fn|e5m2|FP8|fp8|quantization|Quantization|comfy\.cli_args|cli_args|startup-flags" -S .

Repository: Comfy-Org/docs

Length of output: 427

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== Relevant file excerpt =="
cat -n development/comfyui-server/startup-flags.mdx | sed -n '90,145p'

echo "== Search for FP8/e8m0fnu references in repo =="
rg -n "e8m0fnu|e4m3fn|e5m2|FP8|fp8|quantization|Quantization|comfy\.cli_args|cli_args|startup-flags" -S .

Repository: Comfy-Org/docs

Length of output: 50371

🌐 Web query:

NVIDIA FP8 formats e4m3fn e5m2 sign exponent mantissa bits

💡 Result:

NVIDIA FP8 formats consist of two distinct 8-bit floating-point encodings, E4M3 and E5M2, both designed for efficient deep learning workloads [1][2]. Each format is comprised of a single sign bit, a specific number of exponent bits, and a set number of mantissa bits [1][3]. The bit breakdown for these formats is as follows: E4M3 (E4M3FN) 1 sign bit, 4 exponent bits, and 3 mantissa bits [1][4]. This format prioritizes precision and is typically used for weights and activations in the forward pass [1][5]. It does not support infinities and has a limited set of NaN patterns to extend its dynamic range [1]. E5M2 1 sign bit, 5 exponent bits, and 2 mantissa bits [1][4][3]. This format provides a wider dynamic range at the cost of lower precision [5][2]. It is often used for gradients in the backward pass, where magnitude variations are more significant [1][5]. It follows IEEE 754 conventions for special values, including infinities [1][2]. Summary Table Format | Sign Bits | Exponent Bits | Mantissa Bits | Total Bits --- | --- | --- | --- | --- E4M3 | 1 | 4 | 3 | 8 E5M2 | 1 | 5 | 2 | 8 Note: In technical documentation, the "mantissa" bits refer to the explicit significand bits, excluding the implicit leading bit common in IEEE 754-style floating-point numbers [6][3].

Citations:

🌐 Web query:

"e8m0fnu" FP8 PyTorch sign bits E8M0fn2 "0-8-0"

💡 Result:

The term E8M0 refers to a specialized 8-bit format primarily used for block-scaling factors in low-precision matrix multiplication, rather than as a general-purpose floating-point data type for activations or weights [1][2]. Key technical characteristics of E8M0/E8M0fnu include: Format Definition: The E8M0 format consists of 8 exponent bits and 0 mantissa bits (an unsigned 8-bit floating-point representation) [3][2]. It is interpreted as powers of two with a biased exponent [4]. Technical Specifications: - Exponent: 8 bits [3]. - Mantissa: 0 bits [3]. - Exponent Bias: 127 [4][3]. - NaN (Not a Number): Represented by the bit pattern 0xFF (255) [4][3]. - "FNU": The suffix "fnu" stands for "finite," "no sign," and "unsigned" (or "no-sign"), indicating that the format does not support negative values, infinities, or specialized sign-bit encodings [3][2]. - Use Case: It is frequently utilized as a scale factor type in micro-scaling (MXFP) operations, where it encodes the scaling factors for underlying FP8 or FP4 tensor data (e.g., E4M3FN weights) [1][5][2]. In software implementations such as vLLM and various CUDA math APIs, this format is often referred to as float8_e8m0fnu [1][6][4]. Because it lacks a sign bit and mantissa, it acts strictly as an exponent-based scale, allowing for efficient range adjustment in block-quantized kernels [7][2].

Citations:

🌐 Web query:

ComfyUI --fp8_e8m0fnu-unet fp8_e8m0fnu unet

💡 Result:

In ComfyUI, --fp8_e8m0fnu-unet is a command-line argument used to store UNET model weights in the fp8_e8m0fnu data format [1][2]. The fp8_e8m0fnu format (8-bit floating-point with 8 exponent bits, no mantissa, and no sign bit) is a technical data type often utilized as a scale factor for quantization, such as in MXFP (Microscaling Formats) [3]. Key technical context regarding this flag and FP8 in ComfyUI: 1. Purpose: Like other --fp8-unet flags (e.g., e4m3fn or e5m2), this argument is designed to reduce the VRAM footprint of the model during inference [4][1]. 2. Implementation: ComfyUI supports multiple FP8 variants [4]. When using these flags, the application typically converts model weights to the specified FP8 format upon loading, eliminating the need to download separate, pre-quantized FP8 checkpoints [4]. 3. Status: The --fp8_e8m0fnu-unet option was merged into ComfyUI to provide additional flexibility in weight storage [5][2]. However, users should be aware that different FP8 formats may have varying impacts on model output quality or compatibility depending on the specific model architecture being used [4][5]. If you encounter errors or performance issues, standard formats like e4m3fn are often the recommended default [4].

Citations:

🌐 Web query:

Python pytorch fp8_e8m0fnu dtype PyTorch

💡 Result:

In PyTorch, at::kFloat8_e8m0fnu (or torch.float8_e8m0fnu) exists primarily as a specialized internal data type used in low-level CUDA operations for scaling factors [1][2]. It is not typically exposed as a standard, high-level tensor data type for general model weights or arithmetic in the same way as float32 or bfloat16 [3][4]. The E8M0FNU format is an 8-bit floating-point representation with an 8-bit exponent, no mantissa, and no sign bit [5][6]. It is specifically designed for use as a scaling factor in microscaling (MX) formats, such as those used in scaled matrix multiplication (scaled_mm) operations [1][7][5][8]. Key points regarding its presence and use: - Internal Usage: It appears in PyTorch's internal C++ implementation, specifically within ScaledBlas.cpp, to support block-wise scaling in specialized matrix multiplication kernels [1][2]. - Functionality: It is used to define the format of scale tensors when performing operations with other low-precision types (like FP8 or FP4) [1]. - Documentation: While it is referenced in technical implementations and internal ATen headers, it is generally not listed in the standard public-facing documentation for tensor attributes (e.g., in torch.dtype enumerations available for end-user code) [1][9][3]. If you encounter this type, it is likely within the context of hardware-specific optimizations or advanced quantization research (such as MXFP) where explicit control over scale tensor formats is required [7][5][8].

Citations:

Correct the FP8 definitions and remove em dashes.

e4m3fn and e5m2 are 1-sign-bit NVIDIA FP8 encodings, not e4+m3 or e5+m2. e8m0fnu has no sign bit and is an unsigned scale-like FP8 encoding, so describing it as “IEEE-compatible” misses the key distinction. Keep the MDX short and dash-free.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@development/comfyui-server/startup-flags.mdx` around lines 109 - 112, Update
the FP8 flag documentation to describe e4m3fn and e5m2 as NVIDIA 1-sign-bit FP8
encodings, and e8m0fnu as an unsigned, scale-like encoding without a sign bit.
Remove the “IEEE-compatible” framing and all em dashes while keeping the
explanations concise.

Source: Coding guidelines


### VAE precision

| Flag | Description |
Expand All @@ -125,6 +130,8 @@ python main.py --disable-auto-launch
| `--fp32-text-enc` | Store text encoder weights in fp32. |
| `--bf16-text-enc` | Store text encoder weights in bf16. |

See the [UNET precision](#unet-precision) table for the explanation of each FP8 format suffix.

### Other inference options

| Flag | Default | Description |
Expand Down
Loading