docs: explain FP8 format suffixes (e4m3fn, e5m2, e8m0fnu) in startup flags page#1313
Conversation
📝 WalkthroughWalkthroughChangesFP8 precision documentation
🚥 Pre-merge checks | ✅ 2✅ Passed checks (2 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@development/comfyui-server/startup-flags.mdx`:
- Around line 109-112: Update the FP8 flag documentation to describe e4m3fn and
e5m2 as NVIDIA 1-sign-bit FP8 encodings, and e8m0fnu as an unsigned, scale-like
encoding without a sign bit. Remove the “IEEE-compatible” framing and all em
dashes while keeping the explanations concise.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 6a1549a0-04db-457e-a3a6-a480c0ddc2a9
📒 Files selected for processing (1)
development/comfyui-server/startup-flags.mdx
| The suffix in each FP8 flag identifies the IEEE-compatible format: | ||
| - **e4m3fn** — 4 exponent bits, 3 mantissa bits. Good balance of precision and range for model weights on NVIDIA Ada Lovelace and newer GPUs. | ||
| - **e5m2** — 5 exponent bits, 2 mantissa bits. Wider dynamic range but lower precision than e4m3fn. Useful when weight magnitudes vary widely. | ||
| - **e8m0fnu** — 8 exponent bits, 0 mantissa bits. Maximum dynamic range with no fractional precision (acts as a per-tensor scale factor). |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Locate file =="
git ls-files | grep -F 'development/comfyui-server/startup-flags.mdx' || true
echo "== Relevant file excerpt =="
if [ -f development/comfyui-server/startup-flags.mdx ]; then
sed -n '90,145p' development/comfyui-server/startup-flags.mdx | nl -ba -v90
fi
echo "== Search for FP8/e8m0fnu references in repo =="
rg -n "e8m0fnu|e4m3fn|e5m2|FP8|fp8|quantization|Quantization|comfy\.cli_args|cli_args|startup-flags" -S .Repository: Comfy-Org/docs
Length of output: 427
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Relevant file excerpt =="
cat -n development/comfyui-server/startup-flags.mdx | sed -n '90,145p'
echo "== Search for FP8/e8m0fnu references in repo =="
rg -n "e8m0fnu|e4m3fn|e5m2|FP8|fp8|quantization|Quantization|comfy\.cli_args|cli_args|startup-flags" -S .Repository: Comfy-Org/docs
Length of output: 50371
🌐 Web query:
NVIDIA FP8 formats e4m3fn e5m2 sign exponent mantissa bits
💡 Result:
NVIDIA FP8 formats consist of two distinct 8-bit floating-point encodings, E4M3 and E5M2, both designed for efficient deep learning workloads [1][2]. Each format is comprised of a single sign bit, a specific number of exponent bits, and a set number of mantissa bits [1][3]. The bit breakdown for these formats is as follows: E4M3 (E4M3FN) 1 sign bit, 4 exponent bits, and 3 mantissa bits [1][4]. This format prioritizes precision and is typically used for weights and activations in the forward pass [1][5]. It does not support infinities and has a limited set of NaN patterns to extend its dynamic range [1]. E5M2 1 sign bit, 5 exponent bits, and 2 mantissa bits [1][4][3]. This format provides a wider dynamic range at the cost of lower precision [5][2]. It is often used for gradients in the backward pass, where magnitude variations are more significant [1][5]. It follows IEEE 754 conventions for special values, including infinities [1][2]. Summary Table Format | Sign Bits | Exponent Bits | Mantissa Bits | Total Bits --- | --- | --- | --- | --- E4M3 | 1 | 4 | 3 | 8 E5M2 | 1 | 5 | 2 | 8 Note: In technical documentation, the "mantissa" bits refer to the explicit significand bits, excluding the implicit leading bit common in IEEE 754-style floating-point numbers [6][3].
Citations:
- 1: https://arxiv.org/pdf/2209.05433
- 2: https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/examples/fp8_primer.html.md
- 3: https://docs.nvidia.com/cuda/archive/12.8.2/cuda-math-api/cuda_math_api/struct____nv__fp8__e5m2.html
- 4: https://www.baseten.co/blog/fp8-efficient-model-inference-with-8-bit-floating-point-numbers/
- 5: https://developer.nvidia.com/blog/floating-point-8-an-introduction-to-efficient-lower-precision-ai-training/
- 6: https://huggingface.co/papers/2209.05433
🌐 Web query:
"e8m0fnu" FP8 PyTorch sign bits E8M0fn2 "0-8-0"
💡 Result:
The term E8M0 refers to a specialized 8-bit format primarily used for block-scaling factors in low-precision matrix multiplication, rather than as a general-purpose floating-point data type for activations or weights [1][2]. Key technical characteristics of E8M0/E8M0fnu include: Format Definition: The E8M0 format consists of 8 exponent bits and 0 mantissa bits (an unsigned 8-bit floating-point representation) [3][2]. It is interpreted as powers of two with a biased exponent [4]. Technical Specifications: - Exponent: 8 bits [3]. - Mantissa: 0 bits [3]. - Exponent Bias: 127 [4][3]. - NaN (Not a Number): Represented by the bit pattern 0xFF (255) [4][3]. - "FNU": The suffix "fnu" stands for "finite," "no sign," and "unsigned" (or "no-sign"), indicating that the format does not support negative values, infinities, or specialized sign-bit encodings [3][2]. - Use Case: It is frequently utilized as a scale factor type in micro-scaling (MXFP) operations, where it encodes the scaling factors for underlying FP8 or FP4 tensor data (e.g., E4M3FN weights) [1][5][2]. In software implementations such as vLLM and various CUDA math APIs, this format is often referred to as float8_e8m0fnu [1][6][4]. Because it lacks a sign bit and mantissa, it acts strictly as an exponent-based scale, allowing for efficient range adjustment in block-quantized kernels [7][2].
Citations:
- 1: https://docs.vllm.ai/en/v0.22.0/api/vllm/model%5Fexecutor/layers/quantization/utils/marlin_utils_fp8/
- 2: https://github.com/NVlabs/cutile-rs/blob/main/cuda-core/src/dtype.rs
- 3: sbrunk/modular@8693038
- 4: https://docs.nvidia.com/cuda/cuda-math-api/cuda_math_api/struct____nv__fp8__e8m0.html
- 5: [Bugfix][DeepSeek V4] Enable cross-node TP=16 FP8 serving vllm-project/vllm#41312
- 6: https://vllm.readthedocs.io/en/latest/api/vllm/model%5Fexecutor/layers/quantization/utils/marlin_utils_fp8/
- 7: https://www.generalcompute.com/blog/fp8-training-and-inference-the-precision-sweet-spot
🌐 Web query:
ComfyUI --fp8_e8m0fnu-unet fp8_e8m0fnu unet
💡 Result:
In ComfyUI, --fp8_e8m0fnu-unet is a command-line argument used to store UNET model weights in the fp8_e8m0fnu data format [1][2]. The fp8_e8m0fnu format (8-bit floating-point with 8 exponent bits, no mantissa, and no sign bit) is a technical data type often utilized as a scale factor for quantization, such as in MXFP (Microscaling Formats) [3]. Key technical context regarding this flag and FP8 in ComfyUI: 1. Purpose: Like other --fp8-unet flags (e.g., e4m3fn or e5m2), this argument is designed to reduce the VRAM footprint of the model during inference [4][1]. 2. Implementation: ComfyUI supports multiple FP8 variants [4]. When using these flags, the application typically converts model weights to the specified FP8 format upon loading, eliminating the need to download separate, pre-quantized FP8 checkpoints [4]. 3. Status: The --fp8_e8m0fnu-unet option was merged into ComfyUI to provide additional flexibility in weight storage [5][2]. However, users should be aware that different FP8 formats may have varying impacts on model output quality or compatibility depending on the specific model architecture being used [4][5]. If you encounter errors or performance issues, standard formats like e4m3fn are often the recommended default [4].
Citations:
- 1: https://huggingface.co/spaces/reach-vb/Chroma/resolve/main/comfy/cli_args.py?download=true
- 2: https://github.com/Comfy-Org/ComfyUI/blob/master/comfy/cli_args.py
- 3: https://apple.github.io/coreai-optimization/api/generated/coreai_opt.coreai_utils.DType.html
- 4: Support FP8 ComfyUI#2157
- 5: Add option for using fp8_e8m0fnu for model weights. ComfyUI#7733
🌐 Web query:
Python pytorch fp8_e8m0fnu dtype PyTorch
💡 Result:
In PyTorch, at::kFloat8_e8m0fnu (or torch.float8_e8m0fnu) exists primarily as a specialized internal data type used in low-level CUDA operations for scaling factors [1][2]. It is not typically exposed as a standard, high-level tensor data type for general model weights or arithmetic in the same way as float32 or bfloat16 [3][4]. The E8M0FNU format is an 8-bit floating-point representation with an 8-bit exponent, no mantissa, and no sign bit [5][6]. It is specifically designed for use as a scaling factor in microscaling (MX) formats, such as those used in scaled matrix multiplication (scaled_mm) operations [1][7][5][8]. Key points regarding its presence and use: - Internal Usage: It appears in PyTorch's internal C++ implementation, specifically within ScaledBlas.cpp, to support block-wise scaling in specialized matrix multiplication kernels [1][2]. - Functionality: It is used to define the format of scale tensors when performing operations with other low-precision types (like FP8 or FP4) [1]. - Documentation: While it is referenced in technical implementations and internal ATen headers, it is generally not listed in the standard public-facing documentation for tensor attributes (e.g., in torch.dtype enumerations available for end-user code) [1][9][3]. If you encounter this type, it is likely within the context of hardware-specific optimizations or advanced quantization research (such as MXFP) where explicit control over scale tensor formats is required [7][5][8].
Citations:
- 1: https://github.com/pytorch/pytorch/blob/e9ebbd3b/aten/src/ATen/native/cuda/ScaledBlas.cpp
- 2: trichmo/pytorch@5ebf74a
- 3: https://docs.pytorch.org/docs/2.3/tensor%5Fattributes.html
- 4: https://docs.pytorch.org/docs/2.2/tensor%5Fattributes.html
- 5: https://mojolang.org/nightly/docs/std/builtin/dtype/DType/
- 6: https://mojolang.org/docs/std/builtin/dtype/DType/
- 7: https://apple.github.io/coreai-optimization/api/generated/coreai_opt.coreai_utils.DType.html
- 8: https://github.com/NVlabs/cutile-rs/blob/main/cuda-core/src/dtype.rs
- 9: https://docs.pytorch.org/docs/2.12/tensor_attributes.html
Correct the FP8 definitions and remove em dashes.
e4m3fn and e5m2 are 1-sign-bit NVIDIA FP8 encodings, not e4+m3 or e5+m2. e8m0fnu has no sign bit and is an unsigned scale-like FP8 encoding, so describing it as “IEEE-compatible” misses the key distinction. Keep the MDX short and dash-free.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@development/comfyui-server/startup-flags.mdx` around lines 109 - 112, Update
the FP8 flag documentation to describe e4m3fn and e5m2 as NVIDIA 1-sign-bit FP8
encodings, and e8m0fnu as an unsigned, scale-like encoding without a sign bit.
Remove the “IEEE-compatible” framing and all em dashes while keeping the
explanations concise.
Source: Coding guidelines
1 participant
Summary
Multiple users asked the AI assistant what the different FP8 format flags (e4m3fn, e5m2, e8m0fnu) mean and how they differ. The startup-flags page listed the flags but provided no explanation of the format names. This change adds a brief explanation of each FP8 format suffix under the UNET precision section and a cross-reference from the Text encoder precision section.
Changes
development/comfyui-server/startup-flags.mdx: Added format suffix explanation (bit allocation, use case) for e4m3fn, e5m2, and e8m0fnu under UNET precision. Added cross-reference note to Text encoder precision section.Source
Mintlify AI Assistant analytics data (2026-07-17)
Checklist