Submission declined on 11 May 2026 by Samoht27 (talk). This draft appears to contain text generated by a large language model (such as ChatGPT). You cannot use LLMs to generate article content. LLM-generated pages with certain obvious signs of being machine generated may be deleted without notice. Instead, only summarize in your own words a range of independent, reliable, published sources that discuss the subject. See the advice page on large language models for more information. If you would like to continue working on the submission, click on the "Edit" tab at the top of the window. If you have not resolved the issues listed above, your draft will be declined again and potentially deleted. If you need extra help, please ask us a question at the AfC Help Desk or get live help from experienced editors. Please do not remove reviewer comments or this notice until the submission is accepted. Where to get help If you need help editing or submitting your draft, please ask us a question at the AfC Help Desk or get live help from experienced editors. These venues are only for help with editing and the submission process, not to get reviews. If you need feedback on your draft, or if the review is taking a lot of time, you can try asking for help on the talk page of a relevant WikiProject. Some WikiProjects are more active than others so a speedy reply is not guaranteed. How to improve a draft Wikipedia:Contributing to Wikipedia – a basic overview on how to edit Wikipedia. Help:Wikitext – how to use the markup Help:Referencing for beginners – how to include references Wikipedia:Article development – how to develop your article Wikipedia:Writing better articles – how to improve your article Wikipedia:Verifiability – make sure your article includes reliable third-party sources You can also browse Wikipedia:Featured articles and Wikipedia:Good articles to find examples of Wikipedia's best writing on topics similar to your proposed article. Improving your odds of a speedy review To improve your odds of a faster review, tag your draft with relevant WikiProject tags using the button below. This will let reviewers know a new draft has been submitted in their area of interest. For instance, if you wrote about a female astronomer, you would want to add the Biography, Astronomy, and Women scientists tags. Add tags to your draft Editor resources Easy tools: Citation bot (help) | Advanced: Fix bare URLs Declined by Samoht27 2 months ago. Last edited by Samoht27 2 months ago. Reviewer: Inform author. Resubmit Please note that if the issues are not fixed, the draft will be declined again.

StreamDiffusion is an open-source diffusion model inference pipeline for real-time, interactive image generation. It was introduced by Akio Kodaira, Chenfeng Xu and coauthors in a 2023 preprint and was published in the proceedings of the IEEE/CVF International Conference on Computer Vision in 2025.^[2][1]

The method is a pipeline-level system rather than a new base generative model. It is designed to improve the throughput and latency of streaming image generation by restructuring inference around Stream Batch, residual classifier-free guidance, stochastic similarity filtering, input-output queues, pre-computation and model acceleration tools.^[1] In the authors' reported experiments, StreamDiffusion reached up to 91.07 frames per second on a single RTX 4090 GPU and exceeded the throughput of the Diffusers AutoPipeline baseline by more than 59.56 times under the benchmarked conditions.^[1]

A related system, StreamDiffusionV2, was introduced by Tianrui Feng, Zhi Li, Shuo Yang and coauthors in 2025 as a training-free streaming system for interactive video diffusion. StreamDiffusionV2 extends the general streaming-inference idea to video diffusion models and focuses on service-level objectives such as time-to-first-frame, per-frame deadlines, temporal consistency, jitter and multi-GPU scaling.^[4]

Diffusion models generate data by learning to reverse a gradual noising process. In a common formulation, a clean sample ${\displaystyle x_{0}}$ is corrupted into a noisy sample ${\displaystyle x_{t}}$ according to

${\displaystyle q(x_{t}\mid x_{0})={\mathcal {N}}\left(x_{t};{\sqrt {{\bar {\alpha }}_{t}}}x_{0},(1-{\bar {\alpha }}_{t})I\right),}$

where ${\displaystyle {\bar {\alpha }}_{t}}$ is determined by the noise schedule.^[5] A neural network is then trained to predict either the added noise, the clean sample, or another equivalent parameterization. A simplified denoising objective is

${\displaystyle {\mathcal {L}}(\theta )=\mathbb {E} _{x_{0},\epsilon ,t}\left[\left\|\epsilon -\epsilon _{\theta }(x_{t},t,c)\right\|_{2}^{2}\right],}$

where ${\displaystyle c}$ denotes optional conditioning information such as text, an image, a pose map, a depth map, or another control signal.

Diffusion models can produce high-quality images, but sampling usually requires repeated neural-network evaluations. Several earlier methods reduced this cost. Denoising diffusion implicit models showed that sampling could be accelerated by using non-Markovian sampling procedures with the same training objective as denoising diffusion probabilistic models.^[6] Latent diffusion models reduced computation by applying diffusion in the latent space of an autoencoder rather than in pixel space.^[7] Classifier-free guidance improved conditional generation by combining conditional and unconditional model predictions without requiring a separate classifier.^[8]

StreamDiffusion addresses a different part of the efficiency problem. Instead of only reducing the number of denoising steps or changing the base model, it changes how the inference pipeline handles a continuous input stream.^[1]

The first StreamDiffusion preprint was submitted to arXiv on 19 December 2023 and later revised on 8 July 2025.^[2] The corresponding ICCV 2025 paper was published in the open-access proceedings of the Computer Vision Foundation, with the final proceedings version listed as pages 12371–12380.^[1]

The official StreamDiffusion repository is hosted on GitHub under the Apache License 2.0. The repository describes the project as a diffusion pipeline for real-time interactive generation and lists Stream Batch, residual classifier-free guidance, stochastic similarity filtering, input-output queues, pre-computation for key-value caches and model acceleration tools among its key features.^[3]

StreamDiffusionV2 was submitted to arXiv on 10 November 2025 and revised on 22 February 2026.^[4] The StreamDiffusionV2 repository states that the work was accepted to Machine Learning and Systems 2026 and that the package was made available on PyPI in March 2026.^[9]

StreamDiffusion is designed for cases where a system receives a sequence of inputs, such as webcam frames, drawings, rendered frames, or other changing visual signals, and must continuously generate outputs. Conventional image diffusion pipelines usually process one input through all denoising steps before returning the result. StreamDiffusion instead treats the denoising process as a pipeline in which multiple frames can be in flight at different noise levels.^[1]

The system combines several mechanisms:

• Stream Batch, which reformulates sequential denoising as a batched process over frames at different denoising stages;
• residual classifier-free guidance (R-CFG), which reduces redundant unconditional passes in classifier-free guidance;
• stochastic similarity filtering (SSF), which probabilistically skips computation for frames similar to recently processed inputs;
• input-output queues, which handle differences between input rate, model throughput and display rate;
• pre-computation and caching, including prompt embeddings and key-value caches where applicable;
• model acceleration, including tools such as TensorRT and lightweight autoencoders in the authors' implementation.^[1][3]

In a standard image-to-image diffusion pipeline with ${\displaystyle n}$ denoising steps, the system denoises frame ${\displaystyle I_{t}}$ through all ${\displaystyle n}$ steps before starting frame ${\displaystyle I_{t+1}}$. Stream Batch changes the schedule by accepting a new input after each denoising step. The resulting active batch contains several frames at different denoising indices.^[1]

This is analogous to a pipeline in computer architecture: the latency of an individual frame remains tied to the number of stages, but throughput increases because several frames are being processed simultaneously. The authors reported that Stream Batch achieved approximately 1.5 times higher throughput than sequential denoising under their experimental settings.^[1]

The paper also describes a cross-frame attention operation within the stream batch. In simplified notation, for a query ${\displaystyle Q_{t,i}}$ from frame time ${\displaystyle t}$ and denoising index ${\displaystyle i}$, the keys and values can be concatenated across frames in the active batch:

${\displaystyle K_{\mathrm {batch} }=[K_{t+i,0},\ldots ,K_{t,i},\ldots ,K_{t-(n-1-i),n-1}],}$

${\displaystyle V_{\mathrm {batch} }=[V_{t+i,0},\ldots ,V_{t,i},\ldots ,V_{t-(n-1-i),n-1}].}$

Attention is then computed as

${\displaystyle \operatorname {Attn} (Q_{t,i},K_{\mathrm {batch} },V_{\mathrm {batch} })=\operatorname {softmax} \left({\frac {Q_{t,i}K_{\mathrm {batch} }^{T}}{\sqrt {d}}}\right)V_{\mathrm {batch} }.}$

The purpose of this operation is to allow a latent at one denoising stage to attend to information from neighboring frames and denoising stages available in the streaming batch.^[1]

Classifier-free guidance combines conditional and unconditional denoising predictions. A common expression is

${\displaystyle \epsilon _{\mathrm {cfg} }=\epsilon _{\bar {c}}+\gamma (\epsilon _{c}-\epsilon _{\bar {c}}),}$

where ${\displaystyle c}$ is the positive condition, ${\displaystyle {\bar {c}}}$ is the unconditional or negative condition, and ${\displaystyle \gamma }$ is the guidance scale.^[8] This improves conditional control but usually requires both conditional and unconditional model evaluations.

StreamDiffusion introduces residual classifier-free guidance, which approximates or reuses the negative branch rather than recomputing it at every denoising step. If the initial latent is expressed as

${\displaystyle x_{\tau _{0}}={\sqrt {\alpha _{\tau _{0}}}}x_{0}+{\sqrt {\beta _{\tau _{0}}}}\epsilon _{0},}$

then, under the paper's latent-consistency-style derivation, a residual noise term can be written as

${\displaystyle \epsilon _{\tau _{i},c'_{i}}={\frac {x_{\tau _{i}}-{\sqrt {\alpha _{\tau _{i}}}}x_{0}}{\sqrt {\beta _{\tau _{i}}}}}.}$

The R-CFG estimate can then be written as

${\displaystyle \epsilon _{\tau _{i},\mathrm {cfg} }=\delta \epsilon _{\tau _{i},c'_{i}}+\gamma (\epsilon _{\tau _{i},c}-\delta \epsilon _{\tau _{i},c'_{i}}),}$

where ${\displaystyle \delta }$ is a moderation coefficient. The paper describes variants in which the negative branch is computed once or not computed separately, reducing the number of denoising evaluations from ${\displaystyle 2n}$ to ${\displaystyle n+1}$ or ${\displaystyle n}$ for ${\displaystyle n}$ denoising steps.^[1] In the authors' benchmarks, R-CFG produced speedups of up to 2.05 times compared with conventional classifier-free guidance.^[1]

StreamDiffusion uses stochastic similarity filtering to avoid running the denoising pipeline on frames that are very similar to previously processed frames. The filter computes a cosine similarity between an input frame ${\displaystyle I_{t}}$ and a reference frame ${\displaystyle I_{\mathrm {ref} }}$:

${\displaystyle S_{C}(I_{t},I_{\mathrm {ref} })={\frac {I_{t}\cdot I_{\mathrm {ref} }}{\|I_{t}\|\|I_{\mathrm {ref} }\|}}.}$

The skip probability is

${\displaystyle P(\mathrm {skip} \mid I_{t},I_{\mathrm {ref} })=\max \left(0,{\frac {S_{C}(I_{t},I_{\mathrm {ref} })-\eta }{1-\eta }}\right),}$

where ${\displaystyle \eta }$ is a similarity threshold.^[1] The stochastic form avoids a purely deterministic freeze in nearly static streams, since similar frames can still be recomputed with non-zero probability.

In the authors' energy-consumption experiments, stochastic similarity filtering reduced computation for static image flows and produced reported reductions in power consumption of 2.39 times on an RTX 3060 and 1.99 times on an RTX 4090.^[1]

The StreamDiffusion paper describes input-output queues that decouple frame ingestion from model throughput and output display. The implementation also pre-computes reusable quantities such as text embeddings and negative prompt embeddings, and uses cache information where available.^[1]

The official repository reports benchmark results on a system with an RTX 4090 GPU, an Intel Core i9-13900K CPU and Ubuntu 22.04.3 LTS. Under those repository benchmarks, SD-Turbo with one denoising step achieved 106.16 frames per second for text-to-image and 93.897 frames per second for image-to-image; LCM-LoRA with KohakuV2 at four denoising steps achieved 38.023 frames per second for text-to-image and 37.133 frames per second for image-to-image.^[3] These figures are implementation- and hardware-dependent and are not general performance guarantees.

The ICCV paper compared StreamDiffusion against the Diffusers AutoPipeline baseline and reported up to 91.07 frames per second on a single RTX 4090 for image generation, more than 59.56 times the baseline throughput in the tested configuration.^[1] The same paper reported that Stream Batch increased throughput by approximately 1.5 times relative to sequential denoising, while R-CFG improved speed by up to 2.05 times relative to standard classifier-free guidance.^[1]

The results depend on several factors, including the base diffusion model, the number of denoising steps, image resolution, hardware, implementation framework, use of TensorRT, and the autoencoder used for latent encoding and decoding. The authors describe StreamDiffusion as complementary to model-level acceleration methods, meaning that faster samplers, distilled diffusion models, lightweight autoencoders and other acceleration techniques can be combined with the pipeline-level approach.^[2]

StreamDiffusionV2 is a related streaming system for dynamic and interactive video generation. It was proposed by Tianrui Feng, Zhi Li, Shuo Yang, Haocheng Xi, Muyang Li, Xiuyu Li, Lvmin Zhang, Keting Yang, Kelly Peng, Song Han, Maneesh Agrawala, Kurt Keutzer, Akio Kodaira and Chenfeng Xu.^[4] The paper describes the method as a training-free pipeline for adapting video diffusion models to live streaming.^[4]

Whereas the original StreamDiffusion is primarily image-based, StreamDiffusionV2 is designed around video-diffusion workloads. The authors identify four main problems in adapting offline video diffusion systems to live streaming: fixed-size input chunks that do not meet real-time service-level objectives, drift over long time horizons, quality degradation under fast motion, and inefficient multi-GPU scaling under per-frame latency constraints.^[4]

StreamDiffusionV2 combines several system-level mechanisms:^[4]

• SLO-aware batching, which adapts chunk and batch sizes to meet time-to-first-frame and per-frame deadline constraints;
• adaptive sink tokens and RoPE refresh, which update temporal anchors and reset positional offsets to reduce long-horizon drift;
• motion-aware noise scheduling, which adapts the denoising schedule according to estimated motion magnitude;
• pipeline orchestration, which partitions diffusion transformer blocks across devices for multi-GPU inference;
• DiT block scheduling, which balances the workload across pipeline stages;
• Stream-VAE, a low-latency video VAE variant that processes short frame chunks and caches intermediate features;
• asynchronous communication overlap, which uses separate compute and communication streams to reduce pipeline stalls.

In simplified scheduling notation, if ${\displaystyle L(m,b)}$ is the latency for a chunk of ${\displaystyle m}$ frames at stream batch size ${\displaystyle b}$, a real-time system must choose ${\displaystyle m}$ and ${\displaystyle b}$ such that

${\displaystyle L(m,b)\leq D,}$

where ${\displaystyle D}$ is the frame or chunk deadline. The StreamDiffusionV2 paper frames this problem as an SLO-aware scheduling task rather than a conventional offline throughput-maximization task.^[4]

The motion-aware noise controller estimates motion between consecutive latent frames and adjusts the denoising intensity. In simplified notation, a frame-difference motion score may be written as

${\displaystyle d_{t}={\frac {1}{N}}\|z_{t}-z_{t-1}\|_{2}^{2},}$

where ${\displaystyle z_{t}}$ and ${\displaystyle z_{t-1}}$ are latent frame or chunk representations and ${\displaystyle N}$ normalizes by the number of latent elements. The paper states that fast motion receives more conservative denoising to reduce tearing and ghosting, while slow or static motion can receive stronger refinement.^[4]

The StreamDiffusionV2 paper reports time-to-first-frame values of 0.47 seconds at 16 frames per second and 0.37 seconds at 30 frames per second in the tested video-to-video setting.^[4] It also reports 42.26 frames per second at 480p and 61.57 frames per second at 512×512 resolution for a 1.3-billion-parameter model on H100 GPUs with NVLink, and approximately 16 frames per second at 480p and 24 frames per second at 512×512 on RTX 4090 GPUs with PCIe.^[4]

For a 14-billion-parameter configuration, the authors reported 39.24 frames per second at 480p and 58.28 frames per second at 512×512 across four H100 GPUs.^[4] The paper also reports that StreamDiffusionV2 achieved a 0.2% miss rate under a one-second service-level objective in an H100 workstation experiment, compared with 99.9% for the CausVid baseline used by the authors.^[4]

The paper evaluates quality with text-image CLIP, temporal CLIP and warp error. In the reported comparison, StreamDiffusionV2 scored 29.29 on text-image CLIP, 98.51 on temporal CLIP and 73.31 on warp error, compared with 26.48, 95.24 and 117.01 respectively for StreamDiffusion in the same table.^[4]

The StreamDiffusion paper discusses applications that require low-latency generation from continuous inputs, including augmented and virtual reality, video-game graphics rendering, live video streaming and broadcasting.^[1] These use cases differ from ordinary text-to-image generation because the output must update continuously as input signals change.

StreamDiffusion has also been used in creative-coding and live-visual workflows. StreamDiffusionTD, a TouchDesigner operator documented by Dotsimulate, integrates StreamDiffusion into TouchDesigner and supports features such as SDXL-Turbo, IP Adapter, ControlNet, TensorRT acceleration and cloud or local backends.^[10] In a SIGGRAPH 2025 Real-Time Live summary, ACM SIGGRAPH described a laser-installation system that used StreamDiffusion with TouchDesigner to generate and refine real-time visuals before converting them into laser paths.^[11]

StreamDiffusion can be deployed locally or through hosted inference services, depending on the implementation. Local deployment generally requires a CUDA-capable NVIDIA GPU and a compatible Python, PyTorch and CUDA environment. Dotsimulate's StreamDiffusionTD documentation lists Windows 10/11, an NVIDIA GPU with at least 6 GB of VRAM, Python 3.11.9 or 3.10.9, and CUDA 12.8 as local-mode requirements for its v0.3.1 workflow.^[10]

A hosted deployment moves diffusion inference from the local machine to remote GPU infrastructure. In the StreamDiffusionTD ecosystem, Daydream describes a hosted StreamDiffusion mode in which the TouchDesigner plugin runs against the Daydream API while inference is performed on remote GPUs.^[12] Daydream states that this mode works on macOS and Windows and can run on laptops without a discrete GPU, while preserving real-time control over prompt, seed, ControlNet and IP Adapter values from TouchDesigner.^[12]

Dotsimulate's installation guide distinguishes between Daydream Cloud installation and local installation. It describes the cloud mode as requiring no local Python, CUDA or GPU installation, while noting that some features, including FX Processors, StreamV2V and custom processor support, are available only with the local backend.^[13] The practical trade-off is that hosted inference can reduce local setup and hardware requirements, while local inference provides offline operation and fuller control over the local GPU pipeline.^[12][13]

The original StreamDiffusion system is primarily image-centric. Although it can process a stream of image inputs, the StreamDiffusionV2 paper argues that image-based streaming diffusion systems can suffer from flicker, drift and weaker temporal consistency when applied to long video streams.^[4]

The benchmark results reported for StreamDiffusion and StreamDiffusionV2 are dependent on hardware, model selection, resolution, denoising steps and implementation details. StreamDiffusion's highest reported image-generation throughput was measured on an RTX 4090 in the authors' experimental configuration,^[1] while StreamDiffusionV2's multi-GPU results were measured on H100 and RTX 4090 systems under the paper's stated settings.^[4]

Hosted implementations also introduce deployment dependencies not present in local inference. In the Daydream and StreamDiffusionTD case, cloud operation requires an internet connection and an API key, while local operation requires compatible GPU and software setup.^[12][13]

• Diffusion model
• Stable Diffusion
• Latent diffusion model
• ControlNet
• Computer vision
• Image-to-image translation
• Video synthesis
• Real-time computing
• TouchDesigner

• StreamDiffusion GitHub repository
• StreamDiffusionV2 GitHub repository
• StreamDiffusionV2 project page

Category:Diffusion models Category:Generative artificial intelligence Category:Computer vision software Category:Image processing software Category:Real-time computing Category:Artificial intelligence art