[Pipelines] Add DreamLite text-to-image and image-edit pipelines

[Carlofkl]

Context

This PR integrates DreamLite — ByteDance's text-to-image / image-edit diffusion model — into diffusers, following an invitation from @NielsRogge to release the model on the Hub in diffusers format.

Related issue: ByteVisionLab/DreamLite#3 (comment)

Model cards (public, ungated):

• Base (3-branch dual CFG): https://huggingface.co/carlofkl/DreamLite-base
• Mobile (distilled, single forward): https://huggingface.co/carlofkl/DreamLite-mobile

Both repos use a diffusers branch (loaded via revision="diffusers") to keep the original ByteDance-internal main branch intact for backward compatibility with existing users.

What's added

src/diffusers/
├── models/unets/unet_dreamlite.py            # DreamLiteUNetModel
├── pipelines/dreamlite/
│   ├── __init__.py
│   ├── pipeline_dreamlite.py                  # DreamLitePipeline (3-branch dual CFG)
│   ├── pipeline_dreamlite_mobile.py           # DreamLiteMobilePipeline (distilled)
│   └── pipeline_output.py
└── (registered in src/diffusers/__init__.py, models/__init__.py,
    pipelines/__init__.py, utils/dummy_*.py)

docs/source/en/api/pipelines/dreamlite.md
tests/pipelines/dreamlite/
├── test_pipeline_dreamlite.py
└── test_pipeline_dreamlite_mobile.py

Architecture highlights

• DreamLiteUNetModel — UNet-based denoiser conditioned on Qwen3-VL text/vision embeddings.
• DreamLitePipeline — runs 3 forward passes per step (text-cond / image-cond / uncond) and combines them with a dual-CFG schedule for high-fidelity text-to-image and image edit.
• DreamLiteMobilePipeline — distilled single-pass variant; no CFG; designed for on-device inference. Pairs with AutoencoderTiny.
• Both pipelines use FlowMatchEulerDiscreteScheduler.

Testing

• Loading smoke test against carlofkl/DreamLite-base with revision="diffusers" — all 6 sub-modules resolve to the correct diffusers.* namespace.
• Inference smoke test — generates a 1024×1024 image in ~0.6s/step on a single A800; output stats sane (std≈93, no NaN/Inf).
• Standard pipeline tests in tests/pipelines/dreamlite/.

Before submitting

• Did you read the contributor guideline?
• Did you read our philosophy doc?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case. → Release DreamLite on Hugging Face ByteVisionLab/DreamLite#3
• Did you make sure to update the documentation with your changes? Here are the documentation guidelines.
• Did you write any new necessary tests?

Who can review?

cc @sayakpaul @yiyixuxu @DN6 — thanks in advance for the review!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Carlofkl]

Hi @sayakpaul @yiyixuxu — pushed a small follow-up commit (032412c) that fixes the two check_repository_consistency failures from the previous run:

1. Registered DreamLiteAttnProcessor2_0 in docs/source/en/api/attnprocessor.md (was missing — check_support_list.py).
2. Split combined height / width and guidance_scale / image_guidance_scale docstring entries in both pipelines into separate lines, and added a complete Args block to DreamLiteTransformer2DModel.forward (was tripping check_forward_call_docstrings.py).

No behavioral change — docs/docstrings only. Verified both lints pass locally.

Whenever convenient, could you re-approve the workflows? Thanks!

[Carlofkl]

Hi @yiyixuxu @DN6 @sayakpaul — quick update: CI is now fully green
(all 15 checks passing) and I've just marked the PR as ready for review.
Whenever you have a moment to take a look, I'd really appreciate it. Thanks!

[dg845]

Thanks for the PR! Left an initial design review :).

[dg845]

Also, if I test out the example using the following script:

import torch
from diffusers import DreamLitePipeline
from diffusers.utils import load_image

model_id = "carlofkl/DreamLite-base"
device = "cuda"
dtype = torch.float16

pipe = DreamLitePipeline.from_pretrained(model_id, revision="diffusers", torch_dtype=dtype)
pipe.to(device=device)

# Text-to-image
image = pipe(
    prompt="A serene mountain lake at sunrise",
    generator=torch.Generator(device=device).manual_seed(42),
).images[0]

image.save("dreamlite_t2i.png")

# Image-to-image (instruction-based edit)
image_url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/astronaut.jpg"
init_image = load_image(image_url)
edited = pipe(
    prompt="make it snowy",
    image=init_image,
    generator=torch.Generator(device=device).manual_seed(42),
).images[0]

edited.save("dreamlite_i2i.png")

I get the following T2I sample:

and the following I2I sample:

Is the sample quality expected? The T2I image in particular has a weird block pattern.

[Carlofkl]

@dg845 Thanks for testing!

Confirmed: the artifact only appears with transformers >= 5.0, which changed the underlying tokenization/processor logic for Qwen3-VL in a way that breaks DreamLite. With transformers==4.57.3 it reproduces cleanly on my end (A800, fp16, carlofkl/DreamLite-base) — see attached samples. I've pinned transformers==4.57.3 in the DreamLite README accordingly.

[Carlofkl]

@dg845 thanks for the initial review — pushed updates in 62a5db6 addressing all the inline comments, please take another look when you have time.

@yiyixuxu friendly ping — would love your eyes on this whenever you get a chance, especially the single-file unet decision in src/diffusers/models/unets/unet_dreamlite.py (per dg845's comment).

[dg845]

Thanks for iterating! Left some follow up comments.

[Carlofkl]

@dg845 follow-up review fully addressed across two commits (9fd711a2a + 63304257e):

-1. resnet (DepthwiseSeparableConv + ResnetBlock2DDreamLite) inlined into unet_dreamlite.py; resnet_dreamlite.py deleted.
-2.DreamLiteAttnProcessor2_0 moved into unet_dreamlite.py; attention_processor.py cleaned up; the autodoc entry in attnprocessor.md repointed to the new path.
-3. Dropped the hasattr(F, "scaled_dot_product_attention") ImportError — diffusers already requires torch>=2.0.
-4. Dropped the deprecated scale handling (*args, **kwargs + deprecate(...)); the new signature matches your suggestion exactly.
-5. Switched SDPA to dispatch_attention_fn with _attention_backend / _parallel_config class attrs (mirrors Flux2AttnProcessor). Follow-up fix in 63304257e: kept Q/K/V in (batch, seq, heads, head_dim) layout — that's what dispatch_attention_fn expects, the previous (batch, heads, seq, head_dim) layout collided with the dispatch's internal transpose and broke inference.
-6+7. Renamed block dispatch keys to match the Python class names: DreamLiteCrossAttn{Down,Up}Block2D, DreamLiteCrossAttnNoSelfAttn{Down,Up}Block2D, DreamLite{Down,Up}Block2D, DreamLiteUNetMidBlock2DCrossAttn. The DreamLiteUNetModel.__init__ defaults, the test fixtures, and the unet/config.json on the diffusers branch of carlofkl/DreamLite-{base,mobile} are all updated in lock-step.
-8. retrieve_latents inlined locally in pipeline_dreamlite.py with a # Copied from marker; the cross-pipeline import is gone — same shape as pipeline_dreamlite_mobile.py.
-9. Added check_inputs(prompt, image, height, width) to DreamLitePipeline and to DreamLiteMobilePipeline (mobile uses # Copied from diffusers.pipelines.dreamlite.pipeline_dreamlite.DreamLitePipeline.check_inputs); both pipelines call it from __call__. The image-type validation moved out of prepare_image_latents into check_inputs, plus we now also check that prompt is a str and warn if height/width aren't multiples of vae_scale_factor.

Re-tested end-to-end T2I + I2I locally on A800 / bf16 against carlofkl/DreamLite-base (diffusers branch) with transformers==4.57.3 — both modes still produce clean outputs.

Note on CI: the 5 failing PR test jobs (Fast PyTorch Pipeline / Models CPU tests, PyTorch Example CPU tests, Hub tests, LoRA tests with PEFT main) appear to be failing on main itself as well — looks like an environment / dependency issue upstream, not introduced by this PR. Happy to investigate further if it persists after main goes green.