Official implementation of SyncLight: Single-Edit Multi-View Relighting (NeurIPS 2026).
Authors: David Serrano-Lozano, Anand Bhattad, Luis Herranz, Jean-François Lalonde, Javier Vazquez-Corral
Paper · Project page · Model weights · Dataset
SyncLight relights a set of multi-view images consistently from a single light edit. Given a reference image, additional viewpoints, and a lightmap describing the edit in the reference view, the model generates all views under the new lighting in a single step, using a multi-view diffusion model (Latent Bridge Matching with MVDream-style 3D attention).
git clone https://github.com/CVC-Color/synclight
cd synclight
pip install -e .Python 3.10+ and a CUDA-capable GPU are required. For CPU-only inference use --device cpu (slow). To also work with the dataset, install the extra dependencies with pip install -e ".[dataset]" (see Dataset).
The pretrained weights are hosted on the Hugging Face Hub at davidserra9/synclight. davidserra9/synclight is the default value of --model_weights, so the weights are downloaded automatically the first time you run test.py, the Gradio demo or get_model. They are then cached in ~/.cache/huggingface.
To keep a local copy instead, for example on a machine without internet access, download it into ckpt/ and point --model_weights to that folder:
hf download davidserra9/synclight --local-dir ckpt
python test.py ... --model_weights ckpt/--model_weights accepts any folder that contains a .yaml config and a .safetensors or .ckpt weight file. If both kinds are present, the .safetensors file is used.
A lightmap describes the edit in the reference view. It is a float32 array of shape (H, W, 4), non-zero only on the pixels of the lights being edited:
| Channel | Meaning | Range |
|---|---|---|
| 0 | Activation: −1 turn the light off · 0 no change · 1 set the light to the values below | −1, 0 or 1 |
| 1 | Target intensity L, with intensity = (L + 1) / 2 |
−1 to 1 |
| 2 | Target colour, CIELAB a / 128 |
−1 to 1 |
| 3 | Target colour, CIELAB b / 128 |
−1 to 1 |
When a light is turned off (activation −1), channels 1–3 are 0. The lightmap does not need the same resolution as the images; it is resized to the model's input size.
Lightmaps can be stored as .npy or as 16-bit PNGs, the format used in input_images/ and in the dataset. In a 16-bit PNG each value x is stored as round(x · 32767) + 32768. This keeps −1, 0 and 1 exact and every other value within 1.6·10⁻⁵, at a fraction of the size. Image viewers show these PNGs as semi-transparent because channel 3 is stored in alpha. To read and write either format:
from src.synclight.lightmap_io import load_lightmap, save_lightmap_png
lightmap = load_lightmap("input_images/example_1_0_lightmap.png") # (H, W, 4) float32
save_lightmap_png("my_lightmap.png", lightmap)Use the interactive lightmap editor:
python lightmap_creator.py --image path/to/reference.pngControls:
- Left-click / drag: place a circular light source
- Scribble mode: click and drag to draw a freeform light
- Right-click: delete a light
- Scroll: resize the selected light
- Turn ON / Turn OFF: whether the selected light is switched on (with the intensity and colour below) or off
- Intensity slider: target intensity
- White / Temperature / ab Picker: light colour
- Save NPY: write the lightmap to disk
The editor saves <image>_lightmap_XXXX.npy and a preview <image>_lightmap_XXXX_preview.png in the current directory.
python gradio_demo.py --device cuda --port 7860Open http://localhost:7860. Load an example from the sidebar, or upload your own images and a lightmap (.npy or .png). Pass --model_weights ckpt/ to use a local copy of the weights.
python test.py \
--image_paths input_images/example_1_0_ref.png \
input_images/example_1_1.png \
input_images/example_1_2.png \
--lightmap_path input_images/example_1_0_lightmap.png \
--output_dir outputs/ \
--num_inference_steps 1 \
--device cudaOutput images are saved to outputs/output_0.jpg, output_1.jpg, etc., one per input view, in the order given.
Options:
| Flag | Default | Description |
|---|---|---|
--image_paths |
required | Input images: the reference view first, then the additional views |
--lightmap_path |
required | Lightmap of the reference view (.npy or 16-bit .png) |
--model_weights |
davidserra9/synclight |
Local weights folder or Hugging Face Hub model name |
--output_dir |
./outputs |
Output directory |
--num_inference_steps |
1 |
Diffusion steps (1 is fast and works well) |
--device |
cuda |
cuda or cpu |
--torch_dtype |
bfloat16 |
bfloat16 or float32 |
from PIL import Image
from src.synclight.inference import evaluate, get_model
from src.synclight.lightmap_io import load_lightmap
model = get_model("davidserra9/synclight", device="cuda") # or a local folder such as "ckpt/"
images = [Image.open(p).convert("RGB") for p in image_paths] # reference view first
lightmap = load_lightmap("lightmap.png") # or .npy; (H, W, 4) float32
outputs = evaluate(model, images, lightmap_image=lightmap, num_sampling_steps=1)
# outputs: list of PIL Images, one per input viewThe data used to train and evaluate SyncLight is hosted on the Hugging Face Hub at davidserra9/synclight (CC BY 4.0, 242 GB). It covers 408 scenes from three sources:
| Source | Content | Scenes (train / test) |
|---|---|---|
| Infinigen | procedurally generated rooms | 320 / 9 |
| BlenderKit | artist-made indoor and outdoor scenes | 37 / 3 |
| Real | multi-view RAW photographs of real rooms | 35 / 4 |
The dataset is released as one image per light source: linear HDR renders (EXR) for the synthetic scenes and RAW photographs for the real ones. Light is additive, so dataset/generate_pairs.py can render training pairs for any combination of lights, intensities and colours. You control the number of pairs, the colour palette (from neutral white to saturated colours), the types of edits, the tone mapping and the resolution. The provided configs reproduce the paper's training data: about 1 M pairs with the same sampling, colours and tone mapping.
pip install -e ".[dataset]"
# 1. download and extract the per-light images (here: the BlenderKit test split, 0.5 GB)
bash dataset/download.sh data/raw blenderkit test
# 2. render relighting pairs with the paper's settings
python dataset/generate_pairs.py --config dataset/configs/blenderkit.yaml \
--input data/raw/blenderkit/test --output data/pairs/blenderkit/test --workers 8See dataset/README.md for the full guide. It covers:
- the file layout and the output format;
- how pairs are sampled, and every generation parameter;
- the settings that reproduce the paper's data;
- speed and disk requirements;
- how to feed generated pairs to SyncLight.
synclight/
├── src/synclight/ # Core library
│ ├── models/
│ │ ├── lbm/ # LBM diffusion model
│ │ ├── unets/ # MVDream UNet with 3D attention
│ │ ├── vae/ # VAE wrapper
│ │ └── embedders/ # Conditioners (lightmap concat)
│ ├── inference/ # Inference utilities
│ └── lightmap_io.py # Read/write lightmaps (.npy and 16-bit .png)
├── dataset/ # Dataset tools
│ ├── README.md # Detailed dataset guide
│ ├── download.sh # Download and extract the dataset from the Hub
│ ├── generate_pairs.py # Render relighting pairs from per-light images
│ └── configs/ # Generation settings matching the paper (one per source)
├── ckpt/
│ └── config.yaml # Model config
├── input_images/ # Example inputs and lightmaps
├── gradio_demo.py # Interactive web demo
├── lightmap_creator.py # Interactive lightmap editor
└── test.py # Command-line inference script
@inproceedings{serrano2026synclight,
title = {SyncLight: Single-Edit Multi-View Relighting},
author = {Serrano-Lozano, David and Bhattad, Anand and Herranz, Luis and Lalonde, Jean-Fran{\c{c}}ois and Vazquez-Corral, Javier},
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
year = {2026}
}This work builds on the Latent Bridge Matching framework by Jasper AI and the MVDream multi-view diffusion architecture. The synthetic scenes were created with Infinigen and BlenderKit.
The code is released under the Apache 2.0 license. The dataset is released under CC BY 4.0.