VisualCloze: A Universal Image Generation Framework via Visual In-Context Learning
[💥💥💥 New 2025-5-15] VisualCloze has been merged into the official pipelines of diffusers. See Model Card for details.
[🥳🥳🥳 New 2025-5-18] We have released the LoRA weights supporting diffusers at LoRA Model Card 384 andLoRA Model Card 512.
📋 Quick Start Guide:
- Adjust Number of In-context Examples, 0 disables in-context learning.
- Set Task Columns, the number of images involved in a task.
- Upload Images. For in-context examples, upload all images. For the current query, upload images exclude the target.
- Click Generate to create the images.
- Parameters can be fine-tuned under Advanced Options.
🔥 Task Examples:
Click the task button in the right bottom to acquire examples of various tasks. Each click on a task may result in different examples. Make sure all images and prompts are loaded before clicking the generate button.
💻 Runtime on the Zero GPU:
The runtime on the Zero GPU runtime depends on the size of the image grid. When generating an image with the resoluation of 1024, the runtime is approximately [45s for a 2x2 grid], [55s for a 2x3 grid], [70s for a 3x3 grid], [90s for a 3x4 grid]. Deploying locally with an 80G A100 can reduce the runtime by more than half. Disabling SDEdit upsampling by setting the upsampling noise to 1 or reducing the upsampling steps can also save computation time, but it may lead to a decrease in generation quality.
Note: For better quality, you can deploy the demo locally using the model, which supports a higher resolution than this online demo, by following the instructions in the GitHub repository.
In-context Example 1
Query
In-context Example 3
In-context Example 4
In-context Example 5
❗❗❗ Before clicking the generate button, please wait until all images, prompts, and other components are fully loaded, especially when using task examples. Otherwise, the inputs from the previous and current sessions may get mixed.
If you find VisualCloze is helpful, please consider to star ⭐ the Github Repo. Thanks!
📝 Citation
If our work is useful for your research, please consider citing:
@article{li2025visualcloze,
title={VisualCloze: A Universal Image Generation Framework via Visual In-Context Learning},
author={Li, Zhong-Yu and Du, ruoyi and Yan, Juncheng and Zhuo, Le and Li, Zhen and Gao, Peng and Ma, Zhanyu and Cheng, Ming-Ming},
journal={arXiv preprint arXiv:2504.07960},
year={2025}
}
📋 License
This project is licensed under apache-2.0.
📧 Contact
Need help or have questions? Contact us at: lizhongyu [AT] mail.nankai.edu.cn.
Task Examples
Each click on a task may result in different examples.