152 lines
4.9 KiB
Markdown
152 lines
4.9 KiB
Markdown
|
||
# Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis
|
||
|
||
|
||
[arXiv](https://arxiv.org/abs/2412.08603) | [Project Page](https://style3d.github.io/design2garmentcode/)
|
||
|
||
Feng Zhou, Ruiyang Liu, Chen Liu, Gaofeng He, Yong‑Lu Li, Xiaogang Jin, Huamin Wang. *CVPR 2025 .*
|
||
|
||
|
||
we propose a novel
|
||
sewing pattern generation approach Design2GarmentCode
|
||
based on Large Multimodal Models (LMMs), to generate parametric pattern-making programs from multi-modal
|
||
design concepts
|
||
---
|
||
|
||
## Installation
|
||
### 1. Clone the repository
|
||
```bash
|
||
git clone https://github.com/your-org/design2garmentcode.git # ← replace with the real URL
|
||
cd design2garmentcode
|
||
```
|
||
|
||
### 2. Create the Conda environment
|
||
An `environment.yml` file is provided in the project root with all required Conda and PyPI dependencies (Python 3.9.19, Torch 2.4.0 + CUDA 12.1, etc.).
|
||
|
||
```bash
|
||
conda env create -f environment.yml
|
||
conda activate d2g
|
||
python -m pip install --upgrade pip # optional: upgrade pip
|
||
```
|
||
|
||
### 3. (Optional) Enable 3‑D simulation
|
||
If you need local cloth simulation and 3‑D visualization, follow the installation instructions for **GarmentCode Warp Simulator**:
|
||
<https://github.com/maria-korosteleva/NvidiaWarp-GarmentCode>
|
||
|
||
---
|
||
### 4. Language‑Model API
|
||
`Design2GarmentCode` communicates with large multimodal models.
|
||
Follow the steps **in the given order**:
|
||
|
||
1. **Provide API credentials**
|
||
- **Environment variable (recommended)** – defaults to *ChatGPT‑4o*
|
||
```bash
|
||
export OPENAI_API_KEY="sk‑..."
|
||
```
|
||
- **Edit `system.json`** (project root) – manually specify `api_key`, `base_url`, and `model` if you prefer a file‑based approach.
|
||
2. **Download the required models**:
|
||
- First, download the base model [Qwen2-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct/tree/main).
|
||
Place the entire folder at:
|
||
`lmm_utils/Qwen/Qwen2-VL-2B-Instruct/`
|
||
|
||
- Next, download the fine-tuned weights file [model.pth](lmm_utils/Qwen/qwen2vl_lora_mlp/model.pth),
|
||
and place it in:
|
||
`lmm_utils/Qwen/qwen2vl_lora_mlp/`
|
||
---
|
||
|
||
## Quick GUI Demo
|
||
|
||
```bash
|
||
python gui.py
|
||
```
|
||
|
||
- Input: Free‑form prompt or an image/sketch
|
||
- Output: GarmentCode JSON, preview image, and (optionally) physics simulation
|
||
|
||
---
|
||
|
||
### 1. Text-Guided Pattern Generation
|
||
|
||
- Go to the PARSE DESIGN tab.
|
||
- In the input box at the bottom ("Describe your design..."), type a natural language description of the garment.
|
||
e.g., a T-shirt
|
||
- Click SEND to generate patterns based on your description.
|
||
|
||
---
|
||
|
||
### 2. Image-Guided Pattern Generation
|
||
|
||
- Click the upload icon inside the input box to upload a reference image or sketch.
|
||
- Once the image is uploaded, click SEND to parse the design and generate corresponding patterns.
|
||
|
||
---
|
||
|
||
### 3. Modify Patterns in the GUI
|
||
|
||
Once a pattern is generated, you can refine it directly inside the GUI:
|
||
|
||
1. Focus the input box at the bottom.
|
||
2. Type `modify: <your-instruction>`
|
||
- e.g., `modify: make sleeves shorter`
|
||
3. Press Enter – the system will regenerate the pattern accordingly.
|
||
|
||
---
|
||
## Batch Inference
|
||
### 1. Text Guided Generation
|
||
|
||
Use `test_text_batch.py` to process a list of text descriptions from a JSON file.
|
||
|
||
```bash
|
||
python lmm_utils/test_text_batch.py \
|
||
--input assets/test_text/examples.json \
|
||
--output assets/test_text_result \
|
||
--sim
|
||
```
|
||
|
||
- `--input`: Path to your input JSON file containing multiple garment description texts.
|
||
- `--output`: Directory where the output `.json` files will be saved.
|
||
- `--sim`: Enable or disable physical simulation output.
|
||
Supports physical simulation (enabled by default in script).
|
||
|
||
---
|
||
|
||
### 2. Image Guided Generation
|
||
|
||
Use `test_picture_batch.py` to process all image files in a directory.
|
||
|
||
```bash
|
||
python lmm_utils/test_picture_batch.py \
|
||
--input assets/test_img/examples \
|
||
--output assets/test_image_result/examples \
|
||
--sim
|
||
```
|
||
- `--input`: Folder containing multiple image files.
|
||
- `--output`: Output folder where results will be saved.
|
||
- `--sim`: Enable or disable physical simulation output.
|
||
|
||
---
|
||
|
||
|
||
## Get 3D Garment Patterns
|
||
### 1. Generate from a pattern.json
|
||
After generating the pattern data, you can simulate the corresponding 3D output directly from the pattern's JSON file.
|
||
```bash
|
||
python test_garment_sim.py --pattern_spec $INPUT_JSON
|
||
```
|
||
### 2. Generate from gui
|
||
You can also run the simulation directly on the GUI to obtain 3D data.
|
||
```bash
|
||
python gui.py
|
||
```
|
||
### Citation
|
||
```bash
|
||
If you find this work useful, please cite:
|
||
|
||
```bibtex
|
||
@article{zhou2024design2garmentcode,
|
||
title={Design2GarmentCode: Turning Design Concepts to Tangible Garments Through Program Synthesis},
|
||
author={Zhou, Feng and Liu, Ruiyang and Liu, Chen and He, Gaofeng and Li, Yong-Lu and Jin, Xiaogang and Wang, Huamin},
|
||
journal={Computer Vision and Pattern Recognition (CVPR)},
|
||
year={2025}
|
||
}
|
||
``` |