Official implementation of GaMO (version 1)
🔔 Note: A fully integrated version with merged environments and a one-click end-to-end pipeline is planned to be released in 2026 February.
- ✅ Release multi-stage code and environment setups
- ✅ Provide example dataset:
Replica_6/office_2 - ⬜ Merge multiple conda environments and resolve dependency incompatibilities
- ⬜ Integrate a one-click bash script for end-to-end pipeline execution
- ⬜ Add more evaluation datasets
Before running GaMO, make sure the pretrained weights exist under:
gamo/check_points/
Required:
- GaMO pretrained model
(Download) → pretrained_model.zip
Save to:gamo/check_points/(unzipped) - DUSt3R ViTLarge checkpoint
(Download) → DUSt3R_ViTLarge_BaseDecoder_512_dpt.pth
Save to:gamo/check_points/
Additional optional downloads (depending on your environment):
-
Stable-Diffusion-2-1-base
→ place under:gamo/check_points/
(Download) → Stable-Diffusion-2-1-base -
MASt3R model weights
→ place under:gamo/submodules/MASt3R-SLAM/thirdparty/mast3r/weights/
(Download) → MASt3R GitHub
# 1. 3DGS environment
conda env create -f env/env_3dgs.yml
# 2. GaMO environment
conda env create -f env/env_GaMO.yml
# 3. Mask / Init environment
conda env create -f env/env_opamask.yml
# For 3dgs and opamask
conda activate 3dgs
pip install -e 3dgs/submodules/diff-gaussian-rasterization
pip install -e 3dgs/submodules/simple-knn
# For GaMO
conda activate GaMO
pip install -e gamo/submodules/MASt3R-SLAM
pip install -e gamo/submodules/MASt3R-SLAM/thirdparty/mast3r
pip install -e gamo/submodules/MASt3R-SLAM/thirdparty/in3d
Dataset must follow:
3dgs/data/Input/Duster/{ROOT}/{SCENE}/
├── images/
├── images_test/
└── sparse/
├── 0/
└── test/
Example:
3dgs/data/Input/Duster/Replica_6/office_2/
conda activate opamask
bash Point.sh Replica_6 office_2
mkdir -p 3dgs/data/Input/Duster/Replica_6/office_2/sparse/0
cp dust3r_results/Replica_6/office_2/sparse/0/points3D.ply \
3dgs/data/Input/Duster/Replica_6/office_2/sparse/0/
conda activate 3dgs
bash Pipeline.sh --step 1 Replica_6 office_2
# masks
conda activate opamask
bash Pipeline.sh --step 1b Replica_6 office_2
# GaMO Outpaint
conda activate GaMO
bash Pipeline.sh --step 2 Replica_6 office_2
conda activate GaMO
bash Pipeline.sh --step 3 Replica_6 office_2
conda activate opamask
bash Pipeline.sh --step 3.5 Replica_6 office_2
conda activate 3dgs
bash Pipeline.sh --step 4 Replica_6 office_2
bash Pipeline.sh --step 5 Replica_6 office_2
| Step | Environment | Command | Description |
|---|---|---|---|
| 0 | opamask | Point.sh | Initial DUSt3R pointcloud |
| 1 | 3dgs | --step 1 | Scale cameras, train GS |
| 1b | opamask | --step 1b | Render masks |
| 2 | GaMO | --step 2 | GaMO diffusion outpainting |
| 3 | GaMO | --step 3 | Alignment / seed init |
| 3.5 | opamask | --step 3.5 | DUSt3R refined pointcloud |
| 4 | 3dgs | --step 4 | Final GS training |
| 5 | 3dgs | --step 5 | Final GS rendering |
If you encounter issues, open a GitHub Issue. """
If you find our work useful, please cite this paper and give us a ⭐️.
@article{gamo2024,
author = {Huang, Yi-Chuan and Chien, Hao-Jen and Lin, Chin-Yang and Chen, Ying-Huan and Liu, Yu-Lun},
title = {GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstruction},
journal = {arXiv preprint arXiv:2406.01570},
year = {2024}
}