<p align="center">
  <img src="assets/vi2g.png" width="200" alt="VI2G Logo">
</p>

<h1 align="center">VI2G — Viclant Image-to-Gauss</h1>

<p align="center">
  A lightweight Rust desktop application for real-time 3D Gaussian Splatting viewing<br>
  and a graphical frontend for <a href="https://github.com/apple/ml-sharp">Apple's SHARP (ml-sharp)</a> single-image-to-3DGS inference pipeline.
</p>

<p align="center">
  <strong>Author:</strong> <a href="https://github.com/Viclant">Viclant</a> &nbsp;|&nbsp;
  <strong>License:</strong> MIT &nbsp;|&nbsp;
  <strong>Language:</strong> Rust
</p>

> **What it does:** You give it an image; it calls Apple's SHARP model to generate a 3D Gaussian Splatting `.ply` scene, then lets you explore the result in a GPU-accelerated viewport — all from one window.

---

## Screenshots

<p align="center">
  <img src="sample/VI2G-sample-1-1.png" width="48%" alt="Sample scene 1 — view A">
  <img src="sample/VI2G-sample-1-2.png" width="48%" alt="Sample scene 1 — view B">
</p>
<p align="center"><em>Same scene, different camera angles — real-time GPU-rendered 3D Gaussian Splatting</em></p>

<p align="center">
  <img src="sample/VI2G-sample-2.png" width="80%" alt="Sample scene 2">
</p>
<p align="center"><em>A different scene generated from a single image via ml-sharp</em></p>

### Demo Video

The demonstration is available at [`sample/V12G-sample-3.mp4`](sample/V12G-sample-3.mp4).

---

## Features

- **Real-time 3DGS rendering** — GPU-accelerated Gaussian Splatting via compute-shader preprocessing, GPU radix sort, and rasterization (powered by [`wgpu-3dgs-viewer`](https://github.com/yurionion/wgpu-3dgs-viewer)).
- **One-click ml-sharp setup** — built-in button to automatically `git clone` + `pip install` Apple's ml-sharp. No Apple code is bundled; it is fetched from Apple's repository at the user's request.
- **Dual navigation modes**
  - *Blender-style*: MMB orbit · Shift+MMB pan · scroll zoom · Shift+B box zoom
  - *FPS-style*: WASD movement · mouse look · Space/Ctrl vertical · Shift sprint
- **Adjustable controls** — sliders for move speed, look sensitivity, pan speed, and FOV.
- **Preset system** — save / load / delete named control presets, persisted to JSON across sessions.
- **Multilingual UI** — English (default) and 中文. Language switcher in the top bar.
- **PLY metadata camera** — reads embedded extrinsic/intrinsic matrices from the PLY header to initialize the viewport at the original camera pose.
- **Drag & drop** — drop a `.ply` file onto the window to open it.
- **Sample included** — `sample/sample.ply` and `sample/sample.png` are provided to try immediately after building.

---

## Quick Start

1. Download the current Windows package from [viclant.com/software/vi2g](https://viclant.com/software/vi2g).
2. Extract the archive and run `VI2G.exe`.
3. Open or drag a `.ply` file into the viewport.
4. Optional: click **Setup ml-sharp** to install the external inference project. This requires Git and Python 3.13; CUDA is recommended, while prediction can also run on CPU.

To build from a local source checkout, run `cargo run --release` or use `run.bat` on Windows.

---

## Project Structure

```
VI2G/
├── src/
│   ├── main.rs             # Entry point, wgpu device config
│   ├── app.rs              # UI, input handling, ml-sharp setup & invocation
│   ├── camera.rs           # View/projection, orbit/pan/zoom/FPS controls
│   ├── gpu_renderer.rs     # egui ↔ wgpu-3dgs-viewer bridge
│   ├── ply_loader.rs       # PLY header metadata extraction
│   └── i18n.rs             # Multilingual string table (EN / ZH)
├── assets/
│   ├── vi2g.png            # App logo
│   └── vi2g.ico            # Windows icon (embedded in exe)
├── sample/                 # Sample files & screenshots (Git LFS)
│   ├── sample.png          # Sample input image
│   ├── sample.ply          # Sample output (3DGS scene)
│   ├── VI2G-sample-*.png   # Screenshots
│   └── V12G-sample-3.mp4   # Demo video
├── build.rs                # Windows resource compiler (icon embedding)
├── Cargo.toml
├── LICENSE                 # MIT
├── README.md
└── run.bat                 # Windows convenience script
```

### Source Files

| File | Lines | Purpose |
|---|---:|---|
| `src/main.rs` | 60 | Entry point, wgpu device setup with raised storage buffer limits |
| `src/app.rs` | 854 | UI layout, input handling, navigation modes, presets, ml-sharp setup & invocation |
| `src/camera.rs` | 161 | View/projection matrices (glam), orbit/pan/zoom/FPS, `wgpu_3dgs_viewer::CameraTrait` impl |
| `src/ply_loader.rs` | 141 | PLY header metadata extraction (extrinsic, intrinsic, image size) |
| `src/gpu_renderer.rs` | 85 | `egui_wgpu::CallbackTrait` — preprocess, sort, and render via `wgpu-3dgs-viewer` |
| `src/i18n.rs` | 168 | Compile-time string table for English and Chinese UI |

---

## Architecture

```
┌──────────────────────────────────────────────────────┐
│  VI2G  (Rust / eframe + egui)                        │
│                                                      │
│  ┌──────────┐   ┌──────────────┐   ┌──────────────┐ │
│  │ Side     │   │ Camera       │   │ GPU Renderer  │ │
│  │ Panel UI │   │ (glam)       │   │              │ │
│  │ (egui)   │   │ Blender/FPS  │   │ wgpu-3dgs-   │ │
│  │ i18n     │   │ CameraTrait  │   │ viewer       │ │
│  └──────────┘   └──────────────┘   └──────────────┘ │
│        │                                  ▲          │
│        ▼                                  │          │
│  ┌──────────┐                     ┌───────┴──────┐  │
│  │ ml-sharp │  subprocess call    │ wgpu         │  │
│  │ (Python) │ ──────────────────► │ Compute +    │  │
│  │ sharp    │  produces .ply      │ Render Pass  │  │
│  │ predict  │                     └──────────────┘  │
│  └──────────┘                                        │
└──────────────────────────────────────────────────────┘
```

---

## Third-Party Acknowledgements

### GPU Rendering — `wgpu-3dgs-viewer`

The real-time Gaussian Splatting rendering is powered by **[wgpu-3dgs-viewer](https://github.com/yurionion/wgpu-3dgs-viewer)** (MIT license), a Rust library that implements:

- **Compute-shader preprocessing** — projects 3D Gaussians to 2D screen-space splats, calculates spherical harmonics colors and 2D covariance matrices on the GPU.
- **GPU radix sort** — sorts splats by depth via a parallel radix sort compute pipeline for correct back-to-front alpha blending.
- **Splat rasterization shader** — renders sorted splats as alpha-blended quads in a single draw-indirect call.

All WGSL shader code is authored by and originates from the `wgpu-3dgs-viewer` crate. **VI2G does not contain or modify any shader source; it consumes the library as a compiled Cargo dependency.**

### 3DGS Inference — Apple `ml-sharp` (SHARP)

The image-to-3DGS conversion is performed by **[Apple's SHARP](https://github.com/apple/ml-sharp)** (Apple Sample Code License), a PyTorch-based model that predicts a 3D Gaussian Splatting scene from a single input image.

> **Important:** `ml-sharp` is distributed under the **Apple Sample Code License**, which is *not* an OSI-approved open-source license. It permits use, modification, and redistribution but **does not grant patent rights** and **prohibits using Apple's name or trademarks** to endorse or promote derivative works.

VI2G invokes `ml-sharp` as an **external subprocess** (`sharp predict`). **No Apple code is compiled into, linked with, or distributed as part of VI2G.** The built-in "Setup ml-sharp" button downloads the code directly from Apple's GitHub repository at the user's request — equivalent to the user running `git clone` themselves.

### UI & Graphics Stack

| Component | License | Role |
|---|---|---|
| [egui](https://github.com/emilk/egui) / [eframe](https://github.com/emilk/egui/tree/master/crates/eframe) | MIT OR Apache-2.0 | Immediate-mode GUI framework |
| [wgpu](https://github.com/gfx-rs/wgpu) | MIT OR Apache-2.0 | Cross-platform GPU API (Vulkan / DX12 / Metal) |
| [glam](https://github.com/bitshifter/glam-rs) | MIT OR Apache-2.0 | Linear algebra (matrices, quaternions) |
| [rfd](https://github.com/PolyMeilex/rfd) | MIT | Native file dialogs |
| [serde](https://serde.rs/) / serde_json | MIT OR Apache-2.0 | Preset serialization |
| [bytemuck](https://github.com/Lokathor/bytemuck) | Zlib OR Apache-2.0 OR MIT | Safe byte casting |
| [image](https://github.com/image-rs/image) | MIT OR Apache-2.0 | PNG decoding (logo/icon) |
| [winresource](https://github.com/nicohman/winresource) | MIT | Windows exe icon embedding (build-time) |

---

## Prerequisites

- **Rust** ≥ 1.75 (edition 2021)
- **GPU** with Vulkan, DirectX 12, or Metal support
- **Git LFS** — required to clone sample files; install from [git-lfs.com](https://git-lfs.com)
- **Python 3.13** (recommended by ml-sharp) and **git** — only needed for inference, not for viewing
- **CUDA GPU recommended** for fast inference; CPU inference is supported but slower

---

## Usage

1. **Open a PLY** — click "Open PLY" or drag-and-drop a `.ply` file. Try `sample/sample.ply` to get started.
2. **Navigate** — switch between Blender and WASD modes in the side panel.
3. **Switch language** — click "English" / "中文" in the top-right corner.
4. **Set up ml-sharp** — click "Setup ml-sharp" in the app (one-time, downloads from Apple's repo).
5. **Run inference** — select input image and output directory, then click "Run Predict".
6. **Save presets** — type a name and click "Save" to persist your control settings.

---

## Author

**UI & Application**: [Viclant](https://viclant.com)

Detailed authorship, third-party boundaries, user instructions, privacy notes, and release guidance are maintained in [`docs/`](docs/README.md).

## License

VI2G is released under the **MIT License** — see [LICENSE](LICENSE).

Third-party dependencies are under their respective permissive licenses (see table above).

**Apple ml-sharp** is under the [Apple Sample Code License](https://github.com/apple/ml-sharp/blob/main/LICENSE) and is **not bundled** with this project. Users must accept Apple's license terms when downloading it (via the in-app setup button or manually).
