|
|
--- |
|
|
title: 🎬 BackgroundFX Pro - SAM2 + MatAnyone |
|
|
emoji: 🎥 |
|
|
colorFrom: indigo |
|
|
colorTo: purple |
|
|
sdk: streamlit |
|
|
sdk_version: 1.32.0 |
|
|
app_file: streamlit_app.py |
|
|
pinned: false |
|
|
license: mit |
|
|
tags: |
|
|
- video |
|
|
- background-removal |
|
|
- segmentation |
|
|
- matting |
|
|
- SAM2 |
|
|
- MatAnyone |
|
|
--- |
|
|
|
|
|
# 🎬 BackgroundFX Pro — Professional Video Background Replacement |
|
|
|
|
|
BackgroundFX Pro is a GPU-accelerated app for Hugging Face Spaces (Docker) that replaces video backgrounds using: |
|
|
- **SAM2** — high-quality object segmentation |
|
|
- **MatAnyone** — temporal video matting for stable alpha over time |
|
|
|
|
|
Built on: **CUDA 12.1.1**, **PyTorch 2.5.1 (cu121)**, **torchvision 0.20.1**, **Streamlit 1.49.1**. |
|
|
|
|
|
--- |
|
|
|
|
|
## ✨ Features |
|
|
|
|
|
- Replace backgrounds with: **solid color**, **AI-generated** image (procedural), **custom uploaded image**, or **professional backgrounds** |
|
|
- Optimized for **T4 GPUs** on Hugging Face |
|
|
- Two-stage pipeline: SAM2 segmentation → MatAnyone refinement → compositing |
|
|
- Caching & logs stored in the repo volume: |
|
|
- HF cache → `./.hf` |
|
|
- Torch cache → `./.torch` |
|
|
- App data & logs → `./data` (see `data/run.log`) |
|
|
- **FFmpeg** — video format conversion and frame extraction |
|
|
|
|
|
--- |
|
|
|
|
|
## 🚀 Try It |
|
|
|
|
|
Open the Space in your browser (GPU required): |
|
|
https://huggingface.co/spaces/MogensR/VideoBackgroundReplacer2 |
|
|
|
|
|
--- |
|
|
|
|
|
## 🖱️ How to Use |
|
|
|
|
|
1. **Upload a video** (`.mp4`, `.mov`, `.avi`, `.mkv`). |
|
|
2. Choose a **Background Type**: Image, Color, Blur, Professional Backgrounds, or AI Generated. |
|
|
3. If using custom background, upload your image or select from professional options. |
|
|
4. Click **🚀 Process Video**. |
|
|
5. Preview and **💾 Download Result**. |
|
|
|
|
|
> Tip: Start with 720p/1080p on T4; 4K can exceed memory limits. |
|
|
|
|
|
--- |
|
|
|
|
|
## 🗂️ Project Structure (key files) |
|
|
|
|
|
- `Dockerfile` — CUDA 12.1.1 + PyTorch 2.5.1 container |
|
|
- `requirements.txt` — Python dependencies |
|
|
- `app.py` — Main Streamlit application |
|
|
- `integrated_pipeline.py` — Two-stage processing pipeline |
|
|
- `models/sam2_loader.py` — SAM2 model loader with HF Hub integration |
|
|
- `models/matanyone_loader.py` — MatAnyone model loader |
|
|
- `utils/` — Utility functions |
|
|
- `data/` — Created at runtime for logs/outputs |
|
|
- `tmp/` — Created at runtime for processing jobs - `video_pipeline.py` — Core video processing logic (SAM2 + MatAnyone integration) |
|
|
- `video_pipeline.py` — Core video processing logic (SAM2 + MatAnyone integration) |
|
|
|
|
|
|
|
|
--- |
|
|
|
|
|
## ⚙️ Runtime Notes |
|
|
|
|
|
- Binds to `PORT` / `STREAMLIT_SERVER_PORT` (defaults to **7860**) |
|
|
- File upload limit: 200MB via `--server.maxUploadSize=200` |
|
|
- CORS disabled for Docker compatibility: `--server.enableCORS=false` |
|
|
- Memory management with automatic cleanup between stages |
|
|
- If processing fails, check Space logs for detailed error information |
|
|
|
|
|
--- |
|
|
|
|
|
## 🧪 Local Development (Docker) |
|
|
|
|
|
Requires an NVIDIA GPU with CUDA drivers. |
|
|
|
|
|
```bash |
|
|
git clone https://huggingface.co/spaces/MogensR/VideoBackgroundReplacer2 |
|
|
cd VideoBackgroundReplacer2 |
|
|
|
|
|
# Build (Ubuntu 22.04, CUDA 12.1.1; installs Torch 2.5.1+cu121) |
|
|
docker build -t backgroundfx-pro . |
|
|
|
|
|
# Run |
|
|
docker run --gpus all -p 7860:7860 backgroundfx-pro |
|
|
``` |
|
|
|
|
|
Access at: http://localhost:7860 |
|
|
|
|
|
--- |
|
|
|
|
|
## 🔧 Technical Details |
|
|
|
|
|
### Pipeline Architecture |
|
|
1. **Stage 1**: SAM2 generates object masks using click points |
|
|
2. **Stage 2**: MatAnyone refines masks for temporal consistency |
|
|
3. **Stage 3**: Composite foreground with new background |
|
|
|
|
|
### Model Loading |
|
|
- SAM2 models downloaded from Hugging Face Hub automatically |
|
|
- Supports small/base/large variants (small recommended for T4) |
|
|
- MatAnyone loaded from official repository |
|
|
|
|
|
### Performance Optimizations |
|
|
- T4-specific optimizations (fp16, channels_last) |
|
|
- Memory pruning during long video processing |
|
|
- Automatic model unloading between stages |
|
|
|
|
|
--- |
|
|
|
|
|
## 📝 License |
|
|
|
|
|
MIT License - See LICENSE file for details. |
