README.md

metadata

title: 🎬 BackgroundFX Pro - SAM2 + MatAnyone
emoji: 🎥
colorFrom: indigo
colorTo: purple
sdk: streamlit
sdk_version: 1.32.0
app_file: streamlit_app.py
pinned: false
license: mit
tags:
  - video
  - background-removal
  - segmentation
  - matting
  - SAM2
  - MatAnyone

🎬 BackgroundFX Pro — Professional Video Background Replacement

BackgroundFX Pro is a GPU-accelerated app for Hugging Face Spaces (Docker) that replaces video backgrounds using:

• SAM2 — high-quality object segmentation
• MatAnyone — temporal video matting for stable alpha over time

Built on: CUDA 12.1.1, PyTorch 2.5.1 (cu121), torchvision 0.20.1, Streamlit 1.49.1.

---

✨ Features

• Replace backgrounds with: solid color, AI-generated image (procedural), custom uploaded image, or professional backgrounds
• Optimized for T4 GPUs on Hugging Face
• Two-stage pipeline: SAM2 segmentation → MatAnyone refinement → compositing
• Caching & logs stored in the repo volume:
  • HF cache → ./.hf
  • Torch cache → ./.torch
  • App data & logs → ./data (see data/run.log)
• FFmpeg — video format conversion and frame extraction

---

🚀 Try It

Open the Space in your browser (GPU required):
https://huggingface.co/spaces/MogensR/VideoBackgroundReplacer2

---

🖱️ How to Use

1. Upload a video (.mp4, .mov, .avi, .mkv).
2. Choose a Background Type: Image, Color, Blur, Professional Backgrounds, or AI Generated.
3. If using custom background, upload your image or select from professional options.
4. Click 🚀 Process Video.
5. Preview and 💾 Download Result.

Tip: Start with 720p/1080p on T4; 4K can exceed memory limits.

---

🗂️ Project Structure (key files)

• Dockerfile — CUDA 12.1.1 + PyTorch 2.5.1 container
• requirements.txt — Python dependencies
• app.py — Main Streamlit application
• integrated_pipeline.py — Two-stage processing pipeline
• models/sam2_loader.py — SAM2 model loader with HF Hub integration
• models/matanyone_loader.py — MatAnyone model loader
• utils/ — Utility functions
• data/ — Created at runtime for logs/outputs
• tmp/ — Created at runtime for processing jobs - video_pipeline.py — Core video processing logic (SAM2 + MatAnyone integration)
• video_pipeline.py — Core video processing logic (SAM2 + MatAnyone integration)

---

⚙️ Runtime Notes

• Binds to PORT / STREAMLIT_SERVER_PORT (defaults to 7860)
• File upload limit: 200MB via --server.maxUploadSize=200
• CORS disabled for Docker compatibility: --server.enableCORS=false
• Memory management with automatic cleanup between stages
• If processing fails, check Space logs for detailed error information

---

🧪 Local Development (Docker)

Requires an NVIDIA GPU with CUDA drivers.

git clone https://huggingface.co/spaces/MogensR/VideoBackgroundReplacer2
cd VideoBackgroundReplacer2

# Build (Ubuntu 22.04, CUDA 12.1.1; installs Torch 2.5.1+cu121)
docker build -t backgroundfx-pro .

# Run
docker run --gpus all -p 7860:7860 backgroundfx-pro

Access at: http://localhost:7860

---

🔧 Technical Details

Pipeline Architecture

1. Stage 1: SAM2 generates object masks using click points
2. Stage 2: MatAnyone refines masks for temporal consistency
3. Stage 3: Composite foreground with new background

Model Loading

• SAM2 models downloaded from Hugging Face Hub automatically
• Supports small/base/large variants (small recommended for T4)
• MatAnyone loaded from official repository

Performance Optimizations

• T4-specific optimizations (fp16, channels_last)
• Memory pruning during long video processing
• Automatic model unloading between stages

---

📝 License

MIT License - See LICENSE file for details.