screencastgen

Audio-synchronized document readers with text highlighting and lip-sync video. Convert PDF, EPUB, plain-text, and other documents into narrated audio, highlighted readers, or LipSync Reader presentations.

Supports pluggable TTS backends, alignment providers, and lip-sync providers. The supported implementations are Qwen for TTS, WhisperX for alignment, and LatentSync for lip-sync.

Documentation

Read the documentation site for guided workflows, architecture, troubleshooting, and the developer reference. The Markdown source is maintained in docs/.

Try the hosted interactive demo reader to see synchronized narration, text highlighting, and the presenter view.

Preview documentation changes locally with:

pip install -e ".[docs]"
mkdocs serve
mkdocs build --strict

Quick Start

Start with the Getting Started guide for installation, environment verification, and first pipeline commands. The Installation Guide covers Docker-based GPU VM deployments and source installation.

Pipelines

screencastgen includes three document pipelines:

• Lip-sync: generate a LipSync Reader or EPUB from a document, reference voice, and reference face video.
• Highlight: generate synchronized word highlighting over PDF page images or reflowed document text.
• Audio: generate narrated audio from a document.

See the pipeline overview for shared processing stages and the CLI reference for command options.

Model Dependencies and References

The ML providers are maintained independently. Review their repositories, model cards, licenses, and usage restrictions before distributing generated content or deploying commercially.

Component	Used for	Resources
Qwen3-TTS	Default local TTS and voice cloning	GitHub, 0.6B model, 1.7B model
WhisperX	Transcription and word-level alignment	GitHub
LatentSync 1.6	Default diffusion-based lip synchronization	GitHub, model weights, paper

This integration uses LatentSync’s latentsync_unet.pt and Whisper tiny.pt checkpoints. LatentSync lists 18 GB as the minimum VRAM for LatentSync 1.6 inference and publishes its code under Apache 2.0. See the LatentSync README for current runtime requirements and checkpoint details.

Web Application

Full-stack web UI wrapping all three pipelines. Stack: FastAPI + PostgreSQL + Celery/Redis + React/Tailwind.

See Web Application Setup for Docker and local-development instructions.

By default files are stored on the local filesystem. Set P2A_STORAGE_BACKEND to gcs or s3 to store uploads and outputs in a cloud bucket. Pipelines always work against local directories; the storage layer handles downloading inputs and uploading outputs to the bucket.

New web jobs default to the LipSync Reader pipeline. The recommended output is the standalone offline reader ZIP, which preserves the synchronized document, narration, and presenter experience for local playback.

Future Directions

The following ideas are exploratory. They are not committed features and do not have release dates.

• Presenter background replacement: Use person segmentation to isolate the presenter and replace, blur, or stylize the background.
• Subject-aware video effects: Use face and object detection with tracking for overlays, masking, reframing, and other controlled visual effects.
• Diagram-driven presentations: Animate diagrams created with draw.io or Excalidraw in sync with generated narration, with either voice-only output or an optional lip-synced presenter overlay.

Citation

If screencastgen contributes to your research or published work, cite this repository:

@software{screencastgen,
  author  = {Shekhar, Shashank},
  title   = {screencastgen: Audio-Synchronized Document Readers with Text Highlighting and Lip-Sync Video},
  year    = {2026},
  version = {2.0.0},
  url     = {https://github.com/ShaShekhar/screencastgen}
}

This site is open source. Improve this page.