Student Project

Mic & Magic - Live AI assistant for Pen&Paper game masters

Staff

Brief Description

You speak - the AI listens, understands and inspires. Mic & Magic is an innovative practical project that combines the magic of pen & paper role-playing games with state-of-the-art voice AI. The result is a system that transcribes spoken dialogues in real time, recognizes speakers and generates creative suggestions for the game master - without any manual input.

Learning Objectives and Disciplines

The project aims to develop a game management assistance system that:

  • Records live audio from the game session,
  • Automatically recognizes speakers,
  • creates transcriptions in real time,
  • and continuously generates inspiring suggestions for small plots, encounters or twists from the ongoing conversation - based on the rules, world description and previous notes.

The AI acts like a silent co-player, constantly whispering in ideas.

Project Workflow

Phase 1 - Understanding

  • Integration of Whisper/WhisperX for live transcription
  • Speaker recognition with diarization and voice embeddings
  • Assignment of voices to defined characters (embedding cache)

Phase 2 - Thinking

  • Connection of a local LLM (e.g. via Ollama)
  • RAG system with world description, rules, notes
  • Continuous prompt generation for creative impulses

Phase 3 - Support

  • Development of a CLI application for the game management
  • Output of suggestions, character hints, tension building etc.

Bonus phase - Present

  • Optional desktop app to visualize transcript, speaker history, session summaries and AI ideas

Technologies & Tools

  • Audio & Diarization: Whisper, WhisperX, Insanely-Fast-Whisper, pyannote
  • Speech embedding: torchaudio, FAISS/Embedding Cache
  • LLM & RAG: Ollama, LangChain or own pipeline
  • Interface: Python (CLI), optional Tauri/Electron for GUI
  • Audio input: Microphone or combined stream via VoiceMeeter

Who is the project suitable for?

The project is ideal for students who…

  • are interested in practical AI development
  • are interested in audio processing, NLP or role-playing systems
  • like to think across systems (audio, embeddings, LLM, CLI/UI)
  • want to work creatively with technology

Previous knowledge of Python is helpful, but not a prerequisite - having fun with AI, game design and systemic thinking is more important!

Kick-Off Meeting

If you want to participate in the project, you are required to attend the kick-off meeting on TBA at TBA in room TBA. If that is not possible, message us before the meeting.

Material

Moodle course

To join the course you are required to enter a key which will be distributed during the kick-off meeting.