Shadon

Overview

Shadon is a private local AI assistant designed for voice and text interaction, retrieval-augmented generation and persistent memory — keeping inference and knowledge closer to the user.

Problem

Most assistants depend on cloud-only inference. Shadon explores a private alternative with local LLMs, speech interfaces and memory that remains under user control.

Research and requirements

The stack required local model serving, speech transcription, text-to-speech, vector retrieval and short-/long-term memory orchestration through a practical application layer.

Architecture

Voice and text inputs flow through STT, memory and RAG layers into a local LLM served by Ollama. Responses can return as text or speech, with PostgreSQL supporting application persistence.

Solution

Built a multimodal assistant using Ollama-hosted local models, Faster-Whisper transcription, FAISS retrieval, sentence transformers and a Django REST application layer.

Features

• Local LLMs
• Retrieval-augmented generation
• Voice input
• Speech transcription
• Text-to-speech
• Short- and long-term memory
• Private local inference

Challenges

• Balancing local model quality with latency on consumer hardware
• Designing memory systems that stay useful without becoming noisy
• Integrating speech, retrieval and chat into one coherent flow

Results and impact

• Demonstrates applied Generative AI with private local inference
• Validates multimodal voice + RAG assistant design patterns
• Provides a foundation for private enterprise assistant concepts

Gallery