STT Runner

Speech-to-Text transcription using sherpa-onnx + Qwen3-ASR, Text-to-Speech with ZipVoice (zero-shot voice cloning), and LLM/embedding inference with llama.cpp (Granite-4.2 + nomic-embed-text-v1.5).

Installation

python3 -m venv .venv
.venv/bin/pip install -r requirements.txt

Usage

Speech-to-Text

python stt_runner.py [--language=Indonesian] audio1.wav audio2.wav ...

Text-to-Speech

python tts_runner.py [--output=out.wav] [--ref-audio=ref.wav] [--ref-text="..."] "text to speak"

Output defaults to output.wav. The reference audio/text (voice to clone) is set in config/tts.py and can be overridden per-run with --ref-audio / --ref-text (the text must match the audio exactly).

LLM Chat (llama.cpp)

python llm_runner.py "What is the capital of France?"   # single prompt
python llm_runner.py                                     # interactive chat

Embedding (llama.cpp)

python embed_runner.py "text to embed" "another text"
python embed_runner.py "What is TSNE?" --prefix "search_query: "

Configuration

Model paths and inference parameters are hardcoded in config/:

• config/model.py — model paths (conv_frontend, encoder, decoder, tokenizer under models/)
• config/asr.py — inference params: LANGUAGE, HOTWORDS, NUM_THREADS, PROVIDER, SAMPLE_RATE, FEATURE_DIM, MAX_TOTAL_LEN, MAX_NEW_TOKENS
• config/tts.py — TTS model paths, REFERENCE_AUDIO, REFERENCE_TEXT, OUTPUT_FILE, NUM_THREADS, PROVIDER, NUM_STEPS
• config/llm.py — Granite-4.2 model path, N_CTX, N_THREADS, N_GPU_LAYERS, MAX_TOKENS, TEMPERATURE, TOP_P, TOP_K, SYSTEM_PROMPT, CHAT_TEMPLATE
• config/embed.py — nomic-embed-text-v1.5 model path, N_CTX, N_THREADS, PREFIX_QUERY, PREFIX_DOCUMENT, DEFAULT_PREFIX

LANGUAGE defaults to "" (all languages / auto-detect). Passing --language on the CLI overrides it.

Download Model (Qwen3-ASR 1.7B int8)

BASE="https://modelscope.cn/models/zengshuishui/Qwen3-ASR-onnx/resolve/master"
mkdir -p models/model_1.7B models/tokenizer
wget -O models/model_1.7B/conv_frontend.onnx "$BASE/model_1.7B/conv_frontend.onnx"
wget -O models/model_1.7B/encoder.int8.onnx  "$BASE/model_1.7B/encoder.int8.onnx"
wget -O models/model_1.7B/decoder.int8.onnx  "$BASE/model_1.7B/decoder.int8.onnx"
for f in vocab.json merges.txt tokenizer_config.json preprocessor_config.json config.json chat_template.json; do
  wget -O "models/tokenizer/$f" "$BASE/tokenizer/$f"
done

Download Model (ZipVoice TTS)

mkdir -p models/zipvoice
wget -qO- https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/sherpa-onnx-zipvoice-distill-int8-zh-en-emilia.tar.bz2 \
  | tar xjf - -C models/zipvoice --strip-components=1
wget -O models/zipvoice/vocos_24khz.onnx \
  https://github.com/k2-fsa/sherpa-onnx/releases/download/vocoder-models/vocos_24khz.onnx

Download Model (Granite-4.2 8B Q4_K_M)

mkdir -p models/granite-4.2
wget -O models/granite-4.2/granite-4.2-8b-Q4_K_M.gguf \
  "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/resolve/main/granite-4.2-8b-Q4_K_M.gguf"

Download Model (nomic-embed-text-v1.5)

mkdir -p models/nomic-embed-text-v1.5
wget -O models/nomic-embed-text-v1.5/nomic-embed-text-v1.5.Q4_K_M.gguf \
  "https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/resolve/main/nomic-embed-text-v1.5.Q4_K_M.gguf"