llama.cpp contributions

llama.cpp is a cross-platform C/C++ inference project with model loading, local/server inference, CPU and GPU backends, and multimodal inputs. The issue is in libmtmd, the layer that prepares media before it enters a model; it is not in a backend kernel or text generation loop.

For audio, llama-server accepts an OpenAI-compatible input_audio file, mtmd_helper decodes WAV/MP3/FLAC through miniaudio into mono F32 PCM, and libmtmd creates an audio bitmap. mtmd_tokenize then selects a model-specific preprocessor and turns the media into multimodal chunks and audio tokens.

Audio input is experimental and the shared tokenizer can select several preprocessors. That matters because a shared boundary check must not silently expand the accepted input domain for every audio-capable model just because one model may pad a very short signal.

The issue report describes a 16 kHz mono WAV with exactly one decoded sample on the Voxtral path. One mono F32 sample is a legitimate four-byte buffer: nx == 1, buf.size() == sizeof(float), and the buffer remains correctly aligned.

The current tokenizer treats nx == 0 as a recoverable empty-input error, but a later check requires buf.size() > sizeof(float). For one sample, 4 > 4 is false, so GGML_ASSERT reaches ggml_abort rather than returning a request error.

The reported CLI exits and the reported server loses /health after one request. Those runtime results are reported upstream, not reproduced locally: my contribution currently confirms the static path and the remaining acceptance gates, not a Windows reproduction or a fix.

The underlying bitmap is not corrupt. Its byte size is nx * sizeof(float), so a one-sample object satisfies the representation and alignment invariants. The fault is that a user-controlled minimum-length question was expressed as an internal, process-terminating memory assertion.

Simply changing > to >= would stop this specific assertion, but it would also send a one-sample signal into every model-specific preprocessor. Whisper-style processing may pad it; Conformer, Parakeet, Granite Speech, Qwen3 TTS, and other paths have different frame, normalisation, or minimum-length semantics. Static evidence does not justify globally accepting it.

The error boundary also has to account for placeholder bitmaps used by /input_tokens. A fix based only on the real buffer could make token-count and normal completion paths disagree for the same sample count.

No source change has been made. The provisional direction is to reject nx < 2 at the shared audio-tokenization boundary with the existing LOG_ERR + return-error contract, before real-buffer and placeholder paths split. That would turn the abnormal input into a recoverable request failure instead of allowing it to reach an abort.

The existing alignment assertion would remain an internal invariant. Normal nx >= 2 audio would still enter exactly the same preprocessor path, while the patch would avoid defining one-sample numerical behaviour across every audio model.

This is a design candidate, not an implementation claim. Before any code, it needs a baseline reproduction, an explicit choice about zero-sample error wording, real-buffer and placeholder checks, and a regression test that fails before the change and passes afterward.