LFM3-Audio in llama.cpp: a speech LLM with real-time support, six hours after release

Liquid AI published LFM3-Audio-2B on 27 October 2026 at 17:00 UTC, and just under seven hours later the real-time endpoint was merged into llama.cpp. Two things are unusual about that. First, voice support landed in the mainline C++ inference engine rather than in a Python wrapper. Second, it happened on the very night of the release. The support number stated on the model card confirms it: GGUF support begins with build b11960.

The model itself is worth a closer look for one reason: it is the first small speech model with official llama.cpp support whose license, hardware footprint and streaming behavior fit a company's own GPU. In return it brings the constraints that come with shipping a model this young.

What LFM3-Audio-2B actually is

LFM3-Audio-2B is a speech-to-speech LLM with two billion parameters. It listens at 16 kilohertz, speaks at 24 kilohertz, carries a 32K context and runs on a single GPU with 4 GB of VRAM. Liquid trains its own audio codec for it, LiquidCodec, and uses a causal streaming encoder on the input side. The practical effect: the model processes audio in chunks, without first having to see the whole utterance. That is what makes low-latency voice at all possible in a local setup. Beyond English, the model understands and speaks 19 other speech locales. In addition it supports tool calling, so a voice agent can call an internal function mid-conversation, look something up and speak the result.

Six hours from model release to engine support

The sequence that same evening, with timestamps in UTC: at 17:00 the weights and the announcement went live. At 17:47 pull request #31257 was merged, delivering format support for the LFM3-Audio GGUFs. Just under seven hours later, at 00:26 on 28 October, pull request #31269 followed with the /v1/realtime WebSocket endpoint. It adds the OpenAI-compatible real-time interface to llama.cpp's own server: a client connects over WebSocket, streams audio in, receives audio out. A merge that adds a whole protocol endpoint also carries review risk. Here the history is visible: 10 commits, 7 resolved and 1 unresolved review thread, and the author states plainly that the file was written entirely with an AI assistant. The follow-up fix for tool calls in the same interface (PR #31287) shows how fast such a young code path still moves.

For comparison, the usual cadence for new architectures in llama.cpp is days to weeks, measured from the moment the first GGUF conversions exist. The size of the change says nothing about quality, but it does say something about scope: the endpoint is not a demo switch, it is a complete session layer.

What works today, and what does not

Whoever wants to build with it today gets a working path. And a list of clear open items, which I would rather name now than discover later.

Working, in llama.cpp build b11960 and later: loading the GGUF with an llama-server instance, audio input over the WebSocket endpoint, streaming output, tool calling, 20 locales. A test client ships in the repository and needs only pip install "openai[realtime]" numpy to run.

Open: throughput. The pull request contains no measured tokens-per-second figures, and Liquid's own model card only says that the model does not excel at complex reasoning. Until someone measures on their own hardware, every latency or tokens-per-second number for LFM3-Audio in llama.cpp that circulates somewhere is a private benchmark. I am not doing that in this post either. Also open: the barge-in behavior of the server implementation. The repository client plays back replies and sends new input in parallel. Whether and how the interruption is handled server-side is an exercise left open in the code. And quantized versions of the audio encoder still cause a warning on load. Whether that costs quality is likewise unmeasured.

One sentence on the license, because it is the question that comes up immediately in companies. LFM3-Audio stands under the LFM Open License v1.0. Section 5 restricts commercial use above a revenue threshold of 10 million US dollars; below it, commercial use is permitted. For any setup at a company of our size that is unproblematic, but the threshold belongs in a compliance file before the model goes into production.

Model sizes measured, not estimated

The GGUF repository currently ships four quantizations of the language model plus projection files (mmproj) for the audio encoder. The following figures are the actual file sizes from the repository, measured on 28 October in gibibytes. Add the mmproj each time, because without it no audio gets through the encoder:

Measured 28.10.2026, source: file sizes in the GGUF repository on Hugging Face.

Where this leaves us relative to the usual alternatives

Three questions decide the choice. Number one: can the model run in-house at all? That is the real shift that LFM3-Audio plus llama.cpp brings. Before, local voice meant a DIY stack of three components (Whisper for input, text LLM, TTS for output), each with its own latency. Now a single model handles both directions, and on the hardware that llama.cpp runs on anyway. Which is to say: any CUDA card, and given the model's size even cards that were built for something else.

Number two: how much latency is acceptable? The official support answers that question technically; only you can answer it economically. A voice assistant that answers in under 800 milliseconds is something else from a voice bot with three seconds of pause. Measured throughput does not exist yet, so the answer is: measure. The test client in the llama.cpp repository is on the machine within ten minutes and is the fastest path to your own numbers.

Number three: who handles the errors? A speech model at the edge makes mistakes differently than a text model. Liquid lists weak complex reasoning on the model card as a known limitation. In practice, that means voice flows with complex branching belong in a text LLM. The right architecture today: LFM3-Audio for input and output, a text model on the same machine for logic and tools.

What other companies should do with this now

I would treat LFM3-Audio in llama.cpp as a pilot object, not as production technology. The support is there, the path is documented, the first bugfixes are landing daily. What's missing are independent measurements and a few weeks of upstream stability. A pilot on one GPU, with a single flow and a clearly defined purpose, answers all three open questions on your own hardware within a week, without committing to anything. Whoever would rather wait: re-check the situation once the first independent benchmarks exist. The license question, by contrast, is already settled today.

Further reading

• llama.cpp, the C++ inference engine: the project that took in the model within hours, the open-source engine behind many of our own AI lanes.
• Our local AI and GPU hardware hub: comparisons and benchmarks of the cards and machines this model runs on.
• GPU memory bandwidth limits inference speed: why a card's data rate, not its name, decides how fast a local model answers.