Nemotron 3.5 Embed, Rerank, Parse: Nvidia's open RAG models from 1B

On 21 October 2026, at GTC Berlin, Nvidia presented three open models for retrieval pipelines, as The Register reports: an embedding model, a reranker and a document parser. The weights are on Hugging Face, and the model cards give 20 October as the release date. If you run a RAG system yourself, this is an alternative to Qwen3-Embedding and bge-reranker. All benchmark numbers, however, come from the vendor. We found no independent measurement when we searched.

The classification in this post uses three labels: verified (read on the model card or in two sources), inferred (calculated by us from the published figures) and vendor-claimed (a number from Nvidia without independent confirmation).

The three models and their sizes

Embed comes in 8B and 1B, Rerank only in 8B, and Parse is a model of its own. Verified against the model cards:

• Embed 8B has 8.38 billion parameters and produces vectors with 4096 dimensions.
• Embed 1B has 1.55 billion parameters, more than its name suggests, and produces 2048 dimensions. BF16 and NVFP4 versions exist.
• Both Embed models handle a context of 32,768 tokens. Queries are plain text, documents can be text, images or both.
• Parse has 630,800,496 parameters according to its card, about 0.631 billion. The Register says "one billion", which is rounded; unite.ai says 631 million, which matches the card.

The vectors can be shortened: the card describes truncating them at the end (then re-normalising). That saves space in the vector index. The card says truncated vectors remain effective for retrieval (vendor claim); our sources give no figures.

The Register also writes that the models support embeddings for images and text in 34 languages.

Vendor figures on quality

All values below are vendor claims, as relayed by unite.ai on 21 October 2026.

More vendor claims from the same source:

• The 8B reranker scores 60.49 on a combined leaderboard, Qwen3-VL-Reranker-8B scores 59.17. That is a gap of 1.3 points.
• With a reranker in front, token cost per query in an agent setup drops by about 14 percent (Embed 8B) and 13 percent (Embed 1B), with better accuracy. LLM, tools, tasks and corpus were held constant.
• Parse reaches 75.20 on ParseBench at 10.5 pages per second on an H100 (three-run average), or 72.74 at 16.1 pages per second (single run). The card gives slightly different throughput values (10.62 and 15.42 pages per second on 2,078 pages).
• The NVFP4 version of Embed 1B is said to deliver 27.1 percent more throughput on GB200 while keeping 99.85 percent of the accuracy. NVFP4 targets Blackwell cards.

According to a web search summary, Vectorize, Perplexity and Turbopuffer report gains of their own. We did not read those announcements at the source, and all three companies have an interest in the outcome.

License and operation

The models use the OpenMDW License 1.1. The base models under Embed and Rerank are Apache 2.0; Embed, Rerank and Parse themselves are not. The card explicitly allows commercial use, though the NVFP4 repository on Hugging Face only says license:other. We did not check the license text.

For serving, the card pins vllm==0.32.0 and transformers>=5.16.1,<5.20.0. The reranker has to run through the /classify endpoint. The card warns that /score and /rerank can mishandle this checkpoint's image placeholders.

What a switch would mean in our stack

On 24 October 2026 we took a read-only look at what our RAG stack uses today. What we measured:

• Embedding runs on a small GPU host with an RTX 4060 Ti (16 GB) in vLLM 0.30.0. The model is Qwen3-Embedding-0.6B with 32,768 tokens of context, and a test query returned 1024 dimensions.
• The reranker is bge-reranker-v2-m3 in its own container on the same host. Together the two containers use 11,925 of 16,380 MiB.
• LibreChat's RAG service talks to our LiteLLM using the model name embed. The vectors sit in a Postgres database. How the Qdrant collections behind the knowledge MCP embed their content, we did not check.

Inferred from that, without a measurement of our own:

• The dimension grows from 1024 to 2048 or 4096. That means a new table or collection and a complete re-embed.
• The model card requires vLLM 0.32.0, we run 0.30.0. Nemotron would need its own container with its own image.
• Embed 8B in BF16 comes to roughly 16.8 GB of weights on paper and does not fit on a 16 GB card. Embed 1B would be about 3.1 GB. The NVFP4 version targets Blackwell, and the 4060 Ti is an Ada card.
• A rerank call in the usual Cohere format does not match the Nemotron card, which requires /classify.

We have not compared Nemotron with Qwen3-Embedding ourselves. Whether a switch pays off is an open question.

How to test a switch

1. Write down what the stack embeds with today: model name, dimension, maximum length.
2. Collect 50 to 100 questions from your own documents with known answers, and compare the old and the new model in two separate collections, for example by recall@5.
3. Start the new model in its own container with vLLM 0.32.0, apart from the production image, and read the OpenMDW license text first.

Only after that comparison should you move the production index.

Further sources

• The Register: Nvidia woos wary Europeans with more open-weights models: announcement at GTC Berlin, 21 October 2026
• unite.ai: benchmark figures for Embed, Rerank and Parse: vendor claims, 21 October 2026
• Hugging Face: Nemotron-3.5-Embed-8B-BF16: model card, dimensions, vector truncation
• Hugging Face: Nemotron-3.5-Embed-1B-BF16: model card with 1.55 billion parameters
• Hugging Face: Nemotron-3.5-Rerank-8B-BF16: license, vLLM pin, note on /classify
• Hugging Face: Nemotron-3.5-Parse: parameter count and throughput
• Simple Icons: nvidia.svg: logo, CC0
• Videos: at the time of writing we found no video from Nvidia or an established channel that covers exactly these three version 3.5 models, so none is embedded.