NVIDIA Nemotron 3.5 Bolt E4B: The 4GB Open Model for Local AI Agents

NVIDIA just released Nemotron 3.5 Bolt E4B 4GB, its smallest open model yet — built so a conversational, tool-using AI agent can run locally on a robot, a handheld, or an entry-level GPU. It fits agent behavior into roughly 4GB of memory while still calling tools across multiple turns. This guide explains what Bolt E4B is, how it performs, what hardware it needs, and how to run it, so you can decide whether it belongs in your edge stack.

What is Nemotron 3.5 Bolt E4B?

Nemotron 3.5 Bolt E4B is a small, open language model for fast local conversational agents. NVIDIA describes it as its latest open small language model optimized for speed and intelligence per gigabyte in conversational agents. It targets three workloads: device and voice assistants, interactive characters, and embedded or industrial agents.

The key detail is its size. The model has 8 billion total parameters but only 4 billion effective, with a 64K context window and text-only input and output. The "4GB" in the name refers to the memory the quantized build targets in its intended workloads — not the download, which is larger. NVIDIA ships it under the OpenMDW-1.1 license, and the release includes open weights, data, and training recipes.

Bolt E4B isn't trained from scratch. It builds on Google's Gemma 4 E4B with NVIDIA's Nemotron post-training — the "E" denotes effective parameters, so the model carries 8B weights but activates roughly 4B per token.

It arrived on October 23, 2026, the last of three open Nemotron 3.5 models NVIDIA rolled out through the month, alongside the 120B Nemotron 3.5 Super VL vision-language model and the 24GB Bolt 74B-A9B. Where Super VL targets data-center agents, Bolt E4B sits at the opposite end: small enough for robotics controllers and embedded devices.

What Bolt E4B is built to do

NVIDIA designed Bolt E4B around how conversational agents actually behave, not around topping a coding leaderboard. Its headline features:

• Multi-turn tool use — it calls application APIs to get things done across a conversation, with guidance for asynchronous tool calls.
• Conversational mode — a toggle that encourages concise, speech-ready replies for natural back-and-forth.
• Configurable reasoning — turn reasoning on for hard requests or off for faster responses, with options to cap "thinking" to hit latency targets.
• 64K context — enough for conversation history, instructions, tool definitions, retrieved data, and app state.
• Multilingual — support for 35 languages, on a base model pre-trained on more than 140.

The target experiences follow from that: local voice and device assistants (PC, mobile, in-car), interactive characters like game NPCs and companions, and human-machine interfaces for robotics and industrial IoT.

NVIDIA's launch demo makes the edge pitch concrete. A Reachy Mini robot runs on a Jetson Orin Nano 8GB, with Bolt E4B handling conversation and tool calls while Cosmos 3 Edge supplies real-time vision. In a second demo, a blacksmith NPC verifies payment for an item, then autonomously calls tools to check recipes, collect ingredients, craft the item, and hand it over — the kind of interleaved dialogue-plus-action loop these small agents are meant for.

Nemotron 3.5 Bolt E4B benchmarks

NVIDIA groups its evaluations into five capabilities and compares Bolt E4B against similarly sized models: its own Nemotron 3 Nano 4B, Qwen 3.5 4B, and the base Gemma 4 E4B. All scores below are NVIDIA-reported (BF16), so treat them as vendor numbers until third parties reproduce them.

On conversation and tool use, Bolt E4B generally leads its size class:

It doesn't win everything — Qwen 3.5 4B leads on Tau2-Bench (75.1 vs 55.1) and some long-context retrieval, where RULER at 64K favors Qwen (95.1) and Nano (91.5) over Bolt (87.0). NVIDIA's own summary is that Bolt E4B scores roughly 40% higher on average than Nemotron 3 Nano 4B and about 10% higher than other peers it evaluated, while using up to 80% fewer output tokens in reasoning-enabled comparisons. Fewer tokens means less decoding work, lower cost, and often faster responses.

One third-party signal appeared at launch: per an X post from NVIDIA, Scale Labs ranked Bolt first among small models on MultiChallenge at 56.03%, about 12.8 points clear of Gemma 4 E4B — directionally consistent with NVIDIA's own MultiChallenge figure.

How much memory and how fast?

This is where the "4GB" claim gets grounded. NVIDIA measured the Q4_K_M build at batch size 1 on llama.cpp across representative hardware:

So on a desktop RTX card the model answers in about half a second and streams well over 100 tokens/sec in ~3.5GB. On an 8GB Jetson Orin Nano, it still fits — but expect a multi-second first token and roughly 15 tokens/sec, which is why NVIDIA pairs it with a lightweight vision model rather than stacking heavier models on the same board.

Two optimizations help close the gap:

• DFlash parallel drafter — a speculative-decoding drafter that proposes draft tokens in one forward pass; NVIDIA reports up to 21% higher throughput than Gemma 4 E4B's original drafter.
• Q4_K_M with imatrix calibration — NVIDIA quantized the BF16 checkpoint using about 1M tokens of RL rollouts from the same training step, keeping quality close to the full-precision model.

How was Bolt E4B trained?

Bolt E4B is a post-training story. NVIDIA started from Gemma4-E4B-it and applied Nemotron post-training across environments for instruction following, multi-turn tool calling, and conversation. The method is reinforcement learning only — RL with verifiable rewards (RLVR) and from human feedback (RLHF), run on NeMo-RL and NeMo-Gym.

To keep the model fast rather than chatty, NVIDIA emphasized non-reasoning accuracy, kept a 2:1 ratio of non-reasoning to reasoning samples per batch, and added length penalties to curb verbosity as training pushed answers longer. The full recipe and datasets are published alongside the weights.

How to run Nemotron 3.5 Bolt E4B

NVIDIA ships multiple checkpoints — a Q4_K_M GGUF for deployment, a DFlash drafter, and a BF16 HF checkpoint — and lists several ways to get started: Hugging Face, Ollama, LM Studio, Unsloth, Canonical, Jetson AI Lab, and the NVIDIA In-Game Inferencing SDK.

The simplest local paths:

1. Desktop (RTX). Pull the GGUF through Ollama or LM Studio and run it like any other local model — expect sub-second responses on a modern RTX card.
2. Jetson / edge. Use the Jetson AI Lab guide, which runs an llama.cpp container that downloads Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf and serves an OpenAI-compatible API on port 8000.
3. App integration. Because the server speaks the OpenAI API, you can point existing agent code at it by swapping the base URL — the model card documents the enable_thinking and conversational_mode toggles.

A minimal call against a running llama.cpp server is a standard chat-completions request to /v1/chat/completions with the model id nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB.

Should you use it?

Bolt E4B is a strong fit if you're building a voice assistant, game NPC, or robot that has to run locally, respond fast, and leave memory headroom for speech or vision models on the same device. It's the right tool when privacy, latency, or no-internet operation rule out a cloud model.

It's less compelling if you need deep reasoning, long-context retrieval, or agentic coding — its own siblings (Bolt 74B-A9B for multistep work, Super VL for documents and vision) or a frontier model will serve those better. And like the rest of the Nemotron family, the smooth path runs through NVIDIA hardware.

For how this compares with other recent small and open-weight launches, see our breakdowns of Muse Spark Open and Reflection Beam.

Build a local agent team on your own hardware

Bolt E4B shows where edge AI is heading: capable, tool-using agents that run on the device in front of you. Eigent is an open-source Cowork desktop app that orchestrates a team of AI agents locally, so you can wire local models and your own tools into real multi-step workflows instead of gluing scripts together. Explore what you can build with Eigent, or download Eigent and start building your own agent team.