Ollama num_ctx Silent Truncation — Why My Agent Forgot Its Prompt
My local agent kept ignoring its system prompt on long inputs. Past num_ctx, Ollama silently trims the front of the prompt — no error. I measured where it breaks.
Tags
AgentDesign
3 posts
Local LLM Cold Starts — Why First Replies Take 10 Seconds
After idling, my agent's first reply dragged. I pulled Ollama's load_duration across model sizes: 1.5s for 2GB up to 9.7s for 9.6GB, and split it by keep_alive.
Why Local LLMs Slow Down in Long Chats — Prefill vs Generation
A 9,700-token prompt took 55s to its first token, then 65ms on the identical second call. I split Ollama's timings into prefill vs generation to see why.