Control AI Crawlers with robots.txt: Block vs. Allow (2026)
Many sites block GPTBot in one line and call it done. I built a robots.txt that separates training, search, and fetch crawlers, then verified it with a parser.
archive
Columns · Page 6
356 · Page 6
Lighthouse Accessibility 55 to 100: Fixing WCAG Failures
A demo bakery page scored 55 on a Lighthouse accessibility audit. Here's the log of fixing six WCAG failures to reach 100, plus a keyboard trap the tool missed.
8 Agents, One Local LLM — Ollama Concurrency Measured
I fired 8 agents at one local model expecting a speedup. Default Ollama serializes requests, so eight at once matched one. I benchmarked OLLAMA_NUM_PARALLEL.
LocalBusiness JSON-LD: Server-Side Beats JS Injection
Inject LocalBusiness JSON-LD with JavaScript and the raw HTML has zero ld+json blocks. I compare it with server-side output, Google's stance and ranking limits.
Local Reasoning Model Token Cost — thinking ON/OFF Measured
I ran 13 questions on gemma4:12b with thinking ON and OFF. Reasoning got one more right while spending 68x the output tokens and 19x the wall-clock.
Ollama num_ctx Silent Truncation — Why My Agent Forgot Its Prompt
My local agent kept ignoring its system prompt on long inputs. Past num_ctx, Ollama silently trims the front of the prompt — no error. I measured where it breaks.
Local LLM Cold Starts — Why First Replies Take 10 Seconds
After idling, my agent's first reply dragged. I pulled Ollama's load_duration across model sizes: 1.5s for 2GB up to 9.7s for 9.6GB, and split it by keep_alive.
Why Local LLMs Slow Down in Long Chats — Prefill vs Generation
A 9,700-token prompt took 55s to its first token, then 65ms on the identical second call. I split Ollama's timings into prefill vs generation to see why.
The Non-English Token Tax — Korean Costs 1.4x, Measured
I tokenized 285 of my posts across ko/ja/en/zh with three tokenizers. Korean ran 1.38x English tokens, Japanese 1.34x. The non-English token tax, measured.
Local LLM Reproducibility: Testing Temperature and Seed
I sent the same prompt to local Gemma 4 dozens of times. temperature=0 was deterministic, and even at higher temperature a fixed seed collapsed output to one line.
Stop Feeding Raw JSON to LLMs — 9 Data Formats Measured
I serialized 50 records into 9 formats (JSON, YAML, CSV, TSV, XML...) and counted tokens with tiktoken. For flat data, TSV ran 62% cheaper than pretty JSON.
Building a TypeScript MCP Client — @modelcontextprotocol/sdk v1.29
I built a TypeScript MCP client with @modelcontextprotocol/sdk v1.29.0: calling server tools and reading resources programmatically, without Claude Desktop.