Tokens and Context Windows
Large language models bill, truncate, and sometimes fail requests based on token count—not character count. A "128k context" model still has practical limits: output reservation, system prompt overhead, and provider-side safety buffers shrink usable space. Underestimating tokens causes mid-request cutoffs where the model never sees your final instructions.
token-counter estimates how many tokens a prompt consumes for common models. context-window-checker compares that estimate against published context limits so you know whether to trim history or choose a larger model tier. English text uses byte-pair encoding (BPE); token boundaries rarely align with words. Chinese and mixed CJK content often averages one to two tokens per character—never assume len(text) equals token count.
Practical scenario: you paste a 40-page PDF into a support bot. token-counter reports 180,000 tokens; context-window-checker flags overflow for a 128k model. Workflow: split the document with rag-chunk-analyzer, retrieve only relevant sections, re-count, then send. Common mistakes include stuffing entire wiki exports "because the model is big," ignoring tokenizer differences between GPT and Claude families, and forgetting that tool definitions and JSON schema also consume tokens.
Prompt Structure
Well-structured prompts separate concerns. The system message sets role, tone, safety rules, and output constraints. The user message states the concrete task with examples. Prior assistant turns carry conversation history—each turn re-sent on every API call in stateless integrations.
prompt-template stores reusable patterns—support triage, SQL generation, release note drafting—with placeholder slots for variables. system-prompt-builder assembles policy-heavy system blocks: PII handling, refusal criteria, citation requirements. Specify output format explicitly: "Return valid JSON matching this schema" with field types reduces parse failures compared to "respond in JSON."
Step-by-step refinement loop: draft system + user messages → token-counter → test on five representative inputs → measure parse success rate → tighten instructions where the model drifts → document winning prompts in prompt-template for the team. Real example: adding "Do not invent product SKUs; use UNKNOWN if missing" cut hallucinated inventory IDs in ERP queries by ninety percent in internal tests.
Cost and Quality
Longer prompts cost more and can degrade answer quality—the "lost in the middle" phenomenon where models overlook facts buried in centermost context. llm-pricing-calculator estimates spend per million tokens for input and output, helping you choose between a cheaper model for classification and a premium model for final synthesis.
context-budget-planner splits retrieval snippets versus instructions versus reserved output tokens. RAG pipelines should use rag-chunk-analyzer to verify chunk sizes: too small loses semantics; too large wastes budget and dilutes relevance. Curate ten high-signal paragraphs over dumping one hundred mediocre pages.
Quality checklist: cite sources when factual, temperature low for extraction tasks, higher for brainstorming only with human review, and always validate structured output with structured-output-validator before writing to databases. Commercial API calls upload prompt content to providers—redact customer names, account numbers, and unreleased financials even when counting tokens locally in Towalles.
Privacy
Prompts become training-adjacent telemetry unless your enterprise agreement opts out. Treat every API call as potential data disclosure. Towalles token-counter and context-window-checker run in the browser, so sizing drafts does not upload them for computation—but clicking "send" on OpenAI or Anthropic still does.
Use local models (Ollama, on-prem inference) for highly sensitive workflows. Apply pii-scanner to prompts before external calls. Establish team policy: synthetic fixtures in shared prompt-template libraries, no production database dumps in chat, and immediate incident review if someone pastes a credential rotation file into a cloud model thread. Local-first tooling reduces accidental exposure; it does not replace data classification and access controls.