System Prompts, Agent Memory & Guardrails: LLM Ops

Prompt work fails in the unglamorous layers: versioning, context budgets, PII review, and orchestration diagrams. These five tools cover that LLM ops loop: System Prompt Compressor, Prompt Diff Comparator, AI Agent Memory & Buffer Simulator, LLM Guardrails & PII Redactor, and Agent Tool Flowchart Builder.

Trim tokens with deterministic rules, not another model call

compressSystemPrompt applies conservative, fully client-side rules: strip common filler phrases, normalize whitespace and bullet styles, deduplicate repeated sentences, and apply a safe abbreviation list — then reports before/after character counts with approximate tokens at chars ÷ 4. No LLM call and no network, so results are reproducible and your prompt never leaves the browser. The estimate is explicitly approximate, which keeps you from treating a 12% saving as exact billing.

const result = compressSystemPrompt(systemPrompt);
// paste back result.text, keep the rule log for review

Compress first, then re-run your evals — trimming filler can shift instruction-following, and the diff comparator below is how you check that nothing semantic was lost.

Diff versions and score A/B candidates side by side

diffPromptLines computes line-level LCS changes so added, removed, and equal lines render separately from noise. extractPlaceholders and comparePlaceholders list template variables per version and flag renames that would break your fill logic. analyzePrompt adds instruction counts and approximate tokens, while validateABMatrix checks a filled A/B output matrix of expected outputs per candidate — all without sending data anywhere.

Gate every prompt edit on a placeholder comparison: a renamed user_id fails silently at runtime, while a reworded instruction usually just degrades gracefully.

Simulate memory budgets and redact PII before shipping

simulateAgentMemory replays a message list through three strategies — last-N sliding window, compact summary placeholder, and keyword-scored top-k vector retrieval — with token costs at chars ÷ 4. compareMemoryStrategies runs all three against a sample query side by side, exposing when a summary buffer drops the one fact the task needs. Size the window from real transcripts rather than round numbers: N=8 keeps chit-chat alive but can starve long tool traces.

scanGuardrails runs deterministic local detectors — email, phone, T.C. identity, Luhn-validated credit cards, Turkish IBAN, IPv4, SSN — plus custom regexes, each with mask, hash, or remove strategies. exportGuardrailRules serializes the set as versioned JSON for review and importGuardrailRules restores it. Test redaction with adversarial spacing (4111 1111 1111 1111 versus 4111-1111-1111-1111) before trusting any single rule in production.

Diagram the tool-call graph before you code it

buildAgentToolFlowchart models start, tool (with parameter templates), decision (two labeled exits), and end nodes as a directed graph. validateFlowchart flags unreachable nodes and decisions missing an exit; detectCycles separates accidental loops from intentional retries by requiring a max-iterations annotation on loop edges. toMermaid then emits a paste-ready flowchart for documentation, for example a search step feeding a found-decision with a retry edge capped at 3 iterations back into search.

Try Them

• System Prompt Compressor — rule-based token trimming with a reviewable rule log.
• Prompt Diff Comparator — version diffs, placeholder checks, and A/B matrices.
• AI Agent Memory & Buffer Simulator — window, summary, and vector memory comparison.
• LLM Guardrails & PII Redactor — TCKN, Luhn, IBAN, and custom regex redaction.
• Agent Tool Flowchart Builder — tool-call DAGs with validation and Mermaid export.