Secrets, Hallucinations, Budgets, Fakes & Bashisms
September 5, 2026 · DevTools
Leaked keys, confident-sounding nonsense, surprise invoices, flaky fixtures, and scripts that only run on one machine are five flavors of preventable regret. This guide walks through the Offline Secret Leak Scanner, Hallucination Entropy Analyzer, LLM Cost Forecaster, Synthetic Data Faker, and Bash POSIX Portability Checker — all running locally in your browser.
Catch keys before they reach the remote
scanSecretLeaks takes named files plus options and returns per-file findings with rule names, line numbers, masked matches, and entropy scores. Eight built-in rules cover the usual suspects:
| Rule | Pattern shape |
|---|---|
| AWS access key | AKIA plus 16 uppercase alphanumerics |
| GitHub token | ghp_, gho_, ghu_, ghs_, or ghr_ prefix |
| Google API key | AIza prefix plus 35 characters |
| Slack token | xoxb, xoxa, xoxp, xoxr, or xoxs prefix |
| OpenAI API key | sk- prefix plus 20 or more characters |
| Stripe secret key | sk_live_ prefix plus 16 or more characters |
| JWT | eyJ with three base64url segments |
| Private key | -----BEGIN ... PRIVATE KEY----- banner |
Beyond patterns, assignments with secret-like keys and values of at least 12 characters that meet the default entropy threshold of 4 are flagged even when no named rule fires. Matches are masked, so reports are safe to screenshot — treat every hit as guilty until proven to be a fixture.
Put a number on model uncertainty
When a model emits a token with 99 percent probability it is committed; when the top candidates split 30/30/25 it is guessing. The Hallucination Entropy Analyzer turns that intuition into arithmetic: analyzeTokenDistributions consumes top-k probability distributions and reports per-token Shannon entropy, perplexity (2 raised to the entropy), and surprise, plus cumulative uncertainty across the whole span.
The scenarioDistributions presets ("confident", "uncertain", "uniform") calibrate your eye: compare their entropy columns to learn what healthy output looks like. Then the dangerous case stands out — low entropy on a wrong answer, the confident hallucination no sampling trick fixes. Flag spans where entropy spikes mid-answer and require retrieval, citations, or refusal there.
Price the workload, seed the fixtures, port the script
forecastLlmCosts prices one workload across five models (price snapshot 2025-02-14): GPT-4o mini at $0.15/$0.60, GPT-4o at $2.50/$10.00, Claude 3.5 Sonnet at $3.00/$15.00, Gemini 1.5 Pro at $1.25/$5.00, and DeepSeek V3 at $0.27/$1.10 per million input/output tokens. Supply daily requests, average token counts, cache hit rate, and a monthly budget; cached tokens take the effective discount (default 90 percent) and over-budget models are flagged budgetExceeded. A CSV export carries the comparison into budget reviews.
Deterministic fixtures come from generateSyntheticData: a mulberry32 seeded generator (default seed 42, default count 10) fills schema fields with names, example.com emails, UTC dates, UUIDs, IPv4 addresses, integers, and text, and formatSyntheticData serializes to JSON, JSONL, or CSV — same seed, same dataset, every machine.
Finally, checkBashPosixPortability flags bashisms with line numbers and portable rewrites. Bash-only first, portable second:
[[ $x == foo ]] && function greet { echo hi; }
source ./env.sh
[ "$x" = foo ] && greet() { echo hi; }
. ./env.sh
Double-bracket conditionals become single brackets joined with && or ||, the function keyword disappears, and source becomes the dot command.
Try Them
- Offline Secret Leak Scanner — gitleaks-style rules plus entropy scoring.
- Hallucination Entropy Analyzer — entropy, perplexity, and surprise per token.
- LLM Cost Forecaster — monthly budgets with cache discounts and CSV export.
- Synthetic Data Faker — seeded datasets in JSON, JSONL, and CSV.
- Bash POSIX Portability Checker — bashism detection with portable fixes.