Show HN: PII-Shield – Log Sanitization Sidecar with JSON Integrity (Go, Entropy)

19 points · aragoss · 1 day ago

What PII-Shield does: It's a K8s sidecar (or CLI tool) that pipes application logs, detects secrets using Shannon entropy (catching unknown keys like "sk-live-..." without predefined patterns), and redacts them deterministically using HMAC.

Why deterministic? So that "pass123" always hashes to the same "[HIDDEN:a1b2c]", allowing QA/Devs to correlate errors without seeing the raw data.

Key features: 1. JSON Integrity: It parses JSON, sanitizes values, and rebuilds it. It guarantees valid JSON output for your SIEM (ELK/Datadog). 2. Entropy Detection: Uses context-aware entropy analysis to catch high-randomness strings. 3. Fail-Open: Designed as a transparent pipe wrapper to preserve app uptime.

The project is open-source (Apache 2.0).

Repo: https://github.com/aragossa/pii-shield Docs: https://pii-shield.gitbook.io/docs/

I'd love your feedback on the entropy/threshold logic!

aragossOP1 day ago
Update: Seeing some folks pulling the Docker image. Just a heads up — the default entropy threshold is 3.8, which is tuned for API keys. If you are testing with simple words like 'test', it might not catch them (by design). Check the README for tweaking PII_ENTROPY_THRESHOLD.
maxbond1 day ago
Cool project!
- Wouldn't this censor UUIDs? I want UUIDs to remain in my logs.
- The never "PII Shield" makes me think this would censor entities like names or social security numbers, rather than secrets. Not a big deal though.
gritspants1 day ago
Thanks for this. Showed some of my colleagues who thought it was cool also.
[deleted]

news.ycombinator.com/item?id=46873308