PII Sanitizer

How PII Sanitizer works

The distinction that makes this tool usable rather than destructive is between a pattern and a detector. Sixteen digits in a row is a pattern; a payment card is sixteen digits that satisfy the Luhn checksum, where every second digit counting from the right is doubled, nine is subtracted from any doubled result above nine, and the total is divisible by ten. That single arithmetic step rejects roughly nine out of ten arbitrary digit runs, which is the difference between flagging a card and flagging every order number, request id and byte count in the file. The same principle applies everywhere there is structure to check: an address is not four numbers separated by dots, it is four numbers none of which exceeds 255.

Some classes are defined by administrative history rather than arithmetic. A US Social Security number has never been issued with an area of 000, 666, or anything from 900 upward, nor with a middle group of 00, nor with a final serial of 0000 — those combinations were reserved or set aside for other identifier programmes. Checking them removes most of what a nine-digit pattern would otherwise sweep up, and one further rule removes most of the rest: nine bare digits with no nearby cue are not treated as a Social Security number at all, because in an operational file they are overwhelmingly likely to be something else entirely.

Most secrets, though, have no shape whatsoever. There is nothing about the string `hunter2` that marks it as a password, and nothing about `abc123` that marks it as a key. What identifies them is the name of the thing they are assigned to. So one detector keys off the identifier rather than the value, recognising the assignment forms configuration is written in — colon-separated, equals-separated, and quoted inside JSON — wherever the name contains a word such as password, secret, token or credential. In practice that one rule catches more genuine leaks than all the shape-based classes put together, because it is how secrets are actually written down.

A fixed list of classes is nevertheless insufficient by construction, and not because the list is too short. Every organisation has identifiers that exist nowhere else: a badge number in a format someone chose in 2011, a customer reference with a checksum nobody documented, a naming convention for internal hosts. No shipped detector can know them, which is why patterns you supply yourself are treated as first-class citizens here rather than as an escape hatch. A rule you write becomes a detector like any other — it competes for contested spans on the same terms, it produces the same numbered tokens, and it appears in the same report — and where it overlaps a built-in, yours takes precedence, on the grounds that you wrote it precisely because the built-in was doing the wrong thing.

Overlap is not a rare corner case, and resolving it is a required pass rather than a refinement. A token in an authorization header is simultaneously a credential and a JSON Web Token; a database URL contains an address, a username and a password inside one string; a phone number written with dashes contains a tail that resembles other things. Detectors run independently and propose candidate spans, then a single sweep sorts them and keeps one winner per region of text: longest first, then by which detector is more informative about what was found. Without that step the replacements are computed against offsets that earlier replacements have already invalidated, and the output is quietly corrupted while the report beside it still looks correct.

How to use this tool

1. 1

   Paste the text, or drop the file

   A log excerpt, a stack trace, an API response, a configuration file. Nothing is uploaded and nothing is stored — the scan runs in the page you are looking at, which is the only reason it is reasonable to paste something sensitive into it.

2. 2

   Choose how values are replaced

   Numbered tokens keep the document readable by giving each distinct value a stable label. Flat labels are the safest. Keyed digests stay consistent between separate files. Partial reveals keep a fragment visible and are the only option that can be partially undone.

3. 3

   Read the per-class counts before you read the output

   Each class shows how many times it fired. A count that looks wrong for the file in front of you — several phone numbers in something that contains none — is a false positive, and finding it here costs nothing compared with finding it after the text has been sent.

4. 4

   Add anything of your own that was missed

   Write a pattern for the identifiers only your organisation uses, give it a label and a token, and it joins the built-in set on equal footing. Rules can be exported as JSON so a team agrees on one set rather than each person maintaining their own.

5. 5

   Add exceptions for what should have been left alone

   Values that must survive go in the exception list, one per line, as a literal, a domain, or an address range. It arrives seeded with the ranges and domains that specifications set aside for documentation, plus the card numbers payment providers publish for testing.

Worked examples

Frequently asked questions