AtomicoPIIFilter 1.0: a 70M on-device model that beats Grok on PII spans
A bilingual DeBERTa-v3-xsmall encoder hits exact-span F1 0.897 English and 0.939 Spanish — above Grok 4.5 High on the same evals. The English-only predecessor did not clear the bar. Bilingual data did.
Prefer the full version? Download the whitepaper (PDF). Methodology, per-type tables, hybrid decode, ONNX parity, and the exact claim bounds are there. HTML: WP-2026-02.
The claim
On our held-out exact-span PII evals, AtomicoPIIFilter 1.0 exceeds Grok 4.5 High:
| Eval | Grok 4.5 High | AtomicoPIIFilter 1.0 |
|---|---|---|
| English (n=400) | 0.893 | 0.897 |
| Spanish (n=300) | 0.895 | 0.939 |
The model is a ~70M DeBERTa-v3-xsmall token classifier, not a hosted LLM. It runs as a BIO tagger plus a regex union (email, phone, IPv4, IBAN, Luhn cards).
That is a real result. It is also a bounded one: in-house evals, exact character spans, PERSON still hard. The whitepaper states the bound in section 7.
Why this architecture
We ran a size ladder first. Generative specialists failed to clear Grok:
- SmolLM2-135M: F1 0.504
- Qwen2.5-0.5B: F1 0.871
- English-only encoder: F1 0.865 — close to 0.5B, still under 0.893
The encoder was the right shape (no JSON, no sampling, 512-token pass). English-only weights were not enough. We froze that checkpoint as an immutable fallback, then trained a new bilingual run.
What bilingual training changed
English did not regress. It went up (0.865 → 0.897), which also cleared the gate of frozen − 0.01. Spanish landed at 0.939 against a 0.80 target and a 0.895 Grok bar.
The Spanish lift is not “it finally learned names.” PERSON is essentially tied with Grok (0.782 vs 0.786). The gain is recall on localized structure — ADDRESS and DATE_DOB — and fewer misses overall (96 FN vs Grok’s 206).
PERSON is still the problem
On English, Grok still leads PERSON (0.855 vs 0.845). ADDRESS recall is the biggest leak (FN 138). CREDIT_CARD and IBAN have too little gold to rank models; those types are why the regex/Luhn union exists.
On-device status
- ONNX: 50/50 span parity vs PyTorch on a 50-example English slice.
- Core ML: blocked. DeBERTa relative attention hits
sqrt(int32). We are not pretending that export worked. - iOS harness: regex hybrid + BIO decode + highlight/substitute UI, with 20 EN + 20 ES bundled examples.
Weights stay in the lab repo. The product path is ONNX Runtime Mobile until we ablate a DistilBERT backbone for Core ML.
Read the paper
Full protocol, tables, and caveats: AtomicoPIIFilter 1.0 whitepaper (WP-2026-02).
Questions: c@atomicolabs.com.
