Commit Graph
4 Commits
Author SHA1 Message Date
Максименко Никита ВладимировичandClaude Sonnet 5 7dd66d45bd feat: expandable denylist + hot-reload, fix declension for -a surnames
Adds current public figures (Nabiullina, Gref, Putin, ...) so news-
style mentions near CB/banking topics aren't treated as client PII.

More importantly, isWellKnown() only worked for consonant-ending
surnames: "Пушкина" starts with "Пушкин" so it matched, but "Толстого"
doesn't start with "Толстой" (adjectival declension replaces the
ending, doesn't extend it), and neither does any oblique case of a
feminine -a surname ("Набиуллиной" vs "Набиуллина"). Strips the
trailing vowel at load time, same trick already used for given names.

Also lets the list grow without a rebuild: an optional external file
(pdguard.well-known-file, default config/well-known.txt) is merged on
top of the bundled list and re-read on change, mirroring how
SystemsConfig watches systems.json.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-21 21:29:57 +03:00
Максименко Никита ВладимировичandClaude Sonnet 5 f7eee7577c feat: validate ADDRESS_CITY against a real city dictionary
The rule matched any capitalized word after "г."/"город" — no check
that it's an actual place. ToponymDictionary checks the match against
1111 Russian cities (pensnarik/russian-cities) plus CIS capitals,
matching by stem so declined forms work ("Москве" against "Москва").

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-21 21:29:57 +03:00
Максименко Никита ВладимировичandClaude Sonnet 5 a758c5ef05 test: independent generated benchmark + tests on large mixed text
benchmark-generated.txt: a fresh dataset (not used to tune the rules)
covering every PD type from the spec plus format variations — date
order, "серия ... номер ...", register-independence. Surfaced two real
gaps worth tracking: FIO detection needs a capitalized first letter,
so "клиент иванова мария петровна" (all lowercase) isn't found; the
NER cascade flags plain capitalized nouns ("Портрет", "Рим") as names
when the base rules alone don't.

LargeTextTest: the existing large-text test just repeated one
email+card sentence 4000 times, so 27 of 28 PD types never ran at
scale. Replaces it with ~400,000 characters built from shuffled lines
of the new dataset — recall on it holds at 0.970, matching small-scale
numbers, and the NER cascade stays sub-second on the whole thing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-21 21:29:57 +03:00
Максименко Никита ВладимировичandClaude Sonnet 5 832738891c feat: self-tuning concurrency limit instead of a fixed Semaphore
Under 0.5-CPU containers, a static Semaphore(2000) never tripped —
latency ballooned to 1.5-2s instead of the service answering 429.
Runtime.availableProcessors() can't help pick a number either: it
ignores the cgroups --cpus quota and reports full host cores.

AdaptiveConcurrencyLimiter reacts to observed latency instead of
guessing capacity: starts at min-concurrent, grows by one per
adjustment window when latency stays under target, halves it the
moment it doesn't. Adjustment is gated by wall-clock time, not by
request count — an earlier per-request version let the limit race to
the ceiling in milliseconds under high RPS, before any real overload
had a chance to show up in the samples.

Verified under load (native image, 250MB/0.5 CPU): p50 latency at 3x
overload dropped from ~1.3s to under 4ms; normal-load p95 unaffected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-21 21:29:57 +03:00