From raw data to structured findings
Behind the public database, Political Prisoner Watch runs a dedicated Python ML microservice — country-specific models served through a REST API — to score case risk, forecast arrest trends, detect coordinated campaigns, and produce statistics that legal teams can cite — across Russia, Belarus, Georgia, and Central Asia. Every score links back to the source records it was computed from.
Every case enters a multi-stage pipeline. Each stage is documented below — expand "How it works" on any stage to inspect the techniques involved.
5,400+
Documented cases
7
Countries tracked
10
Source organizations
25+
API endpoints
The ten-stage pipeline
A case enters at the top, exits at the bottom. Each stage adds structure, evidence, or context that the next one builds on.
Automated data ingestion
The system continuously ingests data from established human rights sources. Sync scripts keep our PostgreSQL database current with the latest arrests, prosecutions, sentencing, and prisoner locations across the regions we cover.
- Primary sources: OVD-Info and Political Prisoners Support. Memorial (Russia), Viasna (Belarus), Political Prisoners of Georgia and Civil Georgia, Open Dialogue Foundation and RFE/RL regional services (Central Asia), Uzbek Forum for Human Rights, Mediazona, and the Committee to Protect Journalists
- Secondary sources: court record aggregators and direct family / legal counsel reports
- Structured database with a normalized schema for prisoners, cases, articles, locations, and outcomes
- Automated sync runs daily to detect new arrests, status changes, and releases; every record keeps its source URL
- Geolocation enrichment for coordinate standardization and region mapping
- Native-to-English machine translation (Russian & Belarusian) with the original text preserved alongside
Named entity recognition
Zero-shot multilingual entity recognition (GLiNER) extracts the legal actors from unstructured case text — judges, prosecutors, lawyers, and courts — enabling statistical analysis of the judicial system itself.
- Zero-shot named-entity recognition using GLiNER (multilingual gliner_multi-v2.1) — extracts judges, prosecutors, courts, and defense counsel from case text in Russian, Belarusian, and English
- Rule-based validation (name morphology, patronymic patterns) filters low-quality extractions before anything is stored
- Per-judge analytics: average and maximum sentence, case count, and deviation from same-country peers
- Statistical fingerprinting per actor: article-by-article sentencing compared against peers using Mann-Whitney U tests — flagged only when the actor has ≥3 cases and peers ≥5
XGBoost risk classification
Our core models use XGBoost gradient-boosted trees to predict two critical probabilities for each case: the urgency level (whether immediate advocacy action is required) and the risk of torture while in custody. Russia and Belarus are trained as separate models so legal patterns do not bleed across borders.
- Gradient-boosted tree classifiers (XGBoost, logistic objective) trained per country, with a global model as fallback
- Inputs: age, gender, arrest location, case category, and criminal articles (multi-label encoded); text-augmented torture model adds TF-IDF features from case summaries
- Class imbalance handled with positive-class weighting; decision thresholds tuned on held-out data rather than assumed at 0.5
- Probabilities calibrated post-hoc (isotonic regression, or Platt scaling on small classes) so a "70%" means 70%
- Every prediction ships with exact TreeSHAP attributions — the per-feature breakdown of why the model scored a case the way it did
- Case-page priority score is a weighted blend: urgency (0.45), torture signal (0.20), sentence severity (0.35)
Surveillance tech attribution
A dedicated classifier predicts the likely surveillance technology used to identify and detain a political prisoner — surfacing patterns in state surveillance infrastructure across regions and case types.
- Every case is labeled with what the prosecution's evidence primarily rests on — online speech, financial support, device evidence, informant testimony, physical presence — with a confidence score and a verbatim source quote, so each label can be checked against the record
- A gradient-boosted classifier trained on labeled cases predicts the likely capture vector for new ones (features: arrest location, case category, arrest year and month)
- Output is a full probability distribution over vectors, not a single guess
- Aggregated by region and over time, this exposes how enforcement actually finds people — e.g. the share of cases resting on online speech alone
Charge inflation detection
This model detects prosecutorial overreach by measuring the gap between what a person actually did (based on case summaries) and the charges filed against them. Distinct severity mappings exist for the Russian and Belarusian Criminal Codes.
- Severity mapping: criminal articles scored on a 1–10 scale, maintained per legal framework (Russian and Belarusian codes)
- A TF-IDF text classifier estimates the probability that the underlying act was non-violent from the case description
- Inflation score = charged severity × P(non-violent act) — high when a severe article is stacked on conduct the record describes as peaceful
- A keyword safeguard caps the non-violent probability when the record itself describes violence
- Example: an anti-war post charged under Art. 207.3 "military fakes" (severity 8) with a clearly non-violent record scores near the top of the scale
Outcome prediction
Given a case's features, this model predicts the likely sentence length. Log-transformed regression guarantees positive predictions and produces a confidence interval, helping legal professionals anticipate outcomes.
- Gradient-boosted regression on log-transformed sentence length (guarantees positive predictions), features: articles, case category, gender, location, age, and charge severity
- Outputs: expected sentence in years with a confidence band, compared against similar documented cases
- A separate Isolation Forest model flags statistical outliers across sentence length, charge severity, age, and charge count — surfacing disproportionate punishments without assuming what "normal" looks like
- Cases whose actual sentences deviate far from the model's expectation are flagged for review as possible political escalation
Prophet trend forecasting
Facebook Prophet time-series models forecast arrest trends up to 90 days into the future — generating isolated models for global trends, Russia-specific trends, and Belarus-specific trends.
- Prophet time-series models on daily arrest counts, per country and per high-use article, with automatic changepoint detection
- A curated calendar of repression events (2022 invasion and censorship laws, mobilization, the 2020 Belarus crackdown, Navalny's death) enters the models as exogenous regressors
- Forecasts are backtested before publication — models that fail an accuracy gate are not served — and confidence bands are widened by conformal calibration
- Repression-wave detection runs separately: 7-day rolling z-scores on locations and articles, alerting when a spike exceeds two standard deviations
Topic & campaign detection
Dual topic modeling identifies thematic clusters across cases — from anti-war speech patterns to religious persecution campaigns. This reveals coordinated repression strategies that are invisible at the level of individual case analysis.
- Two independent topic layers: TF-IDF + non-negative matrix factorization for fast monthly tracking, and BERTopic (MiniLM sentence embeddings + UMAP + HDBSCAN) for deeper semantic clusters
- Prosecution-template detection embeds every case summary with sentence-transformers (all-MiniLM-L6-v2) and clusters summaries whose cosine similarity exceeds 0.85 — surfacing near copy-paste indictments
- Template clusters are strong evidence of assembly-line prosecution: distinct defendants, near-identical case language
- Monthly topic tracking shows how the focus of repression shifts over time (e.g. war-censorship charges surging after February 2022)
Case network analysis
We build similarity networks linking cases by shared charges, location, timing, and tactics. Community detection algorithms identify clusters that reveal coordinated repression campaigns — groups of people targeted simultaneously.
- Case-similarity graphs built per country so domestic patterns stay distinct
- Edges use a transparent, fixed-weight composite: shared criminal articles (40%, Jaccard), same case category (20%), same location (20%), temporal proximity (20%) — a deliberate design choice: every edge is explainable, none depend on a black box
- Edges below a 0.3 similarity floor are dropped; communities are found with greedy modularity maximization (networkx)
- Each community is profiled — dominant charges, geographic center, time span — and labeled for the case-network explorer
- Clusters are leads for investigating coordinated campaigns, not proof of coordination on their own
Legal evidence & asylum tools
Our most impactful outputs: statistical persecution evidence, individual risk scores for asylum seekers, comparative case finding, and generative document drafting — designed to support real legal proceedings.
- Persecution Evidence Score: for any filter (category, location, article), aggregate statistics with Wilson confidence intervals on torture rates and regression-tested temporal trends — numbers a filing can cite with the math shown
- Sentencing-disparity analysis: minority vs. majority-region sentence distributions compared with Mann-Whitney U tests, reported only where p < 0.05
- Personal Risk Score: a transparent weighted heuristic (documented case density matching the applicant's profile, adjusted by torture rates and trend direction) — explicitly a structured estimate, not a trained model
- Comparative Case Finder: weighted multi-factor search for the closest documented precedents
- Report generation: statistical sections are computed directly from the database; where a language model drafts connecting prose, the underlying numbers are injected verbatim and the draft is reviewed before use
System architecture
Four discrete stages. Data flows from sources at the top to the public platform at the bottom; each stage can be replaced or scaled independently.
Layer 01 / 04
Data sources
Layer 02 / 04
Ingestion & verification
Layer 03 / 04
Analysis & models
Layer 04 / 04
Public platform
Data flows one way — from source documentation to public analysis. Scores never overwrite source records.
A note on predictive models
Probabilistic, not deterministic
Risk scores and forecasts are statistical estimates derived from historical data — not predictions of what will happen to any specific person.
Reporting bias
Our data measures what human rights monitors could document, not everything that happened. A torture score, for example, reflects whether torture was reported in a case — regions with weak reporting will look safer than they are. Where we can, we say so explicitly.
Aid, never replacement
The pipeline aids researchers and legal professionals in prioritization. It does not replace human judgment, attorney review, or the lived expertise of families and witnesses.
Continuously retrained
Every model is retrained as new data arrives. A "high risk" classification reflects a statistical resemblance to past cases — outcomes for an individual may differ.
Classification confidence scores
Where a case label carries a confidence score (e.g. “conf 0.85”), the number is the extraction model’s 0–1 self-assessment of how strongly the cited verbatim evidence supports that label — it is not a measure of the underlying facts’ reliability. Read ≥ 0.8 as the quoted evidence stating the basis explicitly; 0.6–0.8 as the label being inferred from strong context; below 0.6 as suggestive only — verify against the linked source before relying on the label. Every scored label links the verbatim quotation and the source URL it rests on, so the score never has to be taken on trust.