A global registry of 787+ data privacy enforcement cases across 7 jurisdictions, totaling $532M+ in fines. Browse cases, compare enforcement actions side-by-side, explore jurisdictions on an interactive map, and learn what privacy enforcement terms actually mean.
Live site: (https://jury.privacydev.org)
The project has two halves: an automated data pipeline that discovers, downloads, and extracts structured case data from regulator websites using Claude AI, and a React frontend that presents it as an interactive gallery.
Regulator Websites (FTC, ICO, PDPC, OAIC, EU DPAs, CA DOJ)
|
v
Collectors (jurisdiction-specific scrapers)
|
v
Document Discovery Log (PostgreSQL: discovered_documents)
|
v
Downloader (raw PDFs/HTML stored as BLOBs in PostgreSQL)
|
v
Text Extractor (pymupdf: PDF β clean text, locally, no API cost)
|
v
Phase 1: Extraction (Claude Haiku β structured fields from document)
|
v
Phase 2: Enrichment (Claude Haiku β company context, narratives, impact estimates)
|
v
URL Resolver (finds official case page URLs per jurisdiction)
|
v
Validation & Normalization β PostgreSQL (cases table)
|
v
Export β generatedCases.json β React Frontend (static build)
| Step | Command | API Cost | Description |
|---|---|---|---|
| Collect | --step collect |
Free | Scrapes regulator websites to discover new case document URLs |
| Download | --step download |
Free | Fetches PDFs/HTML, stores as BLOBs in PostgreSQL |
| Text | --step text |
Free | Converts PDFs to clean text via pymupdf (local) |
| Extract | --step extract |
~$0.01/doc | Claude extracts structured fields (company, fines, violations, legal bases) |
| Enrich | --step enrich |
~$0.01/doc | Claude adds company context, narratives, impact estimates |
| URL Fill | --step url_fill |
~$0.005/doc | Resolves official regulator case page URLs |
| Save | --step save |
Free | Writes validated data to PostgreSQL cases table |
| Export | --step export |
Free | Generates generatedCases.json for the frontend |
Phase 1 β Document Extraction: Reads the document text and extracts fields directly stated in the document: case name, jurisdiction, year, company, sector, violation type, legal bases violated, enforcement outcomes, penalty amounts, individuals affected, legal findings, and severity.
Phase 2 β AI Enrichment: Uses Phase 1 data plus outside knowledge to fill contextual fields: whatTheyDid (195-char narrative), whyTheyWereWrong (195-char harm description), company description, company valuation, founding year, current company status, impacted individuals (estimated when not in document), and claims vs reality.
Each phase is independently re-runnable. You can re-enrich all cases without re-extracting, or re-extract specific documents without touching enrichment.
- Documents are deduped by
(jurisdiction_id, document_url)β same URL won't be re-downloaded - Raw files are deduped by SHA256 hash β same content won't be re-processed
- Existing cases in the database are skipped during extraction
Each case gets a deterministic severity rating (1-5) based on:
- Data sensitivity (1-3): Health/biometric/children/financial = 3, Location/identity/credit = 2, Everything else = 1
- People impacted (0-2): 1M+ = 2, 10K-999K = 1, <10K or unknown = 0
- Final score = data + people, clamped to [1, 5]
| Page | Route | Description |
|---|---|---|
| Cases | / |
Searchable, filterable grid of all cases with jurisdiction, sector, violation type, and sort controls |
| Case Detail | /case/:id |
Full case breakdown β what they did, why they were wrong, claims vs reality, legal findings, outcome, attached PDFs |
| Compare | /compare |
Matrix view (patterns across jurisdictions/violations/sectors) and side-by-side comparison of up to 3 individual cases |
| Explore | /explore |
Interactive world map highlighting 7 jurisdictions; click a region to see its enforcement framework, key laws, and dataset statistics |
| Leaderboard | /leaderboard |
Rankings β top companies by fines, most active jurisdictions, most common violations and sectors |
| Learn | /learn |
Educational glossary explaining enforcement outcomes, violation types, and key legal concepts with cross-references |
| About | /about |
Project information and attribution |
| Jurisdiction | Abbreviation | Region |
|---|---|---|
| Federal Trade Commission | US FTC | United States |
| California DOJ | CA DOJ | United States (California) |
| Information Commissioner's Office | UK ICO | United Kingdom |
| Personal Data Protection Commission | SG PDPC | Singapore |
| General Data Protection Regulation | EU GDPR | European Union |
| European Data Protection Board | EU EDPB | European Union |
| Office of the Australian Information Commissioner | AU OAIC | Australia |
- React 18 + TypeScript 5 β UI framework
- Vite 5 β Build tool and dev server
- Tailwind CSS 3 β Utility-first styling
- shadcn/ui + Radix UI β Component library (50+ components)
- react-simple-maps β Interactive SVG world map on Explore page
- Recharts β Data visualization charts
- TanStack React Query β Data fetching/caching
- React Router DOM β Client-side routing
- Lucide React β Icons
- Python 3 β Pipeline orchestration
- Anthropic SDK β Claude API for document extraction and enrichment
- PostgreSQL + psycopg2 β Database and raw document storage (BLOBs)
- PyMuPDF (pymupdf) β PDF text extraction (local, no API cost)
- BeautifulSoup4 β HTML parsing for collectors and HTML documents
- python-dotenv β Environment variable management
- GitHub Pages β Static hosting
- GitHub Actions β CI/CD (auto-deploys on push to main)
- Custom domain β jury.privacydev.org
PrivacyGallery/
βββ src/
β βββ pages/ # Page components
β β βββ Index.tsx # Home β case gallery with search/filter/sort
β β βββ CaseDetail.tsx # Individual case breakdown
β β βββ Compare.tsx # Matrix & side-by-side case comparison
β β βββ Explore.tsx # Interactive jurisdiction map
β β βββ Leaderboard.tsx # Rankings and statistics
β β βββ Learn.tsx # Educational glossary
β β βββ About.tsx # Project information
β β βββ NotFound.tsx # 404 page
β βββ components/
β β βββ TopNav.tsx # Yellow navigation bar
β β βββ CaseCard.tsx # Case summary card with red fine stamp
β β βββ ControlBar.tsx # Filter/sort controls
β β βββ SearchBar.tsx # Search input
β β βββ JurisdictionMap.tsx # SVG world map (react-simple-maps)
β β βββ JurisdictionDetail.tsx# Jurisdiction stats and info panel
β β βββ JurisdictionLogos.tsx # Jurisdiction logo/icon display
β β βββ ScrollToTop.tsx # Scroll reset on route change
β β βββ ui/ # 50+ shadcn/ui components
β βββ data/
β β βββ cases.ts # Type definitions, utilities, case loading
β β βββ generatedCases.json # 787+ cases exported from PostgreSQL
β β βββ jurisdictionInfo.ts # Jurisdiction metadata (laws, authorities)
β β βββ glossary.ts # Learn page glossary content
β βββ hooks/ # Custom React hooks
β βββ lib/utils.ts # Tailwind class merge utility
β βββ main.tsx # App entry point
β βββ index.css # Global styles and CSS variables
βββ pipeline/ # Automated data pipeline
β βββ config.py # DB URL, API key, model settings
β βββ db.py # Connection pool, schema init
β βββ models.py # Dataclasses for all entities
β βββ runner.py # CLI entry point (--step, --jurisdiction, --limit)
β βββ migrate.py # One-time migration + jurisdiction seeding
β βββ collectors/ # Jurisdiction-specific scrapers
β β βββ base.py # BaseCollector ABC
β β βββ registry.py # Collector factory
β β βββ ftc.py # US FTC
β β βββ ico.py # UK ICO
β β βββ pdpc.py # Singapore PDPC
β β βββ oaic.py # Australia OAIC
β β βββ gdpr.py # EU GDPR (per-country DPAs)
β β βββ california_doj.py # California DOJ
β β βββ edpb.py # EU EDPB
β βββ processing/ # Download, dedup, text extraction
β β βββ downloader.py # Fetch documents, store as BLOBs
β β βββ dedup.py # SHA256 hash deduplication
β β βββ text_extractor.py # PDFβtext (pymupdf), HTMLβtext (bs4)
β β βββ url_resolver.py # Find official case page URLs
β βββ extraction/ # AI extraction (two-phase)
β β βββ prompts.py # Phase 1 + Phase 2 prompt templates
β β βββ extractor.py # Phase 1: structured fields from text
β β βββ enricher.py # Phase 2: company context + narratives
β β βββ parser.py # JSON response parsing
β βββ validation/ # Field validation and normalization
β β βββ validator.py # Missing values, impossible fines, dates
β β βββ normalizer.py # Jurisdiction/sector/violation mapping
β βββ export/ # Output generation
β βββ to_postgres.py # Write to cases table
β βββ to_frontend.py # cases β generatedCases.json
βββ files/ # Legacy pipeline scripts (archived)
β βββ agent.py # Original PDF β Claude β PostgreSQL
β βββ export_to_frontend.py # Original PostgreSQL β JSON export
β βββ ...
βββ .github/workflows/
β βββ deploy.yml # GitHub Actions β GitHub Pages
βββ package.json
βββ vite.config.ts
βββ tailwind.config.ts
βββ tsconfig.json
Requires Node.js 20+ β install with nvm
git clone https://github.com/AISmithLab/PrivacyGallery.git
cd PrivacyGallery
npm install
npm run devThe site runs at http://localhost:8080. Case data is already included in generatedCases.json.
Only needed if you want to collect new cases or rebuild the dataset.
Prerequisites: Python 3.12+, PostgreSQL running locally.
# Install dependencies
pip install -r pipeline/requirements.txt
# Configure environment
cp files/.env.example files/.env
# Edit files/.env with your ANTHROPIC_API_KEY and DATABASE_URL
# Initialize database tables and seed jurisdictions
python3 -m pipeline.migrateRun the full pipeline:
python3 -m pipeline.runnerRun individual steps:
python3 -m pipeline.runner --step collect # discover new documents
python3 -m pipeline.runner --step download # fetch PDFs/HTML
python3 -m pipeline.runner --step text # extract text locally
python3 -m pipeline.runner --step extract # Phase 1: Claude extraction
python3 -m pipeline.runner --step enrich # Phase 2: Claude enrichment
python3 -m pipeline.runner --step url_fill # resolve case page URLs
python3 -m pipeline.runner --step save # write to PostgreSQL
python3 -m pipeline.runner --step export # generate frontend JSONFilter by jurisdiction or limit:
python3 -m pipeline.runner --jurisdiction us_ftc # one jurisdiction only
python3 -m pipeline.runner --step extract --limit 5 # process max 5 documents
python3 -m pipeline.runner --dry-run # preview without changes| Variable | Description |
|---|---|
ANTHROPIC_API_KEY |
Claude API key for document extraction and enrichment |
DATABASE_URL |
PostgreSQL connection string |
EXTRACTION_MODEL |
Model for Phase 1 extraction (default: claude-haiku-4-5) |
ENRICHMENT_MODEL |
Model for Phase 2 enrichment (default: claude-haiku-4-5) |
The pipeline uses 6 PostgreSQL tables:
| Table | Purpose |
|---|---|
jurisdictions |
Registry of data sources (regulator URLs, crawl frequency, access method) |
discovered_documents |
Every document found by collectors (URL, status, jurisdiction) |
raw_documents |
Downloaded files stored as BLOBs with extracted text |
extraction_runs |
Audit log for each AI call (phase, model, tokens, response) |
pipeline_log |
Monitoring and failure logs |
cases |
Final structured case data (feeds the frontend export) |
The site auto-deploys to GitHub Pages on every push to main via GitHub Actions. The workflow:
- Checks out code
- Installs Node.js 20 and dependencies
- Builds with
npm run build - Copies
index.htmlto404.htmlfor SPA routing - Deploys to GitHub Pages
Custom domain configured via public/CNAME β jury.privacydev.org
interface EnforcementCase {
id: string;
company: string;
sector: Sector; // Technology, Healthcare, Finance, etc.
jurisdiction: Jurisdiction; // US FTC, UK ICO, EU GDPR, etc.
year: number;
fineAmount: number;
fineDisplay: string;
violations: ViolationType[]; // Misrepresentation, Failure to disclose, etc.
severityForIndividuals: number; // 1-5 calculated score
impactedIndividuals: string;
whatTheyDid: string; // Plain-language summary
whyTheyWereWrong: string; // Why it matters
claimsVsReality: ClaimVsReality[]; // What they said vs what they did
regulatoryFindings: RegulatoryFinding[];
outcome: string;
outcomeSummary: string; // Complaint Filed, Consent Order, etc.
attachedPDFs: AttachedPDF[]; // Links to source documents
// ... 40+ additional fields
}This project is maintained by AISmithLab.