Multilingual privacy infrastructure

MAPA: multilingual anonymization for sensitive documents and governed AI data

The Multilingual Anonymisation Toolkit for Public Administrations developed open, deployable technology for detecting and transforming sensitive information across all official European Union languages.

Discuss a data anonymization project Explore the project results

24 languages Coverage for every official language of the European Union

€950,000 Connecting Europe Facility project funding

2020 to 2021 Two year development, training and deployment programme

7 partners Language technology, research and public sector organisations

Pangeanic led Project coordination and multilingual technology development

The sensitive data challenge

Valuable documents cannot be safely reused while personal information remains exposed

Public administrations, hospitals, legal institutions and regulated companies hold large collections of documents containing names, addresses, identifiers, dates, organisations and other information connected to identifiable individuals.

These records may contain substantial value for research, analytics, model development and better public services. Their reuse is constrained when personal information cannot be detected and transformed consistently.

Manual anonymization is slow, expensive and difficult to scale across languages. MAPA addressed the problem through reusable multilingual models and controlled document processing workflows.

Sensitive information

Names, addresses, health references, case identifiers and contextual details can identify individuals directly or indirectly.

Multilingual complexity

Entity structures, naming conventions, morphology and document practices vary significantly between European languages.

Domain variation

Medical records, court decisions and administrative documents contain different entities, structures and privacy risks.

Reuse requirements

Removing every meaningful detail may protect privacy while leaving documents unsuitable for research or downstream processing.

The MAPA approach

Detect sensitive entities, apply controlled transformations and preserve document utility

MAPA treated anonymization as a connected AI data workflow combining taxonomy design, multilingual annotation, model training, evaluation and deployable processing.

Define sensitive entities

Establish entity categories and annotation rules for personal, professional, medical, legal and administrative information.

Build multilingual corpora

Collect and prepare representative documents across European languages and relevant institutional domains.

Annotate sensitive content

Human annotators identify entities and contextual information needed to train and evaluate detection models.

Train entity detection models

Deep learning based Named Entity Recognition models learn to locate sensitive information across languages and document types.

Transform detected entities

Sensitive content can be removed, masked, obfuscated or replaced with realistic alternatives according to the intended use.

Validate privacy and utility

Detection quality and document usefulness are assessed before anonymized material is released or reused.

Controlled transformation

Privacy protection requires more than deleting names

Different use cases require different treatment of detected entities. MAPA supported several transformation strategies to balance privacy with document utility.

MASK

Masking

María López → [PERSON]

Replace an entity with a generic category when the original value is unnecessary.

REDACT

Redaction

45872136X → █████████

Remove visible information when no replacement is required for subsequent processing.

REPLACE

Pseudonymization

Valencia → Zaragoza

Replace an entity with a realistic value of the same type to retain linguistic and analytical structure.

KEEP

Selective retention

1987 → 1987

Preserve permitted information when it remains useful and does not create unacceptable identification risk.

Pangeanic in MAPA

Project coordination, multilingual data operations and privacy technology

Pangeanic coordinated the consortium and connected linguistic resources, annotation, model development, evaluation and practical deployment into a common European toolkit.

LEAD

Consortium coordination

Pangeanic led project planning, partner coordination, technical delivery and alignment with European public administration requirements.

DATA

Multilingual data preparation

Documents and linguistic resources were collected, normalized and prepared to support annotation, training and evaluation.

ANNO

Annotation workflows

Human annotation procedures converted sensitive document collections into structured datasets for entity recognition.

NER

Sensitive entity detection

Multilingual models were trained to identify personal and contextual information across legal, medical and administrative content.

Evaluation and quality control

Detection precision, recall and transformation behaviour were evaluated against annotated reference data.

DEPLOY

Operational deployment

The project delivered reusable components suitable for controlled institutional environments and domain adaptation.

A multilingual AI data programme

The toolkit depended on annotated datasets designed around privacy risk

Reliable anonymization requires models to recognise far more than conventional names and places. Entity taxonomies must reflect how people can be identified inside real documents.

MAPA therefore involved a substantial data operation: document acquisition, legal and ethical review, annotation guideline development, multilingual labelling, model training and the construction of separate evaluation sets.

The resulting annotated corpora covered all 24 official EU languages and supported a customizable toolkit capable of detecting and substituting sensitive information in different domains.

Explore Data for AI Explore datasets for AI

Document acquisition

Representative administrative, legal and medical texts from relevant language environments.

Sensitive entity taxonomy

Consistent categories for direct and contextual identifiers across languages and domains.

Expert annotation

Human labelling of entities following documented guidelines and quality procedures.

Model training

Monolingual and multilingual entity detection systems trained on annotated corpora.

Gold standard evaluation

Reference datasets used to measure missed entities, false positives and domain performance.

Governed document workflow

From sensitive source document to controlled reusable data

Anonymization becomes operational when detection, transformation, review and release are connected within a traceable process.

Secure ingestion Documents enter a controlled processing environment

Entity detection Models identify sensitive and identifying information

Transformation Entities are masked, removed or replaced according to policy

Human validation Reviewers inspect uncertain cases and critical content

Governed release Approved documents become available for the intended use

Project results

Open multilingual resources for detecting and substituting sensitive information

MAPA produced annotated language resources and customizable technology that could be adapted to new document domains and institutional requirements.

Official EU languages

Annotated resources and multilingual detection capacity spanning every official European Union language.

Initial document domains

Administrative, legal and medical content provided demanding environments for development and testing.

NER

Deep learning detection

Named Entity Recognition models locate sensitive information according to configurable entity categories.

Open

Reusable toolkit

Open components support institutional deployment, testing and adaptation to additional use cases.

Initial validation domains

Tested where privacy requirements and document complexity are high

The project focused on environments where inaccurate anonymization can expose individuals or remove information required for legitimate analysis.

HEALTH

Medical and health documents

Clinical narratives, medical references, patient information, dates, locations and institutional identifiers.

Patient references
Clinical context
Healthcare professionals
Dates and facilities

LEGAL

Legal and judicial documents

Court decisions, case files, procedural records and documents involving multiple parties and contextual identifiers.

Parties and witnesses
Legal representatives
Case identifiers
Addresses and events

ADMIN

Public administration records

Correspondence, decisions, applications and institutional documents created through public service delivery.

Citizen information
Contact details
Administrative references
Institutional metadata

European consortium

Language resources, NLP research, privacy expertise and public sector validation

The consortium brought together multilingual technology companies, research organisations, universities and institutional partners.

Coordinator

Pangeanic

Project coordination, multilingual data operations, anonymization technology, integration and deployment.

Technology partner

Tilde

Multilingual NLP, language resources and technology for lower resource European languages.

Language resources

ELDA

Language data, corpus development, annotation resources and multilingual research support.

Public sector partner

SEDIA

Spanish public sector participation and connection with national language technology programmes.

Research partner

LIMSI CNRS

Natural language processing research, entity recognition and multilingual model development.

Academic partner

University of Malta

Linguistic research, multilingual evaluation and expertise in Maltese language processing.

Technology centre

Vicomtech

Applied AI, medical use cases, document processing and institutional validation.

MAPA in 2026

Privacy engineering is becoming part of AI data engineering

Organisations increasingly need to prepare internal documents for model training, retrieval, analytics and controlled sharing. Sensitive information must be addressed before those workflows can be operated safely.

MAPA demonstrated the technical foundation of this process: multilingual entity taxonomies, annotated data, domain specific models, transformation policies and human review.

These capabilities now support a broader approach to governed AI data operations in which privacy, provenance, quality and intended use are considered before information reaches a model.

PII

Sensitive data detection

Identify direct and contextual personal information across documents and languages.

DATA

Safe data preparation

Transform enterprise and public sector documents before training, testing or retrieval.

EVAL

Privacy evaluation

Measure missed entities, false detections and residual identification risk against reference data.

SOV

Controlled deployment

Operate sensitive document processing within private, on premise or sovereign infrastructure.

From research to production

What MAPA demonstrates for regulated organisations and AI teams

MAPA provides a practical foundation for processing sensitive multilingual documents before they are shared, analysed or used in AI systems.

For public administrations

Prepare documents for transparency, research, interdepartmental exchange and multilingual public services.

For healthcare organisations

Detect and transform patient information before clinical text is reused for analytics, research or model development.

For legal and compliance teams

Apply configurable privacy policies to case files, contracts, decisions and disclosure workflows.

For enterprise AI teams

Prepare document repositories for model training, evaluation and retrieval augmented generation under controlled conditions.

Public evidence

Project documentation, academic publications and external coverage

These references document the consortium, funding, language coverage, toolkit and multilingual datasets produced by MAPA.

Academic project paper

The Multilingual Anonymisation Toolkit for Public Administrations

EAMT publication describing the project objective, domains, language coverage and development period.

Read publication →

Results publication

Open datasets and customizable anonymization models

Technical paper presenting the annotated corpora and toolkit developed for 24 European Union languages.

Read results paper →

European project profile

MAPA project facts

Public profile documenting project coordination, consortium, duration, funding and technical focus.

View project profile →

Consortium partner

Vicomtech project reference

Partner description of medical and legal anonymization and deployment in European public administrations.

View partner reference →

Language data partner

ELDA project archive

Project description covering multilingual Named Entity Recognition and anonymization for all EU languages.

View project archive →

Industry coverage

Multilingual data management and anonymization

External discussion of Pangeanic’s multilingual data and anonymization work, including MAPA.

Read coverage →

Pangeanic project page

MAPA use case

The canonical Pangeanic project record and its connection to current privacy preserving data operations.

View project →

European portfolio

European AI and language technology projects

MAPA within Pangeanic’s wider work in multilingual data, model development and European digital infrastructure.

Explore all projects →

Operational capability

Data for AI

Governed data preparation, annotation, evaluation and privacy workflows for enterprise and public sector AI.

Explore Data for AI →

Privacy preserving AI data

Prepare sensitive multilingual documents for safe and governed AI use

Pangeanic supports enterprises and public institutions with multilingual PII detection, document anonymization, pseudonymization, human validation and controlled deployment.

Discuss your anonymization requirements Explore Data for AI Explore datasets for AI