← WorkMSc DataScale · joint project with Youcef Bouagada
Rules and an LLM that check each other: extracting a verifiable knowledge graph
French scientific text → RDF/OWL triples: NER, syntactic rules, LLM fallback, ontology validation and OWL-RL reasoning. F1 0.81 against 0.73 (rules) and 0.75 (LLM alone).
- Role
- Co-designer and co-developer
- Period
- 01 — 03 · 2026
- Status
- Academic project — 50-sentence corpus
- Tools
- spaCyLlama-3RDFLibOWL 2 / owlrlStreamlit
01Overview
Turning French scientific text into coherent RDF/OWL triples. Linguistic rules are precise but cover little; LLMs cover a lot but invent relations.
MSc DataScale, Université Paris-Saclay / UVSQ — joint project with Youcef Bouagada.
What I personally did
- 01Co-designed the five-module pipeline and co-wrote the code, the report (47 pp.) and the defence.
02Constraints
- Open standards RDF, RDFS, OWL 2; triples traceable back to their sentence.
- Rate limits on the generation API.
03Architecture
- 01
Entities
spaCy, EntityRuler, heuristics, mention normalisation and filtering.
- 02
Rule-based relations
Syntactic dependencies, subject–verb–object, a dictionary to ontology properties.
- 03
LLM fallback
Llama-3 when no rule covers the relation.
- 04
Validation and reasoning
Domains, ranges and classes checked; OWL-RL inference; export and a Streamlit interface.
Five-module neuro-symbolic pipeline.
04Implementation decisions
- 01
Let the ontology refuse
A triple that breaks a property's domain or range is rejected, whatever its source — rule or LLM.
05Challenges
Merged entities
An abbreviation followed by a capitalised pronoun became one entity: segmentation rules and an abnormal-mention filter.
Hallucinated relations
A city linked to a product with no textual evidence: rejected by ontology constraints and subject/object typing.
Rate limits
LRU cache, exponential back-off and a fallback.
06Evaluation
150 reference triples hand-annotated by two annotators (agreement κ = 0.87), over 50 French sentences.
| Configuration | Precision | Recall | F1 |
|---|---|---|---|
| Rules only | 0.92 | 0.61 | 0.73 |
| LLM only | 0.67 | 0.84 | 0.75 |
| Full hybrid | 0.84 | 0.78 | 0.81 |
Scores specific to this academic protocol.
07Outcomes
- The best trade-off between coverage and semantic control, measured.
Limits
- 50 sentences: not a validation on thousands of industrial documents.
08What I took from it
- 01Having a symbolic checker control a generator is more robust than tuning the generator alone.
09Evidence