STKGFS: Spatio-Temporal Knowledge Graph from Unstructured Texts for Food Security Monitoring

Table of Contents

Overview

Key Features

Pipeline Architecture

1. Data Collection 2. Data Processing 3. KG Exploitation
┌─────────────────┐ ┌──────────────────────────┐ ┌─────────────────────┐
│ Press Articles │ │ Spatial & Temporal Entity │ │ Evaluation │
│ (15,000 articles)│───>│ Recognition (CamemBERT + │───>│ Indicators │
│ │ │ HeidelTime) │ │ │
│ Food Security │ │ │ │ │ Spatial │
│ Lexicon (433 │───>│ Risk Indicator Detection │ │ Visualization │
│ terms) │ │ (Lexicon-based) │ │ of Risk Indicators │
│ │ │ │ │ └─────────────────────┘
│ Open Data │ │ Relationship Rules │
│ (Wikidata) │───>│ Definition & Generation │
│ │ │ │ │
│ Administrative │ │ Knowledge Graph │
│ Hierarchy │───>│ Construction & Update │
└─────────────────┘ └──────────────────────────┘

Installation

Prerequisites

Step-by-step Setup

# 1. Clone the repository
git https://github.com/CharlemagneBrain/STKG-FS.git
cd stkgfs
# 2. Create and activate virtual environment
python -m venv venv
source venv/bin/activate # Linux/Mac
# or: venv\Scripts\activate # Windows
# 3. Install Python dependencies
pip install -r requirements.txt
# 4. Verify Java installation (required for HeidelTime)
java -version

HeidelTime Configuration

treeTaggerHome = /path/to/your/TreeTaggerLinux

pip install py-heideltime

Neo4j Setup

Troubleshooting

Project Structure

stkgfs/
├── Fine-Tuning/ # Step 1: CamemBERT fine-tuning for NER
│ ├── CamemBERT_FT.ipynb # Training and evaluation notebook
│ └── annotations/ # BIO-tagged training data
│ ├── train_extended_bio_feb.json # 59,900 training sentences
│ ├── val_extended_bio_feb.json # 14,758 validation sentences
│ └── test_extended_bio_feb.json # 11,594 test sentences
│
├── Spatial_Annotation_Detection/ # Step 2: Spatial entity detection
│ ├── Spatial_Pipeline.ipynb # CamemBERT vs GLiNER evaluation
│ └── data/
│ └── df_sample.csv # 1,000 annotated articles for evaluation
│
├── Temporal_Entities_Detection/ # Step 3: Temporal extraction
│ ├── Heideltime_Detection.ipynb # HeidelTime processing notebook
│ ├── config.props # HeidelTime/TreeTagger configuration
│ └── data/
│ ├── df_extended_intersection.csv # Segments with spatial + temporal annotations
│ └── new_heaideltime_today.csv # HeidelTime extraction results
│
├── Triplet_Formation/ # Step 4: Triplet extraction
│ ├── Triplet_Algo.ipynb # Projection-based triplet algorithm
│ └── data/
│ ├── lexique_enrichi_final_july.csv # 433-term food security lexicon
│ ├── annotated_data.csv # 1,133 expert-annotated triplets
│ └── reconstruct_df_new.csv # Reconstructed articles
│
├── Events_to_Graph/ # Step 5: Knowledge graph construction
│ ├── Preprocessing_for_Graph.ipynb # Data analysis and visualization
│ ├── enrich_geo_wikidata_complete.py # Wikidata geocoding enrichment
│ ├── generate_neo4j_graph.py # Neo4j Cypher query generation
│ └── data/
│ ├── df_preprocessed.csv # Preprocessed triplets with labels
│ ├── processed_wd.csv # Wikidata-enriched locations
│ ├── annotations_df.csv # Consolidated annotations
│ ├── stkg_dtb.cypher # Generated Neo4j import file
│ └── stkg_dtb_events.csv # Event summary CSV
│
├── Preprocessing/ # Step 0: Text preprocessing
│ ├── Preprocess_News_Papers.py # Article segmentation with overlap
│ └── raw_articles.csv # Source articles (available on request, see below)
│
├── csv_to_map.py # Step 7: Generate interactive Folium map from query results
│
├── images/ # Figures for paper and README
│ ├── extract_kg_overview.png # Knowledge graph overview (Figure 2)
│ ├── H1_Ouaga_historical_events.png # Temporal monitoring (Figure 4)
│ └── map_env_agr_eco_risks_2009_annotated.png # Risk map (Figure 5)
│
├── results_csv/ # Intermediate evaluation results
│ ├── entites_df.csv
│ ├── filtered_df_nelson.csv
│ ├── segmented_csvfile.csv
│ └── spatial_entities_with_pos.csv
│
├── requirements.txt # Python dependencies
└── README.md # This file

Data Description

Source Corpus

Training Data for NER

Food Security Lexicon

Manual Annotations and Corrections

Reproducibility Guide

Step 1: Preprocess News Articles

cd Preprocessing
python Main.py

Step 2: Fine-tune CamemBERT for Spatial NER

cd Fine-Tuning
jupyter notebook CamemBERT_FT.ipynb

Step 3: Evaluate Spatial Entity Detection

cd Spatial_Annotation_Detection
jupyter notebook Spatial_Pipeline.ipynb

Step 4: Extract Temporal Entities

cd Temporal_Entities_Detection
jupyter notebook Heideltime_Detection.ipynb

Step 5: Form Triplets

cd Triplet_Formation
jupyter notebook Triplet_Algo.ipynb

Step 6: Build Knowledge Graph

cd Events_to_Graph
# 6a. Analyze and visualize preprocessed data
jupyter notebook Preprocessing_for_Graph.ipynb
# 6b. Enrich locations with Wikidata coordinates and hierarchy
python enrich_geo_wikidata_complete.py data/df_preprocessed.csv data/processed_wd.csv
# 6c. Generate Neo4j Cypher import file
python generate_neo4j_graph.py data/processed_wd.csv data/stkg_dtb.cypher

Step 7: Import to Neo4j

// In Neo4j Browser or cypher-shell:
:source /path/to/Events_to_Graph/data/stkg_dtb.cypher

cat Events_to_Graph/data/stkg_dtb.cypher | cypher-shell -u neo4j -p <password>

Step 8: Generate Interactive Risk Map

python csv_to_map.py

Neo4j Queries and Visualizations

Graph Schema

Queries for Paper Figures

MATCH (n:Event)-[c:CONCERNS]->(r:Risk)
MATCH (n)-[li:LOCATED_IN]->(l:Location {name : "Bobo-Dioulasso"})
MATCH (n)-[on:OCCURRED_ON]->(t:Time)
MATCH (n)-[rec:PRECEDES]->(n1:Event)
MATCH (n)-[irec:IS_RECURRENT]->(n2:Event)
MATCH (n2)-[on2:OCCURRED_ON]->(t2:Time)
MATCH (n2:Event)-[rel3:IS_SYNCHRONOUS]->()
MATCH (l)-[isf:IS_FROM_PROVINCE]->(l4:Location)
MATCH (n1)-[loacf:OCCURRED_ON]->(to:Time)
MATCH (n1)-[css:CONCERNS]->(rss:Risk)
MATCH (n)-[cs:CONCERNS]->(rs:Risk)
MATCH (n2)-[cr:CONCERNS]->(rr:Risk)
MATCH (l5)-[dept:IS_FROM_DEPARTEMENT]->(l)
RETURN n, c, r, li, l, on, t, rec, n1, n2, irec, on2, t2, isf, l4, loacf, to, css, rss, cs, rs, cr, rr, l5, dept

// --- BLOC 1: Événements clés du 1er septembre (inondation principale) ---
MATCH (e_flood:Event)-[c_flood:CONCERNS]->(r_flood:Risk {name: 'inondation'})
MATCH (e_flood)-[o_flood:OCCURRED_ON]->(t_flood:Time {datetime: '2009-09-01'})
MATCH (e_flood)-[li_flood:LOCATED_IN]->(l_ouaga:Location {name: 'Ouagadougou'})
WITH collect(e_flood)[0..3] AS flood_events,
 collect({
 event: e_flood,
 risk: r_flood,
 time: t_flood,
 concerns: c_flood,
 occurred: o_flood,
 located: li_flood // <-- CONSERVÉ UNIQUEMENT ICI
 })[0..3] AS flood_triplets,
 l_ouaga
// --- BLOC 2: Événements synchrones du 1er septembre (autres risques) ---
OPTIONAL MATCH (e_sync:Event)-[c_sync:CONCERNS]->(r_sync:Risk)
MATCH (e_sync)-[o_sync:OCCURRED_ON]->(t_sync:Time {datetime: '2009-09-01'})
MATCH (e_sync)-[:LOCATED_IN]->(l_ouaga)
WHERE r_sync.name <> 'inondation'
WITH flood_events, flood_triplets, l_ouaga,
 collect({
 event: e_sync,
 risk: r_sync,
 time: t_sync,
 concerns: c_sync,
 occurred: o_sync
 })[0..4] AS sync_triplets
// --- BLOC 3: Événements AVANT le 1er septembre (précurseurs) ---
OPTIONAL MATCH (e_before:Event)-[c_before:CONCERNS]->(r_before:Risk)
MATCH (e_before)-[o_before:OCCURRED_ON]->(t_before:Time)
MATCH (e_before)-[:LOCATED_IN]->(l_ouaga)
WHERE t_before.datetime < '2009-09-01' 
 AND t_before.datetime >= '2009-01-01'
WITH flood_events, flood_triplets, sync_triplets, l_ouaga,
 collect({
 event: e_before,
 risk: r_before,
 time: t_before,
 concerns: c_before,
 occurred: o_before
 })[0..10] AS before_triplets
// --- BLOC 4: Événements APRÈS le 1er septembre (conséquences) ---
OPTIONAL MATCH (e_after:Event)-[c_after:CONCERNS]->(r_after:Risk)
MATCH (e_after)-[o_after:OCCURRED_ON]->(t_after:Time)
MATCH (e_after)-[:LOCATED_IN]->(l_ouaga)
WHERE t_after.datetime > '2009-09-01' 
 AND t_after.datetime <= '2009-09-15'
WITH flood_events, flood_triplets, sync_triplets, before_triplets, l_ouaga,
 collect({
 event: e_after,
 risk: r_after,
 time: t_after,
 concerns: c_after,
 occurred: o_after
 })[0..3] AS after_triplets
// --- BLOC 5: Hiérarchie spatiale ---
OPTIONAL MATCH (l_ouaga)-[hp:IS_FROM_PROVINCE]->(province:Location)
OPTIONAL MATCH (province)-[hr:IS_FROM_REGION]->(region:Location)
// --- BLOC 6: Relation IS_FROM_VILLAGE entre Ouagadougou et ses villages ---
OPTIONAL MATCH (l_village:Location)-[hv:IS_FROM_VILLAGE]->(l_ouaga)
WITH flood_events, flood_triplets, sync_triplets, before_triplets, after_triplets,
 l_ouaga, province, hp, region, hr,
 collect({relation: hv, village: l_village})[0..3] AS villages_ouaga
// --- BLOC 8: Village Bogodogo avec ses événements ---
OPTIONAL MATCH (e_village:Event)-[li_village:LOCATED_IN]->(l_village:Location {name: 'Bogodogo'})
OPTIONAL MATCH (e_village)-[c_village:CONCERNS]->(r_village:Risk)
OPTIONAL MATCH (e_village)-[o_village:OCCURRED_ON]->(t_village:Time)
OPTIONAL MATCH (l_village)-[hp_village:IS_FROM_PROVINCE]->(province)
OPTIONAL MATCH (l_village)-[hv_bogodogo:IS_FROM_DEPARTEMENT]->(l_ouaga)
WITH flood_events, flood_triplets, sync_triplets, before_triplets, after_triplets,
 l_ouaga, province, hp, region, hr, villages_ouaga,
 collect({
 event: e_village,
 risk: r_village,
 time: t_village,
 location: l_village,
 concerns: c_village,
 occurred: o_village,
 located: li_village,
 is_from_province: hp_village,
 is_from_village: hv_bogodogo
 })[0..2] AS village_events
// --- BLOC 9: Collecter tous les événements ---
WITH flood_triplets, sync_triplets, before_triplets, after_triplets,
 l_ouaga, province, hp, region, hr, villages_ouaga, village_events,
 [e IN flood_triplets | e.event] + 
 [e IN sync_triplets | e.event] + 
 [e IN before_triplets | e.event] + 
 [e IN after_triplets | e.event] AS all_events
// --- BLOC 10: Relations IS_RECURRENT et IS_SYNCHRONOUS ---
UNWIND all_events AS ev1
UNWIND all_events AS ev2
OPTIONAL MATCH (ev1)-[rec:IS_RECURRENT]-(ev2)
WHERE id(ev1) < id(ev2)
OPTIONAL MATCH (ev1)-[sync_rel:IS_SYNCHRONOUS]-(ev2)
WHERE id(ev1) < id(ev2)
WITH flood_triplets, sync_triplets, before_triplets, after_triplets,
 l_ouaga, province, hp, region, hr, villages_ouaga, village_events,
 all_events,
 collect(DISTINCT CASE WHEN rec IS NOT NULL 
 THEN {rel: rec, from: ev1, to: ev2} END) AS recurrent_rels,
 collect(DISTINCT CASE WHEN sync_rel IS NOT NULL 
 THEN {rel: sync_rel, from: ev1, to: ev2} END) AS synchronous_rels
// --- BLOC 11: Relations PRECEDES - UNIQUEMENT CHAÎNE DIRECTE ---
// On ne garde que les relations PRECEDES où il n'existe pas d'intermédiaire
UNWIND all_events AS ev_from
UNWIND all_events AS ev_to
OPTIONAL MATCH (ev_from)-[prec:PRECEDES]->(ev_to)
WHERE ev_from IS NOT NULL AND ev_to IS NOT NULL
 // FILTRE CRUCIAL: exclure si un intermédiaire existe dans notre ensemble
 AND NOT EXISTS {
 MATCH (ev_from)-[:PRECEDES]->(ev_mid)-[:PRECEDES]->(ev_to)
 WHERE ev_mid IN all_events
 }
WITH flood_triplets, sync_triplets, before_triplets, after_triplets,
 l_ouaga, province, hp, region, hr, villages_ouaga, village_events,
 recurrent_rels, synchronous_rels,
 collect(DISTINCT CASE WHEN prec IS NOT NULL 
 THEN {rel: prec, from: ev_from, to: ev_to} END) AS precedes_rels
// --- RETOUR DU SOUS-GRAPHE ---
RETURN 
 flood_triplets AS Evenements_Inondation_1er_Sept,
 sync_triplets AS Evenements_Synchrones_1er_Sept,
 before_triplets AS Evenements_Avant_Aout,
 after_triplets AS Evenements_Apres_2_15_Sept,
 l_ouaga AS Location_Ouagadougou,
 province AS Province_Kadiogo,
 hp AS IS_FROM_PROVINCE,
 region AS Region_Centre,
 hr AS IS_FROM_REGION,
 villages_ouaga AS Villages_IS_FROM_VILLAGE,
 village_events AS Evenements_Village_Bogodogo,
 [r IN recurrent_rels WHERE r IS NOT NULL] AS Relations_IS_RECURRENT,
 [s IN synchronous_rels WHERE s IS NOT NULL] AS Relations_IS_SYNCHRONOUS,
 [p IN precedes_rels WHERE p IS NOT NULL] AS Relations_PRECEDES_Chaine_Directe;

MATCH (e:Event)-[:CONCERNS]->(r:Risk)
WHERE r.theme = 'environment' OR r.theme = 'economique'
MATCH (e)-[:OCCURRED_ON]->(t:Time)
WHERE t.year = 2009
MATCH (e)-[:LOCATED_IN]->(loc:Location)
RETURN
 loc.name AS Lieu,
 loc.type AS type,
 loc.latitude AS latitude,
 loc.longitude AS longitude,
 
 // ============= RISQUES ENVIRONNEMENTAUX =============
 SUM(CASE WHEN r.name = 'inondation' THEN 1 ELSE 0 END) AS Flood,
 SUM(CASE WHEN r.name = 'incendie' THEN 1 ELSE 0 END) AS Fire,
 SUM(CASE WHEN r.name = 'sécheresse' THEN 1 ELSE 0 END) AS Drought,
 SUM(CASE WHEN r.name = 'tempête' THEN 1 ELSE 0 END) AS Storm,
 SUM(CASE WHEN r.name = 'tornade' THEN 1 ELSE 0 END) AS Tornado,
 SUM(CASE WHEN r.name = 'crue' THEN 1 ELSE 0 END) AS FloodPeak,
 SUM(CASE WHEN r.name = 'orage' THEN 1 ELSE 0 END) AS Thunderstorm,
 SUM(CASE WHEN r.name = 'désertification' THEN 1 ELSE 0 END) AS Desertification,
 SUM(CASE WHEN r.name = 'déforestation' THEN 1 ELSE 0 END) AS Deforestation,
 SUM(CASE WHEN r.name = 'éboulement' THEN 1 ELSE 0 END) AS Landslide,
 SUM(CASE WHEN r.name = 'écroulement' THEN 1 ELSE 0 END) AS Collapse,
 
 // ============= RISQUES ÉCONOMIQUES =============
 
 // Campagne agricole (seul)
 SUM(CASE WHEN r.name = 'campagne agricole' THEN 1 ELSE 0 END) AS AgriculturalCampaign,
 
 // Augmentation des prix (agrège tous les types de hausses)
 SUM(CASE WHEN r.name IN [
 'augmentation des prix',
 'augmentation du prix',
 'hausse du prix',
 'hausse du coût',
 'hausse des prix',
 'montée du prix',
 'prix élevés',
 'élévation du coût',
 'inflation'
 ] THEN 1 ELSE 0 END) AS PriceIncrease

Results :

Results and Interpretation

Spatial Entity Recognition (Table 1)

Triplet Formation Evaluation

Knowledge Graph Statistics

Metrics Explained

Notes on Divergences Between Code and Paper

Pre-trained Models

HuggingFace Model and Datasets

Model Card Summary

Usage Example

from transformers import CamembertTokenizerFast, CamembertForTokenClassification
import torch
# Load model and tokenizer
tokenizer = CamembertTokenizerFast.from_pretrained("CharlesAbdoulaye/BF_NER")
model = CamembertForTokenClassification.from_pretrained("CharlesAbdoulaye/BF_NER")
model.eval()
# Entity labels
label_list = [
 "O",
 "B-country", "I-country",
 "B-region", "I-region",
 "B-departement", "I-departement",
 "B-province", "I-province",
 "B-village", "I-village"
]
id2label = {i: label for i, label in enumerate(label_list)}
# Example text covering all 5 administrative levels
text = "La crise alimentaire au Burkina Faso a frappe la region des Hauts-Bassins et la province du Houet. La ville de Bobo-Dioulasso et le village de Sya sont particulierement touches."
# Tokenize with offset mapping for span extraction
inputs = tokenizer(text, return_tensors="pt", truncation=True, return_offsets_mapping=True)
offset_mapping = inputs.pop("offset_mapping")[0].tolist()
with torch.no_grad():
 outputs = model(**inputs)
preds = torch.argmax(outputs.logits, dim=2)[0].tolist()
tokens = tokenizer.convert_ids_to_tokens(inputs["input_ids"][0])
# Reconstruct entities with spans (handles subword tokens)
entities, current = [], None
for idx, (pred_id, (start, end)) in enumerate(zip(preds, offset_mapping)):
 if start == 0 and end == 0:
 if current: entities.append(current); current = None
 continue
 label = id2label[pred_id]
 is_subword = not tokens[idx].startswith("\u2581")
 if label.startswith("B-"):
 if current: entities.append(current)
 current = {"type": label[2:], "start": start, "end": end}
 elif label.startswith("I-") or (is_subword and current):
 if current: current["end"] = end
 else:
 if current: entities.append(current); current = None
if current: entities.append(current)
# Merge consecutive same-type entities and clean boundaries
merged = []
for ent in entities:
 if merged and merged[-1]["type"] == ent["type"]:
 gap = text[merged[-1]["end"]:ent["start"]]
 if gap in ("", "-", " -", "- "):
 merged[-1]["end"] = ent["end"]; continue
 merged.append(dict(ent))
for ent in merged:
 ent["text"] = text[ent["start"]:ent["end"]].rstrip(".,;:!?")
for ent in merged:
 print(f'{ent["text"]:20s} | {ent["type"]:15s} | span=({ent["start"]}, {ent["end"]})')

Burkina Faso | country | span=(24, 36)
Hauts-Bassins | region | span=(60, 73)
Houet | province | span=(92, 97)
Bobo-Dioulasso | departement | span=(111, 125)
Sya | village | span=(143, 146)

Publishing to HuggingFace

# Install huggingface_hub
pip install huggingface_hub
# Login to HuggingFace (you'll be prompted for your token)
huggingface-cli login

from huggingface_hub import HfApi, create_repo
# Create model repository
create_repo("CharlesAbdoulaye/BF_NER", repo_type="model", private=False, exist_ok=True)
# Upload model checkpoint
api = HfApi()
api.upload_folder(
 folder_path="/data/charles/agile/camembert-ner-finetuned/checkpoint-15000",
 repo_id="CharlesAbdoulaye/BF_NER",
 repo_type="model"
)

from huggingface_hub import HfApi, create_repo
import os
# Create dataset repository
create_repo("CharlesAbdoulaye/BF_NER_datasets", repo_type="dataset", private=False, exist_ok=True)
# Upload each dataset file
api = HfApi()
dataset_files = [
 "Fine-Tuning/annotations/train_extended_bio_feb.json",
 "Fine-Tuning/annotations/val_extended_bio_feb.json",
 "Fine-Tuning/annotations/test_extended_bio_feb.json"
]
for file_path in dataset_files:
 api.upload_file(
 path_or_fileobj=file_path,
 path_in_repo=os.path.basename(file_path),
 repo_id="CharlesAbdoulaye/BF_NER_datasets",
 repo_type="dataset"
 )

Limitations

Citation

@article{ngom2026stkgfs,
 title={Spatio-Temporal Knowledge Graph from Unstructured Texts: A Multi-Scale Approach for Food Security Monitoring},
 author={Ngom, Charles Abdoulaye and Rajaonarivo, Landy and Valentin, Sarah and Teisseire, Maguelonne},
 journal={AGILE: GIScience Series},
 year={2026},
}

Acknowledgments

Contact