Deep learning-based ensemble for temporal knowledge graph embedding in link prediction
Author
An, Ning
Term
4. term
Education
Publication year
2023
Pages
12
Abstract
Question answering (QA) has gained momentum since ChatGPT. QA systems built on knowledge graphs (KGs) and temporal knowledge graphs (TKGs) rely heavily on link prediction—estimating missing connections in a graph. To improve this, the thesis investigates ensemble learning, which combines multiple models. We use neural networks to predict how much weight each base model should get in an ensemble for TKGs (treating these weights as hyperparameters). Two neural network architectures are proposed and evaluated with six metrics. Results show that the neural network–based ensemble outperforms all individual models and also an ensemble built with grid search. Among base models, ATiSE performs best, but a grid-search ensemble improves HITS@1 (the share of test cases where the correct answer is ranked first) by 33% over ATiSE. The neural ensemble then improves HITS@1 by a further 8% over the grid-search ensemble. Moreover, when predicting only one element of a TKG fact (head/subject, relation, tail/object, or timestamp), the neural ensemble yields larger gains for tail and timestamp.
Spørgsmål–svar (QA) er kommet i fokus siden ChatGPT. QA-systemer, der bygger på vidensgrafer (KG) og tidslige vidensgrafer (TKG), afhænger stærkt af linkforudsigelse – dvs. at gætte manglende forbindelser i en graf. For at forbedre dette undersøger afhandlingen ensemblelæring, hvor flere modeller kombineres. Vi bruger neurale netværk til at forudsige, hvor meget vægt hver basismodel skal have i et ensemble for TKG’er (behandlet som hyperparametre). Vi foreslår to neurale netværksarkitekturer og evaluerer dem med seks metrikker. Resultaterne viser, at det neurale ensemble slår alle enkeltmodeller og også et ensemble bygget med grid-søgning. Blandt basismodellerne klarer ATiSE sig bedst, men et grid-søgnings-ensemble forbedrer HITS@1 (andelen af testtilfælde hvor det korrekte svar ligger nr. 1) med 33% i forhold til ATiSE. Det neurale ensemble forbedrer HITS@1 yderligere med 8% i forhold til grid-søgnings-ensemblet. Desuden, når man kun forudsiger én del af en TKG-fakta (subjekt/head, relation, objekt/tail eller tidsstempel), giver det neurale ensemble størst forbedringer ved forudsigelse af objekt (tail) og tid.
[This apstract has been rewritten with the help of AI based on the project's original abstract]