GenAI governance: Analyzing Large Language Models in Public Administration Contexts
Author
Andris, Nikolaos
Term
4. semester
Publication year
2024
Submitted on
2024-06-03
Abstract
This thesis examines the use of large language models in Denmark’s public administration through a collaboration with the Danish Agency for Development and Simplification (UFST). The core question is how an LLM‑based, user‑friendly conversational service can deliver accurate, context‑specific tax information while addressing challenges such as data quality, trust, and integration. A proof‑of‑concept is developed using Retrieval‑Augmented Generation (RAG), built on LangChain and supported by vector databases. The system retrieves and grounds answers in content from skat.dk, focusing on English pages for people who want to work or study in Denmark, to improve access to relevant information and reduce customer service workload. The thesis reviews governance and implementation considerations for generative AI in the public sector, explains the RAG pipeline (indexing, retrieval, generation), and discusses grounding techniques to mitigate hallucinations. It outlines evaluation approaches (including corrective, self‑reflective, and adaptive RAG), notes limitations, and proposes future development such as expanding data sources and continuous assessment prior to potential deployment. Based on the prototype, the work indicates potential for more efficient citizen service and easier access to correct information, without presenting detailed empirical performance in the provided text.
Denne afhandling undersøger, hvordan store sprogmodeller kan anvendes i dansk offentlig forvaltning gennem et samarbejde med Udviklings- og Forenklingsstyrelsen (UFST). Den centrale problemstilling er, hvordan en LLM-baseret, brugervenlig samtaleservice kan levere korrekt og kontekstuel skatteinformation og samtidig adressere udfordringer som datasikkerhed, nøjagtighed og tillid. Der udvikles et proof-of-concept med Retrieval-Augmented Generation (RAG), opbygget på LangChain og understøttet af vektordatabaser. Systemet henter og anvender indhold fra skat.dk, med fokus på engelsksprogede sider målrettet personer, der ønsker at arbejde eller studere i Danmark, for at lette adgang til relevante svar og reducere belastningen på kundeservice. Afhandlingen gennemgår relevante styrings- og implementeringsspørgsmål for generativ AI i den offentlige sektor, beskriver RAG-processen (indeksering, hentning og generering) og diskuterer teknikker til at begrænse hallucinationer via grounding. Der skitseres evalueringsmetoder (bl.a. corrective, self‑reflective og adaptive RAG) og identificeres begrænsninger samt mulige videreudviklinger, herunder udvidelse af datakilder og løbende vurdering før en eventuel implementering. På baggrund af prototypen peger arbejdet på et potentiale for mere effektiv borgerservice og bedre adgang til korrekt information, men uden at fremlægge detaljerede empiriske resultater i den givne tekst.
[This apstract has been generated with the help of AI directly from the project full text]