EphèseLe catalogue de la bibliothèque de l'ECAM

Titre :	Enhancing Information Retrieval by training a Sentence-Transformer with LoRA (Low-Rank Adaptation of Large Language Models)
Type de document :	Travail de fin d'études
Auteurs :	Nick KUIJPERS, Auteur ; Cedric RAEMDONCK, ; François Defrance,
Editeur :	ECAM
Année de publication :	2024
Langues :	Français (fre)
Mots-clés :	Intelligence artificielle
Index. décimale :	TFE - Informatique (ECAM)
Résumé :	This study commenced with the development of a Retrieval-Augmented Generation (RAG) system during an internship at Siemens. This system integrates a Large Language Model (LLM), a vector database, and user interaction software, designed to empower AI assisting agents. We addressed the challenges associated with fine-tuning LLMs, especially in scenarios with limited datasets. Drawing inspiration from state-of-the art methods such as Generative Pseudo Labeling (GPL) and LoRa (Low-Rank Adaptation of Large Language Models), we devised innovative strategies to optimize embedding generation, a critical component within the vector database. Our study represents a significant advancement by introducing LoRa, a novel technique, into the training process of sentence-transformers. This pioneering approach marks a substantial contribution to the field, promising enhanced performance and adaptability in AI-driven applications.

Enhancing Information Retrieval by training a Sentence-Transformer with LoRA (Low-Rank Adaptation of Large Language Models) [Travail de fin d'études] / Nick KUIJPERS, Auteur ; Cedric RAEMDONCK, ; François Defrance,  . - ECAM, 2024.
Langues : Français (fre)

Mots-clés :	Intelligence artificielle
Index. décimale :	TFE - Informatique (ECAM)
Résumé :	This study commenced with the development of a Retrieval-Augmented Generation (RAG) system during an internship at Siemens. This system integrates a Large Language Model (LLM), a vector database, and user interaction software, designed to empower AI assisting agents. We addressed the challenges associated with fine-tuning LLMs, especially in scenarios with limited datasets. Drawing inspiration from state-of-the art methods such as Generative Pseudo Labeling (GPL) and LoRa (Low-Rank Adaptation of Large Language Models), we devised innovative strategies to optimize embedding generation, a critical component within the vector database. Our study represents a significant advancement by introducing LoRa, a novel technique, into the training process of sentence-transformers. This pioneering approach marks a substantial contribution to the field, promising enhanced performance and adaptability in AI-driven applications.