EphèseLe catalogue de la bibliothèque de l'ECAM

Enhancing Information Retrieval by training a Sentence-Transformer with LoRA (Low-Rank Adaptation of Large Language Models) / Nick KUIJPERS 

Public ISBD Titre : Enhancing Information Retrieval by training a Sentence-Transformer with LoRA (Low-Rank Adaptation of Large Language Models) Type de document : Travail de fin d'études Auteurs : Nick KUIJPERS, Auteur ; Cedric RAEMDONCK, ; François Defrance, Editeur : ECAM Année de publication : 2024 Langues : Français (fre) Mots-clés : Intelligence artificielle Index. décimale : TFE - Informatique (ECAM) Résumé : This study commenced with the development of a Retrieval-Augmented Generation (RAG) system during an internship at Siemens. This system integrates a Large Language Model (LLM), a vector database, and user interaction software, designed to empower AI assisting agents. We addressed the challenges associated with fine-tuning LLMs, especially in scenarios with limited datasets. Drawing inspiration from state-of-the art methods such as Generative Pseudo Labeling (GPL) and LoRa (Low-Rank Adaptation of Large Language Models), we devised innovative strategies to optimize embedding generation, a critical component within the vector database. Our study represents a significant advancement by introducing LoRa, a novel technique, into the training process of sentence-transformers. This pioneering approach marks a substantial contribution to the field, promising enhanced performance and adaptability in AI-driven applications. Enhancing Information Retrieval by training a Sentence-Transformer with LoRA (Low-Rank Adaptation of Large Language Models) [Travail de fin d'études] / Nick KUIJPERS, Auteur ; Cedric RAEMDONCK, ; François Defrance,  . - ECAM, 2024. Langues : Français (fre) Mots-clés : Intelligence artificielle Index. décimale : TFE - Informatique (ECAM) Résumé : This study commenced with the development of a Retrieval-Augmented Generation (RAG) system during an internship at Siemens. This system integrates a Large Language Model (LLM), a vector database, and user interaction software, designed to empower AI assisting agents. We addressed the challenges associated with fine-tuning LLMs, especially in scenarios with limited datasets. Drawing inspiration from state-of-the art methods such as Generative Pseudo Labeling (GPL) and LoRa (Low-Rank Adaptation of Large Language Models), we devised innovative strategies to optimize embedding generation, a critical component within the vector database. Our study represents a significant advancement by introducing LoRa, a novel technique, into the training process of sentence-transformers. This pioneering approach marks a substantial contribution to the field, promising enhanced performance and adaptability in AI-driven applications.

Exemplaires

Code-barres	Cote	Support	Localisation	Section	Disponibilité
aucun exemplaire

Documents numériques

Ce document n'est visible qu'après identification KUIJPERS_Nick_-_5MIN.pdf Adobe Acrobat PDF

---

1 (1 - 1 / 1) Par page : 25 50 100 200