Loading...
Publication
Uso de Retrieval-Augmented Generation para Auxílio em Atividades de Suporte Técnico
| Name: | Description: | Size: | Format: | |
|---|---|---|---|---|
| 6.78 MB | Adobe PDF |
Authors
Advisor(s)
Abstract(s)
Esta dissertação investiga o uso de Retrieval-Augmented Generation (RAG) para apoiar
atividades de suporte técnico em contexto empresarial, onde a rotatividade de elementos, o
volume de incidentes e a dispersão do conhecimento di cultam o acesso rápido à informação.
Seguindo a metodologia Design Science Research (DSR), foi concebido e implementado um
protótipo composto por uma SPA em React e uma API em Spring Boot/Spring AI, suportada
por uma base de dados vetorial Qdrant e uma base de dados relacional Microsoft SQL Server.
A solução integra-se com o Con uence através de webhooks, para sincronização em tempo
real (indexação, reindexação e exclusão), e inclui um processo periódico de extração de
dados históricos do Control-M, desenvolvido em Java, publicando relatórios estruturados no
Con uence para posterior recuperação.
A avaliação adotou o modelo Goal Question Metric (GQM) e métricas LLM-based para
medir correção, relevância e consistência das respostas em cenários de documentação geral
e de dados do Control-M. Os resultados evidenciam: (i) melhor relação desempenho-custo
em modelos de chat de menor dimensão (gpt-4.1-mini e gpt-4o-mini); (ii) superioridade do
modelos de embedding de grande dimensão (text-embedding-3-large); (iii) impacto negativo
de similarity thresholds acima de 40% e ganhos com top-k mais elevados; (iv) melhoria na
recuperação de dados do Control-M com reescrita de queries, embora aquém da meta de
precisão de 90% de nida. Complementarmente, o código apresentou boa qualidade técnica
e o feedback dos utilizadores foi maioritariamente favorável. Conclui-se que o protótipo
é e caz para documentação geral e tecnicamente viável, sugerindo como trabalhos futuros
a exploração da recuperação híbrida, de estruturas alternativas de armazenamento do
conhecimento e a realização de testes operacionais.
This dissertation investigates the use of Retrieval-Augmented Generation (RAG) to optimize technical support activities in a business context, where sta turnover, the volume of incidents, and the dispersion of knowledge make it di cult to access information quickly. Following the Design Science Research (DSR) methodology, a prototype was designed and implemented consisting of a SPA in React and an API in Spring Boot/Spring AI, supported by a Qdrant vector database and a Microsoft SQL Server relational database. The solution integrates with Con uence through webhooks, for real-time synchronization (indexing, reindexing, and deletion), and includes a periodic process of extracting historical data from Control-M, developed in Java, publishing structured reports in Con uence for later retrieval. The evaluation adopted the Goal Question Metric (GQM) model and LLM-based metrics to measure the accuracy, relevance, and consistency of responses in general documentation and Control-M data scenarios. The results show: (i) better cost-performance ratio in smaller chat models (gpt-4.1-mini and gpt-4o-mini); (ii) superiority of large embedding models (textembedding- 3-large); (iii) negative impact of similarity thresholds above 40% and gains with higher top-k; (iv) improvement in Control-M data retrieval with query rewriting, although falling short of the de ned accuracy target of 90%. In addition, the code showed of good technical quality and user feedback was mostly favorable. It is concluded that the prototype is e ective for general documentation and technically feasible, suggesting as future work the exploration of hybrid retrieval, alternative knowledge storage structures, and operational
This dissertation investigates the use of Retrieval-Augmented Generation (RAG) to optimize technical support activities in a business context, where sta turnover, the volume of incidents, and the dispersion of knowledge make it di cult to access information quickly. Following the Design Science Research (DSR) methodology, a prototype was designed and implemented consisting of a SPA in React and an API in Spring Boot/Spring AI, supported by a Qdrant vector database and a Microsoft SQL Server relational database. The solution integrates with Con uence through webhooks, for real-time synchronization (indexing, reindexing, and deletion), and includes a periodic process of extracting historical data from Control-M, developed in Java, publishing structured reports in Con uence for later retrieval. The evaluation adopted the Goal Question Metric (GQM) model and LLM-based metrics to measure the accuracy, relevance, and consistency of responses in general documentation and Control-M data scenarios. The results show: (i) better cost-performance ratio in smaller chat models (gpt-4.1-mini and gpt-4o-mini); (ii) superiority of large embedding models (textembedding- 3-large); (iii) negative impact of similarity thresholds above 40% and gains with higher top-k; (iv) improvement in Control-M data retrieval with query rewriting, although falling short of the de ned accuracy target of 90%. In addition, the code showed of good technical quality and user feedback was mostly favorable. It is concluded that the prototype is e ective for general documentation and technically feasible, suggesting as future work the exploration of hybrid retrieval, alternative knowledge storage structures, and operational
Description
Keywords
IA RAG Control-M Suporte Técnico Bases de Dados Ve toriais Confluence Technical Support Vector Databases