| Name: | Description: | Size: | Format: | |
|---|---|---|---|---|
| 9.12 MB | Adobe PDF |
Authors
Advisor(s)
Abstract(s)
O Instituto Superior de Engenharia do Porto é uma Escola constituída por um número elevado de estudantes, cujo processo de suporte é fulcral para o bom funcionamento da sua comunidade. A disponibilidade do atendimento presencial da Divisão Académica é atualmente reduzida, dificultando o esclarecimento de dúvidas. Isto resulta não só no constrangimento dos processos virtuais, como e-mail e linhas telefónicas, como também na diminuição do nível de satisfação do estudante. Para resolver este problema, foi idealizado um agente conversacional com o intuito de responder às perguntas mais frequentes relativas à Divisão Académica da Escola. O chatbot é constituído por um módulo de interpretação de linguagem, portuguesa, recorrendo a machine learning, particularmente técnicas de compreensão de texto. De forma complementar, para manter o sistema abstraído de qualquer especificação técnica, desenvolveu-se um serviço web que efetua a gestão da base de conhecimento do agente. A avaliação da solução comparou o desempenho de vários modelos de classificação, compostos internamente pelo agrupamento de métodos de pré-processamentos de dados, modelos de linguagem (transformadores), e, finalmente, algoritmos de aprendizagem. Os resultados obtidos acentuam a superioridade do modelo de linguagem da Google, BERT, em junção com o classificador DIET, obtendo um F1-Score de 0.965. O sistema foi implantado através de um protótipo exposto não só aos estudantes da Escola, como também aos membros da Divisão Académica. Para um total de 256 mensagens, obtiveram-se cerca de 30% de respostas não enquadradas. Em contrapartida, as avaliações persistidas foram positivas, com uma média acima de 4.20 e um desvio padrão de 1.23.
Since Instituto Superior de Engenharia do Porto is a school with a high number of students, client support is very important to having a healthy community. This is handled by the academic department, who works under a very limited presential schedule. Virtual support, such as emails and phone lines, end up queuing and students are often dissatisfied with the duration of this process. To solve the problem, a chatbot is idealized with the objective of answering the most frequent student questions. The system is conceptually composed of a portuguese language interpreter module based on machine learning, particularly natural language understanding. In order to have the application abstracted from the technical specifications of the chatbot framework, a web service was also developed with functionalities to maintain and modify the agent’s information store. The solution evaluation compared several intent classifying models, particularly text preprocessing methods, language models (transformers) and deep learning algorithms. The best combination was Google’s language model, BERT, combined with DIET, obtaining a 0.965 F1- Score. The chatbot was deployed through a public prototype exposed to both students from ISEP and to the student support department. The final result was about 30% of non fitted responses, while the remaining 70% were evaluated with a mean of 4.20 and a standard deviation of 1.23.
Since Instituto Superior de Engenharia do Porto is a school with a high number of students, client support is very important to having a healthy community. This is handled by the academic department, who works under a very limited presential schedule. Virtual support, such as emails and phone lines, end up queuing and students are often dissatisfied with the duration of this process. To solve the problem, a chatbot is idealized with the objective of answering the most frequent student questions. The system is conceptually composed of a portuguese language interpreter module based on machine learning, particularly natural language understanding. In order to have the application abstracted from the technical specifications of the chatbot framework, a web service was also developed with functionalities to maintain and modify the agent’s information store. The solution evaluation compared several intent classifying models, particularly text preprocessing methods, language models (transformers) and deep learning algorithms. The best combination was Google’s language model, BERT, combined with DIET, obtaining a 0.965 F1- Score. The chatbot was deployed through a public prototype exposed to both students from ISEP and to the student support department. The final result was about 30% of non fitted responses, while the remaining 70% were evaluated with a mean of 4.20 and a standard deviation of 1.23.
Description
Keywords
Chatbot Universidade Apoio ao Cliente Natural Language Understanding Deep Learning College Client Support Natural Language Processing