ISEP - DM – Engenharia de Inteligência Artificial
Permanent URI for this collection
Browse
Recent Submissions
- Transfer learning applied to government auditing: A focused approach on financial statements in Maranhão, BrazilPublication . Coelho, Heloisa Guimarães; Marreiros, Maria Goreti CarvalhoSince Brazil’s return to democracy, dozens of laws, decrees and normative instructions have been drafted with the purpose of regulating and improving the mechanisms for controlling and monitoring municipal public resources. These regulations are specifically aimed at the process of accountability by elected officials, who currently rely on the help of accountants responsible for preparing and submitting financial statements to the Courts of Auditors. However, according to data from the TCU (Federal Court of Accounts), in 2023, Maranhão was the Brazilian State with the highest number of rejected accounts. There are several reasons that can lead to these processes being challenged, including incorrect application of resources, flaws in documentation, human errors, among others. In practice, the routine of accountants includes repetitive and mechanical activities that requires considerable time to prepare and review documents, hence often leading to errors in classification and issuing of documentation. In this context, this dissertation investigates the use of Transfer Learning (TL) to improve automation and accuracy in the classification of financial commitment notes, an initial document in the public expenditure cycle, with a specific focus on the context of the state of Maranhão. To this end, BERTimbau, a pre-trained language model for Brazilian Portuguese, was fine-tuned to assist government accountants in reducing classification errors and ensuring compliance with local and national financial regulations. The CRISP-DM methodology, widely used in data science, was adopted to structure the development of the project. The dataset used, consisting of several classifications of commitment notes for the year 2023, was thoroughly analyzed and pre-processed. For the fine-tuning process of the model, two samples with a similar number of data were selected, varying only the number of possible classifications, due to the high degree of imbalance between the classes. Even in a multiclass context with datasets with a reduced number of classes, the results obtained indicate that the BERTimbau model presents strong performance in the classification task, achieving 98% accuracy with an error rate of 0.10 in the test set, highlighting the effectiveness of BERTimbau in public financial auditing applications. These results highlight the effectiveness of BERTimbau for public financial auditing applications. It is therefore concluded that TL models have great potential to optimize and improve financial auditing processes, with positive implications for wider adoption in Brazil.
- Leveraging generative artificial intelligence and wearable technology for adaptive health and conversational interventions in the elderlyPublication . Crista, Vítor Rafael Palmeiro; Martinho, Diogo Emanuel PereiraAging is often associated with an increase in loneliness and social isolation, factors that have significant consequences for health. The lack of social interactions can lead to a decline in emotional well-being, the deterioration of cognitive conditions, and a higher risk of physical health problems. Therefore, it is relevant to develop solutions that facilitate interaction and promote the quality of life of the elderly population, encouraging regular physical activity and improving their physical and psychological well-being. Technology attends as an effective tool to address these challenges, enabling the creation of systems that provide continuous support and interactions adapted to individual needs. This dissertation explores the development of a conversational system that uses wearable technology to collect information about users' health levels, aiming to create proactive interactions, encourage healthy habits, and promote active aging. The system also incorporates artificial intelligence, specifically generative artificial intelligence, with the main objective of developing advanced communication mechanisms between the system and the user. These mechanisms allow the system to learn from past interactions to improve the quality of future ones. The work included designing the system architecture, which integrates a Multi-Agent approach, providing a scalable structure that enables continuous improvements in the quality of the communication strategies used by the system. The solution was developed based on a domain model aligned with the common needs of elderly users and includes a detailed description of the essential components for its implementation. To evaluate the system, a study was conducted with five users, providing valuable insights into the system’s performance and impact. The study was conducted with five participants over a week, and to assess the system’s impact, indicators were defined to measure the key objectives of the research. The results indicate a positive impact on user participation and physical activity, with an average acceptance rate of proposed activities at 63.6%. Additionally, the average number of steps increased by 42.93% throughout the study, and the frequency of conversations recorded an average growth of 20.14%.
- Estratégias de aprendizagem por reforço para configuração dinâmica de meta-heurísticasPublication . Oliveira, Vítor José Henriques; Pinto, Tiago Manuel Campelos Ferreira; Ramos, Carlos Fernando da SilvaA eficácia da otimização de problemas complexos está intimamente ligada à configuração de parâmetros em algoritmos meta-heurísticos. Embora já tenham sido propostos métodos automatizados para a escolha dos parâmetros de algoritmos para reduzir a necessidade de ajuste manual, existe ainda um potencial significativo, não explorado, de ajuste dinâmico de parâmetros de algoritmos durante a execução, o que pode melhorar o seu desempenho. Este estudo visa aferir a eficácia da definição manual de parâmetros em comparação com uma abordagem dinâmica baseada em aprendizagem por reforço, reduzindo a necessidade de intervenção humana e aumentando a eficiência operacional dos algoritmos. Para alcançar este objetivo, adaptaram-se os métodos SARSA (State-Action-Reward-State-Action) e Deep SARSA para regular os parâmetros de algoritmos meta-heurísticos, em especial, o algoritmo genético. O modelo adotado é independente do problema a ser otimizado ou do algoritmo meta-heurístico selecionado, por isso, oferece a flexibilidade necessária, sendo apenas crucial escolher os parâmetros a ajustar durante o decorrer do processo de otimização de qualquer problema estudado. Estas metodologias foram testadas em funções benchmark, amplamente reconhecidas na literatura, e aplicadas nesta investigação nos seguintes cenários práticos: a otimização de portfólios de investimentos, na qual um participante possui ou pretende adquirir energia elétrica num mercado de eletricidade e a melhoria relacionada com a alocação de pacientes em Unidades de Cirurgia (UC) e em Unidades de Cuidados Intensivos (UCI), com o intuito de melhorar a eficiência da utilização de recursos limitados. Os resultados demonstram que o algoritmo Deep SARSA, baseado em aprendizagem por reforço e redes neuronais, obtém frequentemente um melhor desempenho em comparação com a configuração manual, de cariz completamente aleatório. Este facto pode ser comprovado pela análise dos resultados das médias do número de execuções, nomeadamente, no problema das UC, onde o valor do teste ANOVA apresentou um 𝑝-value significativo igual a 0.014. Este desfecho sugere que abordagens dinâmicas de ajuste de parâmetros podem ser mais eficazes e oferecer uma alternativa viável a métodos estáticos de configuração, que possam potenciar soluções propostas para enfrentar os desafios em ambientes dinâmicos e incertos.
- AI-driven information retrieval system for candidate screeningPublication . Silva, Vasco Reid Ferreira da; Conceição, Luís Manuel da SilvaEfficient screening and evaluation in the recruitment process are tasks that demand substantial time and effort from Human Resources professionals. These processes often suffer from long waiting periods, inconsistent candidate evaluation, and the potential to overlook qualified candidates. In this context, leveraging state-of-the-art natural language processing architectures, specifically large language models (LLMs), holds significant promise. LLMs can generate evaluations using advanced prompt techniques to improve the accuracy and reliability of the output. This thesis researches the feasibility of employing 7 billion parameter LLMs in candidate screening to reduce response times, decrease workload, and improve evaluation consistency. The study involves a comparative analysis of various state-of-the-art large language models to identify those most suitable for this application. Additionally, it examines different prompt engineering techniques to optimize the performance of these models. A comprehensive analysis of the results is conducted to determine the most effective combinations of LLMs and prompt engineering techniques. This includes a two-way validation process, utilizing both the state-of-the-art GPT-4 model and manual human resources validation, to ensure the robustness and reliability of the findings. The outcomes of this thesis aim to enhance the quality of candidate screening by integrating LLMs into the process. Furthermore, this work aspires to provide valuable insights into the capabilities of 7 billion parameter large language models in the field of human resources and their application in real-world scenarios.
- Development and evaluation of a complex-valued neural network library: the Renplex open-source projectPublication . Alves, Pedro Manuel Ferreira; Faria, Luiz Felipe Rocha deComplex-Valued Neural Networks (CVNN) have shown to be a promising type of Artificial Neural Networks (ANN) when compared to its real-valued counter-parts. However, it has been a research field where authors autonomously developed and tested CVNN with no common tools or library to module them. This Master Thesis presents a library called Renplex capable of modulating CVNN as an open-source project for research and even for small scale applications. Although not suitable for beginners in the field of ANN or programming, the library provides a low-level interactive with Machine Learning (ML) pipeline, in order to accurately control CVNN evaluation. To test the library’s core functionalities, architectures such as Complex-Valued Multi-Layer Perceptron, Auto-encoder and Convolutional Neural Network were trained. These achieved test results that outperformed their real-valued counterparts for the MNIST dataset and a synthetically generated dataset for signal reconstruction. Such improvement on performance, has been previously stated throughout literature. It consisted in greater test accuracy (or lower loss values), more stability in training, faster convergence in terms of epochs needed, greater capability of generalization, and subsequently less prone to over-fitting. This work will introduce a new tool for exploring CVNN, capable of scaling and potentially uncovering many of their hidden potentials for ML-related tasks.
- Intelligent recommendation system to enhance youth football athletes developmentPublication . Fraga, Pedro Filipe Monteiro; Martins, António Constantino Lopes; Matos, Paulo Sérgio dos SantosWith the constant evolution of sports, technological advancements are constantly being made with the objective of maximizing the performance of teams and players. Looking specifically at football, which is the most played sport across the world, teams are constantly looking to develop the best players, and that process starts when they are young. However, only the top football clubs in the world have the funding necessary to create the best conditions to improve the best players. Consequently, for most of the football academies, it is hard to maximize the potential of their young players and enhance their growth. Therefore, this thesis pretends to solve this issue by proposing a football recommendation system for young athletes improvement, where the main objectives were to help the coaching staff by recommending the most relevant skills for a player to improve, using different approaches to achieve these recommendations, such as the input of experts, test them in a real life football academy environment, with the aid of recommendation system evaluation metrics, and discuss the results obtained. In the State of the Art chapter, a systematic literature review with PRISMA methodology was used, to assess the existing recommendation systems in football and their algorithms, as well as the aspects typically included in player modeling. The system was tested in a U14 and a U17 men’s football team, and the results obtained are very good indicators of the effectiveness of the system, showcasing 93.1% of accuracy in the recommendations while maintaining recall, precision and F1-measure values above 80%. The results obtained, combined with an interview with a coach that tested the system, show evidence that the system enhances the growth of the players, by aiding the coaching staff at suggesting the most relevant skills to a player. It also indicates that the system can be implemented in football academies, to complement the coaching staff. There are some limitations to the system, notably not having an interface built, the goalkeeper position not being addressed by the system and the experimentation tests having a relatively small sample size. Future work includes improving the performance of the algorithms, testing on a bigger set of players, adding the goalkeeper position, reanalyzing the skills used to modulate a player and the addition of an interface, which will aid the utilization of the system and enable the customization of the parameters used in the models.
- Metrix analyzer: automatic insights generator for teams performancePublication . Panda, Miguel Francisco Coutinho; Martins, António Constantino LopesThe African continent is witnessing a technological renaissance, with its market competitiveness escalating at an unprecedented pace. In this rapidly evolving landscape, software companies must swiftly adapt to not only survive but thrive. Central to this endeavour is the imperative to enhance team performance, as it is vital for sustaining profitability and driving innovation. As demand for comprehensive performance analysis grows across teams, Agile Coaches are increasingly stretched thin, balancing the urgent need for insights with their ongoing projects. This challenge underscores the critical necessity for advanced automation—a solution that can streamline the generation of actionable insights while requiring fewer human resources. It was within this context that the Metrix project was born. This dissertation examines the detailed and step-by-step process of developing a hybrid expert system solution designed to meet the growing needs of the company. It covers the technical research carried out, the careful analysis of requirements, and the practical application design that laid the groundwork for the project. Additionally, it describes the data preparation and the experimentation phase that led to accuracy results of 85%. By the conclusion of this thesis, the results and insights resultant from this journey will be presented, illuminating the transformative potential of the Metrix project in revolutionizing team performance analysis and paving the way for future advancements.
- Sistema de vigilância inteligente para ambientes domésticos com visão computacionalPublication . Costa, Miguel Ferreira da; Martins, António Constantino LopesNos dias de hoje, num mundo onde a segurança privada é cada vez mais crucial, a Inteligência Artificial emerge como uma aliada poderosa para proteger os nossos lares. No entanto, os sistemas de vigilância para ambientes domésticos ainda enfrentam desafios significativos, nomeadamente, na falta de personalização e na dependência de tecnologias tradicionais, que não conseguem atender às exigências modernas de segurança. Esta dissertação apresenta o sistema iDetect, um sistema de vigilância inteligente com deteção e reconhecimento facial voltado para ambientes domésticos. O sistema utiliza uma combinação de tecnologias tradicionais e deep learning, alcançando uma precisão de 93% na deteção e uma acurácia de 97% para o reconhecimento facial, com base numa seleção criteriosa de algoritmos, utilizando uma versão adaptada da metodologia PRISMA. Devido à crescente utilização de smartphones, o sistema possui a sua aplicação móvel, totalmente configurável e intuitiva, que permite ao utilizador personalizar o sistema e receber notificações e alertas em tempo real. Os resultados mostram que o sistema implementado oferece uma solução eficaz e acessível, que une o poder da Inteligência Artificial à portabilidade dos dispositivos móveis, proporcionando uma resposta mais rápida e eficiente às ameaças domésticas. Foi possível implementar um sistema que supera os atuais sistemas de videovigilância doméstica, fornecendo respostas mais ágeis e precisas, reduzindo significativamente o número de falsos positivos. O sistema desenvolvido representa um avanço no campo da segurança doméstica, ao disponibilizar uma solução inovadora e de fácil utilização que pode ser adaptada às necessidades individuais de cada utilizador.
- Application of active learning on medical images to enhance machine learning modelsPublication . Santos, Maria Inês Salvador dos; Marreiros, Maria Goreti CarvalhoArtificial intelligence has made some huge advancements in the healthcare field, particularly in medical imaging. However, data and annotations in this area are often scarce and expensive to obtain. Labeling images, although essential for machine learning models, is a tedious and time-consuming task. Active learning addresses this challenge by selecting informative samples to try and create a subset of unlabeled data where the model could have more difficulty predicting the labels which are then given to experts to annotate. The goal is to try to use less amount of annotated data, whilst still getting a good model performance. Breast cancer is one of the most common cancers in women. The proposed solution uses the Patch- Camelyon dataset, a variation of the Camelyon16 dataset with patches from histopathologic scans of sentinel lymph node sections for the detection of metastatic tissue of breast cancer patients. This work proposes an active learning approach that includes the division of the unlabeled data into clusters which are then classified based on their level of informativeness (based on Shannon Entropy). Then, from each cluster several samples are selected based on the previously defined informativeness level and each sample is scored based on a formula that includes both entropy and Euclidean distance to the cluster centroid. Finally, samples with the lowest uncertainty score are added to the training dataset with the model’s prediction. The proposed method includes both model uncertainty and data distribution. The solution showed promising results when compared with a random sampling approach. To evaluate the proposed solution, greyscale and Macenko normalization techniques were used in all different approaches (random sampling approach, a variation of the proposed solution with no pseudo label task and the proposed solution). In some iterations, the difference between the F1 score in the proposed active learning solution and random sampling was more than 0,20. With the application of this method, experts can spend less time annotating images while still achieving a high-performance model.
- Ajuste dinâmico de dificuldade em videojogos usando aprendizagem automáticaPublication . Felício, Jorge Emanuel Coelho Mendonça de Anciães; Faria, Luiz Felipe Rocha deIn the constantly evolving field of video games, traditional difficulty settings fail to accommodate the wide range of skill levels among players. The resulting mismatch between the player’s skill and the game’s challenge can make the game boring for skilled players or frustrating for less experienced ones, negatively affecting player engagement. Dynamic Difficulty Adjustment (DDA) seeks to resolve this issue by adapting the game’s difficulty in real time in response to the player’s performance. While advancements in artificial intelligence (AI), particularly machine learning (ML), have enabled more adaptive DDA systems, the full potential of certain advanced techniques or tools has yet to be explored. This thesis thus explores possible innovations in the integration of AI in DDA systems for video games. The research begins by reviewing the techniques used for DDA, focusing on methodologies such as player modeling, rule-based systems, and ML. Based on this research, potential areas for innovation were identified and the application of Deep Reinforcement Learning (DRL) in the Unity game development platform through the usage of the MLAgents toolkit was chosen as a promising approach for this research. Using this methodology, this research aims to implement a DDA system that adjusts a game’s difficulty based on the player’s skills, enhancing their engagement and maintaining a consistent challenge. This project has several critical phases of development, including the creation of a game prototype, data collection for model training, development and integration of the DDA system into the game prototype, and conducting an experiment comparing the prototype with DDA integrated with a version of the prototype that used traditional static difficulty scaling. The experiment conducted was done with 20 participants of varying skill levels and used a combination of collected gameplay metrics and a modified Game Experience Questionnaire (GEQ) survey to evaluate the DDA system’s effectiveness. The results showed that the DDA system demonstrated a statistically significant increase in the player engagement component and appropriately adjusted the difficulty to be harder for participants of higher skill. However, the system sometimes exhibited some issues with drastic adjustments in difficulty between levels, which led to a slightly lower Post-Game positive experience score compared to the static difficulty scaling system. Despite these fluctuations, the proposed system demonstrates the potential of the ML-Agents toolkit in implementing DDA with DRL in games made on the Unity platform. By identifying underexplored areas in the current literature and applying advanced techniques like DRL, this thesis aims to contribute to both academic research and game development regarding the approach to DDA in video games.