ISEP - DM – Engenharia de Inteligência Artificial
Permanent URI for this collection
Browse
Recent Submissions
- Gestão inteligente e distribuída de comunidades de cidadãosPublication . SILVA, RAFAEL DUARTE PEREIRA DA; Gomes, Luís Filipe de Oliveira; Vale, Zita Maria Almeida doA utilização e integração de modelos inteligentes nos edifícios pode transformar as experiências dos utilizadores dentro do edifício, proporcionando a otimização dos espaços e formas eficientes de utilizar e interagir com os recursos do edifício. A utilização de soluções inteligentes traz alguns desafios que devem ser estudados, como a heterogeneidade entre os recursos e a necessidade de adaptar os edifícios já existentes ao conceito de edifícios inteligentes. Embora os edifícios inteligentes possam revolucionar a forma como as pessoas utilizam e interagem com os espaços, o grupo de edifícios, ao criar comunidades, traz novas oportunidades para permitir que os membros interligados atinjam objetivos comuns, modelando papéis cooperativos, colaborativos e, por vezes, competitivos. Esta nova dinâmica em que os sistemas orgânicos podem comunicar e interagir também levanta desafios quanto à modelação dos utilizadores, às suas preferências e à existência de infraestruturas comuns para permitir a implementação de modelos inteligentes ao nível da comunidade, edifício e utilizador. Esta dissertação tem como objetivo conceber, implementar, testar e validar uma infraestrutura baseada em containers, intitulada Caravels, que combina os conceitos de comunidades inteligentes e edifícios inteligentes para desenvolver uma solução sensível ao contexto que considera diferentes utilizadores e edifícios. A solução concebida emprega uma arquitetura distribuída para a gestão de comunidades inteligentes de cidadãos, onde cada membro opera como uma entidade autónoma, enquanto permanece interligado através de uma infraestrutura partilhada. A arquitetura permite serviços tanto a nível local como comunitário, sendo que um membro pode integrar serviços individuais, escolhidos especificamente para esse utilizador, ao mesmo tempo que contribui e beneficia de otimizações a nível comunitário. Central ao projeto está a modelação das preferências do utilizador em ambientes complexos, dinâmicos e multiutilizador. A dissertação explora os desafios psicológicos e cognitivos da representação de preferências, reconhecendo que os utilizadores têm dificuldades em articular ou priorizar as suas próprias preferências. Os modelos propostos podem adaptar-se ao longo do tempo, incorporando feedback e dados comportamentais para apoiar a tomada de decisões proativas e conscientes do contexto. As técnicas de inteligência artificial, incluindo a aprendizagem supervisionada, não supervisionada e por reforço, estão integradas em todo o sistema para permitir a análise preditiva, a otimização e o controlo autónomo. Para validar a arquitetura e as metodologias propostas, foram conduzidos vários estudos de caso em cenários realistas, refletindo as diferentes necessidades dos utilizadores, procura de energia e recursos distribuídos. Os resultados demonstram que o sistema pode modelar o comportamento do utilizador, apoiar a cooperação a nível comunitário e melhorar a eficiência e a inteligência geral do edifício inteligente. Os resultados desta dissertação contribuíram para seis publicações científicas, incluindo uma revista com um fator de impacto de 6,6.
- MASterFLow: Cadeia de sistemas multiagente inteligentes para a criação de pipelines de aprendizagem automática e aprendizagem federadaPublication . BARBARROXA, RAFAEL ALEXANDRE SILVA; Gomes, Luís Filipe de OliveiraThe growing demand for secure, privacy-preserving AI solutions is particularly noticeable in domains such as renewable energy or healthcare, where sensitive data is involved. As society continues to transition to AI-driven systems, the need for decentralized machine learning systems has become increasingly evident. Traditional machine learning methods rely heavily on centralized datasets, often compromising privacy and security. Although federated learning addresses these concerns by enabling decentralized model training while maintaining data privacy, several challenges remain. These include the complexity of creating, configuring, and managing federated learning models, particularly when dealing with a large number of clients and different configurations. As federated learning grows in popularity, there is also a need for more automated solutions that can simplify this process for users with varying levels of expertise. This dissertation presents MASterFLow, a novel system that combines multi-agent systems with large language models to intelligently create machine learning models and federated learning federations. By integrating LLMs and Retrieval-Augmented Generation, MASterFLow provides an efficient way to configure, execute, and analyze FL training simulations. The system streamlines the process by allowing users to interact with intelligent agents that manage different tasks, such as configuring machine learning models, setting up federated learning simulations, and analyzing training logs. MASterFLow is designed with a user-friendly web-based interface that allows users to engage with the system’s agents and configure simulations according to their needs. Through extensive case studies, the dissertation benchmarks various multi-agent frameworks and demonstrates the effectiveness of combining multi-agent systems and large language models to automate the creation of machine learning and federated learning pipelines. The results indicate that MASterFLow provides a more accessible, secure, and adaptable alternative to traditional machine learning methods, offering improved efficiency and usability for AI development.
- Técnicas avançadas de inteligência artificial para a deteção e rastreio de doenças gastrointestinaisPublication . PEREIRA, HUGO SIMÃO DA ROCHA; Martinho, Diogo Emanuel PereiraAs doenças gastrointestinais têm vindo a aumentar devido a vários fatores associados ao estilo de vida moderno, como por exemplo uma alimentação inadequada, sedentarismo e tabagismo. A colonoscopia continua a ser o método de referência para o diagnóstico de patologias intestinais, permitindo a deteção e tratamento de lesões. No entanto, a sua precisão depende fortemente da experiência do médico, resultando em variabilidade nos diagnósticos e potenciais atrasos na deteção de condições críticas. Para além disso, o procedimento de colonoscopia pode exigir a realização de biópsias invasivas, que, embora essenciais para diagnóstico definitivo, acarretam riscos e desconforto para os pacientes. Esta dissertação de tese explora a integração de técnicas de visão computacional e deep learning para otimizar a análise de colonoscopias, com o objetivo de melhorar a precisão na deteção de lesões e apoiar a tomada de decisão clínica. Através do uso de redes neuronais convolucionais (CNNs) e modelos de segmentação como o ResNet, DenseNet e Inception, esta investigação propõe o desenvolvimento de um sistema baseado em inteligência artificial capaz de identificar e classificar lesões colorretais com maior precisão e consistência. O sistema proposto visa complementar a experiência médica, reduzindo a variabilidade nos diagnósticos e otimizando os processos de rastreio. Os resultados desta dissertação mostram que os modelos híbridos, que combinam diferentes arquiteturas convolucionais, superaram os modelos baseados apenas em transfer learning, que apresentaram desempenhos insatisfatórios. A melhor performance foi alcançada pelo modelo híbrido ResNet + EfficientNet + DenseNet, com accuracy de 86,67%. Esses resultados sugerem que a abordagem híbrida é mais eficaz para a deteção de lesões gastrointestinais, podendo contribuir para diagnósticos mais rápidos e precisos, além de reduzir a necessidade de biópsias desnecessárias.
- From relational waters to intelligent oceans: A lakehouse-centric approach to conversational artificial intelligencePublication . FIGUEIREDO, JOANA RODRIGUES; Gomes, Luís Filipe de Oliveiraof handling large volumes of heterogeneous and unstructured data while enabling real-time intelligent decision-making. In the water management domain, where legacy systems and operational complexity often obstruct innovation, there is an increasing need to adopt artificial intelligencepowered solutions that promote efficiency, traceability, and accessibility. Responding to this challenge, this dissertation presents CLARA — a Conversational Lakehouse Architecture supported by Real-time Artificial intelligence. CLARA is a modular solution that integrates modern data infrastructures, artificial intelligence models, and natural language interaction to support intelligent management in water utility operations. CLARA was conceived and developed from scratch, following the data lakehouse paradigm to consolidate structured and unstructured data, such as field images. The infrastructure adopts a medallion architecture (Bronze, Silver, Gold) and includes pipelines for ingestion, loading, and transformation. Particular attention was given to documentation of transformations, and integration of flows for experiment tracking, enabling a robust foundation for artificial intelligence development and data governance. The solution currently features two artificial intelligence models that demonstrate how the lakehouse paradigm can support intelligent reasoning beyond conventional structured data processing. The first is an optical character recognition model, which enables the automated interpretation of water meter readings directly from field images, a type of unstructured data typically excluded from traditional storage systems. This model exemplifies how AI can be embedded into the data architecture to support validation and data quality assurance workflows. The second is a predictive model based on neural networks, designed to anticipate the symptom of the next operational intervention by analyzing historical maintenance sequences. Together, these models illustrate the potential of unifying data storage and artificial intelligence reasoning within a single environment. At the user interaction layer, a custom-built conversational assistant leverages a cascade of large language models to classify and respond to user queries in real-time. The system routes each input to one of four specialized modules: (1) to access structure data in real-time, (2) to execute and access artificial intelligence models, (3) to consult software support manuals, and (4) to provide fallback conversational support only on water-related topics. The assistant also integrates multilingual support and a semantic permission-verification mechanism that maps the user’s intent and role to the structure of the underlying database, preventing unauthorized actions. Developed in partnership with A2O – Água, Ambiente e Organização, Lda., and validated through four real-world case studies, CLARA demonstrated how a carefully orchestrated artificial intelligence pipeline, backed by an efficient data infrastructure, can modernize and improve decision-making, enhance transparency, and simplify access to complex systems through natural language.
- Tetrahedron-Tetrahedron intersection and volume computation using neural networksPublication . PEDRO, ERENDIRO SANGUEVE NJUNJUVILI; Ramos, Carlos Fernando da SilvaThis thesis introduces a framework for fast, learning-based analysis of tetrahedron-tetrahedron interactions, combining scalable dataset generation with an efficient neural model. At its core is TetrahedronPairDatasetV1, a curated collection of one million labeled tetrahedron pairs with ground truth intersection status and volumes, filling a longstanding gap in geometry learning. Built on this dataset, we present TetrahedronPairNet, a neural architecture that adapts PointNet and DeepSets for processing tetrahedron pairs. The model simultaneously predicts intersection classification and intersection volume, achieving real-time performance: over 98% classification accuracy and a mean absolute error of ≈ 0.0012 in volume estimation (R2 = 0.68). It processes over 30,000 samples per second with full preprocessing—orders of magnitude faster than classical algorithms. Unlike traditional symbolic approaches, TetrahedronPairNet is robust to degenerate configurations and requires no handcrafted geometry logic. Its fully batched, differentiable design supports seamless integration into simulation pipelines, CAD tools, and learning-based physics engines. This work reframes geometric intersection as a data-driven inference task, laying the foundation for scalable, real-time, and intelligent geometry processing across computational design, simulation, AR/VR, and scientific computing.
- Melhorar a deteção de anomalias em video com deteção de objetoPublication . PEREIRA, BRUNO ALVES; Soares, Pedro Miguel MachadoVideo Anomaly Detection (VAD) is a critical task in video surveillance and security systems, aiming to automatically identify events that deviate from normal patterns. These systems enable real-time monitoring, offer scalability for processing large volumes of data across diverse environments, and help reduce human error. Despite recent advances, most VAD models rely solely on spatio-temporal features. This project investigates the impact of incorporating contextual information, specifically object-level features, into the pipeline of a State of The Art (SoTA) VAD model. For this aim, we propose modifications in a SoTA model by presenting a new architecture that integrates object detection features. Intermediate and late fusion techniques were explored to determine the most effective method for combining object-level with spatio-temporal features used by the model. The experiments were conducted on a modified version of a SoTA dataset, adapted for weakly supervised training. The findings indicate that integrating object-level features enhances the performance of the baseline model, with improvements observed across three key metrics: Area Under the Curve (AUC), Average Precision (AP), and F1-score, particularly in the late fusion models. Freezing weights of the base model was shown essential to achieve the best results. However, the inclusion of the new channel introduced additional computational costs during training and a slight increase in inference time. Although these factors can affect the scalability of the project, they are not very significant since tasks can be parallelized, or executed in better hardware infrastructures. This work demonstrates that incorporating contextual cues from object detection into existing VAD frameworks can lead to better anomaly discrimination, paving the way for more reliable and context-aware surveillance systems.
- Transfer learning applied to government auditing: A focused approach on financial statements in Maranhão, BrazilPublication . Coelho, Heloisa Guimarães; Marreiros, Maria Goreti CarvalhoSince Brazil’s return to democracy, dozens of laws, decrees and normative instructions have been drafted with the purpose of regulating and improving the mechanisms for controlling and monitoring municipal public resources. These regulations are specifically aimed at the process of accountability by elected officials, who currently rely on the help of accountants responsible for preparing and submitting financial statements to the Courts of Auditors. However, according to data from the TCU (Federal Court of Accounts), in 2023, Maranhão was the Brazilian State with the highest number of rejected accounts. There are several reasons that can lead to these processes being challenged, including incorrect application of resources, flaws in documentation, human errors, among others. In practice, the routine of accountants includes repetitive and mechanical activities that requires considerable time to prepare and review documents, hence often leading to errors in classification and issuing of documentation. In this context, this dissertation investigates the use of Transfer Learning (TL) to improve automation and accuracy in the classification of financial commitment notes, an initial document in the public expenditure cycle, with a specific focus on the context of the state of Maranhão. To this end, BERTimbau, a pre-trained language model for Brazilian Portuguese, was fine-tuned to assist government accountants in reducing classification errors and ensuring compliance with local and national financial regulations. The CRISP-DM methodology, widely used in data science, was adopted to structure the development of the project. The dataset used, consisting of several classifications of commitment notes for the year 2023, was thoroughly analyzed and pre-processed. For the fine-tuning process of the model, two samples with a similar number of data were selected, varying only the number of possible classifications, due to the high degree of imbalance between the classes. Even in a multiclass context with datasets with a reduced number of classes, the results obtained indicate that the BERTimbau model presents strong performance in the classification task, achieving 98% accuracy with an error rate of 0.10 in the test set, highlighting the effectiveness of BERTimbau in public financial auditing applications. These results highlight the effectiveness of BERTimbau for public financial auditing applications. It is therefore concluded that TL models have great potential to optimize and improve financial auditing processes, with positive implications for wider adoption in Brazil.
- Leveraging generative artificial intelligence and wearable technology for adaptive health and conversational interventions in the elderlyPublication . Crista, Vítor Rafael Palmeiro; Martinho, Diogo Emanuel PereiraAging is often associated with an increase in loneliness and social isolation, factors that have significant consequences for health. The lack of social interactions can lead to a decline in emotional well-being, the deterioration of cognitive conditions, and a higher risk of physical health problems. Therefore, it is relevant to develop solutions that facilitate interaction and promote the quality of life of the elderly population, encouraging regular physical activity and improving their physical and psychological well-being. Technology attends as an effective tool to address these challenges, enabling the creation of systems that provide continuous support and interactions adapted to individual needs. This dissertation explores the development of a conversational system that uses wearable technology to collect information about users' health levels, aiming to create proactive interactions, encourage healthy habits, and promote active aging. The system also incorporates artificial intelligence, specifically generative artificial intelligence, with the main objective of developing advanced communication mechanisms between the system and the user. These mechanisms allow the system to learn from past interactions to improve the quality of future ones. The work included designing the system architecture, which integrates a Multi-Agent approach, providing a scalable structure that enables continuous improvements in the quality of the communication strategies used by the system. The solution was developed based on a domain model aligned with the common needs of elderly users and includes a detailed description of the essential components for its implementation. To evaluate the system, a study was conducted with five users, providing valuable insights into the system’s performance and impact. The study was conducted with five participants over a week, and to assess the system’s impact, indicators were defined to measure the key objectives of the research. The results indicate a positive impact on user participation and physical activity, with an average acceptance rate of proposed activities at 63.6%. Additionally, the average number of steps increased by 42.93% throughout the study, and the frequency of conversations recorded an average growth of 20.14%.
- Estratégias de aprendizagem por reforço para configuração dinâmica de meta-heurísticasPublication . Oliveira, Vítor José Henriques; Pinto, Tiago Manuel Campelos Ferreira; Ramos, Carlos Fernando da SilvaA eficácia da otimização de problemas complexos está intimamente ligada à configuração de parâmetros em algoritmos meta-heurísticos. Embora já tenham sido propostos métodos automatizados para a escolha dos parâmetros de algoritmos para reduzir a necessidade de ajuste manual, existe ainda um potencial significativo, não explorado, de ajuste dinâmico de parâmetros de algoritmos durante a execução, o que pode melhorar o seu desempenho. Este estudo visa aferir a eficácia da definição manual de parâmetros em comparação com uma abordagem dinâmica baseada em aprendizagem por reforço, reduzindo a necessidade de intervenção humana e aumentando a eficiência operacional dos algoritmos. Para alcançar este objetivo, adaptaram-se os métodos SARSA (State-Action-Reward-State-Action) e Deep SARSA para regular os parâmetros de algoritmos meta-heurísticos, em especial, o algoritmo genético. O modelo adotado é independente do problema a ser otimizado ou do algoritmo meta-heurístico selecionado, por isso, oferece a flexibilidade necessária, sendo apenas crucial escolher os parâmetros a ajustar durante o decorrer do processo de otimização de qualquer problema estudado. Estas metodologias foram testadas em funções benchmark, amplamente reconhecidas na literatura, e aplicadas nesta investigação nos seguintes cenários práticos: a otimização de portfólios de investimentos, na qual um participante possui ou pretende adquirir energia elétrica num mercado de eletricidade e a melhoria relacionada com a alocação de pacientes em Unidades de Cirurgia (UC) e em Unidades de Cuidados Intensivos (UCI), com o intuito de melhorar a eficiência da utilização de recursos limitados. Os resultados demonstram que o algoritmo Deep SARSA, baseado em aprendizagem por reforço e redes neuronais, obtém frequentemente um melhor desempenho em comparação com a configuração manual, de cariz completamente aleatório. Este facto pode ser comprovado pela análise dos resultados das médias do número de execuções, nomeadamente, no problema das UC, onde o valor do teste ANOVA apresentou um 𝑝-value significativo igual a 0.014. Este desfecho sugere que abordagens dinâmicas de ajuste de parâmetros podem ser mais eficazes e oferecer uma alternativa viável a métodos estáticos de configuração, que possam potenciar soluções propostas para enfrentar os desafios em ambientes dinâmicos e incertos.
- AI-driven information retrieval system for candidate screeningPublication . Silva, Vasco Reid Ferreira da; Conceição, Luís Manuel da SilvaEfficient screening and evaluation in the recruitment process are tasks that demand substantial time and effort from Human Resources professionals. These processes often suffer from long waiting periods, inconsistent candidate evaluation, and the potential to overlook qualified candidates. In this context, leveraging state-of-the-art natural language processing architectures, specifically large language models (LLMs), holds significant promise. LLMs can generate evaluations using advanced prompt techniques to improve the accuracy and reliability of the output. This thesis researches the feasibility of employing 7 billion parameter LLMs in candidate screening to reduce response times, decrease workload, and improve evaluation consistency. The study involves a comparative analysis of various state-of-the-art large language models to identify those most suitable for this application. Additionally, it examines different prompt engineering techniques to optimize the performance of these models. A comprehensive analysis of the results is conducted to determine the most effective combinations of LLMs and prompt engineering techniques. This includes a two-way validation process, utilizing both the state-of-the-art GPT-4 model and manual human resources validation, to ensure the robustness and reliability of the findings. The outcomes of this thesis aim to enhance the quality of candidate screening by integrating LLMs into the process. Furthermore, this work aspires to provide valuable insights into the capabilities of 7 billion parameter large language models in the field of human resources and their application in real-world scenarios.