ISEP - DM – Engenharia de Inteligência Artificial
Permanent URI for this collection
Browse
Recent Submissions
- Modelos híbridos para previsão de resultados de jogos da Premier League usando machine learning e análise de sentimentoPublication . NASCIMENTO, RUBENS FABRÍCIO DO ROSÁRIO SOARES; Ramos, Carlos Fernando da SilvaThis study explores whether combining structured match statistics with pre-match tweet sentiment can enhance probabilistic forecasting of football results. Focusing on English Premier League fixtures, it aligns social signals with each game and compares three families of models: those based solely on statistics, those relying only on tweets, and hybrid approaches that integrate both. The evaluation respects the chronological order of matches, employing sequential training and validation together with a strict 2024/25 holdout. In terms of assessment, Log Loss serves as the primary metric, complemented by calibration measures (ECE, Brier, RPS) as well as accuracy. When comparing different families of models, statistical learners provide the strongest foundation. Within this group, an RBF-SVM delivers a holdout Log Loss of 0.9066 with 58.16% accuracy, while a regularised Logistic Regression remains competitive, suggesting that engineered features capture a substantial linear signal. By contrast, tweet-only models offer useful but weaker contributions. The best-performing configuration, a Linear SVM applied to SBERT-MPNet embeddings, records a Log Loss of 1.0313 and an accuracy of 47.89%, yet generalises consistently across both validation and test. Across the different model families, hybrid approaches provide the most consistent improvements. In particular, Early Fusion with Logistic Regression, which combines sentiment with structured inputs, delivers 59.74% accuracy and a Log Loss of 0.8954 on the holdout, together with a Brier Score of 0.1758 and an RPS of 0.1171. Moreover, Residual Stacking extends these gains by further reducing both Log Loss and Expected Calibration Error compared with the statistical baseline, with the benefits especially clear in lower-confidence fixtures and in predicting draws. The main improvements come from modest probability refinements that reduce error penalties without frequent class flips, while also enhancing calibration. At the same time, certain limitations remain, including the focus on a single league, the risk of temporal drift in team performance, and the presence of noise, ambiguity, and attention bias in social text. Taken together, the findings demonstrate that combining structured match data with curated sentiment yields robust and well-calibrated forecasts, particularly valuable in uncertain fixtures and in outcomes that are traditionally harder to predict.
- Evaluation of explainability AI (XAI) techniques for mitigating ethical and legal challengesPublication . ESQUIÇATO, RAFAEL PORCIDONIO FERNANDES; Marreiros, Maria Goreti CarvalhoThe integration of Artificial Intelligence (AI) into healthcare systems raises significant ethical and legal concerns. This study investigates how Explainable AI (XAI) techniques can enhance the transparency and trustworthiness of medical image classification systems. Through a systematic literature review of 860 papers and experiments using COVID-19 radiography and skin lesion datasets, the research identifies and evaluates XAI methods such as Grad-CAM, SHAP, and ABELE. These methods were assessed for their ability to clarify decision-making processes, improve model accountability, and support regulatory compliance. The study proposes an explainability module that combines different techniques to provide human-readable explanations, aiming to bridge the gap between AI predictions and clinical trust. Findings indicate that XAI not only addresses transparency and bias issues but can also improve diagnostic performance and decision support in critical applications.
- Design and implementation of a low-cost computer vision pipeline for amateur football analysisPublication . ALVES, RAFAEL NUNO DE SOUSA; Matos, Paulo Sérgio dos Santos; Martins, António Constantino LopesThe advancement of computer vision and artificial intelligence has opened new possibilities for sports analytics, particularly in football. This dissertation explores the development of an AI-powered multi-platform application designed to track and analyze amateur football matches without the need for wearable sensors. By leveraging computer vision techniques such as object detection, multi-object tracking, and real-time analytics, this research aims to provide an accessible and cost-effective solution for performance analysis in amateur football. The work presents a systematic review of existing methodologies, identifying key challenges such as occlusion, motion blur, and real-time computational constraints. A methodological framework based on the Design Science Research (DSR) approach guides the investigation, ensuring iterative development, validation, and refinement of the proposed system. The findings of this study lay the groundwork for the future implementation of a fully functional AI-based tracking system. Over the next six months, the research will transition into a practical phase, involving model training, system deployment, and real-world testing. By addressing the identified challenges and leveraging recent advancements in AI and computer vision, this project aims to bridge the gap between professional and amateur sports analytics.
- Ai-driven emotion recognition for mental health diagnoses: Assessing mental health through emotional state evaluationPublication . PRETO, PEDRO MIGUEL PERES; Conceição, Luís Manuel Silva; Figueiredo, Ana Maria Neves Almeida BaptistaMental health conditions remain a concerning challenge across the globe, requiring timely and reliable approaches to correctly make accurate diagnoses and effective interventions. Traditional assessment methods often rely on subjective self-reports and clinical interviews, which may not always capture the full spectrum of an individual’s emotional state. In this context, computational techniques for emotion analysis provide a complementary perspective by identifying patterns in facial expressions, speech, and language. This dissertation evaluates the potential of multimodal emotional state analysis and its contribution to mental health assessment, through the development of a computational application. A systematic review was conducted to evaluate existing methodologies and highlight their strengths, limitations, and applicability in clinical contexts. Building on this review, the present work explores an integration of visual, vocal, textual patterns, assessing the contribution of their combined capacity to improve the consistency and depth of emotional interpretation. An analysis centered on methodological design was conducted by applying techniques such as preprocessing, fine-tuning, and data augmentation on the datasets to enhance the model’s capacity. Ethical and security considerations were also incorporated to strengthen system robustness and ensure responsible deployment in the market. The proposed solution consists of an artificial intelligence based multimodal system that integrates the analysis of emotions present in facial expressions, voice, and text patterns to provide a comprehensive assessment of the user’s emotional state. The application’s modular architecture enables real-time processing and the generation of clinical reports. The experimental validation of the system revealed promising results across several DSM-5 domains, the clinical reference manual that defines diagnostic criteria for mental disorders cases. High F1-scores were recorded in domains such as Anger (0.84) and Personality Functioning (0.87), while more subtle domains, such as Dissociation (0.43) and Repetitive Behaviors (0.52), revealed more modest performance. The overall analysis resulted in an observed agreement level of 71.9% and a Cohen’s Kappa of 0.42, indicating moderate agreement with the DSM-5. The findings underline the promise of computational emotion analysis as a supplementary tool for mental health professionals, while also emphasizing the importance of critical evaluation of its limitations and careful integration into clinical practice.
- Sistema inteligente de apoio à arbitragem: Uma abordagem de visão por computador e aprendizagem profunda para a deteção de faltas no futebol amadorPublication . ALVES, NUNO RAFAEL DE SOUSA; Martins, António Constantino Lopes; Matos, Paulo Sérgio dos SantosRecent advancements in computer vision and artificial intelligence have revolutionized sports technology, particularly in professional football through systems like Video Assistant Referee (VAR). However, a significant technological gap exists between professional and amateur levels of the sport, with professional systems costing thousands of euros remaining inaccessible to grassroots football. This research addresses this disparity by developing an automated referee assistant system using consumer-grade smartphones. The proposed system integrates three specialized YOLO-based models: YOLOv12 for player and ball tracking, a custom-trained YOLO11 Pose model for field keypoint detection, and another YOLO11 Pose model for player pose estimation. A key innovation is our proximitybased processing strategy that triggers pose analysis only when players are near the ball, reducing computational overhead by approximately 65% while maintaining detection accuracy. The system employs dual-camera panoramic stitching to achieve 180-degree field coverage, overcoming parallax challenges through optimized camera positioning guidelines. Our implementation achieves 86.8% mean average precision for player detection and 99.5% for field keypoint detection, though ball detection remains challenging at 51.7% due to object size limitations. The system successfully detects handball violations outside the penalty area and ball out-of-bounds situations in real-time at 15-20 frames per second. We created a custom dataset of 500 annotated images with 27 field keypoints, addressing a critical gap in publicly available football field detection resources. While the system faces limitations in 3D spatial analysis for airborne balls, it demonstrates that meaningful referee assistance is achievable with consumer hardware. This research contributes to democratizing sports technology by providing an accessible, cost-effective solution that brings automated officiating capabilities to amateur football, where the vast majority of matches worldwide currently lack any technological support.
- Bridging automation and customization: MLOps in recommender system developmentPublication . JORDÃO, MIGUEL JOSÉ RIBEIRO; Pereira, Isabel Cecília Correia da Silva Praça GomesRecommender systems have become essential in modern digital platforms, supporting decision making and personalization across domains such as e-commerce, media, and enterprise applications. At BMW Group, MyWorkplace (MWP) is a centralized hub managed by Critical Techworks (CTW) that provides access to hundreds of internal tools. Discoverability remains challenging given the size and heterogeneity of the tools catalog. This creates inefficiencies, highlighting the need for a scalable, reliable, and auditable recommendation solution. This project presents an MLOps-first approach for a recommender grounded in the CRISPML(Q) process model. It characterizes the recommendation problem, available data sources, and success criteria, and proposes a reference architecture integrating automated ETL, feature preparation, containerized training and serving, and CI/CD for continuous delivery. Several content-based approaches are implemented and evaluated under realistic data constraints using established ranking metrics; collaborative and hybrid extensions are outlined for future phases once interaction feedback becomes available. The contributions of this work are both technical and methodological: the design and validation of a recommendation strategy for the hub platform; an assessment of operational and governance requirements, including security and compliance, and the demonstration of the system in a real-world industrial environment. In addition to the deployment within BMW Group, this project advances the understanding of how MLOps principles can be applied to balance automation and customization in recommender systems. Results indicate that an MLOps-first design improves scalability, maintainability, and auditability, and lays the groundwork for collaborative filtering, feedback loops, and, when governance permits, large language model components. The system and methodology are applicable to enterprise-scale recommendation scenarios with similar operational constraints.
- Financial reporting with GenAIPublication . CHEN, MIGUEL HUANG; Santos, Joaquim Filipe Peixoto dosFinancial reporting is a critical but time-consuming activity in the banking sector, traditionally requiring analysts to manually extract data, validate figures, and draft lengthy reports. This thesis investigates the use of Generative AI, specifically a GPT-based API, to automate the reporting workflow. A key contribution lies in the design of structured prompt engineering strategies that constrain outputs, ensure numeric accuracy, and enforce corporate formatting requirements. The proposed framework integrates three components: (i) a data extraction tool for structured retrieval of financial indicators, (ii) a Python-based orchestration layer that preprocesses data, builds prompts, and manages interaction with the Generative AI API, and (iii) a report assembly module that converts the AI’s HTML output into fully formattedWord documents. A Streamlit-based interface centralizes usage, enabling analysts to configure parameters, trigger generation, and download reports seamlessly. Evaluation followed an iterative approach with weekly user feedback cycles. Results show a reduction of over 90% in preparation time compared to the manual workflow, alongside improved consistency and reduced operational risk. The framework also demonstrated adaptability across different use cases, from quarterly statements to daily transaction-level reports. While limitations remain—such as dependence on data quality and model fragility—the study demonstrates that LLM-driven prompt engineering can deliver scalable, auditable, and efficient financial reporting automation.
- AI-based synthesis of bacterial colony evolution imagesPublication . SILVA, MIGUEL ÂNGELO FERRAZ DA; Martinho, Diogo Emanuel Pereira; Marreiros, Maria Goreti CarvalhoThe growing demand for safety and efficiency in healthcare highlights the importance of optimising sterilisation procedures, where delays or errors can compromise patient outcomes. In this context, microbiological analysis of agar plates is a fundamental step, as it allows the identification of microbial growth that may compromise sterilisation quality. However, traditional inspection methods are time-consuming and rely heavily on manual observation, which limits their scalability in clinical environments. Meanwhile, Artificial Intelligence has demonstrated strong potential in image analysis and forecasting, offering opportunities to enhance microbiological analysis and support decision-making in healthcare workflows. This dissertation addresses the problem of detecting and predicting the growth of bacterial colonies on agar plates. Anticipating how colonies evolve is essential to evaluate contamination levels, yet this task remains challenging due to the natural variability of growth patterns, the occurrence of overlapping colonies, and the diversity of experimental conditions that affect microbial behaviour. To tackle this problem, an integrated application was developed and structured into three main modules. The first is a detection module that applies the YOLO object detection architecture to identify bacterial colonies from agar plate images. The second is a synthetic forecasting module based on convolutional autoencoders capable of predicting future colony states from early observations. The third is a contamination analysis module that translates predictions into interpretable indicators such as colony count, average size, growth rate, and coverage. Together, these modules form a complete pipeline designed to combine visual fidelity with biological relevance. The results show that the system can detect colonies with high accuracy, achieving a Precision of 99.1%, a Recall of 91.7%, and an F1 score of 95.3%. In addition, the forecasting module generated realistic predictions of colony growth, and the contamination analysis provided meaningful metrics across different experimental conditions. The exploration of different temporal intervals revealed complementary trade-offs between predictive detail and biological plausibility, reinforcing the flexibility of the proposed methodology. The main conclusion of this dissertation is that Artificial Intelligence can be effectively applied to predict microbial growth in laboratory settings. By integrating detection, forecasting, and contamination analysis within a single framework, this work establishes a technological foundation that supports the transition to more intelligent sterilisation workflows and contributes to the broader vision of safe, efficient, and smart healthcare environments.
- Robótica inteligente no reconhecimento de instrumentos médicosPublication . RIBEIRO, MARIANA DA SILVA; Martinho, Diogo Emanuel Pereira; Santos, Joaquim Filipe Peixoto dosA contagem de instrumentos cirúrgicos antes e depois de uma cirurgia é uma etapa fundamental para garantir que nenhum objeto fica perdido no bloco operatório ou no interior do paciente. Apesar de parecer uma tarefa simples, este processo demora em média cerca de cinco minutos, podendo prolongar-se até dez devido a interrupções, o que afeta a fluidez da cirurgia e pode ter consequências negativas para o paciente. Com o intuito de apoiar esta tarefa crítica, é proposta uma solução integrada que combina Visão Computacional e Robótica para reconhecimento, contagem e manipulação de instrumentos cirúrgicos. A abordagem desenvolvida tem por base o robô educativo DOFBOT-Pi, um manipulador de seis graus de liberdade equipado com câmara, que serviu de plataforma experimental para a execução de tarefas de pick-and-place. Para possibilitar esta integração, foram desenvolvidos módulos de cinemática direta e inversa, calibração da câmara e conversão de coordenadas, assegurando a correspondência entre as deteções visuais e as posições reais no espaço tridimensional. No domínio da Visão Computacional, foi concebido um processo de treino baseado em arquiteturas da família YOLO, explorando variantes das séries YOLOv5, YOLOv8 e YOLOv11. Para tal, recorreram-se a dois conjuntos de dados: o Labeled Surgical Tools, um dataset da literatura com mais de três mil imagens distribuídas por quatro classes de instrumentos (bisturi, pinça, tesoura Mayo reta e tesoura Mayo curva), e o Robo Tools, capturado com a câmara do robô, que permitiu avaliar o desempenho em condições reais. O processo experimental foi estruturado em quatro fases: avaliação de modelos de base, combinação de hiperparâmetros, treino aprofundado das melhores combinações e, por fim, adaptação com imagens reais do robô. Os resultados demonstraram uma evolução clara entre arquiteturas, com o YOLOv5 a revelar maiores dificuldades e as séries YOLOv8 e YOLOv11 a atingirem desempenhos próximos, ambos com valores de mAP50 de 91% em teste. A escolha final recaiu sobre o YOLOv11n, uma vez que alia robustez de deteção a elevada eficiência computacional, sendo adequado para execução em tempo real no Raspberry Pi 5 Em conclusão, a solução proposta comprova a viabilidade da contagem e manipulação assistidas por visão computacional e constitui um primeiro passo para futuras aplicações em contexto cirúrgico. As limitações identificadas, em particular a sensibilidade às variações ambientais, a qualidade da câmara e a precisão limitada da plataforma robótica, apontam para oportunidades de desenvolvimento futuro, com ênfase na utilização de hardware mais robusto e na experiência de modelos de deteção especializados para este campo.
- Personalização no setor de seguros: Segmentação de clientes, recomendação de produtos e previsão de retençãoPublication . PEREIRA, MADALENA ANDREIA MARTINS; Marreiros, Maria Goreti CarvalhoA crescente complexidade do setor segurador, aliada às exigências de clientes cada vez mais informados e diversificados, torna a personalização de produtos e a fidelização dois dos maiores desafios enfrentados pelas seguradoras. Este trabalho explora a aplicação de técnicas de Machine Learning para apoiar a tomada de decisão no setor, através da segmentação de clientes, da recomendação de produtos e da previsão da retenção. A investigação iniciou-se com uma revisão sistemática da literatura, que permitiu mapear as abordagens mais relevantes e identificar lacunas na aplicação de métodos de personalização em seguros. Em seguida, foi desenvolvido um estudo experimental com base num conjunto de dados reais, abrangendo variáveis demográficas, contratuais e de sinistralidade. Foram aplicados algoritmos de clustering para segmentar clientes em grupos homogéneos, modelos supervisionados para identificar o produto mais provável para cada perfil e técnicas de previsão para estimar a probabilidade de retenção. A interpretabilidade dos modelos foi assegurada através do método SHAP, permitindo compreender o impacto relativo de cada variável nas previsões. Os resultados mostraram que a segmentação consegue identificar padrões claros entre perfis de clientes e os produtos contratados. A recomendação de produtos alcançou resultados promissores, destacando variáveis como idade, antiguidade e capitais segurados como determinantes para a escolha. Já a previsão da retenção revelou-se mais desafiante, com desempenhos modestos, mas evidenciou que fatores estruturais e contratuais têm maior influência na renovação de apólices do que a sinistralidade. Conclui-se que a integração de segmentação, recomendação e previsão constitui uma abordagem viável para apoiar a personalização no setor segurador, ainda que persistam desafios técnicos e de qualidade dos dados. Este estudo abre caminho para investigações futuras que explorem datasets mais ricos, modelos mais sofisticados e a validação em contextos reais de seguradoras.