Loading...
Publication
Robótica inteligente no reconhecimento de instrumentos médicos
| Name: | Description: | Size: | Format: | |
|---|---|---|---|---|
| 41.38 MB | Adobe PDF |
Authors
Abstract(s)
A contagem de instrumentos cirúrgicos antes e depois de uma cirurgia é uma etapa fundamental para garantir que nenhum objeto fica perdido no bloco operatório ou no interior do paciente. Apesar de parecer uma tarefa simples, este processo demora em média cerca de cinco minutos, podendo prolongar-se até dez devido a interrupções, o que afeta a fluidez da cirurgia e pode ter consequências negativas para o paciente. Com o intuito de apoiar esta tarefa crítica, é proposta uma solução integrada que combina Visão Computacional e Robótica para reconhecimento, contagem e manipulação de instrumentos cirúrgicos. A abordagem desenvolvida tem por base o robô educativo DOFBOT-Pi, um manipulador de seis graus de liberdade equipado com câmara, que serviu de plataforma experimental para a execução de tarefas de pick-and-place. Para possibilitar esta integração, foram desenvolvidos módulos de cinemática direta e inversa, calibração da câmara e conversão de coordenadas,
assegurando a correspondência entre as deteções visuais e as posições reais no espaço tridimensional. No domínio da Visão Computacional, foi concebido um processo de treino baseado em arquiteturas da família YOLO, explorando variantes das séries YOLOv5, YOLOv8 e YOLOv11. Para tal, recorreram-se a dois conjuntos de dados: o Labeled Surgical Tools, um dataset da literatura com mais de três mil imagens distribuídas por quatro classes de instrumentos (bisturi, pinça, tesoura Mayo reta e tesoura Mayo curva), e o Robo Tools, capturado com a câmara do robô, que permitiu avaliar o desempenho em condições reais. O processo experimental foi estruturado em quatro fases: avaliação de modelos de base, combinação de hiperparâmetros, treino aprofundado das melhores combinações e, por fim, adaptação com imagens reais do robô. Os resultados demonstraram uma evolução clara entre arquiteturas, com o YOLOv5 a revelar maiores dificuldades e as séries YOLOv8 e YOLOv11 a atingirem desempenhos próximos, ambos com valores de mAP50 de 91% em teste. A escolha final recaiu sobre o YOLOv11n, uma vez que alia robustez de deteção a elevada eficiência computacional, sendo adequado para execução em tempo real no Raspberry Pi 5 Em conclusão, a solução proposta comprova a viabilidade da contagem e manipulação assistidas por visão computacional e constitui um primeiro passo para futuras aplicações em
contexto cirúrgico. As limitações identificadas, em particular a sensibilidade às variações ambientais, a qualidade da câmara e a precisão limitada da plataforma robótica, apontam para oportunidades de desenvolvimento futuro, com ênfase na utilização de hardware mais robusto e na experiência de modelos de deteção especializados para este campo.
Counting surgical instruments before and after surgery is a crucial step to ensure that nothing is left behind in the operating room or inside the patient. Although it seems like a simple task, this process takes an average of about five minutes and can take up to ten minutes due to interruptions, which affects the flow of the surgery and can have negative consequences for the patient. To support this critical task, we propose an integrated solution that combines computer vision and robotics for the recognition, counting, and manipulation of surgical instruments. The approach developed is based on the educational robot DOFBOT-Pi, a six-degree-offreedom manipulator equipped with a camera, which was used as an experimental platform for performing pick-and-place tasks. To ensure this integration, direct and inverse kinematics, camera calibration, and coordinate conversion were developed, ensuring correspondence between visual detections and actual positions in three-dimensional space. In Computer Vision, a training process based on YOLO family architectures was designed, exploring variants of the YOLOv5, YOLOv8, and YOLOv11 series. To this end, two datasets were used: Labeled Surgical Tools, a dataset from the literature with more than 3,000 images distributed across four classes of instruments (scalpel, forceps, straight mayo scissors, and curved mayo scissors), and Robo Tools, captured with the robot’s camera, which allowed performance to be evaluated in real conditions. The experimental process was structured in four phases, beginning with the evaluation of base models, moving on to the combination of hyperparameters, followed by in-depth training of the best combinations, and ending with adaptation using real images from the robot. The results showed a clear evolution between architectures, with YOLOv5 revealing greater difficulties and the YOLOv8 and YOLOv11 series achieving similar performances, both with mAP50 values of 91% in testing. The final choice fell on YOLOv11n, as it combines detection robustness with high computational efficiency, making it suitable for real-time execution on the Raspberry Pi 5. It can be concluded that the proposed solution successfully proves the viability of computer vision-assisted counting and manipulation, constituting a first step towards future applications in a surgical context. The limitations identified, such as sensitivity to environmental variations, camera quality, and reduced precision of the robotic platform, point to opportunities for future development, with particular emphasis on the use of more robust hardware and the specialization of detection models for this field.
Counting surgical instruments before and after surgery is a crucial step to ensure that nothing is left behind in the operating room or inside the patient. Although it seems like a simple task, this process takes an average of about five minutes and can take up to ten minutes due to interruptions, which affects the flow of the surgery and can have negative consequences for the patient. To support this critical task, we propose an integrated solution that combines computer vision and robotics for the recognition, counting, and manipulation of surgical instruments. The approach developed is based on the educational robot DOFBOT-Pi, a six-degree-offreedom manipulator equipped with a camera, which was used as an experimental platform for performing pick-and-place tasks. To ensure this integration, direct and inverse kinematics, camera calibration, and coordinate conversion were developed, ensuring correspondence between visual detections and actual positions in three-dimensional space. In Computer Vision, a training process based on YOLO family architectures was designed, exploring variants of the YOLOv5, YOLOv8, and YOLOv11 series. To this end, two datasets were used: Labeled Surgical Tools, a dataset from the literature with more than 3,000 images distributed across four classes of instruments (scalpel, forceps, straight mayo scissors, and curved mayo scissors), and Robo Tools, captured with the robot’s camera, which allowed performance to be evaluated in real conditions. The experimental process was structured in four phases, beginning with the evaluation of base models, moving on to the combination of hyperparameters, followed by in-depth training of the best combinations, and ending with adaptation using real images from the robot. The results showed a clear evolution between architectures, with YOLOv5 revealing greater difficulties and the YOLOv8 and YOLOv11 series achieving similar performances, both with mAP50 values of 91% in testing. The final choice fell on YOLOv11n, as it combines detection robustness with high computational efficiency, making it suitable for real-time execution on the Raspberry Pi 5. It can be concluded that the proposed solution successfully proves the viability of computer vision-assisted counting and manipulation, constituting a first step towards future applications in a surgical context. The limitations identified, such as sensitivity to environmental variations, camera quality, and reduced precision of the robotic platform, point to opportunities for future development, with particular emphasis on the use of more robust hardware and the specialization of detection models for this field.
Description
Keywords
Visão Computacional YOLO Robótica DOFBOT Contagem de Instrumentos Cirúrgicos Computer vision Robotics Counting surgical instruments