Browsing by Author "Carvalho, Daniel Mota"
Now showing 1 - 1 of 1
Results Per Page
Sort Options
- Exploração, migração e teste de tecnologias de armazenamento baseadas em séries temporaisPublication . Carvalho, Daniel Mota; Oliveira, Paulo Jorge MachadoWithin the dynamic landscape of renewable energy, the intersection with artificial intelligence (AI) and machine learning (ML) is pivotal for optimizing energy management efficiency and profitability. As the renewable energy sector continuously grows, Enlitia, an innovative organization leading this trajectory, faces a critical challenge in managing and storing the escalating volume of data collected by its multidisciplinary team. This data, particularly timeseries data, crucial for AI and ML applications, serves as the lifeblood for providing clients with key business insights. The problem at hand revolves around the effective handling of this ever-increasing amount of data, emphasizing the strategic importance of a robust storage infrastructure. As such, Enlitia acknowledges that the success of its AI and ML applications hinges not only on algorithmic sophistication but also on the reliability, scalability and performance of its data storage infrastructure. This thesis addresses Enlitia's imperative need to develop and implement a more advanced data storage infrastructure, emphasizing adaptability to diverse solutions. The objective is to facilitate quick and efficient data retrieval, enabling the extraction of valuable business insights to meet the demands of a data-driven decisionmaking landscape present in the renewable energy sector. To address this, a project of considerable dimension that encapsulates the problem was chosen for migration to a new data infrastructure as a way to test its effectiveness. The necessary requirements were addressed, which culminated in the decision to migrate the previous database technology, MariaDB, to a technology more appropriate for storing time series data, TimescaleDB. Additionally, dbt, a technology specifically utilized for data transformation processes, was chosen to be adopted in order to improve communication between stakeholders. The effectiveness of the migration from TimescaleDB to MariaDB was benchmarked and analysed for various hardware configurations in accordance with relevant query and write operations, with the execution time found to have been massively improved for queries in the simulated workload environment. The success of the adoption of dbt was also investigated through a questionnaire which inquired about various aspects such as ease of use and communication improvements. The results obtained were generally positive, although some aspects, such as making the technology easier to use, require reconsideration.