| Name: | Description: | Size: | Format: | |
|---|---|---|---|---|
| 4.91 MB | Adobe PDF |
Authors
Abstract(s)
A eficácia da otimização de problemas complexos está intimamente ligada à configuração de parâmetros
em algoritmos meta-heurísticos. Embora já tenham sido propostos métodos automatizados
para a escolha dos parâmetros de algoritmos para reduzir a necessidade de ajuste manual, existe
ainda um potencial significativo, não explorado, de ajuste dinâmico de parâmetros de algoritmos
durante a execução, o que pode melhorar o seu desempenho. Este estudo visa aferir a eficácia
da definição manual de parâmetros em comparação com uma abordagem dinâmica baseada
em aprendizagem por reforço, reduzindo a necessidade de intervenção humana e aumentando a
eficiência operacional dos algoritmos.
Para alcançar este objetivo, adaptaram-se os métodos SARSA (State-Action-Reward-State-Action)
e Deep SARSA para regular os parâmetros de algoritmos meta-heurísticos, em especial, o algoritmo
genético. O modelo adotado é independente do problema a ser otimizado ou do algoritmo
meta-heurístico selecionado, por isso, oferece a flexibilidade necessária, sendo apenas crucial escolher
os parâmetros a ajustar durante o decorrer do processo de otimização de qualquer problema
estudado. Estas metodologias foram testadas em funções benchmark, amplamente reconhecidas
na literatura, e aplicadas nesta investigação nos seguintes cenários práticos: a otimização de portfólios
de investimentos, na qual um participante possui ou pretende adquirir energia elétrica num
mercado de eletricidade e a melhoria relacionada com a alocação de pacientes em Unidades de
Cirurgia (UC) e em Unidades de Cuidados Intensivos (UCI), com o intuito de melhorar a eficiência
da utilização de recursos limitados.
Os resultados demonstram que o algoritmo Deep SARSA, baseado em aprendizagem por reforço
e redes neuronais, obtém frequentemente um melhor desempenho em comparação com a configuração
manual, de cariz completamente aleatório. Este facto pode ser comprovado pela análise
dos resultados das médias do número de execuções, nomeadamente, no problema das UC, onde
o valor do teste ANOVA apresentou um 𝑝-value significativo igual a 0.014. Este desfecho sugere
que abordagens dinâmicas de ajuste de parâmetros podem ser mais eficazes e oferecer uma alternativa
viável a métodos estáticos de configuração, que possam potenciar soluções propostas para
enfrentar os desafios em ambientes dinâmicos e incertos.
The effectiveness of optimizing complex problems is closely tied to parameter configuration in meta-heuristic algorithms. Although automated methods for selecting algorithm parameters have already been proposed to reduce the need for manual tuning, there remains significant untapped potential for dynamic parameter adjustment during algorithm execution, which could improve performance. This study aims to assess the effectiveness of manual parameter setting compared to a dynamic approach based on reinforcement learning, reducing the need for human intervention and increasing the operational efficiency of algorithms. To achieve this objective, the SARSA (State-Action-Reward-State-Action) and Deep SARSA methods were adapted to regulate the parameters of meta-heuristic algorithms, particularly the genetic algorithm. The adopted model is independent of the problem to be optimized or the selected meta-heuristic algorithm, thus offering the necessary flexibility, with the only crucial requirement being the choice of parameters to adjust during the optimization process of any studied problem. These methodologies were tested on benchmark functions widely recognized in the literature and applied in this investigation to the following practical scenarios: portfolio optimization, where a participant owns or intends to acquire electricity in an energy market, and improvements related to the allocation of patients in Surgical Units (SU) and Intensive Care Units (ICU), with the goal of improving the efficiency of limited resource utilization. The results demonstrate that the Deep SARSA algorithm, based on reinforcement learning and neural networks, often achieves better performance compared to completely random manual configuration. This finding is supported by the analysis of the average results across multiple runs, particularly in the SU problem, where the ANOVA test yielded a significant p-value of 0.014. This outcome suggests that dynamic parameter adjustment approaches may be more effective and provide a viable alternative to static configuration methods, potentially enhancing proposed solutions to address challenges in dynamic and uncertain environments.
The effectiveness of optimizing complex problems is closely tied to parameter configuration in meta-heuristic algorithms. Although automated methods for selecting algorithm parameters have already been proposed to reduce the need for manual tuning, there remains significant untapped potential for dynamic parameter adjustment during algorithm execution, which could improve performance. This study aims to assess the effectiveness of manual parameter setting compared to a dynamic approach based on reinforcement learning, reducing the need for human intervention and increasing the operational efficiency of algorithms. To achieve this objective, the SARSA (State-Action-Reward-State-Action) and Deep SARSA methods were adapted to regulate the parameters of meta-heuristic algorithms, particularly the genetic algorithm. The adopted model is independent of the problem to be optimized or the selected meta-heuristic algorithm, thus offering the necessary flexibility, with the only crucial requirement being the choice of parameters to adjust during the optimization process of any studied problem. These methodologies were tested on benchmark functions widely recognized in the literature and applied in this investigation to the following practical scenarios: portfolio optimization, where a participant owns or intends to acquire electricity in an energy market, and improvements related to the allocation of patients in Surgical Units (SU) and Intensive Care Units (ICU), with the goal of improving the efficiency of limited resource utilization. The results demonstrate that the Deep SARSA algorithm, based on reinforcement learning and neural networks, often achieves better performance compared to completely random manual configuration. This finding is supported by the analysis of the average results across multiple runs, particularly in the SU problem, where the ANOVA test yielded a significant p-value of 0.014. This outcome suggests that dynamic parameter adjustment approaches may be more effective and provide a viable alternative to static configuration methods, potentially enhancing proposed solutions to address challenges in dynamic and uncertain environments.
Description
Keywords
Algoritmo genético Aprendizagem máquina Aprendizagem por reforço Configuração dinâmica de algoritmos DAC Otimização por enxame de partículas SARSA Deep-SARSA Dynamic algorithm configuration, Reinforcement learning Genetic algorithm Particle swarm optimization Machine learning