Reinforcement learning as data-driven optimization technique for GMAW process

Welding optimization is a significant task that contributes to enhancing the final welding quality. However, the selection of an optimal combination of various process parameters poses different challenges. The welding geometry and quality are influenced differently by several process parameters, with some exhibiting opposite effects. Consequently, multiple experiments are typically required to obtain an optimal welding procedure specification (WPS), resulting in the waste of material and costs. To address this challenge, we developed a machine learning model that correlates the process parameters with the final bead geometry, utilizing experimental data. Additionally, we employed a reinforcement learning algorithm, namely stochastic policy optimization (SPO), with the aim to solve different optimization tasks. The first task is a setpoint-based optimization problem that aims to find the process parameters that minimize the amount of deposited material while achieving the desired minimum level of penetration depth. The second task is an optimization problem without setpoint in which the agent aims to maximize the penetration depth and reduce the bead area. The proposed artificial intelligence-based method offers a viable means of reducing the number of experiments necessary to develop a WPS, consequently reducing costs and emissions. Notably, the proposed approach achieves better results with respect to other state-of-art metaheuristic data-driven optimization methods such as genetic algorithm. In particular, the setpoint-based optimization problem is solved in 8 min and with a final mean percentage absolute error (MPAE) of 2.48% with respect to the 42 min and the final 3.42% of the genetic algorithm. The second optimization problem is also solved in less time, 30 s with respect to 6 min of GA, with a higher final reward of 5.8 from the proposed SPO algorithm with respect to the 3.6 obtained from GA.