A Q-learning algorithm to generate shots for walking robots in soccer simulations

RoboCup, primitively named the J-League, is an yearly robotics and artificial quality (AI) contention organized by the International RoboCup Federation. During RoboCup, robots vie with different robots shot tournaments.

The thought for the contention originated successful 1992, erstwhile Professor Alan Mackworth astatine University of British Columbia successful Canada wrote a insubstantial entitled "On Seeing Robots." In 1993, a probe squad successful Japan drew inspiration from this insubstantial to signifier the archetypal robot soccer competition.

While RoboCup tin beryllium highly entertaining, its main nonsubjective is to showcase advancements successful robotics and AI successful a real-world setting. The robotic systems participating successful the contention are the effect of intensive probe efforts by galore researchers worldwide.

In summation to the real-world competition, machine scientists and roboticists tin trial their computational tools for robot shot astatine the the RoboCup 3D shot simulation league. This is fundamentally a level that replicates the RoboCup situation successful simulation, serving arsenic a virtual "gym" for AI techniques and robotic systems designed to play soccer.

Researchers astatine Yantai Institute of Technology successful China and University of Rahjuyan Danesh Borazjan successful Iran person precocious developed a caller method that could heighten the quality of robots participating successful shot games to sprout the shot portion walking. This technique, presented successful a insubstantial published successful Springer Link's Journal of Ambient Intelligence and Humanized Computing, is based connected a computational attack known arsenic the Q-learning algorithm.

"One of the astir important goals of the teams participating successful the RoboCup3D league is the quality to summation the fig of shots," Yun Lin, Yibin Song and Amin Rezaeipanah, the 3 researchers who developed the technique, wrote successful their paper. "The crushed for this value is that superiority implicit the hostile requires a almighty and precise shot."

Most techniques to make shots successful simulation are based connected 2 approaches called inverse kinematics (IK) and constituent analysis. These are mathematical methods that tin beryllium utilized some to make machine animations and successful robotics to foretell the associated parameters required for a robot to attain a fixed presumption oregon implicit an action.

"The presumption of these methods is that the positions of the robot and the shot are fixed," the researchers explained successful their paper. "However, this is not ever the lawsuit for shooting."

To flooded the limitations of antecedently projected methods, Lin and his colleagues created a caller shooting strategy based connected a Q-learning algorithm, which tin heighten the quality of robots to sprout the shot portion walking. Q-learning algorithms are model-free computational approaches based connected reinforcement learning. These algorithms are peculiarly utile successful instances wherever agents are attempting to larn however to optimally navigate their situation oregon execute analyzable actions.

"A curved way is designed to determination the robot towards the ball, truthful that it volition yet person an optimal presumption to shoot," the researchers wrote successful their paper. "In general, the imaginativeness preceptor successful RoboCup3D has noise. Hence, robot question paramenters specified arsenic velocity and space are much precisely adjusted by the Q-learning algorithm. Finally, erstwhile the robot is successful the optimal presumption comparative to the shot and the goal, the IK module is applied to the shooting strategy."

Lin, Song and Rezaeipanah evaluated their Q-learning algorithm successful a bid of experiments and simulations. Remarkably, they recovered that it allowed robots to sprout the shot portion walking overmuch amended than robots successful astir teams participating successful the RoboCupSoccer league and successful Iran's RoboCup3D league. Ultimately, it could frankincense importantly amended the show of robots during RoboCup shot games.

---

---

More information: Generation a shooting connected the walking for shot simulation 3D league utilizing Q-learning algorithm. Journal of Ambient Intelligence and Humanized Computing(2021). DOI: 10.1007/s12652-021-03551-9

© 2021 Science X Network