Motion Control Method for Small Quadruped Robots Based on Trajectory Modulation and Reinforcement Learning

Abstract

Quadruped robots have gained significant attention in various fields, including scientific research, military applications, and rescue operations, due to their stability and flexibility. As a result, motion control for quadruped robots has become a popular research topic in the field of robotics. Currently, there are two common approaches to control quadruped robots — model-based methods and reinforcement learning-based methods. Model-based methods often require extensive expert knowledge, computation, and debugging. Moreover, these methods lack transferability and require designing different approaches for different scenarios. In contrast, reinforcement learning methods have attracted significant interest as they enable robots to interact adaptively with the environment and learn suitable motion strategies. However, previous reinforcement learning methods often have high hardware requirements, substantial computational costs during training, and perform poorly on small robots with limited computational and perceptual capabilities. To address these challenges, we propose the ARS-Bezier method based on the light-weight Augmented Random Search (ARS) algorithm. The ARS-Bezier method combines the ARS algorithm with Bezier curves to enhance the model’s learning ability, improve learning efficiency, and reduce learning difficulty. Additionally, we introduce the method of environment randomization to enhance the transferability of learned policies. The main contributions of this paper are as follows. Firstly, we focus on the motion control methods of the self-developed robot mouse NeRmo. This includes the modeling of forward and inverse kinematics. Next, we analyze the fundamental properties of Bezier curves and provide a simple proof. Then we deploy Bezier trajectories on the robot mouse. Subsequently, based on the robot mouse’s structure, we design the ARS framework and propose the ARS-Bezier control method, which utilizes Bezier curves to enhance the model’s learning ability. Additionally, we introduce an environment randomization method to enable the robot to learn more versatile motion strategies. Then, we conduct a series of experiments in both simulation and real environments to demonstrate the effectiveness of the ARS-Bezier method. We compare the experimental results and analyze the performance of learned motion strategies through environment randomization in different scenarios. Furthermore, we showcase the process of building the actual robot mouse, burning the algorithm, and verify the effectiveness of the motion strategies in real-world scenarios.