Evolutionary Morphology Towards Overconstrained Locomotion via Large-Scale, Multi-Terrain Deep Reinforcement Learning
Yenan Chen#, Chuye Zhang#, Pengxi Gu#, Jianuo Qiu, Jiayi Yin, Nuofan Qiu, Guojing Huang, Bangchao Huang, Zishang Zhang, Hui Deng, Wei Zhang, Fang Wan*, and Chaoyang Song*
Abstract 🦿
While the animals' Fin-to-Limb evolution has been well-researched in biology, such morphological transformation remains under-adopted in the modern design of advanced robotic limbs. This paper investigates a novel class of overconstrained locomotion from a design and learning perspective inspired by evolutionary morphology, aiming to integrate the concept of "intelligent design under constraints"—hereafter referred to as constraint-driven design intelligence—in developing modern robotic limbs with superior energy efficiency. We propose a 3D-printable design of robotic limbs parametrically reconfigurable as a classical planar 4-bar linkage, an overconstrained Bennett linkage, and a spherical 4-bar linkage. These limbs adopt a co-axial actuation, identical to the modern legged robot platforms, with the added capability of upgrading into a wheel-legged system. Then, we implemented a large-scale, multi-terrain deep reinforcement learning framework to train these reconfigurable limbs for a comparative analysis of overconstrained locomotion in energy efficiency. Results show that the overconstrained limbs exhibit more efficient locomotion than planar limbs during forward and sideways walking over different terrains, including floors, slopes, and stairs, with or without random noises, by saving at least 22% mechanical energy in completing the traverse task, with the spherical limbs being the least efficient. It also achieves the highest average speed of 0.85m/s on flat terrain, which is 20% faster than the planar limbs. This study paves the path for an exciting direction for future research in overconstrained robotics leveraging evolutionary morphology and reconfigurable mechanism intelligence when combined with state-of-the-art methods in deep reinforcement learning.

Parametrically Reconfigurable Additive Design of Overconstrained Robotic Limb
⚙️ Our inspiration comes from the well-researched fin-to-limb evolution in biology
(A) 3D printable components and assembly of an overconstrained robotic limb;
(B) Physical prototype of the fully-assembled quadruped in Bennett limbs;
(C) Parametric reconfiguration as (i) a planar four-bar, (ii) an overconstrained 4-bar, and (iii) a spherical four-bar;
(D) An enhanced design for amphibious locomotion reconfigurable as (i) a legged robot, (ii) a wheel-legged robot, and (iii) a simplified equivalent as an open-chain limb identical if using a belt or slender linkages.
Simulation Setup for Massively Parallel Deep Reinforcement Learning
💻 We adopted the state-of-the-art methods in deep reinforcement learning
With the help of Isaac Sim Environment, we trained robots via massively parallel deep reinforcement learning on a single workstation GPU


Large-scale, Multi-terrain Reinforcement Learning of Overconstrained Locomotion
🗻 The overconstrained quadruped robots were trained on simple slopes, slopes with random noises, stairs, and flat floors with random noises.
The reinforcement learning rewards of the quadruped with Bennett, planar, and spherical limbs over 6K training steps prove our training approach works well and the superior mobility of the trained policy in complex terrains.
Performance Benchmark on Terrain Traverse Task
🚀 Thirty quadrupeds were instructed to traverse a serious of terrains at a speed of 1 m/s.
Quadrupeds with Bennett and planar limbs completed both forward and sideways movements on all terrains. While, those with spherical joints struggled and only managed only the flat terrains.
Bennett limbs (red curve) outperformed other limbs. They achieved higher speeds and lower Cost of Transport (COT 🔋). Here the COT is metrics for energy efficiency.


Learning Wheel-legged Overconstrained Locomotion
🛞 We extended the Bennett limb to a wheel-legged quadruped robot using the same simulation framework.
With the excellent performance, we proved the adaptability and efficiency and supports the design’s potential for various robotic applications
Video 🎥
BibTex📝
@misc{chen2024evolutionarymorphologyoverconstrainedlocomotion,
title={Evolutionary Morphology Towards Overconstrained Locomotion via Large-Scale, Multi-Terrain Deep Reinforcement Learning},
author={Yenan Chen and Chuye Zhang and Pengxi Gu and Jianuo Qiu and Jiayi Yin and Nuofan Qiu and Guojing Huang and Bangchao Huang and Zishang Zhang and Hui Deng and Wei Zhang and Fang Wan and Chaoyang Song},
year={2024},
eprint={2407.01050},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2407.01050},
}