# Optimal control of humanoid robot motions

**Kai-Henning Koch, Katja Mombaur** - IWR**Philippe Souères, Nicolas Mansard - LAAS-CRNS**, Toulouse

Humanoid robots are among the most exciting machines in the world. The past few decades have seen a remarkable development in the field of humanoid robots, producing robots that can walk, climb stairs, avoid obstacles and even lift off the ground for very short periods of time. Famous examples are Asimo (Honda), HRP-2 (Kawada/AIST), Lola and Johnnie (TU Munich), Toyota’s Partner Robots or the Korean robots KHR-2 and Hubo. But despite this technological progress, the motions of humanoid robots are still much slower, less efficient and harder to stabilize than the movement of their biological counterparts - to sum up, they still do not appear fully natural or human-like.

The goal of this project between the University of Heidelberg, and the LAAS-CNRS in Toulouse, France is to address the problem of natural motion generation for humanoid robots from an optimization perspective. In particular we study optimal walking motions for the humanoid robot HRP-2 (No. 14) available at LAAS.

One of the main tasks is the formulation of the corresponding optimal control problem taking into account the full dynamics of HRP-2 as well as all its kinematic, dynamic and stability related constraints. Using different objective functions, we study optimal swing phases for given foot placements as well as completely free (quasi-)periodic walking motions along straight lines or curves. The resulting optimal control problems can efficiently be solved by the optimal control framework MUSCOD II (D.Leineweber, Bock et. al.) developed at IWR. The results will undergo various test simulations in the simulation environment OpenHRP before they are applied to the humanoid robot HRP-2 in experiments.

## Optimization-based motion generation

State of the Art motion generation for complex and redundant dynamic systems as humanoid robots, comprises the complex task to identify the most suitable motion trajectory, among a domain of candidate solutions. The more this approach is simplified, based on premature assumptions, it focuses on a given subspace. A popular and widely used approach is marked by the pattern generators, that simplify the complex full-body dynamics to a one-mass inverted pendulum (Kajita et.al., Morisawa et.al., Takenaka et.al.). Generally these generators are remarkably robust, easily parametrizable and fast enough to perfom in real-time. However they only focus motion generation efforts on a reduced subspace of the physically feasible domain of the humanoid robot at hand and therefore may fail to fully exploit its dynamic motion capabilities. In contrast whole-body optimization-based approaches (Roussel et.al., Hardt et.al., Bessonnet et.al.) drop previously made assumptions and therefore generally tend to much higher complexity of the problem formulation. Consequently, they may potentially exploit the full dynamic capacities of the robotic platform, but at the cost of much higher computation times.

The proposed method, the following studies are based on, allows to restrict assumptions to the physical conditions and the dynamic limits of the robot, the motion is generated for. The complete motion set, including all joint trajectories, controls and system parameters, timings are computed in parallel during the optimization process, with respect to a set of high-level optimization criteria. On top of that further constraints may be easily adopted to analyze their effects on the resulting motion pattern.

### Effect of different optimization criteria on generated walking motions

In a first study this method was applied to generate quasi-periodic walking motion trajectories for the humanoid robot HRP-2, based on biologically inspired optimisation criteria. The purpose of this study was to investigate basic characteristics of single high-level optimization criteria on the resulting walking motion as well as effects of a previously fixed foot-step pattern and constraints on the ZMP-trajectory of the robot during walking.

#### Generation of periodic walking motions

The walking motion below results with respect to 5 single optimization criteria are given - please refer to the publication section below for a complete discussion:

##### Minimum Joint Torque [mov]:

The minimum joint torque criterion (Mombaur et.al.) leads to smooth trajectories with low energy costs and small impacts of the feet on the ground. Hence this objective seems to be well suited for motion generation.

##### Maximum Efficiency [mov]:

Maximum Efficiency (Garcia et.al.), another criterion associated with low energy costs and low feet impacts on the ground. In addition, produced walking motions have a higher forward velocity compared to the mininum torque criterion.

##### Maximum Forward Velocity [mov]:

Maximum forward velocity is an interesting criterion for potential evaluations of the robot's limitations as high forward velocities were reached within the dynamic limits of the robot (in simulation: 0.363 m/s). However high impact of the feet on the ground may overstress and severally deteriorate the mechanical structure of HRP-2.

##### Maximum Postural Stability [mov]:

For maximum postural stability the interest of broadening the already conservatively chosen stability margin (Busse et.al.) at the cost of a low walking speed and high energy costs is difficult to justify. Hence this criterion is considered unsuitable for motion generation for HRP-2.

##### Minimum Joint Velocity [mov]:

Besides of low walking velocities, the criterion minimum joint velocity leads to very stiff, non-smooth and unnatural looking walking motions. Furthermore it produces strong oscillations in global pelvis altitude and roll angle and is therefore not considered suitable for motion generation.

As soon as foot placement was constrained to heuristically chosen positions, variety of the produced gaits declined and hence optimization potential was considerably reduced. Thus we conclude that free-foot placement is generally desirable for optimal walking motions, except on terrains were feasible step-locations are strictly limited. Furthermore, investigation on the effect of the ZMP constraint revealed that as soon as it was relaxed, the generated trajectories tended to a more upright and natural walking motion appearance. However, this may not be an option for HRP-2 as its dynamic equilibrium limits are likely to be violated.

### Overcoming large obstacles

The past few decades have seen incredible advances in the locomotion capacities of humanoid robots. An important aspect is the ability to overcome obstacles, if these robots should ever be exploited to support or entirely replace human operators on hazardous incident sites, remote planets or in the domestic environment. Present works have reported, Jonnie (Seara et.al.), ASIMO (Michel et.al.), BHR-2 (Jarfi et.al.) and HRP-2 (Stasse et.al.) successfully clearing various obstacles during locomotion.

Particularly for HRP-2, different studies have been published. Guan et.al. first published a feasibility analysis of HRP-2 stepping up and down blocks and over obstacles, based on a quasi-static approach combined with various model simplifications. The dynamic case (overstepping an obstacle during walking) was then proposed by Stasse et. al., using the preview control pattern generater with cascaded feedback of the whole body ZMP (Kajita et. al.), successfully clearing obstacles in simulation - height: 25cm, width: 5cm - and real experiments - height ~15cm.

As our proposed optimal control approach for motion generation is not specifically bound to a certain type of motion we proposed and solved a problem formulation based on the whole body dynamic model of HRP-2, clearing an obstacle of comparable size. During the motion generation process, we solved for two mostly contradicting criteria - maximum obstacle height and minimum simulation time - giving priority to the former one. The complete motion trajectory after successful convergence of the optimal control problem is given below. The proposed optimization approach found a maximal obstacle height considerably higher than the values mentioned above in a comparable simulation time.

Our model-based optimization approach, previously used for optimization of walking motion was used to investigate the maximum obstacle height the robot can successfully overcome using the dynamics of its whole body. This result clearly gives evidence to the fact that motion performance of such a complex and redundant robot will be considerably improved, if a complex whole-body motion is allowed. Future work will be to implement this motion on the real HRP-2 robot and to investigate the maximum obstacle height in real experiments. As a precise whole-body model and all known dynamic limitations of the robot have been considered during optimization this task should not pose serious problems. However the resulting motion on the real HRP-2 robot is expected to be slightly different, caused by unknown characteristics of the robot (stabilizer, ankle-elasticity) and environmental conditions that have not been considered during modeling.

## Software

Optimization computation is based on the symbolic dynamic model builder **DYNAMOD** that is developed in this workgroup.

## Publications

- K. H. Koch, K. Mombaur, and P. Souères. Optimization-based walking generation for humanoid robot. In SYROCO, Dubrovnic, Kroatia, 2012 [pdf]
- K. H. Koch, K. Mombaur, and P. Souères. Studying the effect of different optimization criteria on humanoid walking motions. In SIMPAR, Japan, Tsukuba, 2012 [pdf]
- K. H. Koch, K. Mombaur, P. Souères - [Poster] Using optimal control to investigate potential improvements in whole-body walking generation for the Humanoid Robot HRP-2. In HLR, Germany, Heidelberg, 2014
- K. H. Koch, K. Mombaur - Optimization based Exploitation of the Ankle Elasticity of HRP-2 for Overstepping Large Obstacles. In HUMANOIDS, Spain, Madrid, 2014 [pdf]