名前：

彭 彦鴻

学年/肩書：

博士課程後期課程3年

役職：

したっぱ

グループ：

ロボット

趣味：

フリーナ

研究テーマ / Research topic

Modeling Fabric-type Actuator Using Point Clouds by Deep Learning

Introduction

Soft actuators, mainly made of stretchable and flexible materials, have received considerable attention in recent years because of their superior human–machine interface to hard robotics, which has substantial economic benefits. They have many uses in wearable devices and have also been investigated for potential applications in medical rehabilitation equipment. However, there are common limitations in modeling and controlling soft actuators because the material’s structural compliance, nonlinearity action, and the viscoelasticity in the material result in complex and unpredictable behaviors. Over the past several decades, deep learning has made unprecedented progress. The insights that artificial intelligence technology can extract from complex data benefit the field of medicine, autonomous driving, and many other fields. Deep learning is also well-known for effectively solving nonlinear problems in soft robotics. The finite element method (FEM) and position-based dynamics (PBD), as current mainstream methods, have some limitations in modeling, such as the balance between computing power consumption and model accuracy; nonetheless, and deep learning can compensate for these shortcomings to some extent.

This research presents a method of modeling four soft fabric-type actuators by correcting the simulated point clouds through deep learning to make them approach the point clouds obtained from real actuators using four depth sensors. This paper also aims to solve the above limitations of fabric-type actuators by deep learning. PointNet is a robust framework that can transform the three-dimensional (3D) coordinates of point clouds into local or global features.
It has achieved excellent performance recently in point cloud classification and semantic segmentation tasks. In this research, the encoder of PointNet was used to extract the global features of the point clouds, and then the simulated point cloud features of the previous five relevant frames were learned by long short-term memory (LSTM) as time-series data to predict the real point cloud features of the next frame. Finally, using the decoder, we can decode the real point cloud features into 3D point cloud coordinates. By establishing a neural network structure of encoder–LSTM–decoder, we can describe the real point cloud of an actuator from the simulated point cloud of the previous state. In this process, the neural network compensates for nonlinear mechanical actions from the soft actuator design material and geometry difficult to simulate with point clouds. The similarity between the corrected simulated point cloud and the real point cloud was significantly improved. Fig. 1 depicts an example of muscle configuration.

Figure 1: muscle configuration

Aim

This research aims to solve the above limitations of the soft robot by deep learning. The goal of this research is to develop a wearable system using the soft actuator and can assist people in some common actions. There are the following three objectives. In the initial stage, build a simulator that can simulate the deformation of the soft robot. In the second stage, find a control method according to the developed simulator. In the final stage, develop a wearable system to support some people with limited mobility and the elderly.

Method

In this study, the point cloud was converted to a feature dimension that was not affected by the order of points, and the features were associated with LSTM. The structure of the correction system included an encoder, a decoder and the LSTM, and they are shown in Figure 2. The encoder converted point cloud with m points from the simulation into n dimensional features, and the LSTM associated the features of the simulated deformation of the continuous l frames with the features of the predicted deformation. The decoder converted the output with n dimensional features from the LSTM into a point cloud with m points.

Figure 2: Correction system structure (Encoder – LSTM – Decoder)

Result

Figure 3  Figure 4 and showed the Chamfer distance (CD) frequency distribution of actuator before and after correction. CD was used to evaluate the extent to which the simulated point cloud can fit the actual point cloud. Comparing the simulations without the correction system (Figure 3) and the simulations with the correction system (Figure 4), the CD of the point cloud decreased after correction; i.e., the overall value of CD decreased. Therefore, based on our neural network, the correction system can learn high-dimensional features of muscle structure in the actuator, and it is possible to simulate deformations close to those of the actual actuator.

Figure 3: CD frequency (before correction)

Figure 4: CD frequency (after correction)

Figure 5 compares the original simulated point cloud with the real point cloud of the actuator in the overall, and Figure 6 shows the corrected point cloud (CD decreased by 12.41 mm) of the actuator. The actual point cloud and the point cloud obtained using the correction system are almost the same.

Figure 5: Simulated point cloud of actuator (CD: 18.33 mm).
Green point: simulated point cloud. Red point: real point cloud

Figure 6: Corrected point cloud of actuator (CD: 5.92 mm).
Blue point: corrected point cloud. Red point: real point cloud

Publication

Please view the detail on IEEE Xplore.

If this study is useful in your research, please consider citing:

Peng Y., Yamaguchi H., Funabora Y. and Doki S., “Modeling Fabric-Type Actuator Using Point Clouds by Deep Learning,” in IEEE Access, vol. 10, pp. 94363-94375, 2022, doi: https://doi.org/10.1109/ACCESS.2022.3204652

Peng, Y., Sakai, Y., Nakagawa, K., et al. Funabot-Suit: A Bio-Inspired
and McKibben Muscle-Actuated Suit for Natural Kinesthetic Perception.
Biomimetic Intelligence and Robotics (2023), doi: https://doi.org/10.1016/
j.birob.2023.100127.