The KIT Happy Robot team (former happy mini) official web site
Happy mini, our human support robot, is the 3rd place in sympathy and 5th place in design rating in the @Home league in RoboCup2016 Leipzig by the research of Eva Jahn as shown in this web site. Many wonderful robots, such as Pepper, HSR, R1, SocRob, Amigo, Tiago and so on, participated in RoboCup@Home2016.
The KIT Happy Robot team, our former team name was happy mini, has been participating in the RoboCup@Home league of the RoboCup Japan Open since 2012, and participated in the RoboCup 2015, 2016, and 2017 world competition. The team got the 9th, 8th, and 9th places in the competition, respectively. We changed the team name more generally because of making a lot of happy robots for human.
The vision of the KIT Happy Robot team is “Making the World Happy by Making a Kawaii Robot.” “Kawaii” means cute, lovely, or charming. We think that the kawaii robot can solve problems of an aging society in Japan, robots in the RoboCup@Home league should be more kawaii. Happy Mini as shown in Fig.1 is designed in the image of a little child with a lively yellow color. Happy Mini is considered to be the first kawaii designed robot in the RoboCup@Home. The software system was based on ROS, and the state of arts frame works such as Caffe, Darknet, Digits, Hark, Kaldi and so on.
We have been developing 2 kinds of human support robots. One is a childcare robot for children and another is a wheelchair robot for the elderly. To foster the research and development, the Demura Research Laboratory has been participating in RoboCup@Home since 2012 and Tsukuba Challenge since 2008.
The design concepts of Happy Mini are Kawaii (lovely, cute), simplicity, safety, and usability. Kawaii is the most important aspect of human-robot interaction for Mini. Mini is an infant robot, and the ability is limited as well as other robots in RoboCup. So, Mini needs the help of human, that is why kawaii of appearance and voice is crucial for Mini. The exterior of Mini is roundish and lively yellow as shown in Fig.1. The technical information of these robots is described in our team description paper(KITHappyRobotTDP2018).
Happy mini (Left: CAD original design, Right: Final real robot)
Fig.1 Happy Robot Family
Fig.2 Extendable Torso
|Max speed [m/s]||0.7|
Object Recognition & Manipulation: We develop a cascade deep neural network of YOLO (You Only Look Once) and a conventional CNN, such as LeNet, and AlexNet. Firstly, we use a pretrained YOLO for object detection and rough classification. Secondly, the CNN classifies object more precisely using outputs of YOLO. The merit of this cascade deep neural network is very fast learning. Because we just use a pretrained YOLO, only learn the data set of RoboCup@Home competition using LeNet or AlexNet. The result of object recognition is shown in Fig.3. Next, the robot removes the planer and outlier from point cloud data, and calculates the centroid of clustering from point cloud corresponded object image area. This calculation uses Point Cloud Library. Finally, the robot calculates Inverse Kinematics from object center of gravity and grasp it.
Fig. 3 Results of object recognition
Perception: The software system is based on ROS and Caffe. Several algorithms are used for an object and human face recognition. HaarCascade classifier, Flat Clustering filtering, and Deep Neural Networks (DNNs) are used for face recognition (Fig.4). Nesterov’s accelerated gradient and Convolution DNNs are used for gender classification. We trained the network using 700 samples for each category, and accuracy of classification is about 75 %.
The real-time object detection deep neural newtork, YOLO (You only look once) is used for person detection as shown in Fig.5. The algorithm is very fast compared to other DNNs and robust.
Fig.4 Face recognition using HarrCascade and DNN
Fig.5 Person detection
Speech Recognition and Sound Localization: We have developed a speech recognition and a sound source localization system. The speech recognition system uses the Kaldibased gstream server. It is a real-time full-duplex speech recognition server, and
uses a DNN-based model for English trained on the TEDLIUM speech corpus.
The sound source localization system is implemented using the robot auditory library HARK. The HARK easily be programmed by GUI as shown in Fig.6. The system takes multi-channel speech waveform data from the 8ch microphone array, calculates FFT and estimates the sound source direction by the MUSIC method.
Simulator: We have developed the Happy Mini simulator using the Gazebo robot simulator as shown in Fig.7. To make the 3D Happy Mini model in the simulator, we described the URDF (Universal Robot-ics Description Format) file and imported 3D meshes from the 3D CAD data. MoveIt! is easily applied to the 3D model described by the URDF
Fig.7 The Happy Mini Simulator using Gazebo
- Scientific achievements
Developing an autonomous personal mobility: The first author’s laboratory has been developing an autonomous personal mobility called UNiMO AI as shown in Fig. 8 based on the commercially available crawler typed electric wheel chair. The conventional wheelchair is difficult to get over small difference (about 30 [mm]) in level, and it sometimes makes a falling accident. To prevent the accident, the capability of detecting the small difference is required. UNiMO AI can detect the small difference and has the ability to cross steps of 10cm.
Fig.8 An autonomous personal mobility robot: UNiMO AI
Table 2 Specification
|Width x Length x Height [m]||1.00 x 0.695 x 0.88|
|Max speed [m/s]||1.1|
|Drive mechanism||Crawler, 400W AC Motor x 2|
|Computer||NVIDIA Jetson TX2 x 2|
|Sensors||2D Lidar, 3D Lidar, IMU, FOG, Camera,|
Cascade deep neural network of Yolo and CNNs: Detection of a specific person is not easy especially in the outdoor environment. We have developing the detection system that is the cascade deep neural network of Yolo and AlexNet. Firstly, Yolo is one of the state arts of deep neural network, and it can segment regions and classify objects. However, the classification ability is not so good. Thus, we propose a cascade of Yolo and other CNNs such as AlexNet, Goog-LeNet. Fig. 9 shows the results of Yolo and AlexNet. The system is very robust even in the outdoor environment.
Fig. 9 Detection of a specific person in the outdoor environment
Road unevenness detection: We have been developing a wheelchair robot and a driver assist technology system for the elderly. The conventional wheelchair is difficult to get over small difference (about 30 [mm]) in level, and it sometimes makes a falling accident. To prevent the accident, the capability of detecting the small difference is required. We have developed two system to solve the problem.
The first system uses the Kinect V2 sensor as shown in Fig.10, and the second system uses the 3D Rolling Lidar developed by demura.net as shown in Fig.11. As far as our knowledge, it is the most advanced technology to detect small difference in level for a wheelchair size robot. Developing the small difference level detector for a commercial wheelchair and a mobility scooter for elderly, and applying the technology to a human support robot is our near future work.
Fig.11 The results of detection of 25 [mm] difference in level by the Rolling 3D Lidar
- Team Description Paper
- Current out GitHub
- RoboCup2015 @Home League in Heifei, China (9th place)
- RoboCup Japan Open 2014 @Home Simulation League (3rd place)
- SourceCode.tgz by Kensei and Kosei Demura
- RoboCup@Home 2014: What did you say ?
- The Book: Robot Simulation with Open Dynamics Engine (by Kosei Demura)
- Adviser: Kosei Demura
- Member:Koyo Enomoto (Leader), Kazuki Nagashima, Takeaki Yamakawa, Ryo Iwasaki, Syuhei Mashimo
- Previous participation in RoboCup@Home
- RoboCup Japan Open 2012
- @Home Real Robot League: Lost in a preliminary round
- RoboCup Japan Open 2013
- @Home Real Robot League: Lost in a preliminary round
- @Home Simulation League:
- RoboCup Japan Open 2014
- @Home Real Robot League: 8th place
- @Home Simulation League: 3rd place (Team name: Yumekobo Junior)
- RoboCup Japan Open 2015
- @Home Real Robot League: 5th place
- @Home Simulation League: 5th place
- RoboCup 2015, Heifei, China
- @Home: 9th place
- RoboCup Japan Open 2016
- @Home Education: 1st place
- RoboCup 2016, Leipzig, Germany
@Home: 8th place
- RoboCup Japan Open 2017
- @Home Open Platform League: 4th place
- RoboCup 2017, Nagoya, Japan
- @Home Open Platform: 9th place
- RoboCup Japan Open 2012