Overview
This project presents deep learning based approaches for stair detection, statistical filtering on images for the estimation of stair alignment, and novel mechanical design for an autonomous stair climbing robot. The primary objective is to solve the problem of indoor locomotion over staircases with the proposed implementation.
This project was partially sponsored by the IEEE VNIT Student Branch and made possible
through the active efforts of the Branch Counselor, Dr. Pradyumn Chaturvedi.
Novel Contributions
- Novel mechanical design for an Autonomous Stair Climbing Robot.
- Two Deep Learning based approaches for Stair Detection.
- Statistical filtering on images for estimation of stair alignment
Hardware Specifications
Electrical Aspects
- Motor Driver – 18 V, 20 A
- Microcontroller – Arduino Nano
- Bluetooth Module – HC-05
- Battery – 11.1 V, 3S LiPo
- Motors – 150 RPM, Johnson motors (average torque of 5-10 kg-cm)
Mechanical Aspects
- Dimension:
- Height – 20 cm
- Breadth – 30 cm
- Length – 80 cm
- Weight – 5.4 kg
- Materials:
- 2 mm thick Aluminum sheet
- 3 mm thick acrylic sheet
- Plastic wheels
- Rubber gripped conveyor belts,
Design Aspects
- Tracked and segmented design.
- Custom designed 3D printed wheels to avoid slippage.
- Use of hinges and springs for smooth motion.
- Supporting wheels and plates for efficient power transmission.
Cyber Physical Architecture
Stair Detection using YOLOv3
Data Collection and Augmentation
- Data was manually collected. Most of it is from the VNIT campus and some from the Internet.
- 848 images were collected and annotated. Horizontal flipping was done for data augmentation (total – 1696).
- Dataset was split into 1488 images for training and 208 images for testing.
- Sample images and annotations (in yellow).
Transfer Learning was done for tiny-YOLOv3 using the ImageNet trained weights as pretraining. The model was deployed on an Nvidia Jetson TX1 module.
Results
- Over 100 frames per second (fps) on Nvidia GTX 1080Ti using tiny-YOLOv3
- Around 20-25 fps on an Nvidia Jetson TX1 using tiny-YOLOv3
Stair Alignment
To align the tracked robot w.r.t. the stairs, it is essential to estimate the heading of stairs w.r.t current heading and minimize it w.r.t time.
STEPS INVOLVED :
- Fitting of lines on the edges of stairs detected inside the ROI.
- Determination of the midpoint of every stair.
- Line-fitting on the midpoints of stairs.
- Computation of desired heading.
- ROI obtained after Stair Detection
- Gradient image computed from Scharr operator
- Binarization of image based on gradient magnitude
- Binarization of image based on gradient direction
- Hough line fitting without prevention of cluttering
- Lines obtained after prevention of cluttering
- Blue lines represent stair edges.
- White line represents the current heading
- Black line represents desired heading.
- Yellow points represent the midpoints of the lines.
- The angle between white line and black line is known as offset angle.
Control System for Alignment
- PID controller is used to minimize offset angle with time.
- Velocities of right and left wheel are derived using differential drive model.
- IMU sensors are used to compute the current state action of robot. IMU readings indicate whether robot is climbing stairs or traversing plane ground.
Citations
- U. Patil et al., “Deep Learning Based Stair Detection and Statistical Image Filtering for Autonomous Stair Climbing,” 2019 Third IEEE International Conference on Robotic Computing (IRC), Naples, Italy, 2019, pp. 159-166. doi: 10.1109/IRC.2019.00031 IEEEXplore
- N. Panchi, K. Agrawal, U. Patil, A. Gujarathi, A. Jain, H. Namdeo, and S. S. Chiddarwar, “Deep learning-based stair segmentation and behavioral cloning for autonomous stair climbing, ” International Journal of Semantic Computing, vol. 13, no. 04, pp. 497–512, 2019. Journal Paper Link
Team
- Unmesh Patil
- Aniket Gujarathi
- Akshay Kulkarni
- Aman Jain
- Navid Panchi
- Khush Agrawal
- Kartik Paigwar
- Lokesh Malke
- Radhika Tekade