Skip to main content

Heat Follower Robot

rosbot

Description

Heat Follower is a project based on the ROSbot Pro platform, that follows heat by using a 360 degree thermal camera. It utilizes a MLX90641 sensor which is a 16x12 pixels IR sensor array. With some custom hardware it is possible to rotate this sensor and get a surround infrared image.

Hardware

Hardware used in this project:

Software

Thermal Camera

Software for Thermal Camera is written in PlatformIO IDE to ensure compatibility with multiple platforms. MLX90641 API is used to ensure proper communication with the IR Sensor. It is slightly modified to support STM32's hardware. Micro-ROS libraries are mandatory to communicate with ROSbot.

ROS2 Python Node

A simple Python node was written to process the input thermal image with OpenCV and steer the robot. When a heat source is detected, this node sends rotational and linear velocities via /cmd_vel topic.

ROS Description

TopicNode NameDescription
/thermal_image/thermal_cameraRaw thermal image represented with grayscale image (temperature = pixel intensity/100).
/tc_image_devel/thermal_subscriberNormalized image with target object contours.
/tc_goal_angle/thermal_subscriberArrow marker pointing at the target angle relative to the robot.

Additionally there are 3 parameters

ParameterNode NameTypeDescription
target_min_temp/thermal_subscriberintMinimum temperature of object
target_max_temp/thermal_subscriberintMaximum temperature of object
go_to_hottest_piont/thermal_subscriberboolIf true, robot follows the hottest object. If false, robot follows object which temperature is within target_min_temp and target_max_temp.

System schematic diagram:

diagram

The camera sends the main topic /thermal_image - a raw unprocessed image which is then transformed by /thermal_subscriber node. The transformed image published on /tc_image_devel contains a normalized image with a contour of an object selected by the algorithm. The angle of a selected object can also be visualized with /tc_goal_angle marker that publishes an arrow pointing from base_link to the direction of the object.

Building the project

Flashing MCU

Open the project with Visual Studio Code with PlatformIO plugin.

This project is optimized for STM32F4 MCU's - keep that in mind when flashing on your hardware!

After installing the PlatformIO extension to your VSCode, click its icon on the Activity Bar then from Project Tasks select General and Upload.

Running on ROSbot

Clone robot folder to your ROSbot, connect the Thermal Camera via USB and run

cd robot/
docker-compose -f docker-compose.yaml up

After downloading all the necessary docker images ROSbot should follow warm objects nearby.

Due to issues connected with micro-ros agent, user must unplug and plug the Thermal Camera to reset the connection before running the docker compose.

Running Rviz on PC

Clone pc folder to your PC and run:

cd pc/
xhost local:root
docker-compose -f compose.pc.yaml up

Rviz should now pop up showing ROSbots surroundings in deep infrared.

rviz

If images are not visible click Add -> By topic -> /thermal_image

Here's ROSbot facing a kettle:

rb kettle

rviz kettle