ALARMbot: Autonomous Laboratory Safety Inspection and Operable Hazard Intervention Robot Enabled by Foundation Models

The hardware architecture consists of three integrated modules: a mobile chassis, perception sensors, and a manipulation subsystem. A ROS-driven mobile platform integrating a wheeled chassis with RGB-D cameras, LiDAR and a 6-DoF robotic arm fitted with an adaptive gripper. All perception, planning and control modules run on a central compute unit, with SSH-based communication enabling real-time remote monitoring.

The navigation system is designed to enable the robot to autonomously navigate through complex laboratory environments. It utilizes a combination of LiDAR-based mapping and vision-and-language models for semantic navigation and functional zone recognition.

Based on navigation instructions and multimodal sensor inputs—including RGB images and LiDAR scans—the system generates query tokens and image tokens through dedicated Modality Fusion Projectors, integrating them within a unified Observation Encoder. At each timestep, ChemistryNav processes both historical and current observations alongside navigation-oriented tokens to produce explicit navigation actions. Real-time robot positions, acquired via precisely calibrated LiDAR sensors, are dynamically aligned with an evolving Fine-Grained Semantic Map, providing comprehensive semantic and spatial information essential for effective dynamic route planning and targeted risk management tasks.

User prompts trigger the robot to visit each room, record whether it is a laboratory, tag its location, and store the data for designing a comprehensive cross-room inspection route. The robot autonomously explores the entire floor, generating a complete semantic map showing the spatial distribution and functional categorization of work areas. This map is continuously updated as the robot navigates, allowing for real-time adjustments to its path and actions.

Once the robot reaches an appropriate position near a workbench, the robotic arm's camera captures desktop images, which are then processed by Inspect agent, which identifies objects on the workbench and assigns a unique functional-zone label according to predefined criteria. After completing the exploration and classification, the agent compiles a complete semantic map showing the spatial distribution and functional categorization of work areas.

A two-stage detection framework with a YOLO-based Visual Perception Module and a VLM-based Risk Reasoning Module to detect objects and assess hazards in real-time. The results are sent to the Inspect Agent, which plans actions or archives data, and uploads risk assessments and outcomes to the Interaction UI for monitoring.

The Visual Perception Module utilizes a YOLOv8 model trained on real-world, public, and synthetic datasets to detect objects from images collected along the inspection route. The detection results are post-processed into a structured JSON format, paired with prompt queries, and fed into the Detect Agent for risk reasoning. The identified risks are subsequently passed to the Inspect Agent to drive downstream decision-making and action planning.

The Inspect Agent receives structured detection results containing identified objects and inferred risks. It performs risk reasoning and action planning through three stages: Risk Matching, Template Retrieval, and Meta-Action Sequencing. Based on predefined Meta-Action Skills, the system composes executable motion sequences, which are then carried out by the robotic manipulator. Execution results, including success or warnings, are uploaded to the Interaction UI for monitoring and logging.

We evaluate our robotic system in unfamiliar laboratory environments, testing navigation, risk detection, and risk elimination. An intuitive human-robot interface accepts natural-language commands, ensuring stable and precise task execution.

We designed a real-world experiment where the robot autonomously explores an entire floor and searches for a specified laboratory. The robot autonomously explores large-scale spaces, identifies laboratory characteristics, and records the locations of rooms that qualify as laboratories. As a result, the robot successfully navigated the entire floor, planned the exploration route, and found the specified laboratory.

We compared the risk detection success rates compared three configurations: Public only, Public + Real, and Public + Real + Synthetic. By using small models trained on diverse datasets, the system achieves better accuracy in risk identification.

We designed various meta-action-based manipulation experiments to equip our robot with basic laboratory risk mitigation capabilities, including upright fallen bottles, relocate edge-exposed objects, place tipped bottles onto racks, wipe spilled liquids, and clear debris from workbenches. Additionally, we validated some fundamental laboratory operations, such as sorting unclassified reagent bottles and transferring misplaced tubes, enabling it to perform a variety of essential laboratory tasks.
We provide operation videos from multiple perspectives for various tasks.

We achieved high success rates across various tasks, which demonstrate the effectiveness of combining vision-language reasoning with meta-action sequence control for autonomous laboratory risk mitigation, while also identifying opportunities for further improvements in precision-critical manipulations.

We designed a comprehensive inspection experiment. During the inspection process, the robot autonomously plans the inspection route and performs risk identification and handling in each zone. For operable risks, the robot autonomously intervenes; for inoperable risks, the robot reports them to human operators. Additionally, we developed a human-robot interaction interface, allowing operators to control the robot using natural language commands for inspection tasks, while the robot provides real-time feedback on its current status and operation results through the interface.