The modern warehouse is changing.
As automation becomes simpler and cheaper, more and more companies are leveraging autonomous mobile robots (AMRs) for warehouse operations. These AMRs are tackling warehouse operations tasks such as intra-logistics, cleaning, inspection, and maintenance.
In order to deploy and manage a fleet of robots in a warehouse successfully, companies need to adopt a central fleet management system that includes a vendor-agnostic interface to the robots. This interface, which connects end users to robotic devices, enables team members to manage workflows around mapping, mission planning, and traffic management. It also serves as the backbone for the every-day aspects of managing a robotic fleet, providing observability, notifications, and the ability to command and control robots remotely as necessary.
Many robot vendors provide a fleet management system to manage their own specific robots. However, when it comes to a centralized vendor-independent robot fleet management platform, there are very few options available.
With so many different robot vendors running their own architecture, communication and coordination between them is a challenge — a fleet management system needs to be able to speak and translate between all of the various robots.
Robot fleet management platform considerations
Choosing the right robot fleet management platform (or, inadvisably, choosing to build it yourself) can have a big impact on your operations.
A heterogeneous robot fleet management platform needs, at a minimum, the following capabilities:
- Vendor agnostic interface to the robot – One of the most, if not the, most important requirement in heterogeneous fleet management. This allows the fleet management software to be installed on a wide variety of robots and other equipment.
- Central traffic control – Each vendor might be able to provide their own fleet management systems, but in a large warehouse or hospital setup, there are interactions between robots of different vendors. For e.g., a cleaning robot might cross paths with a lifting robot. A central traffic control system with the ability to manage and set maps and routes will ensure that rules drive minimal deadlocks and delays in operations.
- Notifications & interventions – Irrespective of how good the autonomy or fleet management system is there will still be cases where one or more robots need attention. This could be something as simple as an unexpected breakdown to a more complex multi-robot interaction, but getting notified in real-time for issues that need attention is crucial.
- Data observability & analytics – Going back in time and understanding why an issue occurred is crucial to continuously improve the system. A good fleet management solution enables its operators to easily discover opportunities for improvements as well as the capability to debug system failures or provide remote technical support on a need basis.
- Command & control in real-time – Whether a robot mixes up its localization or is stuck in a deadlock with another robot, being able to relocalize or teleoperate a stuck robot in real-time saves employees a lot of time and effort in large warehouses. Not only that, a lot of times it is easier to send advanced commands to the robot remotely than it is through the display screen on the robot. A heterogeneous fleet management system should support the command and control of all the robots in the system.
- Customizable & extensible web interface – Robotics is still in its early days, and user interface requirements and needs are evolving rapidly. The ability to customize and/or extend the capabilities of the solution that you choose is important. Along with customization, it is also important that different roles or users see different types of interfaces. For example, a robot operator might prefer a simple dashboard that enables tech support use cases, while an engineer might want to see a wide array of data displayed in the dashboard for efficient debugging. Hence, the support for role-based access control is equally important.
In addition to the above capabilities, reliability is also vital to ensure a smooth operating experience. Companies with fleets would be wise to seek out fleet-management systems that support standardization such as MassRobotics Interoperability Standard or VDA 5050.
At Formant, our robotic data platform hosts all the necessary capabilities businesses require to accelerate their deployment and scale their heterogeneous fleet of robots in a fast, reliable, and innovative manner while also maintaining compliance with the MassRobotics Interoperability Standard (and could extend that support to VDA 5050 as well).
The Formant robotics data platform architecture
Our data platform enables companies to deploy and manage a heterogeneous fleet of robots by providing vendor-agnostic edge software with an extensible and customizable web interface on top of a bi-directional control and data pipeline.
From a high level, there are two components to the data platform.
- Edge software “agent”
- Cloud platform (also available to self-host, including on-premise)
Edge software
The core component of the edge software is the agent that exposes authenticated and secure HTTP and gRPC APIs to interact with the cloud platform. The agent interacts with the robot using one or more adapters that are specific to the underlying hardware, protocol or device drivers. This enables us to provide a set of common adapters for available systems such as ROS, ROS2, etc out of the box while enabling us or our customers to write custom adapters for new types of systems.
Examples of adapters built for commonly known communication interfaces are ROS, ROS2, USB, MQTT, HTTP, Filesystem, CAN, Modbus, OPC-UA, and more, and can typically be built for a new robot system in a few days.
The agent also comes with a set of data buffering and throttling capabilities that support intermittent connectivity as well as data density requirements. Along with the support for data, the Formant cloud also supports remote commands from the cloud and exposes a local shell through the real-time webRTC connection.
Cloud platform
The cloud platform acts as the backbone for data and control, as well as providing extensibility, customization, and connectivity to third-party platforms, and contains three layers.
The platform layer
The core layer of the cloud platform manages the deployment of the platform and the data pipeline between the robot and the data platform, as well as the identity and access management.
The data pipeline supports both peer-to-peer real-time data and control transportation, as well as data ingestion and control to/from the cloud. Identity and access management supports basic user access management as well as advanced role-based access control.
Applications
The cloud platform comes with a set of default built-in applications – observability, teleoperation, relocalization, notifications & interventions are some of them. It can also be extended by building custom applications.
A few relevant partner applications available on the platform are
- MoveIt Studio provides an interface for supervised autonomy.
- BitwiseAg provides machine learning and enrichment on agricultural data.
- We are starting a partnership with Open Robotics to build a traffic planning interface for Open-RMF as an application on Formant.
- Clearpath provides a mission planning software, primarily for their in-house robots, but extensible for other robots.
Our customers, consultants, and integrators build solutions either directly using the web toolkit and APIs or by stitching together the applications already available on the platform.
Heterogeneous fleet management using Formant
Data observability & analytics
Observability is one of the core parts of Formant. Robot data, especially, as it applies to 3D visualization types such as lidar, URDF, robot transforms, or even audio/video, can be ingested into the platform along with standard data types such as numerics, text, GPS, or JSON and can be visualized on the platform. This is typically used for remote monitoring and tech support, as well as for analysis of a specific mission. We also support streaming or exporting data into another Cloud Objectstore or loading into data warehouses or data marts for further analysis.
Teleoperate or send commands in real-time
Formant teleoperation allows operators to command and control the autonomous device from a browser from anywhere in the world. Perhaps you need to park it in a safe spot or go around an obstacle — whatever it is, you just need a computer, tablet, or even phone, and you’ll have access to real-time, peer-to-peer, low-latency teleop.
Formant allows operators or engineers to execute remote commands on the robot to get more data, states, or insights from the machine without actually logging into the robot. You can set up all the commands that matter to you and your team, so just select which one you want to use from a dropdown.
Customizable & extensible web interface
Every robotics application is different, and Formant’s default views and modules may not satisfy the needs of your application. We provide a suite of tools that enable you to develop your own views and modules that can seamlessly integrate with the platform.
These tools will allow you to:
- Create data widgets that can be displayed in a standard Formant view.
- Create fully custom views with a toolkit of Formant-ready UI components.
- Establish real-time connections with WebRTC.
- Create rich 3D Visualizations.
More importantly, these capabilities enable us and our customers to bring in partners and consultants to build custom applications and solutions on top of Formant.
Integration with productivity and workflow tools
The robots used in warehouses need to be integrated into the existing warehouse productivity, workflow, and management tools to enable streamlined operations and efficient labor allocation. Formant supports extending data within its platform to a set of already existing tools, applications and databases, including HANA. With Formant’s data APIs, the platform supports integration to new tools and applications.
When it comes to heterogenous robot fleet management, you don’t want a single feature that could interrupt your operations. Formant is the leading data platform for managing heterogeneous fleets of robots, and it’s been used on hundreds of different types of robot systems ranging from AMRs, drones, and dogs, to delivery robots and humanoids. With our simple principle of separating the core functionalities from robot and platform extensibility, we have succeeded in adapting to the differing needs of both the robot hardware and its operators.
Notifications, interventions & incident management
The platform comes with a set of eventing and notification capabilities that can either be set via the UI or through the APIs. The platform also supports intervention requests in real-time, where-in a robot can request for an operator to intervene to make a decision at a specific point in time. The platform also comes with support for sending events or notifications to third-party solutions like PagerDuty or Slack.
Central traffic control using Open-RMF application
Users also gain support for partner applications that enable domain-specific or deeper capabilities for our robotics customers. One such application is Open-RMF. RMF, which stands for Robotic Middleware Framework, provides a set of conventions, tools, and software implementations to allow multiple fleets of robots to interoperate with each other and with shared building infrastructure, such as lifts, doors, corridors, and other natural “bottlenecks” to traffic flows and tasks. It utilizes standardized communication protocols to infrastructure, environments, and automation where robots are deployed to optimize the use of critical resources (i.e. robots, lifts, doors, passageways, etc). It adds intelligence to the system through resource allocation and by preventing conflicts over shared resources. Open-RMF is part of the wider open source platforms offered by Open Robotics.
Vendor agnostic edge software using agent <> adapter architecture
Our robot fleet management platform supports remote deployment of adapters directly to the Formant agent. This allows the fleet management software to remotely update or upgrade the adapters as the fleet changes or new requirements pop up without having to physically be near the robot.
The future is multi-robot
At Formant, we firmly believe that the future is multi-robot. As those on the frontier of robotics continue to democratize autonomy, more and more elements of businesses will leverage robots of all shapes and sizes. Warehouse AMRs might be able to operate in a closed system, but what about when they need to interact with autonomous forklifts in the loading bay? Or humanoids, or autonomous trucks?
A heterogeneous robot fleet management platform provides a single plane of glass for your team and enables flexible management of your autonomy systems today and in the future.
