How Cosylab Combined its FPGA and Software Skills to Develop Custom Firmware
The chassis of the AGV is the foundation for all the other system components of the “AI-Driven AGV for Obstacle Detection and Avoidance” use case. Apart from power, the automated guided vehicle also provides connectivity to its built-in programmable logic controller (PLC) that controls movement and gathers sensor data.
The data is exchanged via a Controller Area Network (CAN) bus using the CANOpen protocol. The video camera used for object detection, which is attached to the front of the AGV case, is connected to the Xilinx ZCU104 development board (“the ZCU104“) through the USB interface.
The LiDAR unit is mounted below the camera, at the front of the AGV, and communicates with ZCU104 over the ethernet interface. The ZCU104 can be connected to the development computer via built-in ethernet or a USB Wi-Fi card.
The principal hardware for the control system is the Xilinx ZCU104 development board, based on the Xilinx UltraScale+ MPSoC (MultiProcessor System on a Chip), which includes both the FPGA circuity and a general-purpose CPU with four main ARM Cortex-A cores. The hardware platform runs the operating system and custom firmware developed by Cosylab.
The firmware includes primary input/output (I/O) connectivity on the development board, while the FPGA integrates machine learning processors — DPUs (Deep Learning Processors). With this, neural network processing is offloaded from the main CPU to the DPUs that run custom-compiled neural network models.
Cosylab created a custom version of the Petalinux operating system, which includes application packages, libraries, development tools and configuration files.
The Linux image also includes a development installation of the Robot Operating System 1 (ROS1) Noetic.