Important: These are eight single demonstration runs, not a formal benchmark. Each notebook was run once for its recording, so the results should not be treated as a statistical comparison of GPT-6 Astra, Jev, Grounding DINO, or SAM 3. Timings and outcomes can vary between runs.

What Is the PiPER + GPT-6 Astra + Jev Project?

The project's goal is easy to state:

See an object → understand the task → choose an action → plan a grasp → move the arm → verify the result.

The interesting part is that the eight runs do not all solve this problem in the same way.

• In Run 1, GPT-6 Astra receives camera images directly and acts as the high-level decider.

• In Runs 2–8, Jev acts as the decider while local perception components provide more structured information about the workspace.

• Grounding DINO supplies object detections in Runs 2 and 4.

• SAM 3 supplies segmentation masks in Runs 3 and 5–8.

• STL geometry is added in Runs 6–8 to reason about balance, recover the pose of reflective objects, and align two parts.

This progression matters more than a simple “GPT-6 vs. Jev” framing. It shows that real-world robot performance depends on the complete manipulation stack—not only on the model selecting the next skill.

Source: X @MarianPogran (https://x.com/MarianPogran/status/2101179679319241076)

 

Hardware and Software Stack

The repository describes the following physical setup.

Component	Role in the project	Why it matters
AgileX PiPER robotic arm	Executes the manipulation skills	Provides the 6-DoF physical platform, gripper, CAN control, and joint feedback
Intel RealSense depth camera	Observes the tabletop and supplies color and depth data	Supports object localization, mask-depth fusion, tray measurements, and visual alignment
CAN interface (can0)	Connects the workstation to PiPER	Carries commands and state between the control software and the arm
Workstation with RTX 4090 or similar	Runs local perception models	Accelerates Grounding DINO and SAM 3 inference
Physical emergency stop	Provides immediate human intervention	Essential because perception or model decisions can be wrong

The software stack is modular rather than monolithic.

Software layer	Component used	Responsibility
High-level decision	GPT-6 Astra or Jev	Interprets the task state and selects an available robot skill
Object detection	Grounding DINO	Returns a bounding box for a text-described target
Object segmentation	SAM 3	Returns the pixels belonging to the target object
Geometry reasoning	STL models and project geometry utilities	Estimates pose from silhouette and searches for geometry-aware grasps
Grasp planning	Box-center, mask-based, or geometry-based search	Chooses where and how the gripper should contact the object
Kinematics	PiPER model from MuJoCo Menagerie	Converts target poses into valid arm configurations
Arm control	piper_control over AgileX piper_sdk	Enables the arm and sends commands to the physical PiPER hardware
Experiment interface	Jupyter notebooks	Guides hardware checks, calibration, dry runs, and live execution

PiPER itself is a lightweight 6-DoF arm with Python API support and compatibility with ROS 1 and ROS 2. Developers can review the current specifications and integration options on the official AgileX PiPER product page.

What Are GPT-6 Astra and Jev?

Although both models help decide what PiPER should do, they are different kinds of AI systems.

What is GPT-6 Astra?

GPT-6 is OpenAI's family of general-purpose reasoning models, and GPT-6 Astra is the flagship model used in Run 1. According to the official OpenAI model documentation, Astra supports text and image input as well as function calling and structured outputs. That combination allows it to inspect camera images, reason about the scene, and select a robot skill through a software interface.

GPT-6 Astra is still not a motor controller. In this project, it contributes visual understanding and high-level reasoning; calibrated robotics software remains responsible for grasping, kinematics, limits, and physical execution.

What is Jev?

Jev is TypeSafe's first System One model. Strictly speaking, it is not a conventional generative LLM. The TypeSafe documentation describes it as a structured decision model: it evaluates text or structured application state and returns typed answers with probabilities instead of generating open-ended prose.

That makes Jev suitable for focused questions such as which allowed skill to run, whether a condition is true, or how a situation should be scored. In Runs 2–8, specialized vision components first turn the camera observation into useful scene information; Jev then acts as the high-level decider. It does not replace Grounding DINO, SAM 3, the geometry pipeline, or the robot controller.

Model	Model type	Input used in this project	Main role in the PiPER pipeline
GPT-6 Astra	General-purpose multimodal reasoning model	Camera images and task context	Understand the visual scene and select robot actions
Jev	Structured System One decision model	State produced with help from specialized perception	Select among defined actions using typed decisions

Why model performance is not an absolute comparison

It would be misleading to treat these runs as proof that one model is universally “better.” GPT-6 Astra and Jev receive different inputs and are embedded in different pipelines. A model that interprets images directly is solving a different problem from a decision model that receives information already extracted by Grounding DINO, SAM 3, depth processing, or STL matching.

Robot outcomes also depend on calibration, lighting, object pose, perception quality, grasp planning, motion constraints, latency, and physical contact. A stronger reasoning model cannot recover information that a sensor never captured, while a specialized decision model may be preferable when an application needs constrained outputs, predictable integration, or lower decision overhead.

The right question is therefore not “Which model wins?” but “Which combination of model, perception, and control components is appropriate for this task?” The eight project runs are demonstrations of different architectures — not that benchmark, but a good example for hands-on learning and experiments.

How Does an LLM Actually Control a Robotic Arm?

The phrase “LLM-controlled robot” can create the wrong mental picture. The model is not the entire controller, and it should not be treated as one.

In this project, the AI model sits at the decision layer. It receives either camera images or processed scene information, reasons about the current task, and chooses from robot skills exposed by the software. The robotics stack then checks and executes the selected skill.

A simplified control loop looks like this:

Task instruction

↓

Camera observation

↓

Perception: direct vision, detection, segmentation, or geometry fitting

↓

Structured scene state

↓

GPT-6 Astra or Jev selects a robot skill

↓

Grasp planning + kinematics + safety checks

↓

PiPER executes the motion over CAN

↓

Camera and gripper feedback update the state

↺

1. The model decides what to do

At the highest level, the model reasons about task progress. Has the target been found? Should the arm look again, attempt a grasp, lift the object, move to the tray, release it, or recover after a failed attempt?

That is a good fit for a language or multimodal model because the problem is contextual. The correct next action depends on the instruction, what the system currently perceives, and what happened during previous steps.

The model should choose from a constrained skill set rather than invent arbitrary hardware commands. This gives the rest of the system a defined interface to validate.

2. Perception converts pixels into robot-usable information

A raw image is meaningful to a multimodal model, but a physical robot often needs more precise information:

• Which object is the target?

• Where is it in the image?

• Which pixels belong to it?

• How far is it from the camera?

• What is its orientation?

• Which regions can the gripper contact?

The runs explore three answers. 

GPT-6 Astra can examine the complete camera image and interpret the scene directly;

Grounding DINO locates the target object by drawing a rectangular bounding box around it;

SAM 3 produces a more precise segmentation mask that identifies the pixels belonging to the object.

For known objects, an STL file adds a fourth source: prior knowledge of the object's 3D shape.

3. A grasp planner decides where the fingers should close

Finding an object is not the same as finding a grasp. A box around a cube may be enough to estimate a center grasp. The same rule can fail on a ring, a trowel, or a tool with a long handle: the center of the box might be empty, lie on the blade, or leave no space for both fingers.

The project therefore progresses from box-center grasps to mask-based grasp search and then to geometry-aware grasps. Each step gives the planner more physically useful information.

4. Kinematics turn intent into reachable motion

Once the software has a target pose, it still needs to determine how the PiPER arm joints can reach it. Kinematics and robot-specific constraints translate an intended end-effector pose into joint motion.

This is traditional robotics work, and it remains necessary even when an advanced model chooses the action. The LLM does not remove calibration, reachability, collision risk, joint limits, speed limits, or workspace boundaries.

5. The arm driver executes the command

The project drives PiPER through piper_control, a wrapper around the AgileX piper_sdk. The driver handles the hardware enable sequence and communication cases in which the arm stops responding.

This separation is valuable: high-level model output becomes a request to a controlled robot interface, not an unrestricted message sent directly to the motors.

6. Physical feedback closes the loop

Robots act in a world where objects can slip, move, disappear behind the gripper, or differ from the estimate. After an action, the system must observe what actually happened.

In this project, vision supports scene updates, while the gripper state confirms whether a grasp succeeded. A lift is allowed only after the fingers stop partway in a way that indicates an object is being held.

The key idea is simple:

The AI model helps decide what should happen next. Specialized perception and robotics software determine where and how it can happen, and physical feedback checks whether it really happened.

Eight Real-World PiPER Robot Manipulation Runs

The eight runs form a practical capability ladder. The scene, perception source, grasp method, or available geometric knowledge changes from one stage to the next.

Run	Decider	Perception and prior knowledge	Task	Main lesson
1	GPT-6 Astra	Astra sees camera images directly	Put a cube into a tray	A multimodal model can participate directly in scene understanding and task decisions
2	Jev	Local Grounding DINO	Put a cube into a tray	Perception and decision-making can be separated into inspectable modules
3	Jev	Local SAM 3	Put a cube into a tray	A segmentation mask gives more object-shape information than a box
4	Jev	Local Grounding DINO	Pick up an object by its handle and place it in the tray	A box-center grasp can be a poor match for an irregular object
5	Jev	Local SAM 3	Repeat the handle task with a mask-planned grasp	Mask and depth data can support a grasp across solid, suitably wide material
6	Jev	SAM 3 + the trowel's STL geometry	Hold the trowel where it balances and carry it level	Known geometry and center of mass can improve grasp stability
7	Jev	SAM 3 + the part's STL geometry	Put a chrome pipe fitting into the tray	A silhouette plus known geometry can help when depth sensing fails on reflective metal
8	Jev	SAM 3 + STL geometry for both parts	Slide the fitting over a standing box spanner	Relative visual alignment and geometry support tighter two-part manipulation

Again, the table summarizes demonstrations, not benchmark trials. It is useful for comparing architectures and information flow, but not for ranking model accuracy, speed, or reliability.

Runs 1–3: From Direct Vision to Specialized Robot Perception

Runs 1–3 use the same cube-to-tray task but change how the workspace is perceived.

Run 1: GPT-6 Astra sees the workspace directly

GPT-6 Astra receives the camera images directly, interprets the cube and tray, and helps select the next robot skill. Calibration, grasp generation, kinematics, limits, and motion control still remain in the robotics stack.

Run 2: Jev receives Grounding DINO detections

Grounding DINO answers “Where is the requested object?” Jev uses that structured result to decide “What should the robot do next?” This separation makes the perception and decision layers easier to inspect, replace, and debug independently.

Run 3: Jev receives a SAM 3 object mask

A bounding box places the object inside a rectangle; a SAM 3 mask identifies its pixels. Either may work for a cube, but the mask preserves more shape information for later, less regular objects.

Runs 4–5: Why Object Detection Is Not Enough for Grasping

Runs 4 and 5 replace the cube with a tool that should be picked up by its handle, showing why localization and grasp planning are different problems.

Run 4: A bounding box leads to a center grasp

Grounding DINO provides a box, so the straightforward grasp is its center. That can work for a cube but fail on a tool, where the center may fall on the blade or another unsuitable point.

Run 5: A segmentation mask supports a physical grasp search

SAM 3 provides a target mask. Fused with depth, it lets the planner search for solid material and enough room for both fingers—moving from “Where is the object?” to “Where can the gripper hold it?”

This does not make one perception model universally better: a box may be enough for localization, while a mask is more informative for contact planning.

Runs 6–8: Adding 3D Geometry for More Precise Manipulation

Runs 6–8 combine the observed silhouette from segmentation with an STL model of the object's expected 3D shape.

Run 6: Grasping a trowel near its balance point

An object grasped away from its center of mass tends to rotate. Using the trowel's geometry and center-of-mass metadata, the planner favors a valid contact line near the ferrule, where the blade-heavy tool balances more naturally and can be carried closer to level.

Run 7: Using geometry when depth fails on chrome

Chrome can produce incorrect or missing depth values. Run 7 matches the fitting's SAM 3 silhouette against rendered views of its STL model. The tabletop fixes its height, and the best silhouette match estimates the remaining pose for a geometry-based grasp and reach.

Run 8: Aligning two reflective parts with about 2 mm to spare

PiPER slides the chrome fitting over a standing box spanner. Its 25.5 mm bore and the spanner's 21.3 mm width leave roughly 2.1 mm per side when centered. Both reflective parts use STL geometry; the camera measures them together and corrects their relative alignment so common calibration error largely cancels. The project reports alignment within 0.8 mm before lowering at 5 mm/s and releasing the fitting.

What Changed from Run 1 to Run 8?

The runs gradually increase the physical knowledge available to the system.

Stage	What the robot stack knows	Question it can answer
Run 1	Camera images and multimodal context	What is in the scene, and what should happen next?
Run 2	Detected target location	Where is the requested object?
Run 3	Segmented target pixels	Which visible pixels belong to it?
Runs 4–5	Candidate contact regions	Where can the gripper close on it?
Run 6	Known geometry and center of mass	Where can the gripper hold it more stably?
Run 7	Silhouette, known geometry, and tabletop constraint	What is the pose when depth is unreliable?
Run 8	Geometry and relative alignment of two objects	How can one part be aligned and placed over another?

The lesson is not that every robotics project needs every component. It is that the required representation depends on the physical task. A cube pick-and-place demo may tolerate a rough center estimate. A reflective-part insertion task needs much tighter geometry, pose, and feedback.

How Robot Grasping Works: Look, Reach, Touch

The fixed RealSense camera sees the complete workspace only while the arm is out of the way. As the gripper approaches a target, its fingers begin to hide the object. Continuously trusting every new image can then make the system follow part of its own gripper instead of the target.

The project handles this with a three-stage strategy that resembles human reaching.

Stage	What the system does	Trusted information
Look	Observe the target while the view is clear and plan the grasp	Camera perception, depth, mask, and/or geometry
Reach	Move toward the stored grasp even after the fingers occlude the object	Remembered target pose and robot motion
Touch	Check whether the fingers stopped on an object before allowing the lift	Gripper state rather than the image

Look

The robot plans the grasp while the target is fully visible. It keeps that plan when the fingers enter the image. The system also remembers the tray location while the arm is above it.

Reach

During the final approach, a new detection above the table or larger than the object can be a sliver of the target seen on the gripper—or the gripper itself. The system rejects observations that violate configured height or pixel-size limits and trusts the stored plan through the brief blind phase.

Touch

The gripper supplies the success signal. If the fingers close fully, they probably caught nothing. If they stop partway, Arm.holding() indicates that an object is between the jaws, and only then may the arm lift.

This approach does have a limit: if the target moves after it becomes hidden, the remembered plan becomes wrong. The grasp can close on empty space, after which the decider must open the gripper and look again. The project notes that a wrist camera could reduce this blind interval.

How to Run the LLM Project on PiPER Robotic Arm

The repository supplies eight Jupyter notebooks, one for each demonstration. The commands below summarize the published setup; read the repository's current instructions and every notebook's safety checks before enabling real motion.

1. Prepare the Python environment

python -m venv .venv

source .venv/bin/activate

pip install -r requirements.txt

pip install -e .

2. Configure credentials locally

Copy .env.example to .env and provide only the credentials needed by the selected run:

OPENROUTER_API_KEY= # Jev runs

OPENAI_API_KEY= # GPT-6 Astra run

HF_TOKEN= # gated SAM 3 model

The project states that .env is ignored by Git. Never commit API keys to a repository or notebook.

3. Bring up the PiPER CAN interface

sudo ip link set can0 up type can bitrate 1000000

4. Launch the jupyter notebooks

The notebooks are numbered from the basic Astra cube task through the final fitting-over-spanner task. Each follows the same general sequence: verify the hardware, calibrate the camera, run the safety checks without torque, and only then execute the task on the enabled arm.

For your first experiment, start with Runs 1-3 before moving on to handle grasping, STL-based pose estimation, or the two-part placement task. The later runs introduce additional assumptions and failure modes that are easier to understand once the basic perception-decision-execution loop is working.

Safety: Connecting an AI Model to a Real Robot

A chatbot mistake produces a bad answer. A robot-control mistake can produce unexpected physical motion. The project reduces risk rather than eliminate it. A model may choose the wrong skill, perception may place an object incorrectly, and the arm can move in an unexpected way.

Before every run:

• Keep a physical emergency stop within immediate reach.

• Keep hands, clothing, and cables outside the arm's reachable workspace while it is enabled.

• Remove anything from the table that cannot safely be knocked over.

• Start at speed_percent=10 and increase speed only after a clean run.

• Complete the notebook's dry checks before enabling torque.

• Never leave the enabled arm unattended.

• Keep fingers out of the gripper jaws; they close with real force.

Also consider task-specific safeguards such as workspace bounds, joint and Cartesian limits, allowlisted skills, speed and force limits, pre-motion validation, timeouts, state monitoring, and a human-controlled enable sequence.

What Robotics Developers Can Learn from the Project

1. LLM is the part of a hybrid system

The model is most useful as one component in a hybrid system. Specialized perception, deterministic motion code, safety checks, and physical feedback remain essential.

2. The best perception output depends on the action

A bounding box can be enough to select or roughly locate an object. A segmentation mask is more useful when the grasp depends on visible shape. A known 3D model becomes valuable when balance, hidden surfaces, or reflective materials matter.

3. Robot failures should be diagnosable by layer

Separating perception, decision, grasp planning, kinematics, and execution makes debugging more practical. “The robot missed” is not a diagnosis. The useful question is whether it saw the wrong target, estimated the wrong pose, chose a poor grasp, requested an unreachable pose, or lost the object during execution.

4. Relative measurements can beat absolute calibration

Run 8 uses a powerful robotics pattern. When two objects appear in the same camera frame, their relative offset can be more accurate than two separately transformed world coordinates because common calibration error can cancel.

5. Memory is part of manipulation

The Look–Reach–Touch strategy shows that a robot cannot assume uninterrupted visibility. It needs short-term state: the last trusted target pose, the planned grasp, the tray location, and the result of the most recent action.

6. Demonstrations and benchmarks answer different questions

These single runs show that the architectures can be assembled and exercised on real hardware. They do not establish success rates, robustness across lighting and object layouts, latency distributions, or superiority of one model.

A formal evaluation would require repeated trials, controlled variations, defined success criteria, failure categories, timing methodology, and confidence intervals. That would be a valuable next step, but it is outside the claim made by this project.

Why PiPER Robotic Arm Is a Useful Platform for Robot Control Experiments

PiPER 6-DoF robotic arm brings AI experiments out of simulation and into a manageable physical workspace. Its open development options let researchers combine conventional robot programming with newer perception and reasoning systems.

PiPER is a 6-DoF robotic arm with a 1.5kg payload, 4.2kg body weight, approximately 626mm reach, Python API control, and ROS 1/ROS 2 support. Those characteristics make it relevant to desktop manipulation, education, teleoperation, data collection, and embodied-AI prototyping.

Most importantly, a physical arm exposes problems that a text-only or simulation-only demo can hide:

• camera-to-robot calibration;

• self-occlusion;

• noisy or missing depth;

• grasp uncertainty;

• object balance and contact;

• reach and orientation constraints;

• hardware latency and communication failures; and

• real safety boundaries.

Those are not distractions from embodied AI. They are the work embodied AI systems must eventually handle.

Frequently Asked Questions

Can an LLM model control a robotic arm?

Yes, but usually as part of a layered system. An LLM or multimodal model can interpret instructions, reason about a scene, and select robot skills. Perception, grasp planning, kinematics, safety checks, and the hardware driver should still handle the specialized work required to execute those skills.

Did GPT-6 Astra directly control PiPER's motors?

No. In the RobotKitAI project, GPT-6 Astra acts at the high-level decision and visual-reasoning layer. Robot-specific software handles grasp generation, kinematics, motion, safety limits, and communication with PiPER.

What is the difference between GPT-6 Astra and Jev in these runs?

Run 1 gives GPT-6 Astra the camera images directly. Runs 2–8 use Jev as the decider with perception outputs from Grounding DINO or SAM 3, and later add STL geometry. Because each setup was recorded once, the project is not a benchmark comparing the models' reliability or performance.

Why use SAM 3 instead of only Grounding DINO for robot grasping?

Grounding DINO returns a bounding box, which is useful for locating an object. SAM 3 returns an object mask, which can describe the visible shape more precisely. A grasp planner can combine that mask with depth to look for solid material and room for both gripper fingers. Which method is appropriate depends on the object and task.

Why does the project use STL files?

An STL file supplies known 3D geometry. The project uses geometry to search for a balanced trowel grasp, estimate the pose of reflective chrome parts from their silhouettes, and align two known parts in the final run.

Can a depth camera see chrome objects reliably?

Often not. Reflective surfaces can return incorrect or missing depth. In Run 7, the project estimates the chrome fitting's pose by matching its segmented silhouette to rendered views of its known STL geometry while using the tabletop as a height constraint.

What is Look–Reach–Touch in robot grasping?

It is the project's strategy for handling self-occlusion. The robot looks and plans while the object is visible, reaches using the remembered plan when the gripper blocks the camera, and confirms the grasp using gripper feedback before lifting.

Is this project a benchmark of GPT-6 Astra versus Jev?

No. The repository explicitly describes the experiments as single runs rather than a benchmark. They demonstrate different system designs but do not provide enough repeated trials for a statistical model comparison.

What hardware is needed to reproduce the project?

The published setup uses an AgileX PiPER arm connected over CAN, an Intel RealSense depth camera, an RTX 4090-class workstation for local perception, and a physical emergency stop. The exact requirements can vary with the models and runs you choose.

Does AgileX PiPER support ROS 2 and Python development?

Yes. AgileX Robotics lists Python API control and both ROS 1 and ROS 2 support for PiPER. The RobotKitAI project itself uses notebook-driven Python code and CAN communication rather than requiring every component to run through ROS.

From Language Models to Physical Intelligence

The most useful question is no longer simply, “Can an LLM control a robot?”

Of course it can, but the better question is:

Where should a general-purpose model sit inside the robot-control stack, and which responsibilities should remain with specialized perception, geometry, control, and safety systems?

The piper-astra-jev demonstrations offer eight concrete answers. Direct multimodal vision can support a simple task. Structured detections make the pipeline modular. Segmentation improves the information available to a grasp planner. Known geometry helps with balance and reflective surfaces. Relative visual feedback supports tighter placement. Throughout the process, kinematics, hardware interfaces, constraints, and touch feedback turn a high-level decision into a real physical result.

References: Explore LLM and Build with PiPER Robotic Arm

• Study or reproduce the eight runs: Visit the RobotKitAI piper-astra-jev repository on GitHub for the notebooks, source code, setup details, run documentation, and safety notes. Huge thanks to Michal Kubenka and Marian Pogran for creating this project!

• Build your own AI robotic arm project: Explore the AgileX Robotics PiPER for specifications, Python/ROS development options, and contact information.

Whether you start with a cube pick-and-place task or a geometry-aware manipulation pipeline, begin slowly, validate each layer independently, and keep a human in control of the physical system.

Egocentric data collection is a method of capturing first-person human activity, interactions, and manipulation data using wearable cameras, sensors, or portable data collection devices. In robotics, this data can be used to train Physical AI models by providing large-scale examples of how humans perceive, interact with, and manipulate the physical world.

In 2026, the Robotics market worldwide is expected to witness a significant growth in revenue, reaching a projected value of US$47.22bn; while egocentric data is emerging as an important complement to traditional robot teleoperation and simulation. As robotics companies seek larger and more diverse robot training datasets, first-person human data offers a potentially more scalable way to capture real-world tasks without requiring a robot for every demonstration. The change is being driven by four forces: technology validation, industry demand, falling costs, and the growing need for scalable robotics.

This article examines why egocentric data collection is growing, how it fits into the robot learning pipeline, and how approaches from NVIDIA, JD, AgileX, Appen, Scale AI, and others are shaping the emerging ecosystem.

Egocentric Data Collection System (Pika EGO head belt)

What is Egocentric Data in Robotics?

Egocentric data is first-person data captured from a human’s perspective while interacting with the physical world. It can include RGB/depth video, hand-object interactions, 6D pose, motion trajectories, and task sequences.

For robot learning, this data provides valuable information about what humans see, how they act, and how objects and environments change during a task, making it useful for embodied AI and robot model training.

Egocentric First-person View

Where Egocentric Data Comes In: Robot Data Gap

The rise of egocentric data collection is closely linked to growing interest in large-scale human video for robot model training.

In early 2026, NVIDIA Research published two relevant studies. EgoScale trained a VLA model (vision-language-action) on more than 20,000 hours of action-labeled egocentric human video and found a scalable relationship between human data volume and model performance.

DreamDojo took a broader approach by pretraining a robot world model on approximately 44,000 hours of human egocentric video. It showed that even videos without explicit robot action labels can support physical reasoning, world-model pretraining, and robot planning.

Together, these studies suggest that egocentric data can support both dexterous robot policy training and large-scale embodied AI model training.

Projects such as Ego4D and Ego-Exo4D have also shown that first-person video can capture valuable information about human activities, hand-object interaction, and task structure.

This is why EGO data collection is becoming an important foundation for scalable robot training data.

EgoScale from NVIDIA

How Large Is the Robot Training Data Gap?

Training embodied AI models requires large volumes of multimodal, high-quality physical interaction data.

However, the current situation can be summarized as follows:

• Available robot data: Only a limited amount of high-quality physical interaction data is publicly available.
• Model training demand: General-purpose robot models may require thousands or even tens of thousands of hours of demonstrations.
• Data gap: The gap between available robotics training data and future model training requirements remains extremely large.

For language and image models, data can often be collected from the internet at relatively low marginal cost. Robotics is different. Every robot demonstration requires hardware, space, operators, supervision, and safety management. That is why the industry is increasingly searching for scalable robot data collection methods, including teleoperation data, human demonstrations, simulation data, and egocentric data collection.

Why Real-Robot Data Is Expensive to Scale?

Embodied AI requires a continuous data flywheel:

Real-world deployment → Robot data collection → Model training → Better robot performance → More deployment

The problem is that the first stage is already expensive.

Robot-native data collection usually requires a specific robot embodiment, sensors, calibration procedures, teleoperation equipment, and trained operators. As a result, data collection capacity is directly limited by the number of available robots and the cost of operating them.

Traditional Robot Data Collection vs. Egocentric Data

Traditional collection methods face a difficult trade-off between quality, cost, and scalability.

Collection Method	Data Quality	Cost	Scalability	Main Limitation
Real-robot teleoperation	High	Very high	Low	Expensive hardware and operator time
Simulation & synthetic data	Medium	Low	High	Sim-to-real gap
Internet video	Low	Very low	Very high	Limited action and interaction signals
Egocentric data collection	Medium	Low to medium	High	Requires pose/action extraction and multimodal processing
UMI-based collection	Medium to high	Medium	Medium to high	Requires specialized hardware and robot-action alignment

Real-robot demonstrations provide accurate robot actions but are expensive. Simulation can produce large volumes of synthetic training data but may not fully reflect real-world physics. Internet video is abundant but often lacks structured task information. Egocentric or UMI-based collection provides a middle ground by capturing natural human behavior in realistic environments without requiring a robot to be present in every recording.

Why Egocentric Data Collection Is Growing in Robotics

Hardware Costs are Falling

Traditional robot teleoperation depends on expensive robot hardware and dedicated control systems. Egocentric data collection changes the hardware structure.

Instead of deploying a complete robot platform for every collection task, a data collector can use a lightweight camera and sensor system to record real-world human activity. Depending on the application, the system may include:

• A head-mounted or wearable camera
• RGB and depth cameras
• IMU sensors
• Hand or body-pose tracking
• Optional handheld manipulation interfaces
• Edge computing and cloud storage
• Task management and quality-control software

In-the-Wild Data Collection

This reduces the cost and logistical complexity of robotics data collection. More importantly, it allows data collectors to work in real-world scenarios including homes, offices, workshops, factories, warehouses, and outdoor environments.

The key change is that the data collector is no longer strictly tied to one robot embodiment.

AgileX Robotics’ Pika Pro is one example of this direction. The system combines first-person capture, handheld manipulation hardware, RGB and depth sensing, and data-processing & computing capabilities for robot learning and embodied AI research.

Pika Pro Egocentric Data Collection System

Cross-Embodiment Potential

Unlike robot-native data tied to a specific robot, egocentric data can capture more robot-agnostic information, such as task semantics, object interactions, and spatial relationships.

With appropriate processing and retargeting, the same human demonstrations can potentially support different robot embodiments. Robot-specific data is still needed for precise control and policy fine-tuning.

How Egocentric Robot Data Collection Works

The underlying logic of ego data collection is visual and physical alignment. A human’s first-person view is closer to the visual perspective of a robot than conventional third-person video. It captures:

• What the operator is looking at
• How the hand approaches an object
• The order of task actions
• Contact between the hand and the object
• Changes in object state
• Corrections made during execution
• The relationship between the operator and the environment

However, human data cannot be transferred directly to a robot without processing. A complete human-to-robot data pipeline usually includes:

1. Recording synchronized video and sensor streams.
2. Segmenting long recordings into task episodes.
3. Estimating hand, body, and object motion.
4. Identifying key actions and interaction events.
5. Converting human actions into robot-compatible representations.
6. Aligning the data with robot embodiments, action spaces, and control interfaces.
7. Validating the final data through human or automated quality checks.

Who is Building Egocentric Robot Data Infrastructure

The egocentric data collection ecosystem includes base model developers, robot manufacturers, data collection providers, data annotation platforms, and research organizations.

A Look at the Leading Players of Robot Data Collection

Organization	Egocentric Collection Route	Core Product / Strategy	Data Captured	Scaling Strategy
NVIDIA	Human egocentric demonstration	EgoScale — large-scale egocentric human video collection	First-person video, human-object interactions, manipulation demonstrations	Large-scale human demonstration datasets for Physical AI
JINGDONG	Distributed human-first collection	JoyEgoCam + large-scale distributed contributor network	First-person video, daily activities, human-object interactions	Low-cost distributed collection across diverse real-world tasks
AgileX Robotics	Portable UMI-based robot data collection combining egocentric data	Pika Pro / Pika Sense / Pika Ego — portable egocentric data collection system	RGB / depth, 6D pose, gripper state, manipulation demonstrations	Portable, low-cost hardware for scalable in-the-wild collection
Appen	Workforce-based egocentric collection	Global contributor network + wearable / first-person capture workflows	Egocentric video, human activities, object interactions, task demonstrations	Global workforce for diverse environments, people and tasks
Scale AI	Managed real-world egocentric collection	Physical AI data collection operations & human demonstration programs	First-person human demonstrations and real-world interaction data	Managed data operations for large-scale Physical AI datasets
AgiBot	Portable egocentric manipulation collection	MEgo View / MEgo Gripper + distributed collection ecosystem	First-person video, hand/object interaction, manipulation trajectories	Hardware + distributed collection platform + dataset marketplace

The market is still developing. Some organizations focus on providing robot training datasets, some provide data infrastructure including robotics hardware and accessible systems. In practice, successful robotics model training will likely combine several of these capabilities.

Future: Egocentric + Robot + Synthetic Data

A “Large Brain + Small Brain” Architecture

The future of robot training data will likely combine two layers:

• Small-brain layer: Robot-native teleoperation for precise, short-horizon skills and policy fine-tuning.
• Large-brain layer: Egocentric human data for long-horizon tasks, scene understanding, and real-world diversity.

The first provides precision; the second provides scale.

The Data Fusion Route

A strong robotics training pipeline will combine real-robot data, egocentric human data, and synthetic data. Real-robot data supports control and evaluation, egocentric data supports pretraining and task understanding, while simulation provides low-cost augmentation.

Projects such as Open X-Embodiment demonstrate the value of diverse and cross-embodiment robot data.

Data Quality as a Competitive Advantage

As collection scales, data quality will matter more than raw volume. Useful robot training data must be synchronized, well-labeled, diverse, privacy-compliant, and aligned with the target robot.

Conclusion

The rapid growth of ego data collection is not accidental. The embodied AI data race has only just begun. The long-term winners will not simply be the organizations that collect the most data, but those that can connect human experience, multimodal sensing, and robot action into a reliable robot training data pipeline.

FAQ

What is egocentric data in robotics?

Egocentric data is first-person data captured while humans interact with objects and environments. It can include RGB/depth video, pose, motion, and manipulation signals, helping robot models learn real-world tasks and physical interactions.

How is egocentric data different from teleoperation data?

Robot teleoperation records actions directly from a specific robot, providing precise embodiment-specific data. Egocentric collection captures human demonstrations and can scale more easily across different tasks, environments, and locations.

What is the difference between egocentric data and UMI?

Egocentric data collection broadly refers to capturing interactions from a first-person perspective. UMI-based systems extend this approach by combining visual data with manipulation signals such as pose, motion, and gripper states, making demonstrations more useful for robot learning.

What tools can be used to collect egocentric robot training data?

Depending on the task, researchers may use wearable cameras, RGB-D sensors, pose-tracking devices, or portable manipulation interfaces. AgileX Pika Pro, for example, combines first-person capture, RGB/depth sensing, spatial tracking, and portable computing for in-the-wild robot data collection.

Can egocentric data replace real-robot data?

Not entirely. Egocentric data provides scale and diversity, while real-robot data provides precise, embodiment-specific actions and control signals.

References

1. Ego4D official documentation
2. Ego-Exo4D official documentation
3. Google DeepMind: Scaling up learning across many different robot types
4. Open X-Embodiment project
5. EgoScale
6. DreamDojo
7. AgileX Robotics Pika Pro

The AgileX Robotics Promotes an Upgraded Quality of Scientific Research and Education with ROS2

The birth of ROS has allowed the robotic industry to develop rapidly under a common framework. Based on the creation of a diverse robotic ecosystem and the application of ROS, the robotic control, perception and decision-making can be better organized and operated. With the gradual deepening of robot research, design, and application, robots have been deployed in a variety of environments, such as logistics and distribution, security and inspection, special operations, etc. As a result of diverse industry applications, there are also increasingly high demands for robot navigation, tasks, scheduling, operation, system stability and scalability, and even cost assessment.

When the ROS 1 version faces these demands, problems, such as its single robot system, single platform, poor real-time performance, poor stability, high network requirements and poor confidentiality, have also gradually emerged, and it is gradually failing to meet the task requirements that complex environments demand from robots currently. The birth and application of the ROS 2 provides the possibility and feasibility for robots to break through current bottleneck in development. Based on this, the AgileX Robotics Research and Education Team carefully sorted out the current industry needs and concerns through in-depth exploration and research on the ROS 2 version, designed and developed the world’s first ROS 2 mobile robot open source navigation education kit - ROS 2 EDU Kit.

The ROS 2 EDU Kit is available in 2 versions — ROS 2 EDU Lite and ROS 2 EDU Pro — to meet the research, education, and usage needs of different users. The kit is based on the AgileX autonomous mobile robot platform - SCOUT MINI series, with the ROS 2 Foxy version as the core, equipped with 2D LiDAR, 9-axis IMU, a dual purpose depth camera, and other sensors. The industrial control machine adopts the X86 architecture and provides new features, including NAV 2 and Gazebo 11, which is based on the Ubuntu 20.04 system. The NAV 2 core navigation components are fully compatible with the official new features of NAV, providing a comprehensive and in-depth ROS 2 learning and research integrated robot development platform for scientific education.

What the ROS 2 NAV Kit does.

1) The ROS 2 NAV Kit has a full version of the ROS 2 Foxy, so you can experience the new features of ROS 2 in its entirety.

Figure 1: ROS2 FOXY FITZROY

2) The ROS 2 ROS Kit is installed with Gazebo 11 simulation development environment. You can simulate it in the visual interface, or you can get the ROS 2 simulation package on the official Github platform for AgileX Robotics (all models support ROS 2), which is more efficient and convenient.

Figure 2: Based on Gazebo 11 path simulation

3) The ROS 2 EDU Kit can build a point cloud map based on Cartographer.

Figure 3: Built on Cartographer’s point cloud map

4) The ROS 2 EDU Kit can conduct SLAM algorithm research and demonstration based on NAV 2 Navigation Stack.

Figure 4: Construction of indoor environment SLAM algorithm

Figure 5: NAV2 navigation stack

5) ROS 2 EDU Kit supports NAV 2 navigation stack

Figure 6: Robot barrier avoidance algorithm

Figure 7: Robotic indoor environment barrier avoidance algorithm simulation

6) ROS 2 EDU Kit also supports NOMACTHING remote programming

To make it easier for everyone to see the difference between ROS 2 and ROS 1, we will list the corresponding new features as follows:

The AgileX Robotics NAV Kit adapts all features of ROS2 and supports the user to conduct mobile robot research and development based on ROS2. In recent years, the research has peaked the interests of those concerned with agricultural applications, military applications, outdoor measurement and exploration, patrol and security, and other industrial applications, as well as unmanned vehicle research. It has software and hardware functions for scientific research, teaching, and presentations, and it can be developed, tested, and distributed safely and cost-effectively.

As a leader in the ROS open source ecosystem, AgileX Robotics has a strong understanding of the needs and difficulties of ROS2 scenarios, and, from the customer’s perspective, AgileX’s full series of chassis also supports ROS2 simultaneously. AgileX aims to reduce the ROS2 development threshold, improve the high modularity and multiplexing capabilities of robots, simplify the task volume, and experience the efficiency of human and machine intelligence collaboration.

The detailed information of ROS2 kit can be obtained from the AgileX official Github or from marketing and sales personnel. Everyone is also welcomed to post their experiences and feedback on the official AgileX online community (https://community.agilex.ai). We will also publish technical articles regularly, and everyone can chat on the AgileX community and Github.

Written by GuYueHome

01 What is SLAM

The full name of SLAM is Simultaneous Localization And Mapping, which translates to instant localization and map construction. There are two key words here: localization and map construction, which means that the robot will determine its position in an unknown environment while building a map, and finally output a map like this.

In short, the process of SLAM map construction is to measure the distance information between the robot and the surrounding environment with depth sensors, so as to complete the map construction of the surrounding environment. At the same time, the robot will check the consistency of the environment and detect whether it moves to the place where the map has been constructed, and finally complete the map closed loop to finish the whole map.

The most commonly used SLAM algorithm in ROS is Gmapping, a common open source SLAM algorithm based on the filtered SLAM framework, which is based on the RBpf particle filtering algorithm, i.e., the localization and map building processes are separated, and the localization is performed first and then the map is built. If you are interested in the Gmapping algorithm, you can read the paper on the algorithm at: https://openslam-org.github.io/gmapping.html

02 LIMO Mobile Robot Motion Control

2.1 LIMO multimodal robot

Our common mobile robot movement types are generally four-wheel differential, Ackermann steering, crawler differential, omnidirectional movement, etc., but as the saying goes, adults do not make a choice, Songling Robotics LIMO one to meet all your needs, four types of movement at will any switch.

We will devote to the testing of multimodal motion types in the future. Today's focus is on the implementation of SLAM application functions in the motion state of the LIMO robot with four-wheel differential speed.

2.2 Remote connection to LIMO

You can download NoMachine - Free Remote Desktop For Everybody directly from the official website of LIMO: https://www.nomachine.com/

Once the software is installed, the computer and the cart are connected to the same WiFi so that you can connect with the cart remotely.

2.3 Download and compile the code

The code for the LIMO robot is continuously being updated, as we can see on the Github homepage. To ensure that we are using the latest code, here we use the following command to download the code package to the workspace in the LIMO on-board system, replacing the original code.

Then use the catkin_make command to compile the space. This is a regular operation for ROS, so if you are not familiar with it, you can read "21 Lectures on Getting Started with ROS" by Guyue Jun.

2.4 Make the cart run

To make sure the underlying functionality of the robot is ready to work, let's try the most basic keyboard control to get the cart moving.

Start up a terminal to activate the cart's chassis and sensors.

After the chassis is successfully started, it will subscribe to the cmd_vel topic, just like controlling a small sea turtle, and start the following keyboard control node, you can control the LIMO smooth movement through the keyboard.

 03 Gmapping SLAM function configuration analysis

3.1 Gmapping package

Before using the Gmapping package in ROS, you need to install it using the following command, here is the Melodic version, if you are using other versions of ROS, replace the version name in the command.

The SLAM algorithm is not simple, but Gmapping is a "building block" that has been packaged very well, and we can find the interface on the ROS wiki The description is shown in the following figure.

As can be seen, Gmapping needs to subscribe to the robot joint transformation topic /tf and the LiDAR scan data topic /scan, and then publish the raster map topic /map.

Regarding /tf is further divided into two parts, the TF transformations in the Gmapping feature package are shown in the following figure.

So we can determine that the necessary conditions for the Gmapping algorithm: /tf, /odom, /scan, will be ready to publish/map through the algorithm.

After the start of the previous LIMO robot chassis, these basic conditions have been published or updated normally in the underlying node, so our focus is to configure the Gmapping function node through a launch file.

3.2 SLAM function configuration

Create a feature package called limo_slam under the src file in the workspace, where another launch folder is created, and the launch file about the Gmapping launch configuration is placed here.

The details of the launch file gmapping.launch are as follows. If you use other robots, you will need to change the parts commented in the following file according to your respective situation.

If you are familiar with the SLAM algorithm, these parameters may not be new to you, but if you don't know the SLAM algorithm, don't worry, these parameters have default values and most of the time you can just use the default values or use the configuration of similar robots in ROS.

Since the framework of the Gmapping algorithm is shown below, the main parameters we need to change are odom (odometer coordinate system, required), scan (LIDAR topic, required), base_footprint_frame (robot base coordinate system, required), imu_data (IMU topic, optional), and the names and values of these parameters need to be the same as The names and values of these parameters need to correspond to the topic names and values published in the robot chassis driver limo_start, otherwise Gmapping does not receive these parameters and cannot perform SALM.

04 Gmapping SLAM operation results

After doing the relevant preparations, we can use the LiDAR for SLAM.

First, the code is compiled under the workspace using catkin_make, and then a terminal is started separately to run the following commands.

In order to see the effect of dynamic SLAM, we also need to use the Rviz visual display by opening a terminal and entering the following command.

Then click the ADD button to add the required settings, such as Map, LaserScan, TF, etc., subscribe to the corresponding topic data, for example, when adding Map, we need to change the Topic under Map to /map.

Then we can control the slow movement of the cart for SLAM by keyboard. If we move the cart after starting SLAM, we need to do SLAM again, because the TF of the cart is likely to be wrong.

Pay attention to control the speed of movement of the car, if the speed is too fast when turning, it is easy to appear the phenomenon of map shift.

We can also use the rqt_graph command to see the relationships between the nodes:

At the end of SLAM, the following command is required to save the map：

Due to the limited space, we constructed the map as shown below.

At this point, we have implemented SLAM in a LIMO mobile robot using the Gmapping algorithm! Using a similar approach, we can also implement other SLAM function packages to run, such as hector, cartographer and rtabmap, etc. Each algorithm has its own advantages and disadvantages, so you can choose a suitable algorithm package according to your needs.

Students and researchers from Singapore Polytechnic developed an intelligent outdoor autonomous mobile robot for campus delivery. The robot is capable of navigating to the destination autonomously with collision avoidance. The robot can carry up to 10KG and 30KG of payload based on the robot base SCOUT MINI and SCOUT, respectively.

Picture: Two Outdoor Delivery Robot Based on SCOUT MINI and SCOUT

By combining GPS/RTK navigation with 3D-lidar based collision avoidance, the robot is able to navigate autonomously inside the campus where GPS signal is accessible. The GPS/RTK navigation can achieved up to 1cm accuracy with up to 10HZ updating frequency by talking to the RTK service from Singapore SLA.

Picture: Two Outdoor Delivery Robot Based on SCOUT MINI and SCOUT

Singapore Polytechnic sends their students and researchers to work with Weston Robot engineers to accelerate the pace of the development. The industry-academic alliance is a great tool to help students to bridge the gap between academic and industry. Weston Robot has a trained a dozens of students from local institutes to make them ready to embrace the robotics carrers.

Video: Two Outdoor Delivery Robot Based on SCOUT MINI and SCOUT

Resource from weston robot

For a plant, if it cannot access the necessary nutrients then will make the limit of the overall growth of the plant, which will greatly reduce the economic return to the farmer.  Farmers use a range of strategies for increasing the availability of nutrients for a growing crop, most notably the addition of chemical fertilizers, for example, nitrogen and phosphorus.

Professor Liangxiu Han of Manchester Metropolitan University and her team work to establish the cutting edge "N2 Vision" Nitrogen mapping system which allows farmers to apply fertiliser just where it's needed to maximise yield while protecting the environment.

Together with Europe's agricultural automatic spray system providers Auto Spray Systems, and GMV Group, Liangxiu's team build an advanced soil composition detection and mapping robot using nitrogen and soil sampling technology that could revolutionise the way farmers work in the near future.

Richard Westcott, the BBC's Science Correspondent spent the day finding out more about how autonomous robots can help UK farmers reduce their carbon and nitrogen footprint by working smarter. Checked out the video of the BBC introducing the N2 robotics platform.

https://www.youtube.com/watch?v=cIOWOZ23Tjs

Combined with the robotic arm structure, the advanced soil sampling, analysis and mapping system of the N2 robot is mounted on the AgileX Robotics Scout Mini, an all-terrain and lightweight mobile chassis platform. The team changes the motion model of Scout Mini into the Omni-directional driven mode, which makes it more lightweight, flexible, and energy-effective to meet the needs of agriculture field applications.

The N2 Robot can drive flexibly on the farm autonomously, navigate between the planted rows on the uneven terrain of an agricultural field. Then, it implants the arm detection syetem into the soil for sampling detection and mapping, and precise positioning and judging the status of soil nutrient type and content.

With the "N2 Vision" Nitrogen mapping robot, the farmers are improving efficiency and increasing margins while reducing chemical use, lowering carbon emissions and improving worker safety. The robust N2 robotic platform allows fields to be quickly scanned no matter what the weather.

AgileX Robotics UGV Scout Mini is the better choice for an all-around, lightweight mobile research platform designed for cutting-edge scientific research. The SCOUT MINI supports access to the multi-level autonomous operating systems through the CAN bus protocol, it can mount more sensors system like monitoring and inspection system to extend research and development in more AI agricultural applications, such as the monitoring of soil temperature, moisture, root depth and health, and pest monitoring.

About Liangxiu Han robotics team of Manchester Metropolitan University

Liangxiu Han is a full Professor at School of Computing, Mathematics and Digital Technology,  Manchester Metropolitan University. Prof. Han is Co-Director of Cenre for Advanced Computational Science and MMU Crime and Well-Being Big Data Centre.

About AgileX Robotics

AgileX Robotics is committed to helping company enterprises, researchers, education to quickly deploy and develop the cutting-edge smart robotics research platform to improve productivity in all industries. AgileX Robotics has a full line of UGV chassis that can tackle all terrain environments like an indoor factory, city road, mud road, off-road, rivers, drainage channel, as well as easy to develop an autonomous system like auto-navigation, mapping systems upon the vehicle body. Moreover, AgileX chassis also provides a Gazebo model while supporting ROS packages for industry researchers and education to develop and verify in the physical world.

Read more about us and go build your robotics project with our AgileX professor tech team.

Read more cutting-edge industry cases based on the AgileX robotics chassis platform.

About Agile.XN Community and Developer Award

Welcome to join the Agile.XN Community and sharing your inspiring robotics research story or question with global robotics corporate and educational developers.

The Agile.XN Developer Award was established to encourage global robotics education and research communication. Just easy to share your autonomous robotics subject and project research，tell your robotics things or story in the Agile.XN community to get the developer prizes.

Award1–4 Category:

(1) ROS-based Multi-modal robot LIMO (x1)

(2) LIMO LITE (x1)

(3) A wide range of robotics sensors like EAI LiDAR, Nvidia Jetson nona (x2)

(4) $200 Discount of AgileX products (x2)

Eligibility Requirements

(1)Sign in the community and share you or your team’s robotics research or educational learning post;

(2)The post must be related to the AgileX robotics Platform;

(3)Your case study demos can be including the blog, code, vlog about:

-ROS Deep Learning and Gazebo Simulation

-Robotics Project Development

-Case Study Paper

-Team Working Vlog or Creative Vlog, etc;

(4)Once you win the award, you will agree to share it with more ROS world communities.

Others

Submission Time: Dec.27,2021 — Aug.1st, 2022

The first announcing time：Jan.28, 2022

We appreciate a lot for using the AgileX robotics platform to build your innovative robotics world for all industries or education. Any questions connect us marketing@agilex.ai, Yuki chen,the PR manager at AgileX Robotics.

On November 3, 2021, China, AgileX Robotics, a leading mobile robot chassis manufacturer and autonomous robotics solution provider, launched a tracked-type unmanned driving vehicle BUNKER Pro boasting super high off-road mobility and high-level lP67 protection. It is a better platform to expand robotics applications in challenging terrains like soft soils, steep grades, puddles, box culverts, for industry robotic engineers to quickly deploy the autonomous robot applications spanning inspections, exploration, EOD, video shooting, unmanned transportation and more.

The BUNKER Pro has a high-level ingress protection design for its overall structure, materials and external electrical components, making it meet the IP67 protection requirements against sand, dust and water.

BUNKER Pro waterproof testing-Customer case for mapping in box culvert

The integrated, advanced suspension systems and reinforced shock absorbers bring the BUNKER Pro a unique anti-collision design to tackle tough and damp terrains. The highly flexible BUNKER Pro has a very low centre of gravity, where equipped with the 1500W dual-motor drive to demonstrate super-strong performance in obstacle crossing with a height of 18 cm，and a climbing capability of 30 degrees.

Thanks to multi accessible hardware CAN interfaces and a strong load capacity of up to 120kg，along with preload ROS nodes, the BUNKER Pro can be fully equipped with various inspection and mapping equipment with inclusive sensor LiDAR, Camera, IMU, IPC, and more autonomous system. The BUNKER Pro supports advanced open-source architecture like ROS packages, Autoware and more. AgileX Robotics also provides rich open-source SDK and ROS_PACKAGE and comprehensive user manuals for developers.

Moreover, the powerful onboard battery supports up to 3 hours of continuous operation and can supply power to external devices to provide extra insurance for the robot to carry out different tasks.

AgileX Robotics has helped robotics developers in more than 26 countries deploy more than 1000 robotics applications in all industry. If your team need advice for deploying any robot on AgileX robot chassis, please contact the AgileX Robotics software and hardware team via https://www.agilex.ai/index/contact?lang=en-us. For the Bunker Pro now is available and for more information check the link via

https://www.agilexbot.com/products/bunker-pro

In July 2021, AgileX Robotics,  a leading provider of mobile robot chassis and customized unmanned driving solutions, has completed a round of funding on a cutting-edge commercial mobile robotics project.

The latest round of funding for AgileX Robotics’ 100 million RMB project was completed with the assistance of investors such as Sequoia Capital China, 5Y Capital, Vertex Ventures China, and the Hong Kong X Technology Fund. Each of the investors is amongst the top ventures institutions, having successfully invested and incubated successful Chinese unicorn companies such as DJI, Xiaomi, Alibaba, and JD.com.

"Thanks to the support and trust of all-star capitals, AgileX Robotics will accelerate the development of commercialized outdoor low-speed autonomous driving robotics for full industries to increase the working productivity, backed by AgileX Robotics high-performance mobile chassis matrix and AI software capability," says Jidong Wei, AgileX Robotics Founder and CEO, who also co-founded the DJI RoboMaster Robotics Competition when working for DJI.

“With powerful engineering and manufacture capabilities, AgileX Robotics multiple-load expansion interface standardized robot chassis has transmitted the extremely fragmented outdoor mobile robot requirements to an R&D standard, and through customized perception and autonomous driving solutions, the threshold and cost for mobile robot development and research are greatly reduced, greatly iteration speed is greatly accelerated,” said Peter Chen, Executive Director of 5Y Capital.

Pu Xun, Vice President of Sequoia Capital China, said: "Agilex Robotics is one of a kind in China that has achieved standardization of mobile robot chassis. We believe that Agilex Robotics will bring next-gen low-speed unmanned robotics commercial application products to more outdoor industries soon."

About AgileX Robotics

Founded in 2016, AgileX Robotics is a leading mobile robot chassis and customized unmanned driving solution provider with a vision to enable all industries to improve productivity and efficiency through robot technology. 

AgileX Robotics has developed a full line-up of industry-leading robotics chassis that fully meet all industries’ requirements, including general UGV, indoor AGV, tracked chassis and many of them obtained CE certification. Backed by mature software capabilities, AgileX Robotics also provides customized autonomous driving solutions, including parallel driving solutions, robot R&D kits, mobile manipulators, and autopilot kits to enable customers to accelerate the development and verification of their autonomous driving and robotics projects.

You can learn more about AgileX Robots, their products, solutions, and the latest news about the robot system developments, by visiting the website at https://www.AgileX.ai/index/about?lang=en-us.

END

Notes for the Editor: Please address any press inquiries to Yuki Chen, who can be reached by phone at 008618819436474 or by email at yuki.chen@agilex.ai.

August 16, 2021, AgileX Robotics，a leading robot chassis manufacturer and mobile robot system solution provider, unveils a powerful Mobile Manipulator. Backed by AgileX various adaptable robot chassis, it provides researchers a high-precision and easy-programmable software and hardware platform that easy to kickstart any industrial applications and R&D project in robotics.

The AgileX Mobile Manipulator is a easy configuration system that can achieve robust autonomous performance of accurate grasping operations, autonomous positioning and navigation, route planning and obstacle avoidance. It is compatibles with a 6-degree-of-freedom (DOF) AUBO-i5/xArm6 robotic arm that can meet various requirements for the grasping accuracy, high payload, working radius (819-886mm),close human-robot interaction safety. With a repeated positioning precision of ±0.03mm and a maximum grasping force of 160N, it can grasp up to 5 contact points of an object and auto-adapt to the shape of the target object.

Based on Agilex’s open source ROS package, the Manipulator’s equipped 16 lasers LiDAR and Intel Realsense depth camera allow it to obtain real-time high precision autonomous mapping, navigation, and positioning without additional configuration of LiDAR ROS nodes.

In addition to the official equipped sensors, it is a powerful programmable platform that can easily to be customized with other sensors according to the customer's project needs, and backed by the Manipulator’s built-in open source ROS operating system makes it really a one-stop unparalleled hardware and software platform for researcher.

The Mobile Manipulator can be friendly adaptable to AgileX Robotics’ full lines up of mobile chassis including the tracked-driven Bunker, all-round UGV Scout2.0, Ackerman-steering Hunter2.0, Omni-directional driven Ranger Mini. Hereby, the AgileX Mobile Manipulator not only can easily handle the indoor industry applications, but also broadly tackle though outdoor scenarios such as orchards and agriculture, mining and more, with effortless mobility through soft soils, vegetation, thick muds, and steep grades,and stairs.

About AgileX Robotics

Founded in 2016, AgileX Robotics is a leading mobile robot chassis and customized unmanned driving solution provider with a vision to enable all industries to improve productivity and efficiency through robot technology. We has developed a full line-up of industry-leading robotics chassis that fully meet all industries requirements, including general UGV, indoor AGV, tracked chassis and many of them obtained CE certification. Back by mature software capabilities, AgileX Robotics also provides customized autonomous driving solutions,including parallel driving solutions, robot R&D kits, mobile manipulator, and autopilot kit to enable customers to accelerate the development and verification of their autonomous driving and robotics projects.

With the in-depth cooperation with over a thousand enterprises including Alibaba, Huawei, Honda, the China Construction Third Engineering Bureau, and over fifty global top universities, AgileX Robotics has deployed more than one thousand applications in a range of industries purposes including autonomous driving, academic research, agriculture management, geographic surveying industries, and more.

Press release contact

Yuki Chen

PR Manager at AgileX Robotics

Yuki.chen@agilex.ai

We talked to JD Wei, CEO of AgileX Robotics, a world leading robot chassis manufacturer and mobile robot system solution provider and here is what he said about it.

First of all, how are you and your family doing in these COVID-19 times? 

JD Wei: The Covid-19 epidemic did cause some inconvenience to our life at the beginning of 2020, which happened during the Chinese New Year. So we had to stay at home and stop meeting with relatives and friends. However, the Chinese government has made very strict controls in controlling the epidemic. In my city, the number of infections has always remained within single digits, so my family and I have resumed daily life and work in less than two months. 

Tell us about you, your career, how you founded AgileX Robotics.

JD Wei: I was in charge of the Robomasters division at DJI, which is a global robotics competition for university students. I founded AgileX Robotics in 2016 after I left DJI. We started with parking AGV, but the business model was not fitting the actual market at that period, we gave up this project after two years. In 2018, we started to make outdoor ground mobile robots. At the end of 2018, we released the first ground-based mobile robot product. Unexpectedly, the market feedback and sales were doing good, so we began to deepen the field of outdoor mobile robots. At present, AgileX Robotics is very competitive and famous in this field, we are also in the process of internationalization, setting up business in different countries.

How does AgileX Robotics innovate? 

JD Wei: Outdoor mobile robot is a very new industry. It can be said that the application of autonomous driving has driven the development of outdoor mobile robots. Outdoor mobile robots can also be understood as low-speed automatic driving, application as logistics robots and inspection robots. Because it is an emerging industry, we need to always pay attention to the cutting-edge research of global autonomous driving and mobile robots, maintain cooperation with the world’s top research teams, and also need to be vigilant about market trends. Only in this way can we continue to innovate and inject vitality into the robotics field.

How the coronavirus pandemic affects your business and how are you coping?

In the first quarter, there were indeed some negative effects, because the entire country was shut down. However, because the epidemic has spawned the demand for disinfection robots, some of our R&D returned to the company to develop disinfection robots in February, and we started to sell them in March. 

This product contributed part of our sales in the first and second quarters. Because of the epidemic, the government encourages AI and robots to replace humans to enter complex work areas. Therefore, our outdoor mobile robots have also been widely used and brought us a 100% increasing in 2020 compared to 2019.

Did you have to make difficult choices and what are the lessons learned?

JD Wei: When the epidemic came, I thought I made the right decision, which is to let my employees stayed at home and worked from home. This was not a difficult decision, because my employee is the most important property of AgileX Robotics, and I don’t want my employees to take the risk.

What specific tools, software, and management skills are you using to navigate this crisis?

JD Wei: Because of working from home, we used a lot of online sharing software, such as remote video conferencing software, sharing documents, etc. In addition, offline exhibitions have been postponed due to the epidemic, so we have increased our online promotion efforts and increased promotion expenses on Google, Alibaba, and Baidu. Therefore, our marketing staff has a big improvement of SEO and SEM skills.

How do you see competitors? And how do you plan to stay in the game?

JD Wei: There were not many competitors in the market before 2020. However, there were some competitors that appeared in 2020 because of the strategy changing due to the epidemic impact, and the prospect of outdoor mobile robots. What we can do is to continue to innovate and continue to run at the forefront of the track, so as not to be surpassed by them. Before 2021, we were more focused on hardware. From this year onward, we have paid more attention to the development of the software and application scenarios.

Your final thoughts?

JD Wei: Through this epidemic, people will see more possibilities for robots.

Your website?

www.agilex.ai 

Recently, China Construction Third Engineering Bureau，the world's largest and 18th ranking outstanding vanguard of Chinese construction investment and construction group among the world's top 500 companies, cooperated with AgileX Robotics on the high-precision engineering road measurement robot equipped with AgileX Robotics all-around and compact robot chassis SCOUT 2.0. The AgileX Robotics SCOUT 2.0 has strong operation endurance and flexibly autonomous mobility, and can perfectly adapt to all-weather and help robotics achieve accurate road measurement in high-intensity and high-temperature environments.

With more than 1years development, the high-precision engineering road measurement robot, independently developed by the Engineering Technology Research Institute of China Construction Third Engineering Bureau, completed more than 20 kilometres of road measurement testing in the Wuhan Fourth Ring Road Project, which marks it qualified for road engineering applications to measure and collect road data with higher efficiency and less measurement error.

The high-precision mobile measuring robot for road engineering integrates functions such as automatic driving, automatic levelling, automatic station setting, and automatic measurement. It is applied in the road engineering industry for the first time. The robot's comprehensive measurement accuracy is within 2 millimetres, breaking through the centimetre-level accuracy limits in the global road engineering industry. The comprehensive measurement speed is 8 seconds/point, which rises over 10 times of working efficiency than traditional manual measurement methods.

Automatic driving

The position and navigation system based on multi-sensor fusion can realize high-precision location navigation and automatic driving(include obstacle avoidance and detouring)

Parking accuracy ≤ 0.3m

Course angle accuracy ≤ 0.3°

Automatic leveling

With automatic leveling equipment and control system, the one-key automatic leveling can be realized, and the leveling accuracy reaches 10’, which is far beyond the accuracy required by the total station.

Automatic station setting

Using spatial coordinate to quickly identify and aim at three 360°prism with know position, to realize the rapid and accurate station setting of the total station.

The station setting accuracy rate is 99%, station setting accuracy less than 0.2mm, station setting time is within 30 seconds.

Automatic measuring

Through the laser no contact measuring method, the robot can accurately find the point of the section for measurement, and directly generate the measuring data and forms that meet engineering quality control and check and acceptance requirements.

In addition, the robot can achieve 24-hour road measurement operation under all-weather including high-temperature, extreme cold, and low oxygen environments, to effectively solve the low-efficiency problem of the traditional manual point-by-point road measurement method, reducing labour costs by 80%.

Also, the robot can achieve 24 hours continuous measurement, which is useful for solving traditional road construction and reducing personnel input by 80%. It can also work in high temperature, extreme cold, and low oxygen environments.

Compared with the previous generation, the AgileX Robotics chassis SCOUT 2.0's size is reduced by 25% with strong performance, bringing the advantages of lower consumption and longer operating time. Equipped with a 200W brushless and a new cooling system and safety protection system, plus the four-wheel drive and differential rotation, it can ensure automoving freely indoors and outdoors and in complex environments and weathers.

SCOUT 2.0 has an independent main control system that can complete its own motion control, and can be customized with complex operation modes. The main control system provides a serial port/standard CAN bus as a connected interface for a series of external equipment including differential GPS, inertial navigation system, binocular camera, LiDAR, infrared/ultrasonic obstacle avoidance sensors, to help enterprises and scientific research institutions quickly executive secondary development in multiple robotics applications, such as indoor and outdoor inspections environmental detection, transportation, delivery and distribution.  

About AgileX Robotics

Founded in 2016, AgileX Robotics is a leading robot chassis manufacturer and mobile robot system solution provider in China. At present, AgileX Robotics has launched mobile robot chassis products such as general UGV, indoor small AGV and modified wire-controlled electric robot, which can serve in security inspection, industrial park, agricultural irrigation, logistics and transportation, detection and exploration, scientific research and education. Providing customers with stable performance, reliable quality and 7*12 h worry-free after-sales service.

About  China Construction Third Engineering Bureau Ltd.CO

China Construction Third Bureau, the world's largest and ranks the 18th outstanding vanguard of Chinese construction investment and construction group among the world's top 500 companies. It is China's first industry-wide housing construction general contracting super enterprise and ranks among the top 100 competitive enterprises in China's construction industry. 

The development of mobile robots pursues rapidity and scalability, usually involving conventional development issues such as autonomous navigation and following. The intelligent and autonomous functions of the robot can be realized quickly and free from human factors, so that intelligent robot applications can be truly realized.

Based on this, before realizing the whole functional requirements of the robot, development engineers often need to choose the key part of the mobile robot-the mobile robot platform. A suitable robot mobile platform with rapid secondary development capabilities is undoubtedly very helpful for reproducing these functions.

How fast and convenient can AgileX Robotics help customers complete secondary development? Let's take a look at how the team of the School of Control Science and Engineering of Zhejiang University (hereinafter referred to as the School of Control) used the AgileX mobile robot platform to quickly reproduce the conventional functions of the robot in a week.

Due to research and development needs, the School of Control of Zhejiang University purchased a robot chassis--HUNTER, as shown in Figure 1. The HUNTER used an Ackerman mechanical structure with a built-in underlying control program. It only needs to be connected to the control box developed by us through the CAN bus to realize the control of the chassis, which greatly improves the development efficiency.

Figure 1 HUNTER robot chassis (Click the picture to know more)

Based on the HUNTER robot, a multi-line lidar, IMU and camera were installed, and it took only a week for engineers to reproduce the conventional functions of the robot: following, 3D mapping, autonomous navigation, remote control, etc., the overall structure of the robot has shown in picture 2.

Figure 2 The overall structure of the robot

Function Instruction

01 Following

The following function was implemented on the quadruped robot previously and added a local navigation obstacle avoidance function. You can see that the robot's motion effect is still pretty good.

Video 1 Quadruped robot following function reappeared

For the robot vibrate, the team of the School of Control used IMU data to optimize the following program. Video 2 is about an anti shaking test.

Video 2 Anti shaking test

Video 3 is showing the following function on the chassis of the HUNTER. The following function is complete, and can also be used to extend the reverse parking function.

Video 3 HUNTER robot following function reappeared

02 3D mapping

The 3D mapping function has two modes, offline and online. Offline mode is to collect sensor data first, and then automatically building a 3D map through a mapping program in the background. Online mode is to build the map while collecting sensor data. The map building is shown in video 4.

Video 4 HUNTER robot mapping function reappeared

03 Local navigation

Video 5 is about the local navigation function. This kind of local navigation can make a planning path that does not use map data and prior data. Compared conventional static map planning, the advantage is that there is no need to build the map and can be used very conveniently in an unfamiliar environment.

Video 5 HUNTER robot local navigation function reappeared

It can be seen that through the reproduction functions above, the robot has the basis for engineering implementation. Choosing a suitable mobile robot platform with rapid secondary development capabilities not only greatly shortens the development cycle, but also expands the demonstration and practical application of robots, such as security inspection, logistics and transportation, disinfection and epidemic prevention. At the same time, the mobile robot platform is complete and efficient in supporting various systems and ROS community. If you want to know more about this customer case, you can contact the person through the following contact information!

Name: Zhichen Xu

Contract: 18868805838

Email: 807552127@qq.com

October 24, 2020, undergraduates in industrial design major from the School of System Design and Intelligent Manufacturing (SDIM) of Southern University of Science and Technology of China, as well as 2019 and 2020 graduate students, went to AgileX Robotics (Dongguan) Co., Ltd. under the leadership of Professor Xiaoping Hong to inspect and learn on the spot. AgileX Robotics co-founder, CTO, SDIM corporate mentor  Yixuan Jiang, as well as colleagues from the R&D department and business team gave the students an on-site technology demonstration and industry experience sharing.

Teacher and student of Design and Intelligent Manufacturing (SDIM) of Southern University of Science and Technology of China went to AgileX Robotics for technical exchanges

Yixuan Jiang led the teachers and students to visit the SCOUT 2.0 robot chassis production line, and though the actual operation of representative products such as TRACER(indoor AGV), HUNTER 2.0(front steering UGV), SCOUT MINI(all-terrain high-speed UGV), analysis the characteristics of each series of product, product design process, and the design points of the robot chassis.

Yixuan Jiang analyzed the characteristics of AgileX Robotics various series of products for teachers and students

Yixuan Jiang also explained to the students that the qualities to be a true "system architect" need to possess from the perspective of business, combining his own technical entrepreneurship. Yixuan Jiang said that the system architect is the instructor of the technical route and the soul of the development team. A veritable system architect not only has mastered textbook knowledge and strong skills, but also have the ability of systematic thinking from point to area, and the judgment of insight into the truth in complex environments. He/She should be a person with good at professional  training, rich practical experience, teamwork, communication skills, independent learning ability, and lifelong learning ability.

Yixuan Jiang share the technological entrepreneurial course with students

Through this demonstration, students combined with enterprise product examples, intuitively learned and understood user’s pain points and demand analysis, using design thinking to design products, product development processes, circuit design, Raspberry Pi embedded programmable, sensor applications, signal processing, computer vision, prototype development and iteration process, processing and assemble methods, those knowledge taught by the professors in the course . The students all expressed that they have gained different new experiences when reviewing their book knowledge and learning course,. This is exactly the goal of blended learning in SDIM new engineering teaching model.

SDIM Industrial Design major will be officially launched in the fall of 2020. The learning project of this semester is “Mobile Transportation Vehicle”. Students will learn and master design, machinery, materials, electronics, advanced manufacturing and other disciplines during the process.

Students actually operate and have an intimate understanding of the essentials of robot design

Students visit AgileX Robotics chassis production line on the spot

At the beginning of the establishment of the college, AgileX Robotics established a closed cooperative and interactive relationship with the college. Its founders Jidong Wei and Yixuan Jiang based on their personal experience, strongly agree with the new engineering teaching model. The two young entrepreneurs have been hired as the first batch of corporate mentors at the college's first "Da Vinci Challenge Camp-Winter Camp" in 2019, and they have been to the college many times to provide guidance and exchanges on new engineering teaching. In the future, the two parties will continue to conduct in-depth cooperation in personnel training and technology research and development.

Yixuan Jiang Introduced AgileX Robotics four-whee-drive front and rear Ackerman chassis to students

Recently, the agricultural delivery robots and patrol robots launched by the Tanjiawan Yunshang Agricultural pilot in Wuzhen, Zhejiang Province, China, are all equipped with the Agelix Robotics mobile robot chassis, which can easily move independently inside and outside the agricultural plantation. Combined with advanced clouds intelligent technologies such as IOT and AI, the robots can achieve 24h all-weather agricultural plant growth data inspection and collection, to provide basic data for agricultural precision planting, automatic transportation and intelligent management.

The “Fengqi Model” Cloudfarm is located in Tanjiawan remain relic, Wuzhen, Zhejiang Province and is created by Fengqi tea house, national makerspace recognized by the Ministry of Science and Technology and Wuzhan government. It covers about an area of 700 acres which means requir many amount of labor working.

• Agricultural intelligent delivery robot for efficient picking and delivery

   

Among the clusters of tomatoes, there is a black robot carrying a blue frame and moving in the field. "This is our "Panda" shuttle robot, which can help picking and delivery and help save a lot of labor." The staff introduced.

"Panda" shuttle robot uses the indoor high-speed mobile chassis TRACER of AgileX Robotics, the upper end equipped with functional modules such as Lidar, remote control ground station, jack-up device. It can replace manual realize the transportation of objects, autonomous obstacle avoidance, autonomous navigation, automatic driving in the cloud shed.

TRACER is a two-wheel differential AGV for indoor transportation. Its compact and flexible structure enables it to freely shuttle in various indoor environments. Its has a super high delivering efficiency with high-duty playload capacity and strong power.

• Farm patrol robot for intelligent agricultural data collection

Outside the greenhouse, the fram patrol robot is working beside the golden paddy field. It can collect the data about virus, insect pest, crops and organism growth circle and transmit back the high-definition video by 5G network. The staff can make precise control operations through the visual monitoring and management interface to realize intelligent ecological planting, intelligent insect prevention, and intelligent weeding.

SCOUT 2.0 can be customized with complex operation modes. The main control system provides a serial port/standard CAN bus as a connected interface for a series of external equipment including differential GPS, inertial navigation system, binocular camera, LiDAR, infrared/ultrasonic obstacle avoidance sensors, to help farm research institutions quickly executive secondary development in multiple robotics applications, such as indoor and outdoor inspections environmental detection, fire fighting, disinfection, Geological modelling and surveying.

In addition to shuttle robot and inspect robot, there are two aerial track robots to identify diseased plant and remote diagnose on the top of agriculture proving ground, transmit to the cloud brain in real-time. The outdoor field also equipped with 5G network-based plant protection unmanned aerial vehicle, realize accurate, efficient and real-time disinfection with time delay less than 20ms.

About AgileX Robotics

Founded in 2016, AgileX Robotics is a leading robot chassis manufacturer and mobile robot system solution provider in China. At present, AgileX Robotics has launched mobile robot chassis products such as general UGV, indoor small AGV and modified wire-controlled electric robot, which can serve in security inspection, industrial park, agricultural irrigation, logistics and transportation, detection and exploration, scientific research and education. Providing customers with stable performance, reliable quality and 7*12 h worry-free after-sales service. https://www.agilex.ai/?lang=en-us

Since the activity "Building an AgileX Ecosystem" has launched on June 24, AgileX Robotics has received many cases and open source demos from many users at home and abroad. Previously we shared three simulation videos contributed by Dr. Xianyu Qi from the Bionic Mobile Robot Laboratory of the School of Mechanical Engineering and Automation, BUAA. Dr. Qi used the open-source simulation environment of GAZEBO provided by the AgileX SCOUT 2.0 robot to realize Active SLAM, Pedestrian following and Multi-target navigation. Click the link to see the videos.

What I want to share today is the ROS training course conducted by our Singapore agents at the National University of Singapore. The training course mainly around how to develop robot applications, how to understand the development tools used by robots in different environments, and how to send and receive data from robots.

Robot Operating System(ROS), as an open-source software system, is equivalent to a kind of “Robot Frameworks” in some respects to build a universal robot software platform that can integrate different research results, realize algorithm release and code reuse.

ROS is a system with a steep learning curve, covers a wide range of knowledge. While conducting theoretical knowledge training, there is also a robot(SCOUT 2.0) on the scene, which is convenient for students to follow the actual operation of the lecturer, quickly master main technologies, deeply understand ROS development methods, and how to apply to the actual scenarios.

Students are learning to disassemble commercial robots

students are discussing

Through this training activity, the trainees said that they had a preliminary understanding of the ROS system and the SCOUT 2.0 robot’s actual operation is helpful for the learning of the ROS system, and follow-up learning. Here, we would like to express our gratitude to our Singapore agents for their training at the National University of Singapore and hope that they will open more training courses in local institutions. At the same time, to encourage more users to participate in the AgileX Ecosystem, we will provide more support and assistance.

Since the activity “Building an AgileX Ecosystem” started on June 24, AgileX Robotics has received many cases and open source demos from users at home and aboard in only three months, not only including R&D personnel from high-tech enterprises, but also many professors and students from top universities around the world.

Today we are going to share three simulation videos contributed by Dr. Xianyu Qi from the Bionic Mobile Robot Laboratory of the School of Mechanical Engineering and Automation BUAA, Hope this interesting sharing can bring more inspiration to our developers. So, let’s get started.

01 Active SLAM

Based on the open-source simulation environment of Gazebo of AgileX SCOUT 2.0 robot, using Active SLAM (as opposed to Passive SLAM) method based on boundary exploration, the robot can quickly realize autonomous navigation in an unfamiliar environment.

02 Follow Me

Based on the open-source simulation environment of Gazebo of AgileX robot and using 2D lidar sensors, Kalman filter and global data association for pedestrian detection and tracking, the robot can realize autonomous pedestrians following.  

03 Multi-targets Navigation

Based on the open-source simulation environment of Gazebo of AgileX SCOUT robot, using the Move Base navigation package and specifying multiple target points through the program, the robot can realize multi-target points reciprocating autonomous movement.  

This year, we have a hard time and experience a special new year. Cornavirous epidemic is raging in Hong Kong. In the past few months, the number of local diagnoses in Hong Kong has remained above the level of about 70 cases per day, and has dropped to a single number recently. The situation is worrying, we expect the fourth epidemic seasons. But at the same time, the residents and employees of the company are exposed to a high-risk environment for a long time. Once they are infected, the consequences will be disastrous. Therefore, we believe that prevention and control must be done thoroughly to keep risks to a minimum.

In July 2020, our Bonbon Robotics Company introduced a Disinfection epidemic prevention robot – Thunder. It was specially modified by the AgileX Robotics Company because of the Covid-19 epidemic. The Hunter robot chassis became a THUNDER disinfection and anti-epidemic mechanical vehicle. This is a wheeled-type mechanical vehicle that can remotely control by human. Two buckets of 15-liter disinfection solution can be installed on the left and right sides, and the solution can be sprayed from 2 meters to 8 meters distance. The vehicle uses a diaphragm pump for the food and pharmaceutical industries. It has anti-corrosion properties and can be added to disinfection solutions such as bleach or other disinfection solvent. The equipped smart disinfection spraying kit is equipped with a one-key activation of a large capacity to achieve 360-degree freedom and all-round spraying.

In the busy city like Hong Kong, we rarely see such kind of robots roaming the streets and alleys. This time we came to the footbridge next to the Wan Chai Government Revenue Building. Benefits from barrier-free facilities that the government has built many years ago, it was originally convenient It is designed for people with disabilities. Fortunately, the size of a mechanical vehicle - Thunder is just enough to enter the elevator door for transport.

In Wan Chai Harbour Road Park, the THUNDER machine car climbed up the slope for disinfection and epidemic prevention work. Even if under the scorching sunlight, our staff still can remotely control the machine car for spray disinfection work. The green indicator light above is prompting us that the work is in progress, otherwise it will prompt with a red light.

The THUNDER anti-epidemic mechanical vehicle can walk by itself, and the remote-control operation is simple and easy to use, which is suitable for different people to control and operate. Passers-by are surprised and interesting at the outstanding appearance of robots. It seems that the public is just beginning to realize the behavior toward robots.

We also bring the THUNDER disinfection robot to the church community center of Anda Village near Kwun Tong for rapid air space sanitization. The THUNDER disinfection robot can spray 1.5 liters of solution per minute, and the size of the water droplets is about 100 microns. Therefore, we must spray carefully when we perform disinfection work. To audio equipment, electronic products and so on. However, the THUNDER disinfection robot cannot enter certain indoor locations due to its large size. We also disinfect the toilets. 

Considering the safety of spraying solution, we must choose a suitable disinfection solution. The disinfectant spray (chlorine dioxide) used by the robot has also been tested for many years, after a tremendous of research and laboratory test, and has been certified by SGS, WHO (World Health Organization), FDA (US Food and Drug Administration), EPA (US Environmental Protection Agency). Approved Odorless A1 Food Grade Disinfectant is completely harmless to the human body even in contact with customers and employees. The staff who controls the robots can also be controlled remotely from more than 100 meters with a camera, which greatly reduces the risk of infection of the cleaning staff and saves our manpower. It can also carry up to 30 liters of solution for hotel area, lift lobby, corridors, fitness room and other facilities are thoroughly disinfected.

As for the stairs concern, we equipped the THUNDER with two aluminum ladders, which can facilitate the loading and unloading of the truck. The THUNDER has an anti-falling mechanism. Even if you let go on an uphill halfway, the mechanical vehicle will stay in position. It cannot move back and forth to avoid accidents.

We also tried to put the Thunder disinfection robot in a medium-sized indoor office for disinfection work. From this, we found that some areas need to be improved, as well as the places to pay attention to when disinfecting the indoor environment.

In co-working space office in Kwun Tong, we perform a quick space disinfection before the meeting to provide disinfection and safety protection for the visitors. At present, we use a 50-micron spray head, which is smaller than the previous 100-micron spray head and can perform disinfection work in large spaces.

Thunder disinfection epidemic prevention robot also perform well outdoors. Some streets and alleys in Hong Kong are relatively unclean and smelly. The use of disinfection epidemic prevention robot can used for disinfection of the back alleys and can also deodorize and reduce the work of cleaning workers. Quantities, make nasty work easier.

 Thunder disinfection epidemic prevention robot has been visited many places in Hong Kong. Although the Thunder is large in size, it is very quiet when it travel to different area. It is equipped with an aluminum ladder to facilitate trucks and can be reached in Hong Kong Island, Kowloon, and New Territories. Nowadays, society’s awareness of robots continues to increase. As the 5G technology era is coming, we believe that robot applications will bring many good application cases. At present, the public still needs to know more about these new technologies and take a new step towards the advent of smart cities in Hong Kong.

When the fruit is ripe, a large number of fruit farmers will participate in the transportation and packing of the fruit in the orchard. This is a job that requires a lot of labor. However, in recent years, the rural labor force has shown a trend of aging and high costs, which has caused an increase in transportation costs and a decline in efficiency to a certain extent. In order to alleviate the impact of this situation on fruit production, the Smart Agriculture Innovation Team of the Chinese Academy of Agricultural Sciences has developed an intelligent follow-me delivery robot, which is called the Little Sowing, as shown in Figure 1. The Little Sowing adopts the SCOUT 2.0 all-round robot chassis of Agilex Robotics. And the upper equipped with functional modules such as LiDAR, remote control ground station, follow me module, voice recognition and playback system, etc. Using autonomous driving, obstacle avoidance, stable following, human and machine voice interaction and other technologies, according to the manually set transportation route, cycling driving, to help the farmer achieve repeated fast, time-saving and low cost transportation operations in the specified region to reduce the farmer transportation burden ensures the transportation efficiency at the collection site.

Little Sowing function description

First of all, the Little Sowing has the functions of automatic driving and obstacle avoidance in the orchard. It’s equipped with a remote control ground station, through which it can realize the functions of drawing transportation trajectory, issuing tasks, position and log management. After receiving the task, the Little Sowing can autonomously perform transportation tasks between the rows of the orchard. At the same time, it can also stop loading and unloading and continue operations at any time according to voice instructions. The ground station is simple and convenient to operate, easy to use, and can also be connected to all robots or other loT devices in the orchard.

After receiving the task from ground station, Little Sowing began to follow the route obediently to perform the task. It drove along the track autonomously, carefully and steadily until the task is completed.

Secondly, Little Sowing also has an autonomous follow function. The follow function can enable it to follow the farmer to deliver agricultural materials, fruit and vegetables in real time, thereby reducing the farmer’s labor intensity. After the follow function is turned on, the follow module on Little Sowing will monitor the distance and angle of the label in the hands of the farmer in real time, and then convert it into its own motion control information to achieve follow. The follow module is shown in Figure 2.

In addition, the Little Sowing also has the function of voice interaction. It can introduce itself, drive by voice instructions, introduce the weather, report the farm information, etc. all of this support customization. The real-time voice interaction between the Little Sowing and the farmer not only enhances the practicality and operability of the robot, but also makes the farmer feel fun in the process of using.

About SCOUT 2.0

SCOUT 2.0 is based on a new consumption design and is specially designed for industry applications. The body is compact and small, low consumption, equipped with a new cooling system and safety protection system, indoor and outdoor can move freely, four-wheel drive, differential rotation, abundant power, easy to control complex application environments, standard protocol and CAN bus support rapid secondary development, presenting the best mobile platform solution for you.

At just over three feet (.98 m) tall, the four-wheeled robot may not look like much as it rolls through the office parks of urban Singapore—but don’t let that fool you. Equipped with commercial-grade disinfectant and a double-barreled spray gun, the small robot from Weston Robot is diligently disinfecting any high-touch surface in its path. Whether it is being remotely controlled by personnel offsite or autonomously cruising empty hotel rooms with an infection-zapping UV light, this robot is diligently going where humans no longer should.

Weston Robot designed three new disinfecting and temperature-monitoring robot prototypes. Each took just 10 days.

The disinfecting robot designed by Weston Robot and AgileX. Image credit: AgileX

For decades, humans have been dreaming up ways that robots could make their lives easier by taking over high-risk or tedious jobs, but with these disinfecting robots, the dream is more of a necessity.

The global community has been hit hard and fast by COVID-19, an infectious disease caused by a new coronavirus. The illness was classified by the World Health Organization as a pandemic in March 2020, and the virus had infected over 5 million people worldwide as of June 2020. The virus is spread through respiratory droplets, either directly entering the body through touching our noses and mouths or inhaled from residual particles left in the air at distances closer than six feet (1.8 m).

As scientists race to understand and create a vaccine for this virus, prevention has focused on early detection, contact tracing, and diligent disinfecting of surfaces and hands while extremely limiting human contact.

For Dr. Yanliang Zhang, managing director and chief scientist of Weston Robot, a Singapore-based robot developer and supplier, using robots to help meet the disinfecting standards was an obvious choice. Weston Robot designed three new disinfecting and temperature-monitoring robot prototypes. Each took just 10 days.

Right Place at the (Wrong) Time

“The first day that COVID-19 was announced to be infectious in humans, my team came together and said ‘We design robots, is there something we can do to mitigate the spread of the virus?’ ” says Zhang. “Obviously, we could not totally design a new robot model at that time. Instead, we wanted to take advantage of what we have already: mobile robots.”

Prior to the outbreak of COVID-19, Weston Robot already had a small arsenal of mobile robots, including compact surveillance robots with wheels designed for uneven terrain such as outdoor parks. Instead of starting from the ground up designing new robots to combat the virus, the team retrofitted these existing robots with tools for disinfecting.

“We asked ourselves, ‘Can we add something to these robots, for example, a spray gun to spray chemical disinfectant?’ ” says Zhang.

Although Weston Robot designs other types of robotic solutions, including autonomous cars and exoskeletons, Zhang says that the team quickly decided that small robots like these would be the best solution. Not only were they easier to retrofit, but they could also help lessen the burden on healthcare and other workers by providing easy-to-control, safe options for disinfection and delivery of essential goods.

“When this happened, we saw a lot of demand for robots for disinfection and delivering supplies to combat the spread of the virus,” says Zhang.

“We saw a lot of demand for robots for disinfection and delivering supplies to combat the spread of the virus.”

 -Dr. Yanliang Zhang, managing director and chief scientist of Weston Robot

This robot designed by Weston Robot uses a UV light to disinfect building interiors. Image credit: Weston Robot

Unusual Solutions for Unusual Times

But, even with prefabricated mobile robot bodies, creating new prototypes in a matter of days was no small feat. To get the job done, the team relied on quick modeling and simulation using Model-Based Design with MATLAB® and Simulink®. This approach enabled the team to quickly identify and correct emerging problems in their new designs.

“We didn’t want to start from scratch. We searched for different components in the market and integrated and tested the components.”

- Dr. Yanliang Zhang, managing director and chief scientist of Weston Robot

The first prototype of the disinfecting robot. Image credit: Weston Robot

Another way they sped up their prototyping process was by incorporating ready-made equipment available on the market into their preexisting robots. Zhang says that the initial prototype of their disinfectant-spraying robot used a spray gun designed for washing cars.

“We didn’t want to start from scratch,” says Zhang. “We searched for different components in the market and integrated and tested the components.”

In addition to physically building these robot prototypes, the team also faced a few software challenges—namely, how to effectively communicate with and control these robots to ensure accuracy and safety in the environments where they are deployed.

To overcome these design obstacles, the team decided to use 4G to remotely control the robots in less predictable environments, like the outdoor robots that roam through parks. This approach enables them to better control where disinfectants are sprayed and ensure the safety of the robot as well as any human passersby.

In indoor environments, instead of the robot spraying potentially dangerous disinfectants, the team opted for UV lights and preprogrammed autonomous movement maps—similar to how a robot vacuum would move through a map of your living room to clean.

After rigorous prototyping and trial-and-error testing of each idea, the team emerged with three prototypes: an outdoor robot designed to spray chemical disinfectant in environments such as industrial parks and shopping centers, an indoor robot equipped with a UV light column designed to non-invasively disinfect environments such as dorm rooms and hotels, and a hand sanitizer and tissue delivery robot equipped with an infrared camera to take people’s temperatures.

Each robot weighs about 100 pounds (45 kg) and is no taller than 3.3 feet (1 m). The outdoor robot can carry 8 gallons (30 L) of disinfectant and cover nearly 4 acres (14,800 sq m) in a single charge, and the indoor disinfecting robot can cover 10 to 15 rooms with the same amount of charge.

Taking these robots from prototypes to a finished product took another month and was accomplished with the help of Agile X, a mobile robot manufacturing company in China.

“In the final product, we do redevelop everything,” say Zhang. “Agile X is very good at manufacturing hardware, so they can quickly realize your idea and do product assembly testing there. During lockdown it is very difficult for us to get components from the supply chain, but they are very good at making the whole robot given our design specifications.”

The disinfecting robot can be controlled remotely. Cameras on the robot provide the operator with view of the robot’s surroundings. Image credit: Weston Robot

Preparing for the New Normal

These robots have already begun to roll out in Italy and China, with more expected to be delivered around the world in June. As Weston Robot continues to refine their designs, Zhang says that the team has already begun to learn lessons about what kind of robots the public wants to interact with—and what kind they don’t.

The temperature-monitoring robot wasn’t originally designed to deliver hand sanitizer and tissues, but Zhang says that people found the robot a little too intrusive when it was just a camera.

“People might feel like they’re being monitored all the time,” says Zhang. “So, my team said, ‘Let’s try to make this robot a bit soft.’ For example, we now provide some hand sanitizer and tissue paper with the robot so that people proactively come near the camera and robot. We made the robots more friendly and approachable.”

As these robots continue to be used both through the brunt of the pandemic and afterwards to prevent additional waves of infection, the team wants to ensure that they’re as intuitive to use and easy to interact with as possible. To that end, the team has enabled workers to control them via a computer tablet after a few simple lessons.

Weston Robot is pursuing certifications that will ensure the safety of batteries and other internal components and help make these robots “street legal” in more places across the world. But, those certifications are currently on hold as a result of global lockdowns.

This robot monitors temperatures to detect possible infection. It also offers hand sanitizer and tissues. Image credit: Weston Robot

To ensure that the robots are as intuitive to use and easy to interact with as possible, the team has enabled workers to control them via a computer tablet after a few simple lessons.

“We have to make sure it will pass all the certifications,” says Zhang. “But right now, even the certification center is locked down.”

These certifications are not required during the robots’ trial phase but will be at the top of Weston Robot’s list when countries begin to reopen. Meanwhile, these robots will start their work making sure the “new normal” is as safe as possible.

A Chinese tech company has deployed an army of 5G-controlled robots to help battle the coronavirusoutbreak in China – and their human operators don't even have to be in the same city.

The flu-like disease has so far infected 80,430 people, with more than 3,000 people dying since the disease broke out in Wuhan, central China.

AgileX Robotics, a Guangdong-based tech company, has now invented a mini-rover – equipped with cameras and rotating nozzles to spray disinfectant on the streets – as an innovative approach to reduce cross infections among the public.

The anti-bacterial robots can be controlled by phones more than 2,000 kilometres away using the country's huge 5G network – the largest of its kind.

Brandy, a marketing manager at AgileX Robotics who did not want to provide her last name, told Daily Star Online: "As long as there's 5G network covered, I can control the robot in Beijing while I am here in Shenzhen (roughly 1,367 miles away).

"This is a step of achievement to show how boundless it can be."

So far, the robots have been deployed in Beijing, Shanghai and Shenzhen.

In a video of the robot in action in Shenzhen, the machine is seen operating on a train carriage while spraying anti-bacterial disinfectants onto the seats.

It can also be used on pavements, schools, commercial or resident buildings.

The robot can replace the risk of sending cleaners to coronavirus-hit towns to carry out the sanitation works.

"You can just connect the robot to your 5G-equipped phone or tablet and control it through our designated app," Brandy told this site.

"Alternatively you can control it using a joystick and it works like how you manoeuver a remote-controlled toy car.

"It can reach anywhere until the battery dies, which is about two hours."

She said the robot can work in areas without 5G network, but, the distance would then be restricted to a range of 300 metres.

In China, authorities have deployed spray trucks and installed "disinfectant tunnels" across the cities after thousands of people were tested positive for the deadly virus.

Brandy explained: "Coronavirus can be easily spread through body contact, and the help of autonomous technology can reduce the risk to a minimal level.

"The operator can view the surrounding through the camera on the robot through 5G connection before carrying out the deep clean.

"The whole process is touch-free."

The technology is aimed to replace manual labour to reduce the risk of contracting coronavirus

Daily Star Online can reveal that each robot can carry a 30litre disinfectant container with double nozzles that can spray at a range of 15,000 square-metre.

There are currently 30 robots deployed in the country to carry out the disinfectant work.

But Brandy said they are looking to improve some of the specs in bids to strengthen sanitation and hygiene.

"The nozzles we have at the moment are manually operated, we are hoping to automate it to further reduce the risk of touching infected surface," he explained.

"We will also add 'gimbal', a pivoted support, on the robot for easier control of the nozzles' direction."

COVID-19, the disease caused by the novel coronavirus, has created a global lockdown, the likes of which hasn’t been seen in over a century. Responses in hot spots have varied from recommended social distancing to mandated self-isolation. While the public does its best to heed these directives, critical workers such as first responders, nurses and doctors, transit employees, grocery store employees and others remain at their jobs.

One group being relied heavily on during this time of crisis is the set of people tasked with disinfecting contaminated areas, putting them at great potential risk. To reduce human contact in these affected indoor and outdoor areas, disinfecting robots are being developed.

“Instead of manual disinfection, which requires workforce mobilization and increases exposure risk to cleaning personnel, autonomous or remote-controlled disinfection robots could lead to cost-effective, fast, and effective disinfection,” states an editorial recently published in Science Robotics.

Ten days to create a new robot

The New York Times published a timeline of COVID that starts on December 31, 2019. In less than four months since the first reported cases, more than 2.5 million cases have been confirmed. This pandemic hit fast and technology companies are working hard to keep pace. One in particular, Weston Robot, a developer and supplier of robotics, reacted quickly to help battle the virus. The team developed a prototype in ten days, leading the way for other robotic developers to do the same.

“Back to January, when COVID-19 was first proven to be infectious among humans, the team spent less than 10 days to come up with the first sanitizing robot prototype and to demonstrate the feasibility of using robots. Many robotics companies followed,” says Dr. Zhang Yanliang, managing director and chief scientist at Weston Robot, Singapore.

Two young entrepreneurs based in the Songshan Lake Xbot Park have shown that with a lot of guts, and the ability to humbly shift direction quickly, success is possible in this promising industry. 

Shifting the positioning

During its initial development in 2016-2018, the company put all its efforts into parking robots. It became what’s known in the tech industry as a “learning experience.” Although Wei and Jiang believed their initial idea was “life-changing”, the company had a hard time bringing the product to market. “August 28th, 2018, was the third anniversary of our company, but it was, in fact, the darkest moment for us,” says Wei, thinking back. “I knew we would run out of money in a few months and the investors were not responding well.” As CEO, he had to make a tough decision and accept the cruel reality: the product did not fit the market. “I wasn’t looking for new investment because I couldn’t persuade myself to take a product that I didn’t believe in to investors.” Looking back, Jiang believes he can see what went wrong more clearly. One reason was the limited environment. “Although our product sounded awesome, there were very few fully-automated smart parking lots that allowed for the adoption of our parking robot.” Another was the cost of cultivating user habits and developing a well-rounded support system. “This was not affordable for a start-up. It had to be done by at least a medium-sized enterprise.” It was like “having a small horse drag a heavy truck”, Wei concluded, recognizing that parking robots were not really what they were passionate about. “If you take a look at those successful tech companies today, the founders are usually their own first clients. For instance, the founder of DJI, Wang Tao, had been a huge fan of model airplanes since he was a student. But the parking robot was a whole new field for us.” Wei and Jiang shifted quickly to UGVs. Their thinking was based on three reasons, Jiang says. “Firstly, it’s something that sounds easy to make but is not so easy. Secondly, the product can be implemented in various industrial environments as a basic facility.” Jiang considered their “small unmanned vehicle” had to be “like rice” i.e. something essential that can go with a wide variety of “dishes”. Thirdly, production costs were much lower. AgileX’s UGVs are currently sold at tens of thousands of yuan, the most expensive being just over RMB100,000, whereas parking robots had carried a price tag of RMB700,000. “Lower prices gave us access to a wider market and we could get it to maturity in a shorter time.” The forerunner of Scout was a semi-finished product that Wei put together with an intern. “I just threw this product into a small circle of my acquaintances, and surprisingly, someone sent me a prepayment on the same day. More orders followed.” With RMB60,000 from the first version, Wei started to focus on an upgrade. “I had to let others handle the production. All I did during that time was try to sell like crazy.” The semi-finished “small vehicle” has now been upgraded to AgileX’s star product, Scout, which is a full-featured UGV. With an automatic chassis, wheels and general purpose interfaces, it is used for industrial purposes like spraying agricultural pesticides, inspection, and industrial logistics.

Survive first before getting innovative

Sales, needless to say, were welcome. The company can now afford to move on to mass production and is already selling 50-60 units of different models on average each month. Although AgileX mainly sells standardized products, the company also designs customized products for specific implementation environments, with clients such as consumer-goods behemoth P&G. “There are three levels of UGV business,” says Wei. “The first is a high-end remote-controlled vehicle. The second is to add GPS and all the unmanned features. And the third is the vehicle in a specific industrial environment.” Initially, Scout was intended for as wide a range of customers as possible. Features were carefully selected only to serve essential purposes that would be most commonly needed. Now, Wei points out, the current product aims to match the basic needs of both industrial and non-industrial clients. Instead of putting their eggs in different baskets, Wei emphasized the importance of focusing on only one product and to make it the best, which is essential for attracting customers to ensure the survival of a tech start-up. “I saw some companies who invest in two or three different products targeting different markets. But I only focused on one and made it the best in my targeted customer group.” Wei has a strong development team to give him confidence. All have previously participated in the Asia-Pacific Robot Contest, also known as ABU ROBOCON. Jiang was a member of a champion team when he was a college student. Both members of the post-1990s generation, Jiang believes the keyword of his generation is “change”, yet he also believes young people should learn the spirit of survival from those born in the 70s or 80s. “Wei always tells me stories about many entrepreneurs in Shenzhen who had to sleep under bridges when they were building their businesses from scratch. Many small enterprises established by the post-70s and post-80s generation are very resilient. They can survive for more than ten years with just a tiny production plant for a single product. This is much better than many start-ups nowadays.” Start-ups should aim to survive the first three years and earn “the first bucket of gold”; thereafter they can expand, Jiang says. Wei also stresses that survival capability is essential for a company to attract investment. “Nowadays, many investors who used to invest in ‘concept stocks’ feel insecure. They came to us because we can earn money and do things step by step. Investors are becoming more discrete and choosy these days.” The cooperation and strong bond of the two founders has also been essential for the company’s success. Wei says Jiang is able to “understand what I mean before I finish the sentence”, and Jiang considers his working relationship with Wei as like the chief petty officer and captain on a ship: “He’s the one that makes the decisions and takes the responsibility. My duty is to give him all the advice I have and execute his orders.”

What’s next

 Of course, there is no time to rest on success. The company is now moving on to the next stage, which is to develop more customized products for niche markets, as they have sufficient money for R&D. “The industrial implementation is the real deal. We get to decide whether the industrial environment is big or worthy enough to develop a specific model.” Like any start-up that expands after its initial stage, AgileX is transitioning from a highly self-motivated but rough management style to being more systematic. Jiang admits that it is a challenge. “We can’t just copy the way of the large companies, but we can’t stay the same,” he says. “As we expand, management will become more like neurosurgery – you have to give concrete instructions to the right person at the right spot.” These changes are necessary, however, to ramp up the business. Wei is setting up a new office at the University Town of Shenzhen focused on sales and marketing, to be closer to the market. “In Songshan Lake, it takes a lot of effort [to get out here] for those who truly want to do business with you.” Although currently sales are mainly being boosted by “word-of-mouth”, Wei intends to gain more insights on the market in Shenzhen. “It is unavoidable for a company to make mistakes during development, so we need all kinds of information. I’m here to learn in Shenzhen.” As the mainland city which is most closely connected with Hong Kong and Macau, Wei is also expecting more communication and cooperation with the two SARs under the development of the Greater Bay Area plan. “I think it’s definitely a trend for the cities in the Pearl River Delta to cooperate with Hong Kong and Macau,” he says. Wei believes the mainland and Hong Kong should each do the best in their own fields. “Hong Kong is very strong in finance as well as high-end talents, while the Pearl River Delta is good at the industrial implementation and is supply chain.” Good insight. And good advice for aspiring entrepreneurs on both sides of the border.