Agentic AI Robotics Systems

Traditional robots are deterministic state machines. The control loop is explicit: sense a value, apply a control law, drive an actuator. This works remarkably well for narrow, repetitive tasks in structured environments — a pick-and-place arm on an automotive line can operate at millimetre precision for decades, as long as the part is always in the same location, under the same light, on the same fixture. Remove any one of those constraints and the system breaks. The brittleness is not a bug in the implementation; it is a structural property of the architecture.

The missing layer was always semantic understanding — the capacity to interpret a scene rather than just measure it. Classical CV could detect edges, segment by colour, match templates. What it could not do was generalise. A model trained to detect oranges on a conveyor had no useful representations for apples. Each new object required a new labelled dataset, a new model, a new integration test. The combinatorial explosion of real-world diversity made truly general robot perception seem intractable.

Foundation models trained on internet-scale data carry something qualitatively different: embedded world models. A vision-language model that has processed billions of image-caption pairs has developed internal representations of object categories, materials, spatial relationships, and physical affordances — not through explicit programming, but through statistical regularities in the data. The consequence for robotics is zero-shot generalisation: a model trained on grasping tasks can now interpret "hand me the blue one" and identify the correct object without a labelled dataset for that specific item, because the semantic category and the colour attribute are already factored into the model's representational space.

This changes the design of every layer in the perception-action pipeline. Understanding what changed, and where the hard problems still live, is the starting point for building systems that actually work.

At the sensing layer, not much has changed mechanically — cameras, lidar, IMUs, and encoders are the same transducers they were twenty years ago. What has changed is the expectation placed on them. A classical pipeline needed carefully calibrated, high-SNR input because every subsequent step was hand-coded and non-robust. Modern neural pipelines are far more tolerant of sensor noise and illumination variation, but they impose different failure modes: distribution shift is silent — the model continues to produce confident outputs as the input drifts outside the training domain.

At the feature extraction layer, vision transformers (ViTs) have largely displaced purely convolutional backbones for tasks requiring global context. The DINOv2 backbone from Meta AI, for instance, produces semantically rich patch embeddings without any task-specific supervision — a single pretrained backbone can serve detection, segmentation, depth estimation, and correspondence tasks through lightweight task heads. This dramatically reduces the per-task training burden.

At the planning layer, large language models have introduced a genuinely new capability: the ability to decompose natural language task descriptions into executable action sequences, reason about preconditions and failure modes, and adapt plans when execution deviates from expectation. This is the layer that makes "pick up the red cup and put it next to the sink" tractable without a custom state machine for every possible object-location pair. The hard problem is grounding — mapping abstract language-level plan steps to concrete robot primitives in a way that handles spatial ambiguity and physical constraint.

FarmBot Genesis is the best open-source starting point for autonomous garden weeding. The CNC gantry architecture is the right choice for this task — not because it's simpler than an arm, but because it is fundamentally better suited. A gantry provides sub-millimetre X/Y positioning across the entire working volume, deterministic path planning (no inverse kinematics required), and straightforward tool-head swapping. An arm requires IK solutions, has workspace singularities, and loses positional accuracy near the edges of its reach envelope. For a flat raised bed, the gantry wins on every practical dimension.

Here is the engineering reality of building a functional weed detection and removal system on top of this platform.