Behind every crisp bite of an apple, there is a long season of pruning, tending, and constant vigilance against one of the quietest threats to orchards: weeds. But researchers at Penn State University believe AI may soon offer a precise new way to manage them.
Blanket spraying has long been the standard for orchard weed control, but saturating rows with chemicals can compromise soil health, damage young roots, and leave behind residues farmers would rather avoid. A more targeted method—spraying only what needs to be sprayed—has been difficult to scale. Now, with support from the U.S. Department of Agriculture’s National Institute of Food and Agriculture, the Pennsylvania Department of Agriculture and the State Horticultural Association of Pennsylvania, Penn State researchers have developed an automated weed-management system designed to pinpoint weeds like needles in a haystack.
Their goal is straightforward: instead of spraying indiscriminately, teach machines to see and precisely target, with an AI-driven machine vision model that can detect, outline, and track weeds in the thicket.
“In complex environments like apple orchards, it is difficult to develop weed-detection mechanisms because the tree canopy and low branches block the view from above,” said Long He, associate professor of agricultural and biological engineering and leader of the research team. “A side-view camera can help, but weeds might be partially visible or hidden behind untargeted objects or tree trunks. This causes problems such as misidentifying weeds or losing track of a weed in real time.”
The work, published in the December issue of Computers and Electronics in Agriculture, aims to guide a robotic precision sprayer—an autonomous rover-like device fitted with a spray arm that can apply herbicide only where it’s needed. At Penn State’s Fruit Research and Extension Center in Biglerville, doctoral candidate and study first author Lawrence Arthur recently tested the sprayer across orchard rows, where dandelion, horseweed, common sow thistle, and Carolina horsenettle often cluster at the base of trees.
The challenge is not just spotting weeds but doing so as they slip behind trunks, leaves, or uneven ground cover. Arthur and the team enhanced a commercially available deep-learning model capable of fast object segmentation. They added a module that enables the system to “pay attention” to specific image features—strengthening the patterns that identify weeds and suppressing the clutter of the surrounding environment.
They also integrated a tracking algorithm that allows the system to maintain the identity of each weed across multiple video frames. “The algorithm preserves weed identity across frames and prevents counting the same weed multiple times,” He said. That means the system can momentarily lose sight of a plant and still recognize it when it reappears—an ability that is essential for reliable precision spraying.
The team collected thousands of high-resolution images across orchards in Biglerville and nearby regions to train and test the model. In evaluation trials, the AI achieved 84.9% average precision for segmentation—outlining each weed pixel by pixel—and 83.6% for localization. Tracking performance ranked similarly high: 82% in multiple-object tracking accuracy, 78% in tracking precision, and an 88% identification score, with only six identity switches recorded during the study.
“The model rarely confuses one weed for another as it tracks them,” He said, noting that such consistency is key for minimizing chemical waste.
Weed control in orchards has long been a balancing act. On one side lies mechanical weeding, which can disturb soil structure and expose tree roots. On the other lies chemical spraying, which can lead to herbicide resistance, runoff, and excessive residues. Growers often broadcast spray the entire strip beneath trees—even when weed density is low—to avoid missing anything. But that means large portions of herbicide never reach a target weed. Precision spraying avoids such waste.
Spot spraying systems have existed for years, but most rely on manual operation or top-down imaging, such as drones. Apple orchards have canopies that obscure the ground, and the shading and branches also hinders detection of weeds from overhead angles. By using a side-view camera mounted on a vehicle platform, the researchers are trying to solve a problem that traditional agriculture systems fall short on.
The research also fits into a broader shift in agriculture, as growers navigate rising labor costs, pressure to reduce chemical use, and looming regulatory changes. Overreliance on herbicides has already led to resistant weed species and concerns about environmental contamination. The Environmental Protection Agency has proposed tighter herbicide application guidelines for orchards—rules that could further limit farmers’ options during critical parts of the growing cycle.
Accurate detection and density estimation allow farmers to use herbicides more sparingly while still protecting crops. The researchers say their approach could help chart a path toward sustainable weed management in orchards nationwide.
“By combining better detection and stronger tracking with added density estimation, the model we developed provides more accurate, consistent weed detection, even in difficult orchard conditions,” He said. “By providing actionable data for site-specific weed management, this approach will improve herbicide efficiency and reduce waste.”
Their next steps involve continued testing of the robotic sprayer and refinement of real-time decision-making—to bring the system closer to deployment.
The use of AI in agriculture holds tremendous promise. Researchers at the UGA Institute for Integrative Precision Agriculture are currently conducting studies on how AI can increase productivity of crops, conserve resources, and ensure food security for future generations. Like Penn State, they are also looking at how AI can manage weeds. They are conducting tests at the 250-acre Grand Farm, in Perry, Georgia. Crops will be seeded, cultivated, watered, monitored, and harvested with the assistance of AI, robotics, and predictive analytics.
