DeepFake Weeds: Integration of neural networks and stable diffusion for weed detection in tomato crops

Abstract

Weeds negatively affect crop yield and quality by competing with the crop for resources. Detecting them in time allows us to optimize control through the precise application of herbicides and reduce their environmental impact. However, its detection and classification are challenging due to the great diversity of species and similarities with crops. Deep learning methods have allowed the development of convolutional neural networks (CNN) for the classification of weed species. However, CNNs require large and varied data sets for training. This study proposes an innovative methodology using CNN, YOLOv8l and RetinaNet classifiers augmented with Stable Diffusion data, implemented artificial images of weeds. Three data sets were configured for training CNNs (real, artificial and mixed) in commercial tomato crops infested by Solanum nigrum L., Portulaca oleracea L. and Setaria Verticillata L. The results showed a high degree of agreement with artificial images, reaching a maximum mean Average Precision (mAP) of 0.93 in both networks for the mixed data set. Stable Diffusion made it possible to generate large, high-quality data sets from a limited set of images, reducing the need to obtain a large number of images in the field and their subsequent manual labeling. Furthermore, the method can be adapted to other crops and weed species, thus contributing to the advancement of automated weed management systems.