The paper, authored by ICoNIoT researchers João V. D. Sobrinho (UFRJ) and Miguel E. M. Campista (UFRJ), was presented at the 18th Brazilian Symposium on Ubiquitous and Pervasive Computing at CSBC 2026.
The research grew out of a project related to the automotive industry and investigates an important problem for Artificial Intelligence applications: how to train computer vision models when there is little labeled data available?
In supervised learning, the machine needs examples accompanied by labels—something similar to an answer key. The model receives an image, produces a response, compares that response with the correct label, and, based on the error, adjusts its behavior. The better and more numerous the examples used in training, the greater the model’s ability to perform the task tends to be.
In the project that gave rise to this work, the goal is to use federated learning in the automotive context, particularly due to data privacy concerns. One of the applications being considered is the identification of VRUs (Vulnerable Road Users)—such as pedestrians, cyclists, and skateboarders—so that a system can recognize them and issue alerts to drivers.
For this system to work, however, the model needs to learn how to identify these users in the images. In this context, there is a fundamental question: who will tell the machine what is in each image?
Imagine that thousands of images of streets and roads are collected. To use them for training, someone would need to identify and label, one by one, the presence of pedestrians, cyclists, and other elements. This labeling process is labor-intensive, expensive, and not very scalable. Furthermore, in a real-world scenario, it would be unreasonable to expect the client or end user to perform this work themselves.
One alternative is to use a pre-trained model to automatically generate these labels. The problem is that this model can also make many errors. If the generated labels are incorrect, these errors end up being incorporated into the training and can compromise the performance of the final model.
This is where SAM3 comes in, which is used in this study to generate pseudo-labels. Instead of relying solely on a person to manually identify objects in the images, the model is used to automatically generate these labels. The goal is to evaluate whether these pseudo-labels can improve the quality of the available data and, consequently, the performance of the object detection system in a scenario where labels are scarce.
The term “pseudo-label” stems precisely from the fact that the label was not produced directly by a human, but by the Artificial Intelligence model itself.
There is, however, an important privacy-related issue. SAM3 is a Meta model, and when images are sent to an external solution, concerns arise about what happens to that data. In this work, SAM3 runs locally, reducing privacy risks by avoiding the need to send these images to an external service.
From Research to the Real World
One of the points raised by Miguel Elias M. Campista is that the issue of labeling in federated learning has yet to be fully explored from a practical perspective. There is a significant amount of research on models and methods, but another question remains unanswered: how can these solutions be transformed into systems that can effectively function in the real world?
When a technology moves beyond the controlled environment of academic research and is incorporated into a product or service, various challenges arise that do not always appear in scientific experiments—or that become even more pronounced—such as privacy, data quality, implementation costs, infrastructure, scalability, and interaction with real users.
Thus, this work is not merely about finding a way to improve object detection. It also addresses a broader issue: how to bridge the gap between knowledge produced in academia and its implementation in real-world applications.
