Object Detection in Label-Scarce Settings Using Pseudo-Labels Generated by SAM3

The paper, authored by ICoNIoT researchers João V. D. Sobrinho (UFRJ) and Miguel E. M. Campista (UFRJ), was presented at the 18th Brazilian Symposium on Ubiquitous and Pervasive Computing at CSBC 2026.
The research grew out of a project related to the automotive industry and investigates an important problem for Artificial Intelligence applications: how to train computer vision models when there is little labeled data available?

In supervised learning, the machine needs examples accompanied by labels—something similar to an answer key. The model receives an image, produces a response, compares that response with the correct label, and, based on the error, adjusts its behavior. The better and more numerous the examples used in training, the greater the model’s ability to perform the task tends to be.

In the project that gave rise to this work, the goal is to use federated learning in the automotive context, particularly due to data privacy concerns. One of the applications being considered is the identification of VRUs (Vulnerable Road Users)—such as pedestrians, cyclists, and skateboarders—so that a system can recognize them and issue alerts to drivers.

For this system to work, however, the model needs to learn how to identify these users in the images. In this context, there is a fundamental question: who will tell the machine what is in each image?

Imagine that thousands of images of streets and roads are collected. To use them for training, someone would need to identify and label, one by one, the presence of pedestrians, cyclists, and other elements. This labeling process is labor-intensive, expensive, and not very scalable. Furthermore, in a real-world scenario, it would be unreasonable to expect the client or end user to perform this work themselves.

One alternative is to use a pre-trained model to automatically generate these labels. The problem is that this model can also make many errors. If the generated labels are incorrect, these errors end up being incorporated into the training and can compromise the performance of the final model.

This is where SAM3 comes in, which is used in this study to generate pseudo-labels. Instead of relying solely on a person to manually identify objects in the images, the model is used to automatically generate these labels. The goal is to evaluate whether these pseudo-labels can improve the quality of the available data and, consequently, the performance of the object detection system in a scenario where labels are scarce.
The term “pseudo-label” stems precisely from the fact that the label was not produced directly by a human, but by the Artificial Intelligence model itself.

There is, however, an important privacy-related issue. SAM3 is a Meta model, and when images are sent to an external solution, concerns arise about what happens to that data. In this work, SAM3 runs locally, reducing privacy risks by avoiding the need to send these images to an external service.

From Research to the Real World

One of the points raised by Miguel Elias M. Campista is that the issue of labeling in federated learning has yet to be fully explored from a practical perspective. There is a significant amount of research on models and methods, but another question remains unanswered: how can these solutions be transformed into systems that can effectively function in the real world?

When a technology moves beyond the controlled environment of academic research and is incorporated into a product or service, various challenges arise that do not always appear in scientific experiments—or that become even more pronounced—such as privacy, data quality, implementation costs, infrastructure, scalability, and interaction with real users.
Thus, this work is not merely about finding a way to improve object detection. It also addresses a broader issue: how to bridge the gap between knowledge produced in academia and its implementation in real-world applications.

Read the full article in the CSBC proceedings 

Researcher Dener Ottolini will present a webinar on August 20

The presentation is entilted “Intelligence Across the IoT Continuum: Integration, Adaptation, and Distributed Processing”

The increasing integration of artificial intelligence into Internet of Things (IoT) systems has intensified the challenges of integration, data processing, and decision-making in distributed environments. In this context, the computing continuum enables the distribution of data and services across devices, edge, fog, and cloud infrastructures according to the available resources and the requirements of each application. Drawing on research conducted in the domain of smart agriculture, this presentation discusses architectures for heterogeneous platform integration, dynamic adaptation, service placement, and intelligent data processing. The results highlight the trade-offs involved in selecting where services should be executed and emphasize the importance of considering connectivity, latency, computational capacity, and resilience. This perspective contributes to the development of more autonomous, adaptive, and AI-ready IoT systems.

The speaker

Dener Edson Ottolini Guedes da Silva holds a Ph.D. in Information Engineering and an M.Sc. in Computer Science from the Federal University of ABC (UFABC), as well as a degree in Information Technology for Business Management. He is currently a postdoctoral researcher at FEI University Center, where he investigates the development and deployment of intelligent applications across the computing continuum. His research interests include the Internet of Things, distributed systems, edge, fog, and cloud computing, programmable networks with P4, artificial intelligence, and smart agriculture.

Learn About the FLeer2FLeer Research Project: A Peer-to-Peer Architecture-Based Web Tool for Federated Learning Orchestration

The project, led by researchers João Silva e Costa (GTA-PEE/COPPE-DEL/Poli, Federal University of Rio de Janeiro – UFRJ), Guilherme A. Thomaz, Ronaldo A. Ferreira, and Miguel Elias M. Campista (Facom, Federal University of Mato Grosso do Sul – UFMS), was presented at the SBRC 2026

FLeer2FLeer (F2F) is a web-based tool for orchestrating federated learning based on a P2P architecture. F2F introduces an indexer responsible for discovering, announcing, and enrolling clients in multiple federations, without interfering with the training process.

The tool provides a decentralized orchestration infrastructure for Federated Learning (FL), enabling the training of artificial intelligence models without compromising the privacy of the users who own the data.

The project, led by researchers João Silva e Costa (GTA-PEE/COPPE-DEL/Poli, Federal University of Rio de Janeiro – UFRJ), Guilherme A. Thomaz, Ronaldo A. Ferreira, and Miguel Elias M. Campista (Facom, Federal University of Mato Grosso do Sul – UFMS), was presented at the SBRC 2026 Tools Exhibition and is available in the symposium proceedings.

Federated Learning

This work relates to federated learning, an approach initially developed by Google to address a central challenge in training artificial intelligence models: the need to collect large volumes of user data, which raised privacy concerns.

Instead of centralizing user data on a single server to train models, the federated learning approach reverses this logic. The model is sent to the devices or organizations that hold the data, where training is performed locally. Subsequently, only updates to the trained models are sent back to the central server, which combines (“merges”) these contributions to produce an increasingly accurate and convergent global model. This process depends on the participation of multiple clients training the model in parallel.

The procedure is repeated until the model converges or a stopping criterion is met. Because this distributed training approach preserves user privacy, it is well-suited for situations where training requires sensitive user information, such as data from Internet of Things (IoT) systems, medical data, or data generated by mobile devices.

Peer-to-Peer

Much of the existing literature assumes that these training processes are already underway. However, there is little discussion of a practical question: How does a new participant join a federated training process? How can a company or a user who possesses data find a compatible training process and begin to contribute?

It is precisely this gap that the article seeks to address. The proposal allows a client—whether a company or an end user who owns the data—to identify federated learning models that align with its interests and participate in the corresponding training processes.

By contributing their own data, customers help improve the model and, at the same time, obtain a system that is better tailored to their specific needs. In addition, existing models can continue to be refined on an ongoing basis as new data is incorporated into the training process.

Proposal

This research proposes the development of a web-based tool inspired by P2P architecture, called FLeer2FLeer (F2F), for training federated models. F2F proposes a server that acts as an indexer for discovering and managing federations. Unlike approaches focused on complex changes to the algorithm, the tool prioritizes the user experience and the governance of multiple simultaneous model training sessions.

FLeer2FLeer is an innovative tool that combines federated learning and peer-to-peer architecture to optimize the training of artificial intelligence models. Unlike traditional methods that centralize private information, this solution allows data to remain with the users, promoting greater privacy and security during processing.

The system facilitates collaboration between companies and individuals, enabling interested parties to locate and contribute to specific training programs in a distributed manner. By decentralizing the computational effort, the platform helps reduce operational costs and makes the models more accurate and tailored to the participants’ actual needs. The central focus is on overcoming the challenges of practical implementation by efficiently connecting data owners to processes for orchestrating global models.

Although the system is not an open public service, it can be deployed within companies, allowing different organizations to conduct collaborative training on distributed data without the need to centralize it.

This approach also makes it possible to distribute the computational cost among participants, since processing is carried out in a decentralized manner. The main challenge now is to transform this solution into a technology that can be applied in real-world scenarios, expanding its adoption beyond the research environment.