Multimodal Smart Bins: Fusion of Vision and Sensor Data for Enhanced Waste Classification

Arya S Nair, Dr. Anju J Prakash

Abstract: Vision-based approaches have been utilized for waste classification, a considerable part of waste management, but may possess accuracy limitations in the real world due to environmental factors such as light sources, occlusions or features that look visually similar. We present a new multimodal smart bin utilizing computer vision, weight sensors, and near-infrared (NIR) spectroscopy in an effort to detect and classify urban waste with a high degree of accuracy and robustness. The additional information leveraged from the weight and spectral signature of items with visual features assists in improving accurate classification while mitigating the likelihood of misclassifying similar or visually ambiguous items, e.g. wet paper and food waste or clear plastic and glass. This paper presents a multimodal dataset of image, weight and NIRS readings for urban waste materials that classify waste with a deep learning model using late fusion of vision and multimodal weight-sensor technology. The system can be deployed on edge devices like a Raspberry Pi or NVIDIA Jetson Nano allowing waste detection and classification at bin-level, without the need for cloud server capabilities. The IoT-based integration is capable of providing remote monitoring and centralized analytics, along with actionable insights for the municipal government authority. The results show that the multimodal mechanisms of the system preformed more accurately and robustly than vision only models and furthers the contamination problem of recyclables, allowing for more reliable and automated sorting of urban waste to be undertaken at the source. The work of this dataset further supports both of the United Nations Sustainable Development Goals (SDG 11: Sustainable Cities and Communities, and SDG 9: Industry, Innovation and Infrastructure), towards developing sustainable, green, and smart cities through scalable and sustain- able solutions to the solid waste management systems. Future work can focus on building lighter weight fusion models for cost-effective deployment, adaptive calibration of sensor-driven models for long-term sustainability, and developing engagement features with the recycling and composting processes of incentive- based systems for citizen where they use the waste systems.

Keywords: Multimodal learning, waste classification, smart bins, computer vision, weight sensors, near-infrared spectroscopy, IoT, sustainable cities

How to Cite?: Arya S Nair, Dr. Anju J Prakash, "Multimodal Smart Bins: Fusion of Vision and Sensor Data for Enhanced Waste Classification", Volume 14 Issue 9, September 2025, International Journal of Science and Research (IJSR), Pages: 806-811, https://www.ijsr.net/getabstract.php?paperid=SR25916204727, DOI: https://dx.doi.org/10.21275/SR25916204727