Time Series Forecasting of Air Pollutant PM2.5 Using Transformer Architecture

K. Azhahudurai, Dr. V. Veeramanikandan

Abstract: Transformer architectures are widely used, especially in computer vision and natural language processing. Transformers have been used recently in a number of time-series analysis applications. An overview of the Transformer architecture and its uses in time-series analysis is given in the literature review. To improve performance, the Transformer's primary parts?the encoder/decoder, multi-head, positional encoding, and self-attention mechanism?have been updated. To implement time-series analysis, a few improvements to the original transformer architecture were adopted. Additionally, the optimal hyperparameters values for overcoming the difficulty of successfully training Transformers for time-series analysis are provided in this work. The effectiveness of the Transformer model in forecasting PM2.5 concentrations is examined in this paper. The dataset is pre-processed as a first step. In order to minimize the input parameters while taking into account their statistical significance, multi-collinearity among the independent variables is found using a Variance Inflation Factor (VIF). The proposed model have been trained to forecast PM2.5 concentrations up to one day ahead of time.

Keywords: transformer architecture, time series analysis, self attention, hyperparameters, forecasting PM 2.5, multi-collinearity, Variance Inflation Factor

How to Cite?: K. Azhahudurai, Dr. V. Veeramanikandan, "Time Series Forecasting of Air Pollutant PM2.5 Using Transformer Architecture", Volume 12 Issue 11, November 2023, International Journal of Science and Research (IJSR), Pages: 2075-2082, https://www.ijsr.net/getabstract.php?paperid=SR231125192357, DOI: https://dx.doi.org/10.21275/SR231125192357