Intelligent Data Processing and Automation in End-to-End Machine Learning Pipelines
International Journal of Science and Research (IJSR)

International Journal of Science and Research (IJSR)
Call for Papers | Fully Refereed | Open Access | Double Blind Peer Reviewed

ISSN: 2319-7064

Intelligent Data Processing and Automation in End-to-End Machine Learning Pipelines

Putty Srividya, Dr N. Subramanyam

Abstract: Data processing plays an essential role in the development and performance of machine learning and deep learning models. Traditional machine learning relies on separate, manually designed stages for data preprocessing, feature extraction, feature selection, and model optimization. Deep learning offers a more integrated approach by using neural networks to learn useful features directly from data. However, the quality, quantity, and complexity of training data continue to influence model performance. Data preprocessing, augmentation, and feature engineering remain necessary to address missing values, noisy data, class imbalance, and limited training samples. Feature engineering also helps improve model efficiency and interpretability, particularly in applications such as healthcare, fraud detection, and industrial fault diagnosis. Automated Machine Learning (AutoML) provides methods for combining these tasks into a unified process, reducing manual intervention in data preparation, model selection, and hyperparameter optimization. Neural Architecture Search further supports the automatic design of suitable network structures. The discussion examines the differences between traditional and deep learning pipelines, the continuing need for data processing, and the role of automation in developing efficient, adaptable, and reliable machine learning systems.

Keywords: Machine Learning, Deep Learning, Data Preprocessing, Feature Engineering, Automated Machine Learning

How to Cite?: Putty Srividya, Dr N. Subramanyam, "Intelligent Data Processing and Automation in End-to-End Machine Learning Pipelines", Volume 13 Issue 10, October 2024, International Journal of Science and Research (IJSR), Pages: 2076-2079, https://www.ijsr.net/getabstract.php?paperid=SR241012105719, DOI: https://dx.doi.org/10.21275/SR241012105719

Download Citation: APA | MLA | BibTeX | EndNote | RefMan

Share This Research

Help this article reach readers, researchers and professionals.

Share activity is measured for research-engagement analytics. Only verified, unique public shares can support award tie-breaking.

Confirm Your Share

Enter your details so IJSR can confirm this sharing activity.

Your details are used to validate this share and protect the award process from duplicate or false activity.

Download Article PDF


Rate This Article!

Top

Confirm Your Share

Enter your details so IJSR can confirm this sharing activity.

Your details are used to validate this share and protect the award process from duplicate or false activity.