Abstract

Stock market prediction is a complex and challenging task due to the dynamic nature of financial markets influenced by numerous factors, including economic conditions, political events, and market sentiment. This study proposes a comparative analysis of machine learning and deep learning algorithms for predicting stock market trends, using both continuous and binary data. Four stock market sectors, namely diversified financials, petroleum, non-metallic minerals, and basic metals from the Tehran Stock Exchange, are selected for experimental evaluation.Nine machine learning models, including Decision Tree, Random Forest, Adaptive Boosting (Adaboost), eXtreme Gradient Boosting (XGBoost), Support Vector Classifier (SVC), Naïve Bayes, K-Nearest Neighbors (KNN), Logistic Regression, and Artificial Neural Network (ANN), are compared with two deep learning models: Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM). Ten technical indicators extracted from ten years of historical stock market data are used as input features. Both continuous data and binary-converted data are analyzed to assess model performance.The results demonstrate that deep learning models, particularly RNN and LSTM, outperform traditional machine learning algorithms in predicting stock market trends when using continuous data. In the binary data approach, these models maintain strong predictive accuracy, while some machine learning models, like Random Forest and XGBoost, also show competitive results. Additionally, the study highlights the strengths and limitations of each model in terms of accuracy, computational complexity, and adaptability to market fluctuations. This comprehensive comparative analysis provides valuable insights for investors, financial analysts, and researchers seeking reliable predictive models for informed decision-making. Future work may involve integrating additional economic and sentiment-based data to further enhance prediction accuracy and generalization across different financial markets.