Comparing Optimized Random Forest and XGBoost: Stress Classification using Digital Behavior and Sleep

M. Thoriqul Fadli, M. Rudi Sanjaya, Dwi Rosa Indah, M. Husni Syahbani

Abstract


Increased digital device usage has been associated with various mental health risks, including stress, highlighting the need for predictive approaches capable of objectively identifying digital behavior patterns. This study aims to compare the performance of Random Forest and XGBoost in classifying stress levels based on digital behavior and sleep-related features, while also examining the effects of hyperparameter optimization on the performance of both models. The study uses the Screentime vs Mental Wellness Survey 2025 dataset, comprising 400 respondents, with a class distribution of 85.2% for the "Stressed" class and 14.8% for the "Safe" class. Features suspected of acting as proxies for the target variable, namely Mental Wellness Index and Productivity, were excluded to reduce the risk of data leakage. Class balancing was performed using SMOTENC within the training pipeline, while hyperparameter optimization was conducted using GridSearchCV, with F1-Macro as the primary optimization metric. The main evaluation employed stratified 80:20 training-test splits across 21 random seeds, with F1-Macro reported as the average across all splits. Differences in model performance were analyzed using the Wilcoxon signed-rank test and the 5×2 cross-validation paired t-test. The results show that the average F1-Macro score of Random Forest increased from 0.7721 to 0.7807 following optimization, while XGBoost improved from 0.7370 to 0.7684. Random Forest achieved a higher F1-Macro score in 11 out of 21 data splits. However, neither the Wilcoxon signed-rank test (p = 0.2959) nor the 5×2 cross-validation paired t-test (p = 0.5747) revealed a statistically significant difference in model performance. Feature importance analysis indicated that screen time hours and sleep quality were the dominant features in both models. These findings are limited to the dataset used in this study, and external validation is still required to assess the generalizability of the models to other populations.

Keywords


digital behavior; hyperparameter optimization; Random Forest; stress classification; XGBoost

Full Text:

PDF

References


K. A. Devi and S. K. Singh, “The Hazards of Excessive Screen Time: Impacts on Physical Health, Mental Health, and Overall well being,” J. Educ. Health Promot., Vol. 12, No. 1, Art. No. 413, Nov. 2023, DOI: 10.4103/jehp.jehp_447_23.

R. M. S. Santos et al., “The Associations between Screen Time and Mental Health in Adults: A Systematic Review,” J. Technol. Behav. SCI., Vol. 9, pp. 825–845, Feb. 2024, DOI: 10.1007/s41347-024-00398-7.

A. Khan, G. McLeod, T. Hidajat, and E. J. Edwards, “Excessive Smartphone use is Associated with Depression, Anxiety, Stress, and Sleep Quality of Australian Adults,” J. Med. Syst., Vol. 47, No. 1, Art. No. 109, Oct. 2023, DOI: 10.1007/s10916-023-02005-3.

M. Belal, N. Luong, T. Aledavood, and J. Kulshrestha, “Examining the Association between Internet use and Perceived Stress in Adults: Longitudinal Observational Study Combining Web Tracking Data with Questionnaires,” J. Med. Internet Res., Vol. 28, Art. No. e78775, Jan. 2026, DOI: 10.2196/78775.

C. Pieh et al., “Smartphone Screen Time Reduction Improves Mental Health: A Randomized Controlled Trial,” BMC Med., Vol. 23, No. 1, Art. No. 107, Feb. 2025, DOI: 10.1186/s12916-025-03944-z.

C. Zhong, M. Masters, S. M. Donzella, W. R. Diver, and A. V. Patel, “Electronic Screen Use and Sleep Duration and Timing in Adults,” JAMA Netw. Open, Vol. 8, No. 3, Art. No. e252493, Mar. 2025, DOI: 10.1001/jamanetworkopen.2025.2493.

T. O. Andersen et al., “Nighttime Smartphone Use, Sleep Quality, and Mental Health: Investigating a Complex Relationship,” Sleep, Vol. 46, No. 12, Art. No. zsad256, Sep. 2023, DOI: 10.1093/sleep/zsad256.

M. C. Chandler, O. K. Ellison, A. L. McGowan, K. M. Fenn, and M. B. Pontifex, “Physical Activity and Sleep Moderate the Relationship between Stress and Screen Time in College-Aged Adults,” Journal of American College Health, Vol. 72, No. 5, pp. 1401–1411, Jul. 2024, DOI: 10.1080/07448481.2022.2077110.

L. S. Khoo, M. K. Lim, C. Y. Chong, and R. McNaney, “Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing Approaches,” Sensors, Vol. 24, No. 2, Art. No. 348, Jan. 2024, DOI: 10.3390/s24020348.

D. N. Fahria, K. D. Tania, and R. D. Kurnia, “Analisis Komparatif Algoritma Random Forest, Xgboost, dan Catboost untuk Klasifikasi Tingkat Stres Pengguna Media Sosial,” Rabit : Jurnal Teknologi dan Sistem Informasi Univrab, Vol. 11, No. 1, pp. 1843–1853, Jan. 2026, DOI: 10.36341/rabit.v11i1.7449.

S. A. Tamboto and S. Jaya, “Benchmarking XGBoost and Random Forest for Stress Level Prediction based on Sleep Physiological Parameters,” MALCOM: Indonesian Journal of Machine Learning and Computer Science, Vol. 6, No. 3, pp. 1501–1510, Jun. 2026, Accessed: Oct. 22, 2026. [Online]. Available: https://journal.irpi.or.id/index.php/malcom/article/view/2783

S. W. Sugiharto and Y. Nataliani, “Machine Learning Model Optimization and Interpretability Analysis for Classifying Student Stress Levels,” Sistemasi: Jurnal Sistem Informasi, Vol. 15, No. 5, pp. 1594–1607, May 2026, DOI: 10.32520/stmsi.v15i5.6239.

F. I. S. Ms, F. A. Bachtiar, and E. S. Pramukantoro, “Perbandingan Klasifikasi Stres berbasis Electroencephalography dengan Kombinasi Ekstraksi dan Seleksi Fitur menggunakan Metode Tunggal dan Ensemble,” Jurnal Teknologi Informasi dan Ilmu Komputer, Vol. 13, No. 3, pp. 713–722, Jun. 2026, Accessed: Oct. 22, 2026. [Online]. Available: https://jtiik.ub.ac.id/index.php/jtiik/id/article/view/10457/1604

A. F. Hafin, N. T. Astuti, and S. Zahroh, “Analisis Prediksi Risiko Depresi pada Remaja menggunakan Random Forest dan Logistic Regression berdasarkan Pola Penggunaan Media Sosial,” JIIFKOM (Jurnal Ilmiah Informatika dan Komputer), Vol. 5, No. 2, pp. 67–84, Jul. 2026, DOI: 10.51901/jiifkom.v5i02.735.

D. Pratama Putra, A. Apriyadi, Z. Firmansyah, K. Ditha Tania, and D. Kurniawan, “Perbandingan Model Regresi Machine Learning untuk Prediksi Skor Tingkat Stres berdasarkan Pola Screen Time Tahun 2025,” Jurnal Pendidikan dan Teknologi Indonesia, Vol. 6, No. 4, pp. 733–741, Apr. 2026, DOI: 10.52436/1.jpti.1537.

M. Altalhan, A. Algarni, and M. Turki-Hadj Alouane, “Imbalanced Data Problem in Machine Learning: A Review,” IEEE Access, Vol. 13, pp. 13686–13699, Jan. 2025, DOI: 10.1109/ACCESS.2025.3531662.

S. Matharaarachchi, M. Domaratzki, and S. Muthukumarana, “Enhancing SMOTE for Imbalanced Data with Abnormal Minority Instances,” Machine Learning with Applications, Vol. 18, Art. No. 100597, Dec. 2024, DOI: 10.1016/j.mlwa.2024.100597.

A. A. Huang and S. Y. Huang, “Computation of the Distribution of Model Accuracy Statistics in Machine Learning: Comparison between Analytically Derived Distributions and Simulation-based Methods,” Health SCI. Rep., Vol. 6, No. 4, Art. No. e1214, Apr. 2023, DOI: 10.1002/hsr2.1214.

R. D. Riley and G. S. Collins, “Stability of Clinical Prediction Models Developed using Statistical or Machine Learning Methods,” Biometrical Journal, Vol. 65, No. 8, Art. No. 2200302, Dec. 2023, DOI: 10.1002/bimj.202200302.

A. Apicella, F. Isgrò, and R. Prevete, “Don’t Push the Button! Exploring Data Leakage Risks in Machine Learning and Transfer Learning,” Artif. Intell. Rev., Vol. 58, No. 11, Art. no. 339, Nov. 2025, DOI: 10.1007/s10462-025-11326-3.

M. Rosenblatt, L. Tejavibulya, R. Jiang, S. Noble, and D. Scheinost, “Data Leakage Inflates Prediction Performance in Connectome-based Machine Learning Models,” Nature Communications , Vol. 15, No. 1, Art. No. 1829, Feb. 2024, DOI: 10.1038/s41467-024-46150-w.

S. Kapoor and A. Narayanan, “Leakage and the Reproducibility Crisis in Machine-Learning-based Science,” Patterns, Vol. 4, No. 9, Art. No. 100804, Sep. 2023, DOI: 10.1016/j.patter.2023.100804.

L. Breiman, “Random Forests,” Mach. Learn., Vol. 45, No. 1, pp. 5–32, Oct. 2001, DOI: 10.1023/A:1010933404324.

T. Chen and C. Guestrin, “XGBoost: A Scalable Tree Boosting System,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery, 2016, pp. 785–794. DOI: 10.1145/2939672.2939785.

T. G. Dietterich, “Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms,” Neural Comput., Vol. 10, No. 7, pp. 1895–1923, Oct. 1998, DOI: 10.1162/089976698300017197.




DOI: https://doi.org/10.32520/stmsi.v15i11.7047

Article Metrics

Abstract view : 0 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.