Comparing LoRA on Qwen2.5 and DeepSeek-R1 with IndoBERT Fine-Tuning for Indonesian Sentiment
Abstract
Keywords
Full Text:
PDFReferences
W. J. Kusoema and I. Ibrahim, “Sentiment Analysis on the PT Pertamina Corruption Case using IndoBERT and RCNN Methods,” Sistemasi: Jurnal Sistem Informasi, Vol. 14, No. 5, pp. 2246–2257, 2025, DOI: 10.32520/stmsi.v14i5.5392.
C.-H. Lin and U. Nuha, “Sentiment Analysis of Indonesian Datasets based on a Hybrid Deep-Learning Strategy,” Journal of Big Data, Vol. 10, No. 1, p. 88, 2023, DOI: 10.1186/s40537-023-00782-9.
A. H. Nasution, A. Onan, Y. Murakami, W. Monika, and A. Hanafiah, “Benchmarking Open-Source Large Language Models for Sentiment and Emotion Classification in Indonesian Tweets,” IEEE Access, Vol. 13, pp. 94009–94025, 2025, DOI: 10.1109/ACCESS.2025.3574629.
B. Wilie, K. Vincentio, G. I. Winata, S. Cahyawijaya, X. Li, Z. Y. Lim, S. Soleman, R. Mahendra, P. Fung, S. Bahar, and A. Purwarianti, “IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding,” in Proc. 1st Conf. Asia-Pacific Chapter ACL and 10th Int. Joint Conf. Natural Language Processing (AACL-IJCNLP), 2020, pp. 843–857, DOI: 10.18653/v1/2020.aacl-main.85.
F. Koto, J. H. Lau, and T. Baldwin, “IndoBERTweet: A Pretrained Language Model for Indonesian Twitter with Effective Domain-Specific Vocabulary Initialization,” in Proc. 2021 Conf. Empirical Methods in Natural Language Processing (EMNLP), 2021, pp. 10660–10668, DOI: 10.18653/v1/2021.emnlp-main.833.
D. Suharjanto and M. Mustakim, “Optimizing Emotion Recognition in Indonesian Product Reviews using LoRA on Transformer,” Jurnal Sains, Nalar, dan Aplikasi Teknologi Informasi, Vol. 5, No. 2, pp. 118–127, 2026, DOI: 10.20885/snati.v5.i2.48248.
F. M. Al Simabua and L. Alfat, “Optimization of IndoBERT-Lite Fine-Tuning for Spam Detection in Digital Customer Services,” Sistemasi: Jurnal Sistem Informasi, Vol. 15, No. 5, pp. 1886–1899, 2026, DOI: 10.32520/stmsi.v15i5.6398.
Qwen Team, “Qwen2.5 Technical Report,” arXiv preprint arXiv:2412.15115, 2024, DOI: 10.48550/arXiv.2412.15115.
D. Guo et al., “DeepSeek-R1 Incentivizes Reasoning in LLMs Through Reinforcement Learning,” Nature, Vol. 645, pp. 633–638, 2025, DOI: 10.1038/s41586-025-09422-z.
Z. Han, C. Gao, J. Liu, J. Zhang, and S. Q. Zhang, “Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey,” arXiv preprint arXiv:2403.14608, 2024, DOI: 10.48550/arXiv.2403.14608.
E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-Rank Adaptation of Large Language Models,” in Proc. 10th Int. Conf. Learning Representations (ICLR), 2022, URL: https://openreview.net/forum?id=nZeVKeeFYf9.
X. Wang and A. Azman, “LoRA-based Efficient Fine-Tuning of Transformer Models for Short Text Classification,” in Advances in Visual Informatics (IVIC 2025), Lecture Notes in Computer Science, Vol. 16215. Singapore: Springer, 2026, pp. 168–179, DOI: 10.1007/978-981-95-3808-9_13.
K. I. Roumeliotis, N. D. Tselikas, and D. K. Nasiopoulos, “Optimizing Airline Review Sentiment Analysis: A Comparative Analysis of LLaMA and BERT Models through Fine-Tuning and Few-Shot Learning,” Computers, Materials & Continua, Vol. 82, No. 2, pp. 2769–2792, 2025, DOI: 10.32604/cmc.2025.059567.
H. Jin, N. Ashrafi, A. Abdollahi, W. Liu, J. Wang, G. Gui, M. Pishgar, and H. Feng, “LLM Encoder vs. Decoder: Robust Detection of Chinese AI-Generated Text with LoRA,” arXiv preprint arXiv:2509.00731, 2025, DOI: 10.48550/arXiv.2509.00731.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding,” in Proc. 2019 Conf. North American Chapter ACL: Human Language Technologies, Vol. 1, 2019, pp. 4171–4186, DOI: 10.18653/v1/N19-1423.
M. A. A. Saputra, A. Alamsyah, D. P. Ramadhani, T. S. Siadari, and H. Fakhrurroja, “IndoBERT-Sentiment: Context-Conditioned Sentiment Classification for Indonesian Text,” arXiv preprint arXiv:2604.07057, 2026, DOI: 10.48550/arXiv.2604.07057.
V. R. Joseph, “Optimal Ratio for Data Splitting,” Statistical Analysis and Data Mining: The ASA Data Science Journal, Vol. 15, No. 4, pp. 531–538, 2022, DOI: 10.1002/sam.11583.
J. Opitz, “A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice,” Transactions of the Association for Computational Linguistics, Vol. 12, pp. 820–836, 2024, DOI: 10.1162/tacl_a_00675.
S. Henning, W. Beluch, A. Fraser, and A. Friedrich, “A Survey of Methods for Addressing Class Imbalance in Deep-Learning based Natural Language Processing,” in Proc. 17th Conf. European Chapter ACL (EACL), 2023, pp. 523–540, DOI: 10.18653/v1/2023.eacl-main.38.
Q. McNemar, “Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages,” Psychometrika, Vol. 12, No. 2, pp. 153–157, 1947, DOI: 10.1007/BF02295996.
B. Efron, “Bootstrap Methods: Another Look at the Jackknife,” The Annals of Statistics, Vol. 7, No. 1, pp. 1–26, 1979, DOI: 10.1214/aos/1176344552.
Y. Bestgen, “Please, Don't Forget the Difference and the Confidence Interval when Seeking for the State-of-the-Art Status,” in Proc. 13th Language Resources and Evaluation Conf. (LREC), 2022, pp. 5956–5962, URL: https://aclanthology.org/2022.lrec-1.640.
Y. Kim, M. Lee, and S. Lee, “A Survey on Memory-Efficient Fine-Tuning for Large Language Models,” Transactions of the Association for Computational Linguistics, Vol. 14, pp. 960–982, 2026, DOI: 10.1162/tacl.a.692.
N. T. Bui, G. Savova, and L. Wang, “Assessing the Macro and Micro Effects of Random Seeds on Fine-Tuning Large Language Models,” arXiv preprint arXiv:2503.07329, 2025, DOI: 10.48550/arXiv.2503.07329.
T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “QLoRA: Efficient Finetuning of Quantized LLMs,” in Advances in Neural Information Processing Systems 36 (NeurIPS), 2023, DOI: 10.48550/arXiv.2305.14314.
S.-Y. Liu, C.-Y. Wang, H. Yin, P. Molchanov, Y.-C. F. Wang, K.-T. Cheng, and M.-H. Chen, “DoRA: Weight-Decomposed Low-Rank Adaptation,” in Proc. 41st Int. Conf. Machine Learning (ICML), 2024, pp. 32100–32121, DOI: 10.48550/arXiv.2402.09353.
DOI: https://doi.org/10.32520/stmsi.v15i11.7015
Article Metrics
Abstract view : 0 timesPDF - 0 times
Refbacks
- There are currently no refbacks.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.






