Comparison of The Performance of  Fasttext and Word2Vec Methods in Detecting Fake News: Perbandingan Kinerja Metode FastText dan Word2Vec dalam Mendeteksi Berita Palsu

Try Iksan; Agustinus Jacobus; Fransisca Pontoh

doi:10.35793/jtek.v15i1.62468

Authors

Try Iksan Universitas Sam Ratulangi
Agustinus Jacobus Universitas Sam Ratulangi
Fransisca Pontoh Universitas Sam Ratulangi

DOI:

https://doi.org/10.35793/jtek.v15i1.62468

Keywords:

Bi-LSTM, FastText, Hoax, NLP, Word2Vec

Abstract

Abstract — The spread of fake news (hoaxes) on social media has a significant negative impact on society, such as a decline in public trust and increased uncertainty about information. This study aims to develop and compare accurate and reliable Indonesian-language fake news detection systems, with the hope of improving media literacy among the public. The methods used include collecting several datasets of fake and authentic news, data preprocessing (cleaning, tokenisation, lemmatisation, stopword removal), and applying two word embedding algorithms, FastText and Word2Vec, with two architectures (CBOW and Skipgram). The classification model used is Bi-LSTM, and evaluation is conducted using accuracy, precision, recall, and F1-score metrics. The results show that both algorithms can produce high-accuracy fake news detection models on large datasets (FastText >85%, Word2Vec >87%), but performance decreases on small datasets due to overfitting. This study provides theoretical and practical contributions to the evaluation of word embedding algorithm performance for detecting Indonesian-language fake news based on NLP. In conclusion, the comparison results show that the evaluated word embedding approach is effective in identifying Indonesian-language fake news and can serve as a reference for algorithm selection in the development of future fake news detection technology.

Key words — Bi-LSTM; FastText; hoax; NLP; Word2Vec

Abstrak — Penyebaran berita palsu (hoaks) di media sosial menimbulkan dampak negatif yang signifikan bagi masyarakat, seperti menurunnya kepercayaan publik dan meningkatnya ketidakpastian informasi. Penelitian ini bertujuan untuk mengembangkan dan membandingkan sistem deteksi berita palsu berbahasa Indonesia yang akurat dan andal, dengan harapan dapat meningkatkan literasi media masyarakat. Metode yang digunakan meliputi pengumpulan beberapa dataset berita palsu dan asli, praproses data (cleaning, tokenisasi, lemmatisasi, penghapusan stopwords), serta penerapan dua algoritma word embedding FastText dan Word2Vec dengan dua arsitektur (CBOW dan Skipgram). Model klasifikasi yang digunakan adalah Bi-LSTM, dan evaluasi dilakukan menggunakan metrik akurasi, precision, recall, dan F1-score. Hasil penelitian menunjukkan bahwa kedua algoritma mampu menghasilkan model deteksi berita palsu dengan akurasi tinggi pada dataset besar (FastText >85%, Word2Vec >87%), namun performa menurun pada dataset kecil akibat overfitting. Penelitian ini memberikan kontribusi teoretis dan praktis dalam evaluasi performa algoritma word embedding untuk deteksi berita palsu berbahasa Indonesia berbasis NLP. Kesimpulannya, hasil perbandingan menunjukkan bahwa pendekatan word embedding yang dievaluasi efektif dalam mengidentifikasi berita palsu berbahasa Indonesia dan dapat menjadi acuan pemilihan algoritma untuk pengembangan teknologi deteksi berita palsu di masa depan.

Kata kunci — Bi-LSTM; FastText; hoaks; NLP; Word2Vec

References

[1] M. Deddy Satria and others, “The Phenomenon of Fake News (Hoax) in Mass Communication: Causes, Impacts, and Solutions,” Open Access Indonesia Journal of Social Sciences, vol. 6, no. 3, pp. 980–988, 2023.

[2] D. Susilo Wijayanto et al., “SOCIALIZATION ON PREVENTING THE SPREAD OF HOAX NEWS IN ONLINE NEWS MEDIA AT GROGOL VILLAGE, WERU, SUKOHARJO,” Abdi Masya, vol. 6, no. 1, pp. 42–47, May 2025, doi: 10.52561/ABDIMASYA.V6I1.452.

[3] M. A. B. Al-Tarawneh, O. Al-irr, K. S. Al-Maaitah, H. Kanj, and W. H. F. Aly, “Enhancing Fake News Detection with Word Embedding: A Machine Learning and Deep Learning Approach,” Computers 2024, Vol. 13, Page 239, vol. 13, no. 9, p. 239, Sep. 2024, doi: 10.3390/COMPUTERS13090239.

[4] H. Allam, L. Makubvure, B. Gyamfi, K. N. Graham, and K. Akinwolere, “Text Classification: How Machine Learning Is Revolutionizing Text Categorization,” Information 2025, Vol. 16, Page 130, vol. 16, no. 2, p. 130, Feb. 2025, doi: 10.3390/INFO16020130.

[5] K. Taha, P. D. Yoo, C. Yeun, D. Homouz, and A. Taha, “A comprehensive survey of text classification techniques and their research applications: Observational and experimental insights,” Comput Sci Rev, vol. 54, p. 100664, Nov. 2024, doi: 10.1016/J.COSREV.2024.100664.

[6] L. Galke et al., “Are We Really Making Much Progress in Text Classification? A Comparative Review,” ACM Comput Surv, vol. 1, Jan. 2025, Accessed: Jul. 14, 2025. [Online]. Available: https://arxiv.org/pdf/2204.03954v6

[7] T. B. Hashimoto, D. Alvarez-Melis, and T. S. Jaakkola, “Word Embeddings as Metric Recovery in Semantic Spaces,” Trans Assoc Comput Linguist, vol. 4, pp. 273–286, Dec. 2016, doi: 10.1162/TACL_A_00098.

[8] S. Selva Birunda and R. Kanniga Devi, “A Review on Word Embedding Techniques for Text Classification,” 2021, pp. 267–281. doi: 10.1007/978-981-15-9651-3_23.

[9] E. M. Dharma, F. L. Gaol, H. Warnars, and B. Soewito, “The accuracy comparison among word2vec, glove, and fasttext towards convolution neural network (cnn) text classification,” J Theor Appl Inf Technol, vol. 100, no. 2, p. 31, 2022.

[10] S. Khomsah, R. D. Ramadhani, and S. Wijayanto, “The Accuracy Comparison Between Word2Vec and FastText On Sentiment Analysis of Hotel Reviews,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 6, no. 3, pp. 352–358, Jun. 2022, doi: 10.29207/RESTI.V6I3.3711.

[11] R. Adipradana, B. P. Nayoga, R. Suryadi, and D. Suhartono, “Hoax analyzer for indonesian news using rnns with fasttext and glove embeddings,” Bulletin of Electrical Engineering and Informatics, vol. 10, no. 4, pp. 2130–2136, Aug. 2021, doi: 10.11591/eei.v10i4.2956.

[12] A. Pimpalkar, M. Singh, S. Sheikh, K. Gedam, and A. Khadgi, “Fake news classification using bi-directional LSTM-recurrent neural network,” Journal of Huazhong University of Science and Technology ISSN, vol. 1671, p. 4512, 2021.

[13] I. K. Sastrawan, I. P. A. Bayupati, and D. M. S. Arsa, “Detection of fake news using deep learning CNN–RNN based methods,” ICT Express, vol. 8, no. 3, pp. 396–408, Sep. 2022, doi: 10.1016/j.icte.2021.10.003.

[14] M. Ali Ramdhani, D. S. Maylawati, and T. Mantoro, “Indonesian news classification using convolutional neural network,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 19, no. 2, p. 1000, Aug. 2020, doi: 10.11591/ijeecs.v19.i2.pp1000-1009.

[15] C. N. Tulu, “Experimental Comparison of Pre-Trained Word Embedding Vectors of Word2Vec, Glove, FastText for Word Level Semantic Text Similarity Measurement in Turkish,” Advances in Science and Technology. Research Journal, vol. 16, no. 4, pp. 147–156, 2022, doi: 10.12913/22998624/152453.

[16] N. N. A. Balaji and B. Bharathi, “SSNCSE_NLP@ Fake news detection in the Urdu language (UrduFake) 2020,” Health (Irvine Calif), vol. 100, p. 100, 2020.

[17] H. Padalko, V. Chomko, and D. Chumachenko, “A novel approach to fake news classification using LSTM-based deep learning models,” Front Big Data, vol. 6, p. 1320800, 2024, doi: 10.3389/FDATA.2023.1320800.

[18] Ö. Çelik and B. C. Koç, “TF-IDF, Word2vec ve Fasttext vektör model yöntemleri ile Türkçe haber metinlerinin snflandrlmas,” Dokuz Eylül Üniversitesi Mühendislik Fakültesi Fen ve Mühendislik Dergisi, vol. 23, no. 67, pp. 121–127, 2021.

[19] F. Rollo, G. Bonisoli, and L. Po, “A comparative analysis of word embeddings techniques for italian news categorization,” IEEE Access, vol. 12, pp. 25536–25552, 2024.

[20] J. H. Cabot and E. G. Ross, “Evaluating Prediction Model Performance,” Surgery, vol. 174, no. 3, p. 723, Sep. 2023, doi: 10.1016/J.SURG.2023.05.023.

[21] O. Rainio, J. Teuho, and R. Klén, “Evaluation metrics and statistical tests for machine learning,” Sci Rep, vol. 14, no. 1, pp. 1–14, Dec. 2024, doi: 10.1038/S41598-024-56706-X;SUBJMETA=117,531,639,705;KWRD=COMPUTER+SCIENCE,STATISTICS.

[22] M. K. Anam, S. Defit, Haviluddin, L. Efrizoni, and M. B. Firdaus, “Early Stopping on CNN-LSTM Development to Improve Classification Performance,” Journal of Applied Data Sciences, vol. 5, no. 3, pp. 1175–1188, Aug. 2024, doi: 10.47738/JADS.V5I3.312.

[23] M. Vilares Ferro, Y. Doval Mosquera, F. J. Ribadas Pena, and V. M. Darriba Bilbao, “Early stopping by correlating online indicators in neural networks,” Neural Networks, vol. 159, pp. 109–124, Feb. 2023, doi: 10.1016/J.NEUNET.2022.11.035.

[24] L. Brigato and L. Iocchi, “A Close Look at Deep Learning with Small Data,” 2021.

[25] O. Kwon, D. Kim, S.-R. Lee, J. Choi, and S. Lee, “Handling Out-Of-Vocabulary Problem in Hangeul Word Embeddings,” pp. 3213–3221, 2021.

[26] G. M. Foody, “Challenges in the real world use of classification accuracy metrics: From recall and precision to the Matthews correlation coefficient,” PLoS One, vol. 18, no. 10, p. e0291908, Oct. 2023, doi: 10.1371/JOURNAL.PONE.0291908.

[27] A. Power, Y. Burda, H. Edwards, I. Babuschkin, and V. Misra, “Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets,” Jan. 2022, Accessed: Jun. 13, 2025. [Online]. Available: https://arxiv.org/pdf/2201.02177

[28] A. I. Humayun, R. Balestriero, and R. Baraniuk, “Deep Networks Always Grok and Here is Why,” Proc Mach Learn Res, vol. 235, pp. 20722–20745, Feb. 2024, Accessed: Jun. 13, 2025. [Online]. Available: https://arxiv.org/pdf/2402.15555

[29] S. Takase, R. Ri, S. Kiyono, and T. Kato, “Large Vocabulary Size Improves Large Language Models,” Jun. 2024, Accessed: Jun. 13, 2025. [Online]. Available: https://arxiv.org/pdf/2406.16508

[30] M. Kunilovskaya and A. Plum, “Text Preprocessing and its Implications in a Digital Humanities Project,” 2021. doi: 10.26615/issn.2603-2821.2021_013.

[31] M. Siino, I. Tinnirello, and M. La Cascia, “Is text preprocessing still worth the time? A comparative survey on the influence of popular preprocessing methods on Transformers and traditional classifiers,” Inf Syst, vol. 121, p. 102342, Mar. 2024, doi: 10.1016/J.IS.2023.102342.

[32] S. Rezaei et al., “An experimental study of sentiment classification using deep-based models with various word embedding techniques,” Journal of Experimental and Theoretical Artificial Intelligence, Nov. 2024, doi: 10.1080/0952813X.2024.2384568;PAGE:STRING:ARTICLE/CHAPTER.

[33] “Why is BiLSTM better than LSTM ?. Know the underlying functionality | by Sourasish Nath | Medium.” Accessed: Nov. 09, 2025. [Online]. Available: https://medium.com/@souro400.nath/why-is-bilstm-better-than-lstm-a7eb0090c1e4