Sentiment Classification: Review of Text Vectorization Methods: Bag of Words, Tf-Idf, Word2vec and Doc2vec

Authors

  • Haisal Dauda Abubakar Dept. of Computer Science, Jigawa State Colledge of Education, Gumel, Nigeria
  • Mahmood Umar 2Dept. of Computer Sceince, Faculty of Science, Sokoto State University, Sokoto, Nigeria
  • Muhammad Abdullahi Bakale Dept. of General Studies, Colledge of Agricuture and Animal Science Wurno, Sokoto,, Nigeria

DOI:

https://doi.org/10.56471/slujst.v4i1&2.266

Keywords:

Sentiment Classification, Text Vectorization Methods-Bag of Words, Tf-Idf, Word2vec and Doc2vec.

Abstract

In Sentiment Analysis, there are three (3) approaches namely, machine learning, lexicon-based and ruled based approaches. This study investigates on machine learning approaches which involves text vectorization or word embedding- an essential step in natural language processing tasks since most machine learning algorithms work with numerical input. Text vectorization involves the representation or mapping of words or documents of a corpus to numerical vectors of numbers or real numbers. There are several approaches in the literatures on document/text representation, however this study will focus on three (3) commonly used ones viz; Bag of words, TF-IDF, word2vec and doc2vec, and try to identify the reason behind that for review and recommendation to the researchers in hurry. Review of this study shows that TF-IDF feature vector representations generally outperforms other two (2) vectorization methods word2vec and doc2vec, specifically in book review sentiment classification. And therefore recommended for future studies in book review data set

References

Abubakar, H. D. (2020). A Scheme Of Pairwise Feature Combinations To Improve Sentiment Classification Using Book Review Dataset . Johor Bahru, Malaysia: Universiti Tecknologi Malysia Digital Librarry.

Al-Amin, M., Islam, M. S., & Uzzal, S. D. (2017). Sentiment Analysis of Bengali Comments with Word2vec and Sentiment information of words. 2017 International Conference on Electric, Computer and Communication Engineering(ECCE) (pp. 186-190). United States: IEEE.

Bilgin, M., & Senturk, I. F. (2017). Sentiment analysis on Twitter data with semi-supervised Doc2Vec. Conference: 2017 International Conference on Computer Science and Engineering (UBMK) (pp. 661-666). UMBK: IEEE.

Chen, Q., & Sokolova, M. (2018, May 1). Word2Vec and Doc2Vec in Unsupervised Sentiment Analysis of Clinical. Retrieved from https://arxiv.org/abs/1805.0035: https://arxiv.org/abs/1805.0035

Demidova, L., Klyueva, I., Sokolova, Y., Stepanov, N., & & Tyart, N. (2017). Intellectual Approaches to the Improvement of the Classification Decision Quarlity on the Base of the SVM Classifier. Procedia Computer Science, 222-230.

Doaa, M. E.-D. (2016). Enhancement Bag-of-Words Model for Solving the Challenges of Sentiment Analysis. International Journal of Advanced Computer Science and Applications (IJACSA), 7, 1.

Gulenko, A., Wallschlager, M., Schmidt, F., Kao, O., & & Liu, F. (2016). Evaluating machine learning algorithms for anomaly detection in clouds. Proceedings-2016 IEEE International Conference on Big Data, (pp. 2716-2721).

Gupta, G. (2021, September 27). introduction-to-data-mining-with-case-studies-by-gkgupta-11. Retrieved from DIVINE VASTU: https://divinevastu.net/introduction-to-data-mining-with-case-studies-by-gkgupta-11/

Huspi, D. S., Abubakar, H. D., & Umar, M. (2021). A Scheme of Pairwise Feature Combinations to Improve Sentiment Classification Using Book Review Dataset. International Journal of Innovative Computing, 12(1), 1. Retrieved from . Retrieved from https://ijic.ut

Jiang, L., Wang, S., & Li, C. &. (2016). Structure extended multinomial naive Bayes. Information Sciences.

Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed Representations of Words and Phrases and their compositionality. InProceedings (p. 26). Curran Associates, Inc.

Nawangsaria, R. P., Kusumaningruma, R., & Wibowoa, A. (2019). Word2Vec for Indonesian Sentiment Analysis towards Hotel . 4th International Conference on Computer Science and Computational Intelligence 2019 (ICCSCI) (pp. 360–366). Indonesia: Elsevier.

Prabhu, T. N. (2019, November 11). Understanding-nlp-word-embeddings-text-vectorization-1a23744f7223. Retrieved from Towards Data Science: https://towardsdatascience.com/understanding-nlp-word-embeddings-text-vectorization-1a23744f7223

Q, L., & T, M. ( 2014). Distributed Representations of Sentences and Documents. Proceedings of the 31st International Conference on Machine Learning (ICML-14) (pp. 1188-1196). Beijing: Scientific Research.

Srujan, K. S., Nikhil, S. S., Rao, H. R., Karthik, K., & Harish, B. S. (2018). Classification of Amazon Book Reviews Based on Sentiment Analysis. Nature Singapore Pte Ltd: Springer.

T., i., Sutskever, & K., I. C. (2013). Distributed Representations of Words and Phrases and Their Compositionality. . Proc of the 26th International Conference on Neural Information Processing Systems (pp. 3111-3119). USA: Curran Associates Inc.,.

Team, D. F. (2019, October 14). Data mining. Retrieved from Data Flair: https://data-flair.training/blogs/text-mining/

Tripathy, A., Agrawal, A., & & Rath, S. K. (2016). Classification of sentiment reviews using n-gram machine learning approach. Expert Systems with Applications,.

Trstenjaka, B., Mikacb, S., & Donkoc, D. (2014). KNN with TF-IDF Based Framework for Text Categorization. 24th DAAAM International Symposium on Intelligent Manufacturing and Automation, 2013 (pp. 356 – 1364 ). DAAM: Elsevier.

Yogapreethi, N., & S, M. (2016, August). A REVIEW ON TEXT MINING IN DATA. International Journal on Soft Computing (IJSC), 7, 2/3.

Zhang, X., & & LeCun, Y. (2015). Text Understanding from Scratch. Text Understanding from Scratch, 1–9.

Published

2022-07-20