Evaluation of TF-IDF and Word2Vec Representations Using Support Vector Machine for Multiclass Product Category Classification of Tokopedia Reviews
Abstract
As electronic commerce continues to expand, customer reviews have become an important source for automated text classification. This research evaluates and compares TF-IDF and Word2Vec representations for multiclass product category classification with Support Vector Machine (SVM). The experiment used the Tokopedia Product Reviews 2025 dataset containing 65,543 reviews across six product categories. After text preprocessing and keyword-based text refinement, 11,239 relevant reviews were retained. The dataset was divided into 80% training sets and 20% hold-out test sets. The reviews were transformed using TF-IDF and Word2Vec, while SMOTE was applied to the training data to handle the imbalance among product categories. Classification performance was assessed through five-fold cross-validation and a hold-out test set based on accuracy, macro precision, macro recall, and macro F1-score. Five-fold cross-validation showed that TF-IDF achieved mean accuracy, macro precision, macro recall, and macro F1-score of 95.95%, 93.13%, 92.78%, and 92.93%, respectively, compared with 90.80%, 84.58%, 86.02%, and 85.19% for Word2Vec. On the hold-out test set, TF-IDF achieved 95.42% accuracy and 91.88% macro F1-score, while Word2Vec achieved 91.01% and 84.91%, respectively. The Wilcoxon signed-rank test yielded a p-value of 0.0625, indicating no statistically significant difference at the 5% significance level. Nevertheless, TF-IDF consistently achieved higher descriptive performance than Word2Vec.
Related
- Informative Label Missingness in Multiclass Classification Information Geometry and Excess Risk
- Private and Distributionally Robust Federated Multiclass Classification
- Optimal aggregation in multiclass classification
- Multiclass Classification with Rare Useful Features: Fundamental Limits and the Optimality of Diversity Pursuit Higher Criticism Thresholding
- The classification of taut surface singularities via representations from logarithmic vector fields
BibTeX
@article{septian2026evaluation,
title = {{Evaluation of TF-IDF and Word2Vec Representations Using Support Vector Machine for Multiclass Product Category Classification of Tokopedia Reviews}},
author = {Anggi Dwi Septian and Primandani Arsi and Pungkas Subarkah},
journal = {Journal of Information Systems and Informatics},
year = {2026},
month = oct,
doi = {10.63158/journalisi.v8i5.1820},
url = {https://doi.org/10.63158/journalisi.v8i5.1820},
note = {CC BY},
}
RIS
TY - JOUR
TI - Evaluation of TF-IDF and Word2Vec Representations Using Support Vector Machine for Multiclass Product Category Classification of Tokopedia Reviews
AU - Anggi Dwi Septian
AU - Primandani Arsi
AU - Pungkas Subarkah
PY - 2026
DA - 2026/10/24
JO - Journal of Information Systems and Informatics
DO - 10.63158/journalisi.v8i5.1820
UR - https://doi.org/10.63158/journalisi.v8i5.1820
AB - As electronic commerce continues to expand, customer reviews have become an important source for automated text classification. This research evaluates and compares TF-IDF and Word2Vec representations for multiclass product category classification with Support Vector Machine (SVM). The experiment used the Tokopedia Product Reviews 2025 dataset containing 65,543 reviews across six product categories. After text preprocessing and keyword-based text refinement, 11,239 relevant reviews were retained. The dataset was divided into 80% training sets and 20% hold-out test sets. The reviews were transformed using TF-IDF and Word2Vec, while SMOTE was applied to the training data to handle the imbalance among product categories. Classification performance was assessed through five-fold cross-validation and a hold-out test set based on accuracy, macro precision, macro recall, and macro F1-score. Five-fold cross-validation showed that TF-IDF achieved mean accuracy, macro precision, macro recall, and macro F1-score of 95.95%, 93.13%, 92.78%, and 92.93%, respectively, compared with 90.80%, 84.58%, 86.02%, and 85.19% for Word2Vec. On the hold-out test set, TF-IDF achieved 95.42% accuracy and 91.88% macro F1-score, while Word2Vec achieved 91.01% and 84.91%, respectively. The Wilcoxon signed-rank test yielded a p-value of 0.0625, indicating no statistically significant difference at the 5% significance level. Nevertheless, TF-IDF consistently achieved higher descriptive performance than Word2Vec.
SN - 2656-5935
SN - 2656-4882
ER -