A Semantically Oriented Feature Engineering Approach for Multi-Class Imbalanced Text Classification

Abstract

This thesis investigates the problem of multi-class imbalance in text classification, where conventional learning approaches tend to favour majority classes and often fail to represent minority-class patterns effectively. This limitation is especially important in real-world textual applications, where minority classes may contain the most informative, sensitive, or socially significant signals. To address this problem, the thesis proposes STREAM (Semantically Oriented Vector Based Model), a feature-engineering methodology that combines Pointwise Mutual Information and Semantic Orientation to construct class-sensitive document representations. In the cyberbullying setting, the model is further enriched with predicted user-related features to incorporate contextual information that may support more effective discrimination between classes.

The empirical investigation is carried out using a unified, fold-safe evaluation framework designed to avoid data leakage and support fair comparison across competing approaches. The updated methodology uses nested stratified cross-validation for model selection and performance estimation, followed by separate stratified holdout evaluation to assess generalisation on unseen data. Weighted F1 and Cohen�s kappa are used as the principal evaluation measures, supported by additional metrics for complementary interpretation. In settings where the full optimisation procedure was computationally prohibitive across all datasets and classifier combinations, a standardised emergency-mode protocol was applied to reduce optimisation complexity while maintaining methodological consistency, class stratification, and strict separation between training, validation, and testing stages.

The proposed methodology is evaluated on multiple heterogeneous datasets, including cyberbullying severity detection, New York Times emotional tone classification, Yelp review rating prediction, Reuters topic classification, and 20-Newsgroups categorisation. The results show that semantically oriented feature construction provides a meaningful and interpretable contribution to imbalanced text classification, although the magnitude of improvement varies across domains. STREAM performs particularly strongly on the Cyberbullying, Reuters, and 20-Newsgroups datasets, where it achieves the best or among the most stable results under the reported evaluation settings. On the New York Times and Yelp datasets, STREAM remains competitive and consistently improves over Word2Vec-based alternatives, while strong Bag-of-Words baselines, particularly BoW+SVM, remain highly effective and in some cases superior. These findings indicate that semantically informed feature engineering is especially beneficial when class distinctions are closely aligned with underlying semantic structure, while also showing that strong lexical baselines continue to offer robust performance in some domains.

Overall, the thesis demonstrates that semantically oriented feature engineering offers a robust, interpretable, and practically useful approach to multi-class imbalanced text classification. More broadly, it highlights the importance of rigorous evaluation design, leakage-free experimentation, and careful methodological control when assessing classification models on imbalanced textual data.