Federated Learning to Build Sentiment Analysis Models for Amazon Review Datasets without Labels

In Natural Language Processing, one of the most popular tasks is sentiment analysis which aims to predict the sentiment of a text. It has many practical applications in industries such as marketing and customer service. Performance of sentiment analysis models play a significant role to the success of these applications. To achieve a high accuracy, sentiment analysis usually trains analytical models based on labeled datasets, preferably to large-scale labeled dataset. However, large-size labeled dataset may not be available because of the high-cost in labeling. Therefore, researchers study alternative approaches aiming to learn high accurate and reliable models based on small-scale labeled datasets or using other existing labeled datasets from different categories. A centralized model is a machine learning model that utilizes a large dataset stored on a central server to perform sentiment analysis. Training a centralized model on a small, labeled dataset can result in inaccurate or incomplete predictions. While processing labeled datasets of different categories on a centralized platform, it also comes with many challenges such as data heterogeneity, bias towards the categories that are overrepresented, requirement of large amount of computational power and resources, and the availability of good amount of labeled data for training. In addition, it is difficult to select appropriate data categories to train a reliable model for the new category. In this thesis, we propose a federated learning approach to overcome these challenges. Federated Learning (FL) is a type of decentralized Machine Learning (ML) that lets us train data analytical models on local data without transferring data to a central server. When Federated Learning is applied to sentiment analysis, one server and multiple clients collaborate to train a reliable and accurate sentiment analysis model. In our scenario, each client trains a local sentiment analysis model based on a labeled review dataset of a specific category, and the server makes use of the FedAvg algorithm to aggregate the parameters from the trained client models to build a global model for the new category that has no available labeled dataset. We evaluate the performance of our design based on a prototype implementation using Amazon review datasets. Compared with the centralized sentiment analysis, the proposed FL-based sentiment analysis performance is 10% better. This validates the potential of federated learning in training better data analytical models for categories with no large-scale labeled datasets.