Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Tasks

Link:https://arxiv.org/pdf/2304.10145.pdf

Abstract

We use ChatGPT to relabel 5 seminal datasers covering stance detectionn (2x), sentiment analysis, hate speech, and bot detection and prove it has the potential to annotate data with an average accuracy 0.609 and highest accuracy 0.649 on sentiment analysis.