This paper presents an audit framework for measuring group-differentiated discourse on social media platforms, focusing on Reddit communities discussing generative AI in high school education. It develops a controlled experimental methodology with three algorithmic intervention strategies and evaluates their effects using synthetic data generation and statistical testing.
Key findings
Developed an audit framework for measuring discourse on generative AI in education.
Implemented three algorithmic intervention strategies: diversity boosting, topic balancing, and engagement quality ranking.
Baseline approaches achieved F1 scores around 0.6, with small non-significant effects on group diversity.
Significant effects observed on secondary metrics including topic diversity and engagement patterns.
Limitations & open questions
Algorithmic interventions showed small effects that were not statistically significant on group diversity.
Further research is needed to understand real-time intervention strategies in group-differentiated discourse.