Journal article · EPJ Data Science · 11 June 2026 # Towards Operational Validation of LLM-Agent Social Simulations: A Replicated Study of a Reddit-like Technology Forum **Aleksandar Tomašević**¹, Darja Cvetković¹, Sara Major², Slobodan Maletić³, Miroslav Anđelković³, Ana Vranić¹, Boris Stupovski¹, Dušan Vudragović¹, Aleksandar Bogojević¹, Marija Mitrović Dankulov¹ ¹ Institute of Physics Belgrade, University of Belgrade · ² Faculty of Philosophy, University of Novi Sad · ³ Vinča Institute of Nuclear Sciences, University of Belgrade - [Paper](https://doi.org/10.1140/epjds/s13688-026-00674-x) - [Preprint](https://arxiv.org/abs/2508.21740) - [Code](https://github.com/atomashevic/voat-simulation) - [YSocial client](https://github.com/atomashevic/YClient-Reddit) - [Cite](#cite) **In short:** We ran 30 month-long simulations of a Voat technology forum, populated by language-model agents, and compared them with 30 matched 30-day windows of the real forum. The agents matched the forum’s size, daily activity and topics. They built a looser network than real users did, and put too much toxicity into opening posts and too little into replies. [](https://atomasevic.xyz/images/papers/llm-simulation-validation-figure.webp) **How the study works.** We calibrate the simulation on real Voat data, give each agent a short persona, run the forum for 30 days 30 times, and compare every run with real 30-day windows of the same forum. Figure 1 of the paper. ## Why this question? Researchers increasingly fill simulated social media platforms with language-model agents and watch what happens. Platform data are harder to get than they used to be, and a simulation can be rerun and changed. Yet most of these studies judge a simulation by whether its output looks believable, and nearly all of them report a single run. We wanted to test an LLM simulation the way agent-based modellers test any model: run it many times and compare the results with data from the real community it imitates. ## What we did We rebuilt v/technology, a technology forum on the Reddit-like platform Voat, inside [YSocial](https://atomasevic.xyz/software.html). We took population size, growth, churn, and posting and reply rates from ten 30-day samples of real v/technology data in the MADOC dataset. Each agent got a short persona: education, age, political leaning, two to five tech interests, an action budget and a toxicity propensity. Political leanings came from four right-of-center groups, to approximate Voat’s audience. The agents ran on Dolphin Mistral 24B, an uncensored model, because aligned models tend toward politeness and consensus. They had no memory of their own. Each prompt carried the persona and at most the three previous items in the thread. Discussions were seeded from a catalog of 1,000 links that real v/technology users had shared. - **30** simulation runs - **30** days per run - **30** real Voat windows - **5** validation dimensions We ran the simulated forum 30 times, for 30 days each, and compared every run with 30 separate 30-day windows of real Voat activity on five dimensions: activity, network structure, toxicity, topics and stylistic convergence. ## What we found **The size and rhythm of the forum matched.** Root posts (593 simulated, 569 on Voat), unique users (610 and 591) and daily active users (37.3 and 32.7) all had overlapping 99% confidence intervals. Agents wrote more comments, though (904 against 733), and threads ran a little longer. **The network had the right shape and the wrong details.** Both networks had a dense core and a sparse periphery, and their largest connected components were about the same size (406 and 414 users). But the core held 19.6% of that component in the simulation and 5.1% on Voat. Agents also came back to the same partner less often: 5.9% of simulated connections were repeated, against 8.7% on Voat. **Topics carried over almost completely.** For 99.6% of simulated topics we found a matching Voat topic. Privacy, electric vehicles, AI and Big Tech appeared in both. **Toxicity ended up in the wrong place.** Overall, the simulation was more toxic than Voat (mean toxicity score 0.143 against 0.119). The excess sat in opening posts. Simulated root posts were about 3.8 times as toxic as real ones, while simulated comments were slightly less toxic than real comments. Agents with a higher assigned toxicity propensity did write more toxic content, in all 30 runs. **Agents echoed each other only locally.** Replies reused the wording of the messages just before them, and this faded as earlier turns dropped out of the three-item context. We found no lasting stylistic convergence between agents. ## What it does not show v/technology is small, with short threads and low volume. That makes it an easy first test for agents without memory. We read the results as evidence of validity for a small community. On larger forums we expect the gaps in hubs, repeated interactions and long conversations to grow. With 30 runs, overlapping intervals mean coarse agreement, and smaller differences remain hard to detect. The toxicity comes from the personas and the uncensored model, so we do not claim it emerges from platform dynamics. Results also depend on prompt wording and on the choice of model, and toxicity and text similarity are measured with automated models that were trained on other data. ## Why it matters LLM agents placed in a realistic platform with simple rules reproduced the scale, rhythm and topics of a real forum. Where they diverged, we could trace the gap to a specific design choice, mostly the lack of memory. That makes the divergences useful, because they tell us what to change next. The same validation panel can serve as a baseline for experiments with feed rules or moderation, and for studies where people and agents share a platform. Abstract Validation of LLM-agent social simulations remains underdeveloped, with most studies relying on subjective assessments or single runs. We address this gap by running 30 independent 30-day simulations of a technology forum modeled on Voat’s v/technology, using stateless Dolphin Mistral 24B agents on the Y Social platform, and evaluating operational validity across five dimensions: activity patterns, network structure, toxicity, topical coverage, and stylistic convergence. Against 30 matched, non-overlapping 30-day Voat comparison windows, results show overlapping 99% confidence intervals for unique users, root posts, and daily active users, while comments, average thread length, and mean toxicity remain higher in simulation. Both simulated and empirical networks exhibit core–periphery structure, though simulated cores are larger and more diffuse and repeated interactions are less frequent. Topic alignment is near-complete, but toxicity is misallocated across content layers: simulated root posts are substantially more toxic than real submissions, while simulated comments are less toxic than Voat comments. These findings demonstrate that LLM agents in platform-faithful environments can reproduce familiar online regularities, while systematic divergences—particularly those linked to stateless agent design and content-layer calibration—point to concrete directions for future improvement. ## Cite **Tomašević, A.**, Cvetković, D., Major, S., Maletić, S., Anđelković, M., Vranić, A., Stupovski, B., Vudragović, D., Bogojević, A., & Mitrović Dankulov, M. (2026). Towards Operational Validation of LLM-Agent Social Simulations: A Replicated Study of a Reddit-like Technology Forum. *EPJ Data Science*. ``` @article{Toma_evi__2026, title={Towards operational validation of LLM-agent social simulations: a replicated study of a Reddit-like technology forum}, volume={15}, ISSN={2193-1127}, url={http://dx.doi.org/10.1140/epjds/s13688-026-00674-x}, DOI={10.1140/epjds/s13688-026-00674-x}, number={1}, journal={EPJ Data Science}, publisher={Springer Science and Business Media LLC}, author={Tomašević, Aleksandar and Cvetković, Darja and Major, Sara and Maletić, Slobodan and Anđelković, Miroslav and Vranić, Ana and Stupovski, Boris and Vudragović, Dušan and Bogojević, Aleksandar and Mitrović Dankulov, Marija}, year={2026}, month=June } ``` ## Related - [“Oh No, the Robots Are Here”: AI identity labels, social uptake and sanctioning in mixed human–AI communities](https://atomasevic.xyz/papers/ai-identity-labels.html) (PsyArXiv, 2026) - [Multi-Platform Aggregated Dataset of Online Communities (MADOC)](https://atomasevic.xyz/papers/madoc.html) (Proceedings of the International AAAI Conference on Web and Social Media, 2025) - [All publications](https://atomasevic.xyz/publications.html#2026-towards-operational-validation-of)