Journal article · EPJ Data Science · 24 February 2023

Sustainability of Stack Exchange Q&A communities: The role of trust

Ana Vranić1, Aleksandar Tomašević2, Aleksandra Alorić1,3, Marija Mitrović Dankulov1

1 Institute of Physics Belgrade, University of Belgrade · 2 Department of Sociology, Faculty of Philosophy, University of Novi Sad · 3 Two desperados, Belgrade

In short: We compared Stack Exchange sites on the same topic where the first attempt closed and the second survived. Activity counts in the first months did not tell them apart. The survivors had tighter local clusters, a core of members that stayed put, and higher trust among those core members.

Visual abstract. Questions, answers and comments between four users become links in an interaction network, which is rebuilt over sliding 30-day windows. A bar chart of one user's daily activity sits above a line showing that user's dynamic reputation over 180 days: it jumps after each burst of activity and decays during quiet spells. On the right, two networks: a closed Literature community with a small, loosely connected core, and an active Literature community with a few large, high-reputation core members linked to many peripheral members.
From interactions to trust. Each question, answer or comment links two users. We rebuild these networks for every 30-day window and give each user a reputation that grows with activity and fades with silence. Node size shows reputation; green nodes are the core. Figure 1 of the paper.

Why some communities last

Stack Exchange is a network of question-and-answer sites, each on its own topic. Each new one goes through a public beta, and Stack Exchange then decides whether it graduates, stays in beta or closes. Several topics have had two tries: the first site closed, the second one took off. That gives a natural comparison. Same topic, same platform rules, different outcome. We wanted to know what in the early life of a site separates the two.

The usual answer is activity. Stack Exchange itself reviews beta sites by questions per day, answers per question and the share of questions answered. In our data these numbers often did not point the right way. The closed Astronomy site answered 95% of its questions in the first 180 days and the active one 96%, and the closed site had more answers per question (2.02 against 1.49).

How we measure trust

Stack Exchange's own reputation score only goes up, and the public data only give its final value. So we used a dynamic reputation model built for these communities. Every question, answer or comment adds a point. Bursts of activity on consecutive days add more. Each day without activity, a member's reputation shrinks a little. A member who stops posting fades back towards zero within weeks, and anyone below 1 counts as inactive. We read this score as a proxy for how much the community can rely on that person at a given moment.

What we did

We took four topics with a closed and an active site: astronomy, literature, economics, and physics paired with the closed Theoretical Physics site. We added Mathematics, Mathematica and two closed startup sites as a check.

  • 4matched pairs of closed and active sites
  • 4extra sites as a robustness check
  • 180days of early life per site
  • 150network snapshots per site

For each site we linked two users whenever one answered, commented on or accepted the other's post. We built a network for every 30-day window, sliding one day at a time over the first 180 days. In each network we measured how tightly neighbours were connected, split members into a core and a periphery, and tracked the reputation of each group.

What we found

Survivors were more tightly knit. For most of the period, the active site in each pair had more closed triangles among neighbours than its closed twin, relative to a random network of the same size. Active sites stayed above five times the random level.

Their cores held together. Core membership changed more from month to month in closed sites. In active sites the core was also better connected to occasional members, which suggests a more inclusive community.

Their cores were trusted. Mean reputation among core members was always higher in the active site than in the closed one, and mean reputation across all active members was higher most of the time. The closed Startup Business site is a telling case. It gained active users faster than Physics, but its core members had reputation levels like those of the closed sites, and it closed too.

Limitations

Pairs of closed and active sites on the same topic are rare, so the sample is small and outliers are possible. Astronomy is one: in the first months the closed site looked stronger on several measures. The reputation model treats every interaction as positive and needs its parameters tuned. It suits Stack Exchange, where exchanges are mostly unemotional, and would need adapting for more heated communities. Our networks ignore direction and repetition of contact, and other network choices could call for other core-detection methods.

Why it matters

Volunteer knowledge communities depend on people giving time for no material reward. Our results suggest that counting questions and answers in the first months is not enough to judge whether a site will last. What seems to matter is whether a stable group of trusted members forms early and keeps engaging with newcomers. Early network snapshots alone can also mislead, so following how structure and trust change over time tells more.

Abstract

Knowledge-sharing communities are fundamental elements of a knowledge-based society. Understanding how different factors influence their sustainability is of crucial importance. We explore the role of the social network structure and social trust in their sustainability. We analyze the early evolution of social networks in four pairs of active and closed Stack Exchange communities on topics of physics, astronomy, economics, and literature and use a dynamical reputation model to quantify the evolution of social trust in them. In addition, we study the evolution of two active communities on mathematics topics and two closed communities about startups and compare them with our main results. Active communities have higher local cohesiveness and develop stable, better-connected, trustworthy cores. The early emergence of a stable and trustworthy core may be crucial for sustainable knowledge-sharing communities.

Cite

Vranić, A., Tomašević, A., Alorić, A., Mitrović Dankulov, M. (2023). Sustainability of Stack Exchange Q&A communities: The role of trust. EPJ Data Science, 12(1), Article 4. https://doi.org/10.1140/epjds/s13688-023-00381-x

@article{Vrani__2023, title={Sustainability of Stack Exchange Q&A communities: the role of trust}, volume={12}, ISSN={2193-1127}, url={http://dx.doi.org/10.1140/epjds/s13688-023-00381-x}, DOI={10.1140/epjds/s13688-023-00381-x}, number={1}, journal={EPJ Data Science}, publisher={Springer Science and Business Media LLC}, author={Vranić, Ana and Tomašević, Aleksandar and Alorić, Aleksandra and Mitrović Dankulov, Marija}, year={2023}, month=Feb }

Related