Reddit faces content quality concerns after its Great Mod Purge

Did you know that improper food canning can lead to death? Botulism—the result of bacteria growing inside improperly treated canned goods—is rare, but people can die from it. In any case, they'll certainly get very ill.

The dangers of food canning were explained to me clearly, succinctly, and with cited sources by Brad Barclay and someone going by Dromio05 on Reddit (who asked to withhold their real name for privacy reasons). Both were recently moderators on the r/canning subreddit and hold science-related master's degrees.

Yet Reddit removed both moderators from their positions this summer because the mods refused to end r/canning's protest against Reddit and its new API fees; the protest had made the entire subreddit "read only." Now, the ousted mods fear that r/canning could become subject to unsafe advice that goes unnoticed by new moderators. "My biggest fear with all this is that someone will follow an unsafe recipe posted on the sub and get badly sick or killed by it," Dromio05 told me.

Reddit's infamous API changes have ushered in a new era for the site, and there are still questions about what this next chapter will look like. Ars Technica spoke with several former mods that Reddit booted—and one who was recently appointed by Reddit—about concerns that relying on replacement mods with limited subject matter expertise could result in the spread of dangerous misinformation.

Questions about replacement mods’ expertise

When Reddit announced it would abruptly start charging significant fees for access to its API, many third-party Reddit apps announced they would close (and many have). Some Reddit users, including mods, also quit Reddit. In addition, Reddit revoked the mod badges from long-time moderators and subsequently sought replacements, though some expelled mods worry that the replacements weren't carefully selected or trained.

Barclay told me he moderated r/canning for three years before Reddit nuked his badge. He noted various canning misconceptions, from thinking the contents of a concave lid are safe to eat to believing you don't need to apply heat to food in jars.

He claimed that some new r/canning moderators appointed by Reddit had previously shown a lack of canning expertise before getting the new volunteer gig. For example, Barclay pointed to one mod recommending "citizen science," saying they would use a temperature data logger to "begin conducting experiments to determine what new canning products are safe." Reddit later made that user an r/canning mod.

Dromio05 showed me several posts he deemed questionable since Reddit took away his own mod badge. For example, this post shares a link to an article about "rebel canners," which Dromio05 argues "gives a public platform to people who openly encourage methods and recipes that are known to be unsafe, like canning milk and open kettle canning." The post is labeled unsafe, but Dromio05 would have removed the link to the article.

Another cited example is this recipe for canned sauce. It includes already-canned tomatoes, which experts like the National Center for Home Food Preservation (NCHFP) recommend against, as there's no safe tested process for this. The recipe also includes nuts, though the USDA doesn't have any recommendations for canning nuts, and NCHFP and other experts advise against canning any nuts besides green peanuts.

The new mod team for r/canning declined to comment on this story.

There are many schools of thought around any broad topic such as canning, of course, and the former canning mods I spoke with had slightly different standards for whether or not a post could be considered safe enough for the subreddit. What's critical for Reddit's content quality is not that moderators adopt identical philosophies but that they are equipped to facilitate healthy and safe discussions and debates that benefit the community.

Newly implemented replacement mods represent a fraction of the over 50,000 moderators Reddit says it has. But the hastiness with which these specific replacement mods were ushered in, and the disposal of respected, long-time moderators, raises questions about whether Reddit prioritized reopening subreddits to get things back to normal instead of finding the best people for the volunteer jobs.

Reddit mods are “rarely experts”

Reddit also recently axed r/homeautomation moderators, including Dan (full name withheld because of privacy concerns), who told me that he moderated the subreddit for about seven years. He claims his credentials include networking experience from decades in IT, plus his "extensive smart home" and instruction from a "master electrician." Like the replaced r/canning mods, Dan fears dangerous misinformation taking hold at r/homeautomation, especially when it comes to electricity.

I reached out to r/homeautomation's new moderators, and one responded, requesting anonymity. The mod said their credentials include heavy interest in home automation, "specifically Home Assistant," for the past two years, and frequent home automation conversations outside of Reddit, including with friends in related fields, like electronics, A/V equipment, HVAC systems, and cybersecurity.

The moderator contended that subject matter expertise isn't critical for proper moderation, saying:

A moderator on Reddit is probably rarely an expert on the topic they're moderating over. Instead, they're a passionate member of the community that sees the value in sharing information between community members, and we'll all work together to make sure someone doesn't touch a live wire.

In home automation specifically, it's unlikely you'll find a mod who's both a certified electrician and a cybersecurity expert, the new r/homeautomation mod noted.

Dan, via email, said, "I would argue that while a mod doesn't need to be a subject matter expert, they should at least have above-average knowledge/experience with the topic they are moderating."

In response to concerns that the new r/homeautomation mod team could overlook posts with dangerous misinformation, the anonymous Redditor pointed me to the subreddit's sidebar, which has a disclaimer about the dangers of electricity. However, the disclaimer is only visible on old Reddit. The mod doesn't know why.

The anonymous home automation mod added that "there's been no change in terms of the safety of the information on r/homeautomation."

"As before, we promptly deal with submissions and comments that are inappropriate, including ones that spread false and potentially dangerous information. We lean on tools such as AutoModerator to [flush] out malicious links and spam," the mod said.

When asked for comment, Reddit's director of corporate, policy, and safety communications, Gina Antonini, said via email:

Our platform-wide Content Policy prohibits many kinds of harmful content and other forms of manipulated content. In addition, Reddit’s unique approach to moderation means users and moderators are empowered to set and enforce the rules of their own communities, which can be even stricter than our Content Policy. Users are also empowered to “upvote” and “downvote” content, which affects how visible a piece of content becomes. We find that our deeply engaged users are quick to challenge and downvote harmful or misleading content across the platform.

Questionable vetting process

One of the top complaints I've heard about the Great Reddit Mod Purge is the company's alleged disregard for replaced mods' expertise. The swift, contentious nature of the mod replacements meant that old mods often didn't share advice with new mods. Meanwhile, the users Reddit chose to replace protesting mods may not have been properly vetted.

That includes one of the new mods of the 3D-printing-focused subreddit r/ender3, who requested to only be referred to as the subreddit's top moderator. This person replied to a post by the Reddit employee going by u/ModCodeofConduct and requested to mod the subreddit as a "joke," they said. The user got the job despite telling me:

I have never touched a 3D printer in my life, and there is zero activity on my Reddit account related to 3D printing.

The mod said, though, that most of the other mods are "serious 3D printer people" and are good at moderating, even though the mod team isn't yet "cohesive."

For what it's worth, that mod will step down eventually, "as the joke is starting to wear off." But the story suggests that new mods weren't selected with the utmost care.

So who did that "joke" mod replace? Among r/ender3's Reddit-sacked mods was a user going by the name Deskparser (real name withheld for privacy). Deskparser modded r/ender3 for about four years, the user told me, including ensuring the validity of advice around fire hazards and fire management.

Deskparser's work also involved moderating content around 3D-printing guns, following a rule that permitted non-functional "display and cosplay pieces," Deskparser said.

"Enforcing that rule required attention every week, often several times a week, due to users not really knowing what a truly 'functional' firearm print looked like, which we did get quite a few of," Deskparser explained.

That doesn't seem like the type of thing the subreddit's new joke moderator can spot or even something that Reddit considered when appointing r/ender3's new mod team.

Reddit declined to provide a detailed explanation of how it selected this recent group of moderators to replace protesting ones. The company has a typical process for selecting mods and doesn't seem to have strayed from that with this recent batch of replacement mods.

Lost knowledge

None of the forcibly removed mods I spoke with have worked with or plan to work with replacement mods to pass on knowledge gained through years of experience.

Sarah Gilbert is a postdoctoral associate at Cornell University and research director of Cornell's Citizens and Technology Lab; she studies content moderation, online communities, and research ethics. She's also a moderator for r/askhistorians, a subreddit known for complex modding systems (r/askhistorians is not one of the subreddits with moderators removed by Reddit).

"When you remove entire mod teams, that institutional knowledge is just gone in a flash," Gilbert said.

She added that while knowledge can be rebuilt, it won't be the same, and mod teams recently appointed by Reddit face challenges that most new mods wouldn't.

Dan, whom Reddit relieved of his r/homeautomation mod duties, said, "The sub(s) will carry on, but with lost knowledge, experience, and, above all, love."

In addition to lost knowledge, new and old mods are also dealing with the loss of third-party apps considered helpful for moderating. Like many former and current moderators I've spoken with, Stevie Chancellor, an assistant professor in computer science and engineering at the University of Minnesota who uses big data for research on how people discuss mental health on social media, is concerned that Reddit's API rule changes have left moderators with fewer tools to do their jobs, which can result in overlooked content.

"For mental health, this is really important because people can say hurtful things ... (like calling someone a coward when they have depression) or recommending dangerous or unsafe advice on how to treat or cure their mental illness," she said. "In very rare but extreme cases, some people go to mental health support forums to provoke people to hurt themselves for their own perverted desires. Mod tools make dealing with this content faster and easier, and not having robust mod tooling damages mods' ability to combat these harmful behaviors."

Reddit has shared plans to update its mod tools to include accessibility-focused updates and new desktop tools, such as building macros and updating the Mod Queue interface so more items are visible simultaneously. But many long-time moderators remain discontent, and some of the updates haven't materialized yet. Most recently, though, Reddit announced the Mod Helper Program reward system to encourage moderators to share advice with each other. It's a step toward showing mods a smidge more appreciation and encouraging expertise sharing. Of course, it's unlikely that ousted mods will participate.

Reddit-based research jeopardized

As some question the safety of information shared on Reddit following the forced and voluntary exits of longtime mods, others worry about access to Reddit data for research on important topics related to physical safety.

A 2021 study (PDF) entitled "Studying Reddit: A Systematic Overview of Disciplines, Approaches, Methods, and Ethics" helps illustrate how Reddit can be a valuable resource for scientific studies on medical sciences, drug abuse and alcoholism, public health and safety, and women's health, as well as for studies in the social sciences, like psychology, criminology, and law enforcement. The study notes that, like Twitter, Reddit was starting to become a "model organism" for academic study, due to Reddit's open API fostering easy data collection and Reddit users responding to current world events.

"Reddit’s subreddit structure means that finding relevant research data can be easier than on Twitter, and in contrast to the character limits of Twitter, Reddit offers researchers a qualitative and quantitatively more expansive dataset," it explains, pointing to mental health, depression, and eating disorders as some of the most common topics for Reddit-based research.

Researchers Ars spoke with for this story feel that Reddit's new rules for how researchers and academics can get free or discounted access to Reddit's API are vague. They say that despite the existence of a form for researchers to request API access, actually getting access is harder than it was before.

Reddit declined to say how many research-related API pricing exemptions it has granted since it started charging for API access.

Gilbert said:

Requiring a form is itself problematic because it introduces a system where the platform is gatekeeping what research can be conducted and what research cannot, and currently there's no transparency about Reddit's decision-making process. Who can access it, for example? While they might automatically allow anyone associated with a higher education institution access, what about other groups doing important research, such as data journalists and civil society groups? ...

Some gatekeeping isn't a bad thing—we found examples of studies that I think most people would be uncomfortable with and many would even consider unethical. However, having Reddit—or any big tech platform—in that role creates conflicts of interest that could limit science (including science with goals to prevent or address various kinds of harm).

Reddit's policy says that those who want to use the API for "research in excess of rate limits" need a separate agreement with Reddit. Reddit says its API may be used for research, "provided you use it exclusively for academic (i.e. non-commercial) purposes, don’t redistribute our data or any derivative products based on our data (e.g. models trained using Reddit data), credit Reddit, and anonymize information in published results."

Reddit says it doesn't charge researchers "in most cases" and decides if it will charge researchers on a "case-by-case basis." An example Reddit provides for when it might ask for money is when there's a lot of data involved.

Chancellor's research includes the use of big data sets from Reddit to understand how people talk about severe mental illness and related behaviors, like suicidal thoughts and self-injury.

She said:

Barriers to API access and slower data crawls means that our studies can't run. That means we can't do research on understanding what's going on on Reddit, let alone what kinds of positive interventions could be developed to help people who are looking for social support and help.

Pushshift

The nonprofit Network Contagion Research Institute's Pushshift service archives all Reddit posts, making it popular among moderators and researchers. Chancellor and her team, for example, have relied on Pushshift for data collection related to studies on opioid addiction and recovery, suicidal crisis conversations, and eating disorder behaviors. While Reddit moderators can gain Pushshift access by requesting a single-use token, researchers haven't been able to access it since Reddit's API rule changes took effect.

A July survey of 118 participants, including some health researchers, authored by Gilbert and others, highlighted the detriment to research projects this can have. (Note: Gilbert told me the survey was originally taken to help negotiate with Reddit around API changes rather than for distributing a proper survey. Because of that and due to a lack of consent, there aren't many publicly available details about the survey's participants.)

"Data archives like Pushshift provided needed functionality that the API does not provide, for example, historical Reddit data that could be queried by timeframe," the report says.

A survey respondent is quoted as saying that without Pushshift, they couldn't practically gather the necessary volume of data required for their research, which focuses on people with addictions who aren't in a formal recovery program.

"Qualitative research in this area is overwhelmingly retrospective, with much of the existing qualitative enquiry undertaken many years after individuals initiated recovery," the participant said. "Drawing data [via Pushshift] would have enabled insight into these early stages with no additional risks to participants’ recovery processes. Without that access, the entire purpose of this research and any benefits it could have provided will disappear."

Other topics of research reportedly hindered by researchers losing access to Pushshift include racism, radicalization, gender, and empathy, the July survey found.

Reddit declined to comment on Pushshift access for researchers. But the Network Contagion Research Institute has said it's interested in working with Reddit more to find a way to make Pushshift accessible to researchers.

New Reddit, new normal

This is just a snapshot of a tumultuous period in Reddit's history, and there are many subreddits whose mod team wasn't altered by Reddit. While the company has its work cut out for it in ensuring quality content and moderators, it's too early to make a broad statement about whether Reddit's content quality has suffered since its API changes and the loss of some mods and power users.

I don't doubt that there are replacement moderators who have good intentions and strong expertise in the subreddit topic they're moderating. Even the Redditor who ended up with r/ender3 as a joke told me they want to help rebuild the subreddit's mod team so that appropriate mods are on top.

But amid the urgency to bring Reddit "back to normal," Reddit may have appointed some users to moderate communities about topics they're not experts in. Some argue that expertise isn't critical to good moderation, and it will be interesting to see over the coming months if that's true. Replacement mods could certainly learn about their subreddit topics or even seek the counsel of other mods who are more well-credentialed. But the potential for Reddit to see an increase in misinformation remains. And as with any social media platform or forum, advice on topics with important nuances should be taken with a grain of salt and further researched before being acted upon.

In any case, Reddit's approach to replacement mod appointments has further damaged community trust in Reddit and confidence in the company prioritizing the user experience. Adding to that concern are new obstacles for researchers hoping to tap Reddit as a data source. It's debatable if Reddit should be obliged to provide free API and Pushshift access to research from which it doesn't directly benefit. But providing more transparency on research-related API fee exemptions would go a long way toward showing compassion for researchers' work, which could be valuable for the world beyond Reddit.

Reddit has a reputation for providing unique content that you can't find elsewhere, from real human advice to insight on how humans behave. It's now up to the company, refreshed mod team, and remaining users to ensure New Reddit continues earning that mark.

Advance Publications, which owns Ars Technica parent Condé Nast, is the largest shareholder in Reddit.