Jump to content

Wikipedia:Undetected vandalism

Add links
Page semi-protected
From Wikipedia, the free encyclopedia

Undetected vandalism is vandalism that has remained on a page for a significant amount of time, often months or years. It is an enormous problem – perhaps the single most pressing problem – facing Wikipedia.

How widespread is it?

Donald Rumsfeld expounds upon the nature of Wikipedia vandalism.

There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones. - Donald Rumsfeld

Because undetected vandalism is, by definition, undetected, it is impossible to know how much there is. A fairly recent essay points out that vandalism to low-traffic pages often lasts for days; if only that were the extent of the problem! Wikipedia's records page paints a much more concerning picture. As of November 2024, the longest time that pages have been vandalized without anyone noticing is:

  • Longest undetected vandalism to a featured article: 14 years, 113 days (8 March 2010 to 29 June 2024)
  • Longest undetected vandalism to any article: 15 years, 29 days (4 January 2008 to 2 February 2023)
  • Longest undetected vandalism to a user talk page: 15 years, 82 days (8 October 2006 to 29 December 2021)
  • Longest undetected vandalism to a redirect page: 16 years, 159 days (24 April 2008 to 30 September 2024)
  • Longest undetected vandalism to a talk page: 20 years, 115 days! (27 July 2004 to 19 November 2024)

This should alarm you. More alarming is that undetected vandalism on Wikipedia is not necessarily:

  • Rare. The above cases are just a few of the thousands of known instances of long-undetected vandalism -– much of which is older than the typical Wikipedia vandal.
  • Subtle. Undetected vandalism is often of the usual profane, scatological, insulting, nonsensical, memey, or even defamatory sort.
  • Contained to obscure or small pages. Vandalism has gone undetected for months or years on high-profile pages including (but very much not limited to):

Good/Featured Articles

Vital articles

Other high-profile articles

Why is it bad?

The longer a piece of vandalism remains on Wikipedia, the more people have a chance to see it. A reader encountering vandalism that is profane, scatological, or otherwise prepubescent may be the single most embarrassing scenario for the project. And when that vandalism is insulting to or defamatory against a living person, potential legal issues may arise with speculation about someone's supposed criminal record, sexual predilections, or evil deeds -- as well as lesser but still distasteful insinuations about their physical appearance, their eating habits or their mom.

Wikipedia is also heavily scraped by external sources, used to train AI models, and reproduced in Google search engine summaries that themselves are heavily scraped and AI-generated. It is also a frequent source for journalists, the media, and book authors -- whether or not they admit to that. This can give a piece of vandalism false legitimacy as it leaps from blog posts to news articles to books and even academia, a process called "citogenesis." The more external sources repeat such a thing, the harder it is to identify its origin as vandalism, no matter how "obvious." What seems blatantly stupid in a diff might seem, to an outside blogger, like a quirky fun fact readers will love.

Wikipedia's lists of hoaxes and citogenesis citogenesis incidents are running documents of such worst-case scenarios -- as a hoax is just undetected vandalism at large scale. These include invented medical terms that made it into academic journals, falsehoods about politicians that made it to the Senate floor, and innocent men implicated in decades-old scandals.

How does it happen?

A lot of countervandalism work is done by recent changes patrollers. Some of this is assisted by monitoring plug-ins or automated bots and plug-ins with lists of "bad words" that filter edits by vandalism likelihood. However, these are prone to false positives as well as false negatives. The best-known bot, ClueBot NG instead applies machine learning algorithms to a dataset of known good/bad edits, but even these catch only about 40% of all vandalism (that we know about). And since these tools only patrol new edits, they cannot find vandalism that already exists.

Vandalism can also be obscured by well-meaning editors:

  • Expanding a page: the whole reason we're here, but also something that can make edits seem more trustworthy due to their age.
  • WikiGnomes copy-editing the article or bots making routine, automated edits, making text seem less obviously amiss.
  • Anti-vandals reverting some, but not all existing vandalism on a page. This may happen when two people vandalize a page in succession, and a user uses rollback on only one. It may also happen if vandals make several edits at once, or do a lot of vandalism in one edits.
  • Administrators or bots merging, splitting or translating articles, carrying vandalism with it and making its original source difficult or even impossible to find.

How can I find it?

A previously undetected gorilla

Here are some tips that, while not guaranteed to produce results, will probably produce a depressing amount of stuff.

Watchlists

No one can (or should) watch every page, but as seen above, a surprising amount of vandalism goes undetected on even Good and Featured Articles. These pages are heavily trafficked and edited, but most Wikipedia editors don't rewrite articles wholesale in one go, but make incremental changes. Much like the invisible gorilla from the classic psychological experiment, even the most blatant vandalism can go unnoticed if a reader's attention is elsewhere.

However, these pages are also heavily watched, which is where you come in. If you're watching or improving a page, read through others' edits to make sure nothing has gone awry. Don't assume a "typo fix" is OK – check it. Don't assume that tweak to the references is OK. Check it!

Check common vandalism targets

Certain pages attract more vandalism than others. Wikipedia has a list of most vandalized pages, but it is outdated, and most pages there are already heavily patrolled. Pages that frequently harbor undetected vandalism include:

  • Schools (middle school, high school, universities, districts, etc.) – these receive a near-constant stream of vandalism for obvious reasons, and it can be hard for editors to keep up.
  • Likewise, any author, book, or historical event heavily anthologized in school curricula.
  • Small-to-medium-sized towns (larger ones are likely more patrolled).
  • Celebrities, particularly current sportspeople or musicians

Search for common terms

To find the worst vandalism on Wikipedia, pretend you are a 14-year-old boy. What would you find hilarious to insert into Wikipedia? Profanity, gag names, Internet memes (especially older ones), insulting comments about your bros/your teacher/your crush, misspellings and non-English versions of any and all of them: the sky's the limit, the toilet's the nadir. If you'd prefer that someone else inhabit the internet troll mindset, several users have compiled lists of example common bad terms).

You can either use Wikipedia's built-in search or search engines like Google; you may find it helpful to use both:

  • For Wikipedia search, wildcards (the asterisk *) can be helpful to match many problematic word patterns at once. Regular expressions are sometimes helpful, although they are taxing on the server and likely to time out.
  • For Google, use a template like "site:wikipedia.org (potential vandalism term) -(term to exclude) -(term to exclude)".

Note: This work cannot be automated. This may be tempting in lieu of checking thousands or even tens of thousands of search results, but false positives abound. There are few words used in vandalism that do not have other, legitimate meanings. The ass has been domesticated since Biblical times; at least one man held the title of the Farter; some things do indeed come in 69s or 420s. You will probably collect a long list of false-positive terms (e.g., "poop deck" for "poop") to narrow the list down to a (relatively) manageable couple hundred. But even those results need to be checked one by one.

A downside to this method is that it cannot catch subtle vandalism, only the blatant kind (and any subtle vandalism included with it). But the blatant stuff is really bad.

Triage edit history

Before reverting any edit, always check to make sure it is actually vandalism. Unfortunately, for undetected vandalism, this usually means scouring the dozens, hundreds, thousands, or even tens of thousands of edits in article history. Checking every single diff would be masochistic even by anti-vandalism standards, so triage extensively:

  • Use tools like WikiBlame to search for the offending terms. Note: For articles with large amounts of edits going back many years, this tool may be slow or time out.
  • Check the last revision on an edit history page first, so you can move on to the next page if the vandalism still exists as of that version. (Note: This can be misleading for talk pages, which are often archived.) For more heavily edited articles, try setting revisions per page to 250/500, or changing the datestamp in the URL that appears when clicking "older edits": e.g., changing "20231003163313" to "20091003163313".
  • Look for edits that are roughly the same size as the text added/removed. For instance, if someone has inserted the word "crap" into an article, look for an edit of +5 characters (counting whitespace). This may not always track it down, as vandals often change other things, insert extra space, etc. while editing.
  • Look for editors who have only edited a page once or in one burst of small edits, or who use canned edit summaries.

A note on IPs

Not all IPs are vandals, but the vast majority of undetected vandalism is done by IPs or throwaway accounts. This is not an insult toward IP editors as a group, who do a lot of fine work (including anti-vandalism work). But the fact remains that when you see undetected vandalism, an IP probably did it. Thus, when triaging, make note of IP edits, or edits by throwaway-seeming accounts (no user page, few edits, trollish-looking username, already banned) and check those first. It will save you so much time.

Watch for red flags

Here are some small tip-offs that something questionable may be undetected vandalism:

  • Extraneous whitespace or line breaks; vandals will often insert extra newlines. Sometimes this extra space is only visible in the edit pane. (Example)
  • Broken headers, templates, refs, etc; vandals often put their vandalism in places that will break wiki markup. (Example)
  • Words, especially in references or links, that are capitalized differently from their surroundings. (Example 1, Example 2)
  • Wikilinks that go to drastically different places than the text. Vandals will often change the text but not the link. (Example)
  • Dubious-seeming information in a list or table that is out of alphabetical, numerical, or chronological order; vandals often do not bother to rearrange the entries. (Example 1, Example 2)
  • Information in infoboxes that contradicts article text, or vice versa. Vandals will often change one, but not the other; alternately, well-meaning editors might fix one but miss the other. This is particularly useful for statistics like census or school enrollment figures that can be tedious to fact-check. (Example 1, Example 2)
  • Questionable English text inserted into an article (non-enwiki), paragraph, title, or list in another language. (Example)

Look for the second instance

Editors and administrators do not always check vandals' other edits – even those who have been banned! If you find vandalism, check the user's edit history for more. Note:

  • The "Reverted" tag is useful for triaging, but it has false negatives, especially for reversions that didn't happen right away.
  • The older the vandalism, the more likely that the vandal's other edits will have already been reverted or rewritten.
  • A lot of vandalism comes from shared school IPs. Their edit history is likely the work of dozens or hundreds of different people, many of whom make legitimate edits. Sometimes you'll see a spate of edits at the same time; these are likely (but not always) by the same person, and thus good candidates to check.

Other projects

Querying the wikipedia.org domain on Google will include multiple Wikimedia projects and wikis. Here you must be even more wary of false positives, as some questionable English words are perfectly normal in other languages. Also, Google may eventually rate-limit your results or continually prompt you for a CAPTCHA, because searching Wikipedia for endless variations of the word "poo" is kind of a weird thing for humans to do. But this can be an efficient way to catch things that Wikipedia's search misses.

When you really can't find it

Sometimes you'll see an edit that is almost certainly vandalism but doesn't seem to originate anywhere in the article's history, or originated in a legitimate-seeming edit. A couple things might have happened:

  • It was an honest mistake or typo; when possible, assume good faith.
  • The article was machine translated from another wiki, introducing errors. This is unsightly and should be fixed, but it is not malicious vandalism. Note: If a machine translation has one obvious error, it probably has more; the whole thing may need a rewrite.
  • If a page has been split, merged, or translated, the vandalism may have happened to the original page name. The only way to find it is to track that page down and dig through its history too. In these cases, put the diff of the vandalism in the edit summary to prevent others from having to do the same digging. (Example).
  • If an image, category description, or infobox has suspicious-looking text, the likely culprit is vandalism to its counterpart on Wikimedia Commons (for images) or Wikidata (for infoboxes/etc).
  • If revision deletion has been applied to many edits to a page (and you don't have permission to view it), the vandalism might be in one of those. Again, if you can fix it easily, be bold and do so.

Organize!

WikiProjects such as the Counter-Vandalism Unit, while not as active as they used to be, still exist; however, they primarily focus on recent edits. Significant support exists, at least in theory, for better tooling, a task force, or a whole WikiProject targeting undetected vandalism. If you're reading this, you can help!

Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia:Undetected_vandalism&oldid=1259801775"