Shame on MIT!
Earlier this year the Massachusetts Institute of Technology (MIT) Sloan School of Management published a paper entitled, “Rethinking the Cybersecurity Arms Race When 80% of Ransomware Attacks are AI-Driven” that, to be honest, I didn’t recall hearing much about until recently when well-known security researcher Kevin Beaumont questioned MIT’s findings.
Now, let’s start by saying that most people don’t have the guts to publicly question esteemed institutions like MIT. There is more risk than reward in calling out research from well-known & well-respected institutions. Unless someone is just looking for attention, being correct is literally the only protection from scorn. Well…Kevin was correct in his assertions that the MIT report was critically flawed. Part of the failures in that paper likely came down to editing, with terms like “AI-driven,” “AI-powered,” & “AI-generated” being used interchangably with the much softer “AI-enhanced” & “AI-enabled.” While it appears that most (if not all) of those references are false, the earlier batch that suggest AI as the driving force instead of a helpful tool are likely what brought negative attention to this paper from the start.
Once experts began to review the paper, some illogical claims undermined the entire premise. For instance, while claiming to be data-driven, & reporting on a pool of data from 2,800 ransomware attacks, MIT supported their assertion that 80% of ransomware attacks are AI-driven with examples that included the WannaCry & NotPetya attacks. Both of those attacks were carried out in 2017, which was more than five years prior to the public launch of ChatGPT! For MIT’s conclusions to be accurate, one would have to believe that criminal threat actors (who are, by definition, motivated by money) had a multi-year headstart on OpenAI & decided to use their technological leap to create malware…a skill that the current generation of AI platforms STILL cannot accurtely claim to have mastered.
As the public discussion of this paper continued, Marcus Hutchins (the man credited with actually stopping WannaCry in 2019) pointed out on LinkedIn that MIT “claim(ed) to have analyzed attacks across 2023-2024, but several ransomware groups they cited as ‘definitely using AI’ died out prior to 2023. One even died out before the first GPT model was released.” Now, aside from the audacity of MIT making such a claim, if you clicked on that LinkedIn link that I just provided you probably also noticed that Marcus was laughing about the fact that Kevin Beaumont’s attention appeared to motivate MIT to delete their paper from the internet. I guess we can safely say that MIT is not standing behind this research paper.
Not All Research is Equal
I the midst of all of this activity, Sarah Gooding of cybersecurity company Socket provided a subtle but powerful bit of context about the difference between sponsored research and academic inquiry. As she pointed out, this now discredited research paper came out of the Cybersecurity at MIT Sloan (CAMS) program, which is a consortium of private sector companies, government agencies, and MIT staff. While Sarah accurately pointed out that such public/private efforts are not inherently unethical, the findings of this paper force the reader to consider the possibility that a private sector company may have had undue influence in the outcome.
The authors of this particular research included Michael Siegel (Principal Research Scientist, MIT), Sander Zeijlemaker (Research Affiliate, MIT), Vidit Baxi (Founder & CISO, SAFE), & Sharavanan Raajah (Threat Researcher, SAFE). If you are asking yourself, “Who is SAFE?” you are asking the right question. SAFE is security company that offers a cyber risk quantification platform. This research paper that they participated in creating alongside MIT urged organizations to “embrace AI in cyber risk management,” a recommendation that conveniently aligns with the corporate interests of SAFE. While perhaps only related by the relatively close timing, it’s worth noting that SAFE closed a Series C round of funding for $70M less than four months after their joint publication with MIT that said “80% of Ransomware Attacks are AI-Driven.” That kind of statistic can do a lot to support claims of a $1T global total addressable market (TAM)…which is the kind of opportunity investors like Avataar Ventures probably salivate over.
That’s Not Even the Big Problem
I’m not saying that MIT should be ashamed for publishing the questionable results of their research. In fact, I don’t think anyone should do that. For science to advance, we all must accept that sometimes published papers will eventually be proven wrong. Science is about the advancement of human understanding, which will often inevitably include disproving previously held beliefs. Yes, one could argue that some errors in that paper were egregious. Personally, I’d be interested to know who was responsible for the final edits & decision to publish. But, that kind of inquiry distracts from the much more serious consequences of this publication.
Where MIT deserves ridicule is not in their decision to publish this research but, rather, in how they handled criticism when it arrived. Instead of making a public statement about their findings, or a retraction, MIT quietly pulled the paper from their website without saying a word. In fact, if not for archive.org (AKA The Wayback Machine) we might not be able to find this paper at all.
You may wonder why it matters if the paper quietly disappears as opposed to MIT making a public retraction. It isn’t about MIT being publicly flogged to atone for their sins. The reason there is such a stark difference between a public retraction & a quiet deletion is about integrity, responsibility, & accountability for the impact that this kind of report has today & in the future.
This MIT CAMS paper was available publicly for about 6 months. In that time, dozens of companies & publications have quoted the findings…& those websites aren’t being deleted. A simple Google search of the terms “80% + ransomware” or “80.83% + ransomware” yields those results. Not only do those websites continue to disseminate this falsehood, but they may also be included in the content used to train the next generation of Large Language Models (LLMs) that will be the underpinnings for the AI platforms the public is increasingly relying on as their source of rapid truth.
Will any of those corporate or journalistic websites be deleting their pages? Doubtful. Monumental statistics like this - particularly when built on the highly-respected brand of an institution like MIT - create fear that generates more readers, more clicks, more prospects, & more sales…which translates to more money. Who cares (or will even notice) if the increased fear is based on a lie? Anyone republishing that information, whether they know it to be flawed or not, can stand firmly on the ground that they are citing a research paper from MIT. It’s not their responsibility to question the research of such experts.
Post-Truth World
And THIS is where the rubber meets the road. because all of those active webpages will almost certainly spawn more circular reporting that will all end up appearing to the LLM as validation for the originally dubious claim. Can diligent data scientists fix this kind of problem? Probably. Can they be expected to manually fix 1,000s (or millions) of similar problems throughout the internet today to ensure their LLMs are not poisoned by mis/dis/malinformation? That seems unlikely, particularly when (again) there isn’t much demand for them to do so…& even less regulation or oversight.
AI platforms are already notorious for generating “hallucinations,” the term given when a platform provides an answer with what appears to be valid source that is actually fiction the AI platform created. These “hallucinations” - URL links that go nowhere - fool countless people every day because fact-checking takes time, energy, & focus. If people wanted to apply those traits to their work or personal curiosities, they wouldn’t be relying on AI. Thankfully, users who take the time to aaccess those fake links can still easily identify “hallucinations” & avoid being tricked into believing fiction is fact simply because there a citation was referenced. But, now imagine a world where those links are no longer fake…
In that near-now future, falsehoods supported by real links to respected institutions (like MIT) - further supported by numerous additional links from seemingly unrelated organizations (circular reporting of the original falsehood) - will be blended into the foundation of all training content for all future LLMs. If that happens, eventually AI-generated responses to trivial things like, “Did Curious George have a tail? - & consequential questions about history, science, medicine, economics, & geopolitics - will all be inextricably & undetectably poisoned by falsehoods. Add to that any intentional manipulation of facts by those who would have the resources to alter reality & would most benefit from doing so, & we may find ourselves living in the Orwellian world that every dictator in human history has dreamed of ruling.