Philosophy's Alignment Problem
Reflections on an open letter about philosophers funded by the AI industry
This week, a group of philosophers released an open letter. In light of ties “between philosophers and technology companies producing AI products,” the letter invites “discipline-wide reflection on the norms of research integrity.” It has already drawn over 150 signatories.
Specifically, the letter calls on philosophy journals to require conflict of interest (COI) disclosures from authors with industry ties, an extension of a practice that some journals already follow. Without these disclosures, “philosophical discourse risks problematic influence from commercial interests.” With them, though, journals can “protect the integrity of philosophical research and maintain the trust of the public in philosophical inquiry.”
Why exactly does this matter?
“The history of science is rife with cases where industry problematically shaped research agendas and outputs of scientific fields to protect their own interests.” Tobacco companies funded studies minimizing the health risks of smoking. Shills for the fossil fuel industry manufactured doubt about climate change. Purdue Pharma helped scientists persuade the public that addiction to opioids like OxyContin is rare.
As the letter suggests, COI disclosures are necessary because science runs on trust. A published study describes its methods and its results, but many choices aren’t visible on the page. The article itself can’t tell us whether the researchers exercised proper controls, reported data selectively, invented hypotheses after seeing the results, eliminated inconvenient outliers, selected friendly statistical criteria, and so on.
Because industry involvement increases the likelihood of such questionable research practices, the credibility assigned to funded scientific studies may be misplaced. COI disclosures don’t remove bias, but they alert readers to it and signal a need for independent inquiry.
Imagine studies purporting to show that teenage chatbot use isn’t associated with depression, or that frontier models can’t be used to make bioweapons, or that mass unemployment from AI automation is unlikely. If we find out that the researchers behind these studies are funded by OpenAI, Anthropic, or Google, it doesn’t follow that their conclusions are false, but their credibility slips. We’d at least want confirmation from disinterested researchers.
Philosophers might be useful to AI companies in their own ways. Scientists can be enlisted to distort the likelihood of costly outcomes; philosophers can be enlisted to minimize the costs or inflate the benefits. In need of ethical laundering? Call us.
But do concerns about industry-biased science generalize to philosophy? “I just read a sound argument about AI but couldn’t endorse the conclusion because the author has a grant from Anthropic.”
This disanalogy is real but misleading. The existence of industry funding doesn’t render otherwise sound arguments unsound, but it can distort philosophical inquiry in other ways. Philosophers with industry ties may raise some questions while neglecting others that are just as important. They may give weak objections plenty of air while dispatching strong objections in a few sentences. Or they may draw a stronger conclusion than more fair-minded authors would allow. Readers can evaluate the argument, but these choices likewise aren’t visible on the page.
Industry involvement is also a concern even when research isn’t compromised. In The Misinformation Age, philosophers of science Cailin O’Connor and James Weatherall argue that one common industry tactic is to fund research that is accurate but still misleading. Because real-world data is noisy, some studies will by chance find no harm when harm exists. Industries can also redirect attention toward alternative sources of risk, crowding out risks that are inconvenient.
Likewise, AI companies interested in misleading the public might fund philosophers arguing that cognitive offloading need not erode intellectual virtue (even if it usually does) or that a world without work can still be meaningful (even if it’s mostly bleak). Or they might fund those researching other existential risks, like nuclear weapons or pandemics. In fact, companies do fund research on existential risk from AI itself—maybe because this too serves their bottom line, supporting the perception that the technology is world-historically powerful. The literature as a whole is skewed, even if each article is sound. The industry “floods the zone.”
Just as with industry-funded scientists, philosophers need not be seeking to mislead. Their views might be entirely sincere. But OpenAI funds them rather than others. And their research becomes more visible in journals while contrary research languishes.
For these reasons, it’s reasonable for philosophy journals to require authors with industry funding to make COI disclosures. Even if the research isn’t compromised or misleading, transparency wards off charges of hidden corruption and preserves legitimacy.
Beyond this, the letter may overreach. Requiring disclosures of “close personal relationships with employees of relevant companies” probably casts too wide a net. The call for “retroactive disclosures” backdates a norm that wasn’t at play when authors submitted their work. And failures of disclosure may warrant only “correction” rather than “retraction.” Still, the core idea of disclosure requirements is valid.
I say this even as a philosopher who is enthusiastic about AI. I’ve argued that LLMs can enhance human cognition, that environmental harms are overblown, and that professors should (carefully) incorporate AI into their courses. These are precisely the sorts of conclusions that would serve the industry’s interests, though I haven’t taken any of their funding. Were my finances to change, readers would have a right to know.
Do philosophers tied to the AI industry suffer from a conflict of interest? Yes. Are these conflicts significant? No.
Philosophers are intensely skeptical of AI. The vast majority of work in AI ethics is negative. For example, philosophers have written extensively about algorithmic bias, mass surveillance, manipulation, responsibility gaps, opaque decisions, moral and cognitive deskilling, exploitation of workers, environmental harms, unemployment, artificial companions, and value misalignment.
This glut of criticism is partly due to professional incentives that shape who advances in the field and what research gets conducted, as Peter Königs argues. An article about AI is far more likely to be published if it criticizes the technology than if it commends it. In philosophy, grants are awarded largely insofar as you can articulate ethical problems that merit further research. If a successful career turns on sussing out problems with AI, that’s also a conflict of interest. And as it stands, philosophers are indeed flooding the zone—but with anti-AI research that is uncongenial to the industry.
Academics also have more mundane reasons to engage in motivated reasoning. LLMs are making our jobs more difficult—we’ve been forced to restructure our courses to prevent students from submitting AI-generated essays. Journal editors are being swamped with slop. The technology is sowing doubt about the value of academic research and higher education. No wonder there’s a flood.
What’s more, my lab argues that “the cognitive capital that cultural elites have been monopolizing may soon become cheap and abundant—which threatens to dilute their professional identities and reduce their social status.” As Paul Bloom suggests, asking academics about AI is like asking taxi drivers about Uber.
A few industry-funded enthusiasts hardly skew the literature. They prevent the imbalance from being even more severe.
Incentives aside, whether there’s too much anti-AI philosophy or the right amount depends on how harmful—or beneficial—the technology really is. And anyway, the open letter isn’t proposing to shift the balance by restricting what gets published. (Laissez-faire academic freedom is the best of all bad options, as Derek Anderson argues.) The letter calls only for philosophers to be transparent about their funding. This is standard practice in the sciences, and because philosophical research isn’t fully validated on the page either, it should be standard practice in philosophy too.
Indeed, as the letter states, it should be standard practice for all externally funded research, not just research funded by AI companies. Other institutional funders also “problematically shape research agendas.” When academics take their money, that likewise carries “serious risks to the integrity of their work.”
Yet the letter raises some questions while neglecting others. By focusing on “industry capture,” it passes over far more pervasive ideological capture.
Consider the Mellon Foundation, which decided in 2020 to “prioritize social justice in all of its grantmaking” and holds a virtual monopoly on humanities funding. According to a damning report from The Atlantic, the organization steers which questions get asked, which answers are given, and which alternatives are ignored. To secure funding, academics report “twisting their research into a social-justice pretzel.” Mellon representatives even steer applicants into reorienting their research focus.
If you find out someone’s research is funded by the Mellon Foundation, its credibility slips. That’s true even if the research is sound. It’s likely to be selective or distracting.
The open letter’s case applies far more forcefully to Mellon than to Google. The difference is that COI disclosures mean little without professional backing. Telling academic colleagues that you’re funded by a progressive organization is like telling the choir that you’re funded by the church.
The open letter’s main message is limited to COI disclosures, but the final paragraph promises more:
Finally, we recommend that the discipline of philosophy more generally take this as a moment to reflect on what policies are needed at all levels of organization to protect the integrity of our research in a rapidly changing world. Departments, professional organizations, and other disciplinary groups all may implement protective policies, and journals may wish to do more than require disclosure. Our discipline has a long history of taking research ethics very seriously, and how we continue that history is up to all of us.
Some readers may find this call for further reflection encouraging. I find it unsettling.
For now, it’s just COI disclosures, but what’s next? Opposition to AI is widely seen as urgent, and politically active philosophers tend not to suffer from excess tolerance toward those they regard as antithetical to social justice—that’s been my experience, but I’m hardly alone. (Consider Rebecca Tuvel and another open letter, that one demanding retraction of her Hypatia article on transracialism.) A discipline in the pocket of industry will not earn the public’s trust, but nor will one that polices heterodoxy.
If “philosophical discourse risks problematic influence” from AI companies, as the latest entrants to a history of industry-funded distortion, why would the letter’s authors want only transparency? You don’t want lackeys working for Philip Morris, ExxonMobil, or Purdue Pharma to simply add COI disclosures to the endnotes. You want them to stop. Given the field’s orientation, many signatories of the open letter likely believe that AI is harmful on the scale of these very industries—creating addiction, pushing its products onto children, destroying the climate, and flooding the world with misinformation. In that case, disclosure is hardly enough.
If elimination is where their own premises point, what means will concerned philosophers pursue? The letter itself is a sober and reasonable document, but the signatories may not be of one mind. Some may want journals to “do more” by banning submissions from all philosophers with AI industry funding. Or perhaps from a wider set: those with “close personal relationships with employees of relevant companies.” (Mellon-funded research will be safe.) Once philosophers disclose their funding sources, “protective policies” may include identifying offenders and imposing informal social sanctions on them. We should take this as a moment to reflect on our history of research ethics.
This essay was supported by a grant on AI and politics from the PPE Research Consortium and the John Templeton Foundation. I’m grateful for helpful conversation with Jacob Barrett, Chris Howard, Rachel McKinney, and Derek Anderson.
While I totally agree about non-industry funding sources warranting mentions too (to say nothing about non-funding sources of bias, which I think you're right to point out), aren't people already supposed to disclose that work was funded by this or that grant? (You certainly see stuff about grant funding in the prefaces/acknowledgements for books. I thought we have them at the end of articles too, but I might be misremembering.)
In my work as a journal editor, I’ve seen nearly a dozen papers that read like AI writing (and desk rejected nearly all of them). Interestingly, the plurality of them are making exactly the kind of argument you note the field loves, about how reliance on AI has one or another epistemological problem. Apparently AI-written articles are even more likely to come to this conclusion than non-AI-written articles. Not exactly what one would expect from a naive conflict of interest theory.