The advocates of open-source AI cite the benefits the approach has brought to our digital lives: Open-source software speeds innovation, the code becomes more adaptable and more resilient to attacks, and more people benefit from the software’s capabilities. Open-source advocates believe these same principles should carry through to AI, democratizing its development. They argue that we should know how generative AI systems are being built and how they work, and that this transparency could lead to a greater trust in AI as it becomes a partner in medicine, scientific research, the creative arts, and many other human activities.
Open-source models make their underlying architecture public, allowing for inspection, modification, and reconfiguration. In contrast, closed models keep their architecture, training data, and methods private. Access to these models is restricted, giving developers greater control over their use and preventing potential misuse. It’s also important to distinguish between different levels of openness. For example, open-weight models represent a middle ground by releasing the final trained parameters (weights), providing some insight into the model’s internal workings. However, crucial details like training data and methods remain hidden, limiting the total transparency and making it difficult for those other than the model’s developers to fully understand or reproduce the model’s behavior.
The advocates of a more closed AI hold that AI is a special technology in both the power of its analytical abilities and the potential scale of its effects. If a truly advanced frontier AI model were to be open-sourced, it could place too much potential for harm in the hands of someone who chooses to modify it for nefarious purposes. These advocates argue that keeping AI closed—or, more specifically, not making the model or the model weights accessible to the public—provides a better environment to correct biases in the data, helps ensure the integrity of the model’s output and inferences, and safeguards the model from being reconfigured by bad actors. From this perspective, it would be irresponsible to let large language models (LLMs) be accessible without accountability or limitations on their deployment and use. There are passionate, committed advocates who are thinking of the best approaches to just how open AI should be.
Lawrence Lessig is a professor of law at Harvard University and a longtime open-source advocate who is inserting nuance into a debate that often seems overly binary. “I had an exchange recently with someone who pooh-poohed the idea of any risk, because in their words, they didn’t see a Terminator risk,” Lessig says. “But it didn’t take long to get him to a place where he actually saw that there’s so much more [to worry about] before we get to the Terminator question.” In Lessig’s view, the dangers of AI do not lie in the imagined future of a runaway superintelligence, but in the here and now, when we need to prevent such things as deepfakes, misinformation campaigns, and nonconsensual intimate imagery, as well as abuses by terrorist groups and hostile state actors.
In conversations with policy makers, Lessig has detected misconceptions about how AI is built: “Most people have a crude understanding of this tech, and they assume developers can just write a line in code that says, Do not do bad stuff.” He emphasized how the training of AI is a complex task. “You don’t just tell AIs to behave well, so it is a very complicated thing to figure out how we get the technology to be safe.” Researchers are working extremely hard on the problem that Lessig describes: How do we conform the AI models to human values, and how do we understand the decisions the models make?
One possible solution proposed by Lessig is a two-tier model in which the most powerful AI systems are closed, but elements of open-source experimentation are allowed. “There’s a line below which we ought to be encouraging open-source, because it’s an incredible engine of equality across the world,” he says. “But there’s a level at which it becomes potentially dangerous. I think it drives us to think in a bigger way about the infrastructure within which we can regulate. So the commitment to open-source might be a commitment to find a different way to regulate for safety, not by controlling the models, but by controlling the environment within which the models are running. If that infrastructure existed, I think the freedom for open-source could extend to much more powerful AI models.” Some organizations, such as MLCommons, the Frontier Model Forum, and the nonprofit Partnership on AI, are already working on AI ecosystem safety efforts.
The concerns around open-source only grow larger as the models become more capable, with long context windows, memory, planning, and tool usage enabling “agentic” behavior, which is the ability of an AI model to execute commands in the real world—for good or ill. Monitoring activity on closed models might limit the kinds of abuses that might result.
Another perspective comes from Percy Liang, an associate professor of computer science at Stanford University, who’s also searching for the best way to proceed responsibly with AI and is generally a proponent of open-source approaches. In his role as the director of the Center for Research on Foundation Models, Liang guides a group of scholars who study the deployment and development of major AI models. One of his basic goals, he explains, was to refine the narrative around the open-source conversation. It’s helpful to view AI as part of a progression of general-use, foundational technologies. “We don’t get upset about people using email,” Liang says, “which, in the hands of a malicious actor, people can [use to] generate a lot of spam and disinformation.” He cited computer code itself as another technology that could be considered “very dangerous” since it is used to create viruses and cyberattacks—when it’s not, you know, doing the work of propping up the fundamental processes of modern society. A similar debate about enabling a powerful technology versus protecting the public swirled around encryption in the ’90s. The government wanted access to encrypted communications to protect national security, while civil libertarians argued that individuals had a right to protect their privacy. “Eventually,” Liang says, “it was sort of decided that we were going to live in a world where encryption was available to everyone.”
Liang uses the metaphor of Legos to demonstrate the difference between open and closed models. A closed AI model is like buying a toy from the store and playing with it; an open-weight AI model is like having a collection of Legos and cobbling together a custom creation. When asked to cite a notable benefit of open approaches, Liang described how open models have transformed data analysis in many aspects of scientific research: “You can do things that you could not do before.”
When it comes to the Terminator-like scenarios that make people fearful, Liang counsels “not running and jumping to conclusions” and encourages people to adopt a more measured view of the likely impacts. A headline-making paper published by researchers at MIT in 2023 looked at whether or not LLMs could help build bioweapons. The paper’s conclusion was bleak: “Our results suggest that releasing the weights of future, more capable foundation models, no matter how robustly safeguarded, will trigger the proliferation of capabilities sufficient to acquire pandemic agents and other biological weapons.” Writing in response to the study, Liang and his collaborators described the study as a “cautionary tale” of “what can go wrong when analyses of the risks of open foundation models do not compare against risks from closed models or existing technology (such as web search on the internet).” Subsequent studies suggested that the information acquired on bioweapons through the current level of LLMs was at that point the equivalent to what someone could find through a traditional web search.
Liang envisions a future in which AI is interwoven into our daily lives and our societal institutions. In this scenario, the complete walling-off of the technology seems impractical and hard to implement. He suggests a framework in which we are responsibly assessing the “marginal risk” of open-source AI and then devising collective solutions when the risk is deemed too high. “Unfortunately, there hasn’t been much rigorous study of risk assessment,” Liang says. “What is important here is the marginal risk of releasing an open model compared to, well, just using Google or Wikipedia.”
Part of assessing the risk of an AI model is looking at the potential defenses the model has against misuses. Liang brought up the example of using AI to hack into sensitive databases, such as credit card information. At present, AI can certainly assist in this illegal activity, but AI is also very helpful at defending against these attacks, at looking for security flaws and patching them. In this way, an open model can reinforce its own security. But in other areas, such as disinformation and nonconsensual intimate imagery, the capabilities of AI seem to be overwhelming our defenses, Liang cautions. These are areas in which companies and governments must step in with well-structured policy and effective solutions.
Like so many aspects of artificial intelligence, the question of open-source is a balancing act. Are the benefits of open-source for scientific research such a positive that we accept the potential harms of disinformation? “I absolutely think that we need good policy and we need to understand the ramifications,” Liang says, “but I think a lot of the AI safety worries are a little bit too divorced from reality, and they don’t take the whole system into account.” Lessig, for his part, discussed a possible public-private divide. “AI is an enormously valuable technology for the world,” he says. “I think we should have a kind of Human Genome Project–like case to build it in a way that’s protective of public AI ideals as well as private AI ideals.”
We all want AI that’s ethical, dependable, and secure and helps with important decisions about our health and our climate. We want AI that can accelerate scientific discovery. These goals will require a rational framework to assess the societal impacts of AI models. Right now, both the open and closed models of AI are moving forward at a rapid pace. It’s important to think of AI as along the lines of a public commons—a technology that will affect all aspects of our experience, the future of which depends upon the cooperation of government, private business, and an engaged citizenry.
