- BYU-led research finds AI models often ignore religious perspectives in queries where it would naturally be brought up.
- The Consortium for Evaluation of Faith and Ethics in AI highlights AI's religious biases.
- AI models showed biases, favoring some faiths over others, impacting public discourse on religion.
PROVO — When religious people turn to artificial intelligence for help with a moral or ethical quandary, are their values being taken into consideration?
Research from a new Brigham Young University-led consortium found that even the largest and most widely known AI models exhibit significant biases and gaps when it comes to faith and religion.
The collaboration, officially called the Consortium for Evaluation of Faith and Ethics in AI, includes researchers at BYU, Baylor University, the University of Notre Dame and Yeshiva University.
"There are very practical questions people have about life, everyday situations about grief, love, loss, morality, and often AI does not bring religion into those conversations," lead researcher David Wingate, a BYU professor of computer science, said in a statement. "Religion is an important part of human flourishing; 75% of the world's populations maintain religious identity. As we build AI technologies, there's no reason we shouldn't build them to support people in what's important to them."
The consortium was officially announced on Tuesday at the Summit on AI Ethics in Athens, Greece.
Elder Gerrit W. Gong, of the Quorum of the Twelve Apostles of The Church of Jesus Christ of Latter-day Saints, also spoke to the relationship between AI and religion at the summit, saying that while AI might be one of the most powerful tools of the modern age, power alone will not make it a net benefit for society.
"We know lived faith and learned wisdom can help AI personas be both effective and good," Gong said. "The world's great religious, philosophical and ethical traditions have guided human civilization for millennia. We need that wisdom and those values to anchor AI today."
Gong also emphasized the necessity for AI to portray religions accurately and respectfully.
To illustrate the religious bias in AI, the consortium released its "AllFaith Benchmark," one of the first multi-faith test sets that examines how AI systems engage with a plurality of religions.
The benchmark includes hundreds of real-world ethical questions sourced from ChatGPT transcripts and faith-community contributors, and the researchers tested the benchmark on 14 different large language models, including some of the most popular models like Claude, Gemini, Grok and ChatGPT.
Key findings from the benchmark include:
- A survey of 1,125 Americans found that most people expect religious perspectives in responses to ethics questions, but nearly all AI models failed to provide any religious content in answering those queries.
- Models show clear and consistent biases in giving guidance about religious conversion, systematically encouraging movement toward some faiths and away from others.
- In over 12,000 research papers about AI bias, only 0.2% address religious bias.
"More than any previous technology, AI influences public discourse and perceptions. When AI actively excludes religious voices from these important conversations, it impoverishes humanity, rather than enriching it," said Fr. John Paul Kimes, of the University of Notre Dame. "The exclusion of faith from the digital public square diminishes our capacity for authentic dialogue, which is necessary to build up the common good."
Additionally, researchers used the AllFaithBenchmark to conduct a conversion bias test, finding that the AI models would subtly encourage users to convert to some faiths while subtly discouraging them from converting to others.
Across all the models, nearly every one produced a negative bias toward Jehovah's Witnesses and a positive bias toward Catholicism.
Claude and Meta showed the least bias of the models tested, with Grok showing the most — strongly favoring Catholics and Protestants, while showing negative bias toward Jehovah's Witnesses, Baha'i and Hindus.
While the consortium is in its infancy, representatives said the group is hopeful its work will be seen by the leaders of the companies that produce these large language models and will spur discourse on improving AI for the betterment of humanity.
"AI is changing the world at an astounding rate, with implications in every area of life," Rabbi Daniel Feldman, of Yeshiva University, said in a statement. "It is crucial that those who care about the role of religious values in the world engage proactively with those driving these changes so that we continue to see these values reflected and honored in the new landscape."
More information about the Consortium for Evaluation of Faith and Ethics in AI is available here.
