Grants support new research on role of AI in the arts

Binghamton University faculty are diving into the vast world of AI — from talking robot guide dogs to machine learning-based ASL training tools. But as AI seeps into the sciences, how will it affect fields in the arts and humanities? This is the central question behind the most recent round of Provost Awards for Research Grants, providing up to $100,000 in seed grants to support research addressing the impacts of artificial intelligence and machine learning on arts and creativity.

“It’s really great to see Binghamton supporting this and looking toward AI in the arts and humanities. I think it’s important right now for all of us, because it’s so prevalent in all of our lives now,” said James Budinich, a lecturer in music composition and creative sound technologies who was one of the grant recipients. “We’re constantly being asked to use AI, but it’s important that we find ways to use AI more ethically — as more of a partner in the creative or research-based process, instead of just churning out its own research or creation.”

This year, four projects received funding to investigate the different ethical, philosophical, and creative implications of AI in the humanities and other artistic practices:

1. “Algorithm, Composition & Improvisation: A New Generative Collaboration Via Sensory Percussion and Machine Learning”

This collaboration between Budinich and Gregory Evans, assistant professor of music, will toy with artificial intelligence and improvisation, having generative AI respond in real time to the data of live performances in order to become a new musical “partner” for human instrumentalists.

The project, which received $12,530, will include a workshop in the fall, as well as live performances premiering at Binghamton University, then New York City and Ithaca, with guest violist Stephanie Griffin. Budinich expects not a single one of these performances will be the same.

“Each time it’s performed, it has a whole new liveness to it, a whole new identity. I think that is interesting, because then it may change the way we think about documentation for music, what the role of performance is, and how AI can help that,” Budinich said.

2. “AI as Cinematic System”

The more generative AI advances, the more artists may feel they have only one of two choices: adopt it entirely, or reject it wholly. A collaboration between Magdalena Bermudez and Jason Bernagozzi, assistant professors in the Cinema Department, aims to address this gap in adoption and use. More specifically, they will be investigating how creatives can retain their own agency as artists while using AI technologies.

With $37,500 in funding, Bermudez and Bernagozzi plan to develop a responsive machine learning system that will combine historical and contemporary cinematic and audio technologies in conversation with other AI tools.

3. “Against Detection: Investigating Human and Machine Vision Through Print-Based Practice”

How well can AI really see? Is there any aspect of creation that is so fundamentally human that machines really can’t understand or touch it? This project, a combined effort between the departments of Art and Design, as well as Digital and Data Studies, will experiment with different kinds of artworks — from wearables to painted surfaces — to see what trips up algorithmic interpretations, and what remains “irreducibly human.”

Researchers will use printmaking techniques to generate three classes of images: prints that remain unclassifiable to computer vision, images that humans and machines perceive differently, and traditional artworks that are detectable but remain incomprehensible.

This project is a collaboration between faculty members Christopher Swift and Alexandros Skouras of Art and Design, and Ruth Carpenter and Gregory Hallenbeck of Digital and Data Studies. It received $32,000.

4. “How Do Generative AIs Read Literature?: Designing a RAG Benchmark for Social Knowledge”

Throughout history, literature has been an important marker of social knowledge. Fiction makes its readers grapple with moral conundrums, diverse perspectives and other pressing cultural or philosophical issues, a practice that builds empathy and critical thinking skills, both of which are especially crucial during times of division.

This project, a collaboration of Junting Huang, assistant professor of comparative literature; Sujoy Sikdar, assistant professor in the School of Computing; and William Hayes, assistant professor of psychology, will probe how generative AI systems interpret literary fiction. The team plans to curate at least 30 works of literature from around the world in order to test how multiple Retrieval-Augmented Generation (RAG) configurations fare against more than 150 annotated question-answer pairs.

The goal of this project, which received $17,500, is to test the extent of AI’s capacities for literary interpretation — and where they might fall short.  

The funding for these research projects is provided over an 18-month period. The Provost Awards for Research Grants are made possible thanks to a special endowed fund, established by the state of New York, matching a private donation made to the Binghamton University Foundation.

“This is huge for me. It’s really a great resource that the provost has created for Binghamton’s faculty,” Budinich said. “It’s pushing my practice in a whole new direction, and without this funding, I wouldn’t have the chance to really focus on this project.”