A new startup launched by Carnegie Mellon University alumni can transform a song, podcast, recording or audio clip from a movie into a fully produced video complete with personalized scenery, costumes, moods and shifting camera angles.
Koyal, founded by CMU School of Computer Science graduates Mehul and Gauri Agarwal, produces cinematic videos from any audio clip. The company is offering its platform to the public for the first time, giving creators a quick and easy way to bring audio content to life.
Using artificial intelligence, Koyal generates custom characters, settings and animations in minutes while giving users creative control. Creators can make personalized avatars; change scenery, makeup and costumes; and even edit the tone and mood of the dialogue. Koyal can set a scene from the “Barbie” movie in a castle in Northern Ireland or create a bespoke music video with a choreographed flash mob. The product’s suite of tools helps the user’s imagination come to life in minutes.
“Many AI-generation tools require a lot of trial and error for users to get their desired results, and they often have to enter prompts multiple times,” Mehul said. “We trained Koyal to understand audio-driven storytelling using the best models available, which takes out much of that time-consuming guesswork. We are replacing the need for a camera rather than the need for creativity.”
Mehul earned his master’s degree at the Robotics Institute (RI) in 2024, and Gauri Agarwal completed her bachelor’s degree in computer science in 2020 before continuing her studies at the Massachusetts Institute of Technology. Through their programs at CMU, the Agarwals connected with Jean Oh, an associate research professor in the RI, and Michael Hilton, a teaching professor in the Software and Societal Systems Department, leading to the collaborative development of Koyal.
“Mehul was my student, later a TA and now my collaborator,” Hilton said. “His and Gauri’s work on Koyal shows the dedication and creativity that make CMU students special.”
Hilton added that Koyal helps users manage their own images in AI videos, which has crucial societal implications. As generative AI becomes more sophisticated, concerns about deepfakes, unauthorized image use, identity theft and other malicious intentions are rising. The Koyal team focused on these issues early in product development.
“Koyal started with a strong commitment to protect people’s identity,” Oh said. “Our first goal was to design a security gateway so other people’s photographs cannot be used in generated videos without permission.”
To do so, the team developed Computer Human Assessment for Recreating Characters with Human Actions (CHARCHA), a patented secure verification protocol that asks users to perform a set of randomized physical actions in front of a webcam to confirm their identity. CHARCHA grew out of research the Agarwals pursued while completing master’s degrees. After the team presented CHARCHA at the 2024 Conference on Neural Information Processing Systems (NeurIPS), they worked tirelessly to incorporate the system into Koyal’s generation process.
“If a user wants their likeness to appear in the video, they can use the CHARCHA verification steps to create a personalized avatar,” Gauri said. “But a user can also choose not to incorporate their likeness and instead only use generated models. We developed CHARCHA to be available as a safeguard to prioritize consent.”
What began as a research project at CMU has now transitioned into a robust platform for real-world use. With strong roots in both technical innovation and responsible design, Koyal has already attracted attention from global leaders in the entertainment industry.
The Koyal team recently partnered with Universal Music and T-Series alongside Bollywood production houses like Maddock Entertainment and Collective Artists Network. Mehul demonstrated Koyal’s abilities to producers at the World Audio and Visual Entertainment Summit and, responding to producer requests, the team used Koyal to craft five music videos for Grammy and Oscar-winning artists like A R Rahman, Ricky Kej and Shankar Mahadevan. They also started work with some of the top Indian educational and K-12 companies, including PACE IIT and Medical, Narayana Group and PhysicsWallah.
Building on the early interest across entertainment and education, the team prepared Koyal for its beta product launch. Koyal now offers free 45-second video clips for new users, giving them the opportunity to test the platform’s creative features and built-in safeguards.
“Our hope with Koyal is that it broadens opportunities for a range of creators, not just those who can afford large-scale production,” Gauri said. “AI is moving fast, and we are providing a safe way to explore its potential.”
To demo Koyal, visit the company’s website.
For More Information: Aaron Aupperlee | 412-268-9068 | aaupperlee@cmu.edu