Adobe Premiere Pro to get Firefly genAI video features

Adobe will bring its Firefly generative AI (genAI) video model to its Premiere Pro video editing app later this year, and plans to provide access to third-party AI video creation tools such as OpenAI’s Sora, too. 

Adobe offered a glimpse of the Firefly video AI model in action on Monday, with new features planned for Premiere Pro. 

There’s “object addition and removal,” which lets users replace or remove items from a scene. Here, an AI “smart masking” tool lets users select objects that can then be modified — such as changing the color of an actor’s tie — or taken out of a shot altogether, such as removing a mic boom or a branded coffee mug in a medieval fantasy show.

A text-to-video tool enables the creation of new content in Premiere Pro, which can be useful for creating supplementary B-roll footage, said Adobe.

Finally, the generative extend feature will add new frames to a clip, allowing an editor to hold a shot longer. “You can use the extra media to fine tune your edits, hold for that extra beat, or cover a transition,” an Adobe spokesperson said during a briefing ahead of the announcement. 

The genAI features for Premiere Pro will be available in beta by the end of this year, when the Firefly AI video model launches, Adobe said.

“Generative AI models are increasingly becoming multi-modal — incorporating modalities such as images, speech and video — in both input and output,” said Arun Chandrasekaran, Gartner distinguished vice president analyst. Although existing text-to-video capabilities are “quite nascent today,” the potential is “immense.”

With other AI video creation models also drawing attention, most notably Sora, Adobe appears eager to build out its own capabilities. According to a recent Bloomberg report, Adobe is paying photographers and artists around $3 per minute of content for video content that can be used to train its AI models. 

While Adobe will face competition from the likes of OpenAI and others, it will also benefit from its status as incumbent in the market, said Forrester senior analyst Nikhil Lai. 

“I anticipate demand from editors to access Adobe’s AI features because many have indelible muscle memory for the look and feel of Adobe’s tools,” said Lai.

Adobe also unveiled plans to partner with other vendors, with plans to incorporate third-party models in Premiere Pro. This includes Sora, as well as models from Pika Labs and Runway. These are just the start, said Adobe’s spokesperson during the briefing call. 

“Customers told us that they want choice, and we see a future in which thousands of different customized models are going to emerge, each strong in their own niche,” he said.

Adobe said it will apply its existing “content credentials” feature to assets generated by third-party models. This provides information on how a video was modified using AI, and which models were used. 

The use of multiple third-party tools could increase the risk of generating assets trained on copyright material, however. To help Premiere Pro users avoid adding copyright-infringing content to videos, a green checkmark will be visible in a menu of available third-party modes to show those deemed commercially safe. But customers will still need to do their own due diligence when accessing AI models from other vendors, an Adobe spokesperson said. 

Work to integrate third-part tools is at this point “exploratory,” Adobe said, with no launch date yet set.