Adobe has unveiled its generative neural network, Firefly Video Model, which integrates into the Premiere Pro application. This tool enables users to enhance existing footage and create videos from static images or text prompts.
Generative Extend for Minor Edits
The Generative Extend function, powered by the Firefly neural network, is currently available in beta for Premiere Pro users. It allows video segments to be extended by a few seconds at the beginning, end, or middle of a clip, helping to correct minor flaws during editing. For instance, it can smooth out issues such as a subject’s gaze shifting unexpectedly or small, unnecessary movements in the frame.
However, Generative Extend has some limitations—it can only extend a video by up to two seconds. This function supports resolutions of 720p or 1080p at 24 frames per second. While it also works for audio, it’s restricted to non-verbal sounds. Ambient noises or sound effects can be extended up to 10 seconds, but music tracks or conversations cannot be lengthened in the same way.
New Video Tools: Text-to-Video and Image-to-Video
Adobe has also introduced two additional video generation tools, Text-to-Video and Image-to-Video, now available in the Firefly web version through limited beta testing. As the names suggest, these features allow users to generate videos from either text prompts or static images. However, access to these tools is currently limited and may not be available to all users.
The Text-to-Video tool functions similarly to other AI-based generators like OpenAI’s Sora. Users enter a text description, select a style, and generate a short video. Additional “camera controls” allow for fine-tuning elements such as angles, movement, and zoom effects, giving users more creative flexibility.
With Image-to-Video, users can add static images to complement text prompts for more precise video generation. Adobe recommends this tool for refining segments of existing videos or producing new clips based on individual frames. However, as indicated by test results, this feature isn’t yet a substitute for live reshooting, since the generated videos sometimes fail to accurately replicate objects from the reference images.
Current Limitations and Future Updates
Both the Text-to-Video and Image-to-Video tools currently generate videos capped at five seconds in 720p resolution, with 24 frames per second, notes NIXSolutions. Comparatively, OpenAI’s Sora promises to produce videos up to one minute long, although the tool remains unavailable to the public months after its announcement.
Creating videos using Adobe’s AI tools—whether Text-to-Video, Image-to-Video, or Generative Extend—takes around 90 seconds. Adobe has shared plans to introduce a “turbo mode” to reduce generation times. The company also assured users that the Firefly Video Model is “commercially safe,” as it was trained on content legally accessible to Adobe.
We’ll keep you updated as Adobe continues developing these features and makes them more broadly available.