Articles

Dec 17, 2024

Reading Time: 5min

The State of AI Video Generators What Is the Best One

The top ai generators and how they work

Quick Takeaways

  • Most AI video tools assist with editing and creating YouTube/TikTok-style content from static images rather than full text-to-video generation.
  • The two key players to keep an eye on for AI text-to-video generation are  Sora  and Google’s  Veo 2 .
  • Best free current tool for AI text-to-video is  Luma .
  • AI Video is in its infancy. Check back in a few months.

How does AI text-to-video work?

  • Prompt Engineering : The system breaks down your input into tokens (think individual words) and analyzes grammar, intent, and semantics.
  • Prompt Enhancement : AI fills in the gaps by creating a more detailed prompt, similar to how image generation prompts are expanded from what you give it.
  • Variational AutoEncoder (VAE) : A  VAE  converts the enhanced prompt into usable data points for image creation.
  • Frame-by-Frame Generation : The AI generates the video one frame at a time.
  • Post-Processing : Lighting, physics, and textures are applied to refine the video output.

Fundamental Limitations of AI video generators

Large language models (LLMs) require vast amounts of training data. Unlike text or images, there is simply far less video data available to train on, which creates a ceiling for what can ultimately be created. Not to mention the copyright issues around using AI-generated content.
Hype:  AI Video will replace traditional video content.
Reality:  AI Video will help speed up certain parts of video creation.

AI Video Trends

Initially, Sora had the lead, but Veo 2 is now trending upwards.  Here’s a Google Trends comparison from the past week.
Users with access to both tools seem to agree that  Veo 2 has the edge in quality and capability .
  • AI-Generated Stock Footage : Expect more "pre-generated" short clips for B-roll or background content in YouTube videos and short-form content.
  • Augmented Video Editors : Similar to how AI coding assistants like GitHub’s Copilot help with coding, AI video tools will support editors with faster, smarter tools.
  • Updates to Current Tools : Tools like Adobe Firefly and Canva are implementing AI video within their current products. More will follow.

Our Experiments

We ran our own tests using various AI video generators with the prompt "Kittens flying a spaceship". Check out this  exceptionally soulless blursed result from Synthesia .
If you want to have fun messing with text-to-video, the best free option we found was  Luma . It's free, easy to use, and only takes 2-3 minutes to generate a video. Below is the video generated using Luma.

Recommendations

  • Do Use : AI generators like Sora or Veo 2 for stock footage and creative inspiration.
  • Don’t Use : Current AI video editors and video generators that create videos with AI avatars, fake voices, and AI-generated images. We have not found a single one that does not feel soulless or uncanny.
  • Avoid The AI Trap : Be careful not to fall into the trap of endlessly tweaking prompts—it’s often faster to do it yourself.  Like this guy that spent 1 hour trying to get an AI to push code to GitHub (it takes a human 5 seconds) .
  • Don’t Replace : Avoid replacing your entire content strategy with AI. The more AI slop floods the market, the more "authentic" human content will stand out.
Get the Latest " TheFirmware Update " Newsletter .