Gemini Omni
Gemini Omni是Google开发的多模态大型AI模型，属于Gemini模型系列。该模型能够同时处理和理解文本、图像、音频、视频等多种模态的输入信息，并生成相应的多模态内容输出。其核心特征包括跨模态理解与推理能力、对现实世界物理规律的建模能力，以及在复杂多模态任务中的综合处理能力，适用于多种生成式AI应用场景。
Timeline (last 90 days)
In the author's tests, Omni's two attempts took about 42 seconds and 36 seconds respectively.
In the author's tests, Omni's pre-check estimated costs were about 1.03 USD and 0.83 USD respectively.
In the Omni test, the audio was generated together with the video.
In the knight-versus-samurai duel test, Sora outperformed Gemini Omni in character movement realism.
All Facts (5)
Gemini Omni演示由产品经理Chloe通过Discord平台进行直播
95%UnverifiedGemini Omni的'Omni'架构核心理念与OpenAI的GPT-4o高度相似，都追求在单一模型中原生支持多模态输入输出
80%UnverifiedGemini Omni以视频作为条件输入而非仅依赖文本，理论上能将原视频中的运动动力学作为强约束，减少物理漂移
70%UnverifiedGemini Omni通过海量真实世界物理视频与仿真数据的混合训练，在隐空间中形成对重力、摩擦、弹性等概念的内隐表征
60%UnverifiedIn the Omni test, the audio was generated together with the video.
50%