PaLM-E
PaLM-E是由Google研发的多模态大型语言模型，将视觉感知与语言理解能力相结合，能够处理图像和文本的联合输入。该模型以PaLM为语言基础，融合视觉编码器，主要面向具身智能（Embodied AI）场景，支持机器人任务规划、视觉问答及场景理解等应用，是将大型语言模型能力扩展至物理世界交互领域的代表性研究成果之一。
Core Facts
Timeline (last 90 days)
RT-2 fine-tunes vision-language large models such as PaLM-E into controllers that output robot actions, demonstrating zero-shot transfer capabilities across tasks.
RT-2, PaLM-E, and OpenVLA are robot control frameworks based on pretrained models.
All Facts (4)
VLMs (Vision Language Models) typically consist of a visual encoder (such as ViT), a language model, and a projection layer connecting the two, with representative models including GPT-4V/4o, Claude 3.5, the LLaVA series, and InternVL
80%UnverifiedGoogle's multimodal AI technology roadmap evolved from the early PaLM-E to the Gemini series.
60%UnverifiedRT-2 fine-tunes vision-language large models such as PaLM-E into controllers that output robot actions, demonstrating zero-shot transfer capabilities across tasks.
50%UnverifiedRT-2, PaLM-E, and OpenVLA are robot control frameworks based on pretrained models.
50%