MLLM

新一代多模态大语言模型（MLLM）语义理解能力的提升使得用户不再需要掌握特定的提示词语法即可生成高质量视频

Contact-intensive tasks (such as assembly, insertion, and wiping) require precise force control and high-frequency feedback; visual resolution is insufficient to support sub-centimeter alignment, so real-time adjustment based on proprioception is necessary, and the inference frequency of general MLLMs is usually far below the control frequency required for such tasks.

The key difference between VLA and WAM compared with general MLLMs is that they have undergone action-space alignment, with outputs falling directly within the robot's controllable instruction space.

In stage one, SUPRA-TEX trains an image editing model based on a multimodal large language model (MLLM) to predict illumination-separated PBR material maps under the reference viewpoint, outputting channels such as albedo, roughness, and metallic.

The typical way to attach an external 3D geometry encoder to an MLLM is to inject the geometric features output by the 3D encoder into the LLM's hidden states through a projection layer or cross-attention, and the encoder must be fully run on every forward inference.

GeoThinker-F is a spatial intelligence framework that frees multimodal large language models (MLLMs) from dependence on external 3D encoders, achieving spatial reasoning by having the model intrinsically predict the required spatial representations.

The researchers evaluated 50 MLLMs, and the results show that open-source models as a whole still lag behind closed-source models, and this gap is particularly pronounced on audio-visual understanding tasks.

Even on leading models, precise temporal and spatial localization remains a core unsolved challenge; models perform acceptably on coarse-grained semantic understanding, but have clear gaps on precise localization tasks.

Under the paired setting, when the same target is described in text-only form or guided with text+image, the model's predictions become inconsistent, indicating that its understanding of the target may depend on the prompt form rather than true multimodal semantics.

Currently, the audio encoders and visual encoders of most MLLMs are independent of each other during pre-training, which leads to an easy degeneration during joint reasoning into preferentially trusting one modality.

Existing MLLM-based design generation methods share a common limitation: high-level planning is already fixed before execution, and once the design starts rendering, the model cannot adjust subsequent steps based on the actual effect of the current canvas.

MLLMs are prone to cumulative errors when generating fine layouts over long sequences, and find it difficult to go back and modify content already output during generation, which is one of the technical roots of the limitation that "the plan is fixed before execution".

EmoGenie provides a quantifiable testing tool for the emotional controllability of text-to-image models, and can be used to examine the reliability of MLLMs as AI judges.

EgoOmniWorld's evaluation targets multimodal large language models capable of processing video, audio and text simultaneously.

Evaluation results on current frontier MLLMs show that the models perform strongly on local perception tasks but are clearly weak in long-horizon audio-visual reasoning.

Identity drift and cross-modal misalignment are the two most prominent weak points of current frontier MLLMs in audio-visual captioning tasks.

Multimodal large language models (MLLMs) refer to large-scale models capable of simultaneously processing multiple modality inputs such as text, images, audio, and video and generating text output, with representative works including GPT-4o, Gemini, Qwen-VL, etc.

Multimodal large language models often rely on language priors rather than truly relevant visual evidence when answering visual reasoning questions.

Existing lightweight MLLM solutions reduce costs by replacing the language backbone with a lighter one, but their sparsity and depth allocation are static and cannot flexibly adjust according to the semantic complexity of each token.

Existing MLLM methods directly infer instructional feedback from visual observations and suffer from three categories of deficiencies: generalized suggestions, insufficient interpretability, and physically implausible hallucinations.