Vision-Language Model

VLA directly connects pre-trained vision-language models with robot action policies, enabling robots to process natural language instructions, camera images, and low-level motion control in a unified manner.

VLMs (Vision Language Models) typically consist of a visual encoder (such as ViT), a language model, and a projection layer connecting the two, with representative models including GPT-4V/4o, Claude 3.5, the LLaVA series, and InternVL

RLVR is Reinforcement Learning with Verifiable Rewards, which guides model optimization by designing verifiable reward signals.

Vision-language models (VLMs) can simultaneously process image and text information and establish semantic associations between the two.

A VLM (vision-language model) is a type of model that simultaneously accepts image and text inputs and performs reasoning in a shared representation space.

ZAP (Zero-real Agent Policy) is a robot policy method that uses a pretrained vision-language model to provide world understanding and uses a simulation environment to teach action skills, without using any real robot data throughout.

The mismatch between the discrete token output representation of medical VLMs and the nature of continuous measurement tasks is a general problem.

Existing medical VLMs generate measurements as text token sequences, and the token classification loss does not directly optimize numerical error, so the ability to distinguish close values (such as 1.36 and 1.37) is diluted by the vocabulary structure.

VLMs often lack awareness of rapidly evolving internet memes they have not seen during training.

Human-preference-guided calibration does not change VLM parameters but adjusts their judging framework, making it lower-cost and more interpretable.

Without constraints, VLM judging criteria tend to drift, and sensitivity to fine-grained visual details (such as axis labels and arrow directions) is limited.

Existing chart evaluation benchmarks based on vision-language models (VLMs) generally lack systematic categorization of chart types and apply generic criteria to all charts.

Verifying the equivalence of mathematical steps essentially requires symbolic reasoning ability, whereas current vision-language models mainly rely on pattern matching and language priors and do not possess a strict symbolic computation engine.

Vision-language models may 'hallucinate' in math grading that requires precise derivation, giving annotations that appear reasonable but are wrong.

Vision-language models are relatively sensitive to the spatial layout, superscripts and subscripts, and fraction structures of handwritten symbols, and misreading easily occurs in handwritten scenarios, which is one of the main bottlenecks in the accuracy of applications like Penmentor.

A common engineering mitigation is to have the vision model only recognize and transcribe the handwritten content, then hand the transcription result to a symbolic computation system (such as SymPy or Mathematica) for equivalence verification, and finally have the language model organize the feedback language.

Existing fine-tuning strategies generally lack two key capabilities in object-goal navigation: noticing the target object in the field of view, and accurately estimating the distance to the target object.

General-purpose vision-language models can now take a question about an image directly and return segmentation masks, bounding boxes, and object detection results, replacing work that previously required training a dedicated vision system.

Vision-language models can already perform image segmentation, bounding boxes, and object detection before any fine-tuning, without retraining for a specific dataset, outputting coordinates and labels through prompts alone.

Visual input lets agents read screenshots, operate UIs and understand document layouts; agentic tasks require the model to make multi-step decisions rather than single-turn Q&A.