J-Comm: A New Framework for Jacobian-Guided Latent Communication in LLM Agents

A core challenge in multi-agent systems is: how can different models efficiently exchange intermediate reasoning states? Traditional approaches either share KV caches (which requires identical model architectures) or use text transmission (which incurs significant information loss and high token overhead). J-Comm proposes a new approach—latent communication guided by Jacobian matrices—allowing the receiver model to directly leverage the sender's intermediate computations without sharing caches or requiring equal widths.

Two Components: GEAR and the H+J Interface

J-Comm consists of two parts. The first is GEAR (Geometric Agreement for Latent Reasoning), a training-free latent reasoning method. At each step, it reweights candidate tokens so that their geometric consensus in embedding space is monotonically optimized. In other words, GEAR does not rely on additional training to improve reasoning quality; instead, it leverages the geometric structure of existing model embeddings to select more consistent reasoning paths.

The second is the H+J communication interface. The H channel (Hidden) provides continuous inputs from the feedback distribution as the "raw material" for the receiver's reasoning; the J channel (Jacobian) transmits intermediate state cues as vocabulary-indexed Jacobian lens coordinates, providing additional guidance during the receiver's computation. The receiver reconstructs these cues using its own vocabulary and injects them at intermediate layers during the prefill stage. The key to this design is that communication does not depend on shared KV caches and does not require the two models to have the same width.

The Jacobian Lens is the key background for understanding the J channel. In a Transformer, the partial derivatives of a given intermediate layer's hidden states with respect to the final output logits form a Jacobian matrix, where each element characterizes "how sensitive that hidden dimension is to the prediction of a particular vocabulary token." J-Comm compresses this matrix into low-dimensional coordinates indexed by vocabulary, effectively generating a fingerprint for the sender's intermediate states in a coordinate system of "which outputs will be affected." After receiving these coordinates, the receiver can approximate the corresponding sensitivity profile using its own vocabulary and decoding head, thereby determining which intermediate states are more critical to the result. Since the coordinates are indexed by vocabulary, the communication signal is naturally decoupled from specific model widths—this is why it does not require the two models to have equal width.

Experimental Performance: Leading Across Seven Benchmarks

In the Qwen3-8B to Qwen3-4B communication setting, J-Comm achieves the highest average score across seven benchmarks. Specific results include: 94.80% on MATH500, 54.55% on GPQA-Diamond, 73.33% on AIME2024+2025, and a 62.93% success rate on ALFWorld.

Compared with TextMAS (a text-based multi-agent communication baseline), J-Comm shows performance improvements ranging from 1.29 to 8.59 percentage points while reducing generated text tokens by 19.2% to 74.8%. The token reduction is considerable, demonstrating that latent communication offers a substantial efficiency advantage—it is not trading more computation for performance, but achieving better results with fewer text outputs.

ALFWorld, mentioned in the paper, is a text-based embodied decision-making benchmark where agents must complete multi-step operations in simulated home environments according to natural language instructions (e.g., "put the washed apple in the refrigerator"), with success rate measured as the proportion of completed tasks. Its difference from MATH500 and GPQA-Diamond lies in the fact that the latter two assess single-turn reasoning correctness, while ALFWorld assesses state tracking and planning across multi-turn interactions—thus testing whether communication signals are equally useful in long-horizon decision-making. GPQA-Diamond, meanwhile, is a graduate-level science QA subset with adversarially filtered questions that prevent search or memorization shortcuts, and is often used as a hard benchmark for measuring models' deep reasoning capabilities.

Why It Matters

The dominant paradigm for LLM agent collaboration remains "conversing in natural language," but text as a communication medium has inherent bottlenecks: it compresses continuous intermediate representations into discrete tokens, losing a large amount of computational information. J-Comm's value lies in providing a path that bypasses the text bottleneck while maintaining compatibility with heterogeneous models.

The introduction of the Jacobian lens is also noteworthy. It encodes the sensitivity of the sender's intermediate states to outputs into the communication signal, effectively telling the receiver "which intermediate states are more critical to the final result." This guidance is more targeted than simply transmitting hidden states.

However, current experiments focus on cross-scale communication within the Qwen3 series, from 8B to 4B. The effectiveness of communication at larger scales and across model families remains to be verified. Additionally, whether GEAR's geometric consensus optimization gains on reasoning tasks diminish as model capabilities improve is an open question.

Latent communication differs fundamentally from text communication in information bandwidth. Text communication discretizes the sender's continuous hidden states into vocabulary tokens, effectively decoding the result of lossy compression once more; latent communication directly transmits continuous vectors or their low-dimensional projections, preserving more intermediate computational information. The cost is that the receiver must be able to "read" these vectors—shared KV caches require structural consistency, while J-Comm bypasses this limitation through vocabulary-indexed Jacobian coordinates, enabling heterogeneous models to exchange cues as well. This line of research is related to directions such as "model merging" and "representation alignment," but differs in that it does not modify model weights—it only injects signals at inference time.

Implications for Multi-Agent Architectures

J-Comm's experimental results demonstrate at least one point: latent communication can surpass text communication on reasoning-intensive tasks (mathematics, science QA, embodied decision-making). If this conclusion holds across a broader range of model combinations, the design space for multi-agent systems will expand from "how to write better prompts" to "how to design communication interfaces."

For developers building agent applications, J-Comm's H+J interface approach offers a direction worth considering: communication does not necessarily require transmitting complete states—transmitting "key cues" together with the receiver's own reconstruction capability may be a more economical way to collaborate.

One-line Summary

J-Comm uses Jacobian-guided latent communication to enable heterogeneous LLMs to collaborate efficiently without sharing KV caches.

Paragraph Summary

J-Comm addresses the bottleneck of intermediate state exchange in multi-agent systems by proposing a latent communication scheme consisting of GEAR and the H+J interface. GEAR is a training-free latent reasoning method that reweights candidate tokens through geometric consensus; in the H+J interface, the H channel transmits continuous inputs from the feedback distribution, while the J channel transmits intermediate state cues as vocabulary-indexed Jacobian lens coordinates, which the receiver reconstructs using its own vocabulary and injects at intermediate layers during the prefill stage. This design does not depend on shared KV caches and does not require consistent model widths. In the Qwen3-8B to Qwen3-4B communication setting, J-Comm achieves the highest average score across seven benchmarks, with 94.80% on MATH500, outperforming TextMAS by 1.29 to 8.59 percentage points while reducing generated tokens by 19.2% to 74.8%. The authors note that cross-model-family communication and the pattern of GEAR's gains as model capability changes remain to be verified.