Decoder
A technology podcast produced by The Verge that examines business decisions, power structures, and industry trends within the technology sector. Each episode features in-depth discussions with executives and industry figures on topics shaping the direction of tech companies and the broader industry landscape.
Timeline (last 90 days)
In the decoder route, the context can be placed first and cached; under the causal mask, the context only needs one prefill to obtain the KV cache, which is then reused directly for each subsequent question, computing only the newly added question tokens.
In the tutorial's decoder route, with no fine-tuning whatsoever, the first question, "Is the customer requesting a refund," gives 95% confidence, and the second question, "Which team is responsible," gives 97% attribution to billing.
In decoder practice, if two options share the same first token (such as fraud and fraud review), a simple implementation fails outright and the model can only give 50/50; the author's handling is to throw an error when duplicate first tokens are detected, prompting the user to rewrite the options.
The decoder adds two mechanisms on top of the encoder: masked self-attention and cross-attention; masked self-attention ensures that during generation only previously produced tokens can be seen, while cross-attention allows the decoder to query the encoder's output.
The Decoder episode is the first installment of its Business Future series.
Jonathan Kanter engaged in a conversation on the latest episode of the Decoder podcast about whether AI needs antitrust exemptions.
The computation flow of a single Decoder layer is: the input vector passes through Self-Attention and Add & Norm, then through FFN and Add & Norm, producing the output vector
The current wave of tech backlash differs from previous ones in that it has shifted from localized criticism targeting individual companies to a systemic questioning of the entire tech industry's operating model (a legitimacy crisis).
All Facts (6)
In the decoder route, the context can be placed first and cached; under the causal mask, the context only needs one prefill to obtain the KV cache, which is then reused directly for each subsequent question, computing only the newly added question tokens.
50%UnverifiedIn decoder practice, if two options share the same first token (such as fraud and fraud review), a simple implementation fails outright and the model can only give 50/50; the author's handling is to throw an error when duplicate first tokens are detected, prompting the user to rewrite the options.
50%UnverifiedIn the tutorial's decoder route, with no fine-tuning whatsoever, the first question, "Is the customer requesting a refund," gives 95% confidence, and the second question, "Which team is responsible," gives 97% attribution to billing.
50%UnverifiedThe decoder adds two mechanisms on top of the encoder: masked self-attention and cross-attention; masked self-attention ensures that during generation only previously produced tokens can be seen, while cross-attention allows the decoder to query the encoder's output.
50%UnverifiedThe computation flow of a single Decoder layer is: the input vector passes through Self-Attention and Add & Norm, then through FFN and Add & Norm, producing the output vector
50%UnverifiedThe current wave of tech backlash differs from previous ones in that it has shifted from localized criticism targeting individual companies to a systemic questioning of the entire tech industry's operating model (a legitimacy crisis).
50%