TwelveLabs Pegasus 1.5 Lands on Amazon Bedrock: A New Powerhouse for Video Understanding

TwelveLabs' video-language model Pegasus 1.5 is now officially available on Amazon Bedrock, bringing developers more powerful structured processing capabilities for long videos. The model can convert up to two hours of video content into timestamped JSON metadata, and adds practical features such as video segmentation and multimodal prompting, marking another step forward for video understanding technology toward production-grade applications.

From Long Video to Structured Data: Core Capabilities of Pegasus 1.5

The core value of Pegasus 1.5 lies in transforming unstructured video content into machine-readable structured data. Developers only need to define the key moments or events they care about, and the model returns the timestamps and descriptions of all matching segments in JSON format. This capability is especially critical for scenarios such as video content retrieval, automated editing, and compliance review.

According to the official announcement, the new version supports instant processing of videos up to two hours long, without the need to pre-segment or upload the complete file, significantly improving workflow efficiency. At the same time, the model adds a video segmentation feature that can automatically identify scene changes or content boundaries in videos, providing a foundation for subsequent fine-grained analysis.

Video-Language Model is a class of multimodal models that understand both video visual content and natural language instructions. Unlike traditional video classification models, it typically uses a visual encoder to extract frame-level or clip-level features, then aligns them with a text encoder through cross-modal attention, thereby supporting open-ended queries, description generation, and temporal localization. Pegasus 1.5 belongs to this paradigm; its output is not a single label but timestamped structured JSON, which means the model needs temporal grounding capability—mapping language descriptions back to specific time segments in the video. Such models often rely on large-scale video-text pairs during training and need to handle long contexts during inference, placing high demands on frame sampling strategies and computational efficiency.

Multimodal Prompting and Reference Images: More Precise Search

Pegasus 1.5 introduces multimodal prompting capability, allowing developers to combine reference images to guide video analysis. For example, users can upload a product image and have the model locate all segments in the video where that product appears. This visual-example-based search approach is more intuitive and accurate than pure text prompts, and is especially suitable for applications in retail, media, and security.

The technical logic behind this feature is cross-modal alignment between image features and video frames to achieve fine-grained content matching. Although the official announcement did not disclose specific technical details, based on actual results, Pegasus 1.5 has improved both recall and accuracy in complex scenarios.

Cross-modal Alignment refers to mapping representations from different modalities (such as images, text, and video) into the same semantic space so that similar content is close to each other in vector space. In multimodal prompting scenarios, the model first uses a visual encoder to extract the feature vector of the reference image, then performs similarity calculation or attention interaction with video frame features to locate matching segments. The advantage of this approach is that there is no need to retrain for each query; fine-grained search can be completed using only examples at inference time. Its effectiveness depends heavily on the alignment quality learned during pretraining, as well as robustness under complex backgrounds, occlusion, and viewpoint changes.

Amazon Bedrock Integration: Lowering the Barrier to Video AI Adoption

Pegasus 1.5 is offered through Amazon Bedrock, meaning developers can directly use AWS's managed infrastructure without deploying and managing models themselves. Bedrock's pay-as-you-go model and unified API interface make integrating video understanding capabilities simpler. For enterprises already using the AWS ecosystem, this undoubtedly lowers the barrier to trying and adopting video AI technology.

From an industry perspective, video content is growing at an astonishing rate, but most video data remains in a state of "dark data" that is difficult to use effectively. The emergence of models like Pegasus 1.5 is expected to change this situation, making video search, summarization, and monitoring more automated and intelligent.

Amazon Bedrock is a fully managed generative AI service launched by AWS, providing a unified API to access foundation models from multiple providers, and supporting pay-as-you-go pricing, private fine-tuning, and security and compliance controls. Developers can invoke model inference through Bedrock without managing underlying GPU clusters. For compute-intensive tasks such as video understanding, Bedrock's elastic resource scheduling and managed features can significantly reduce operational complexity, but long video processing still incurs relatively high token or duration charges, so cost optimization needs to combine segmentation strategies and batch processing.

Application Prospects and Challenges

The release of Pegasus 1.5 opens new possibilities for video AI applications. In the media industry, editors can quickly locate footage; in security, systems can automatically flag anomalous events; on e-commerce platforms, merchants can efficiently generate chapter information for product videos. However, the accuracy of video understanding models is still limited by the coverage of training data and scene complexity, and actual deployment may require fine-tuning for specific domains.

In addition, the computational cost and latency of processing long videos are also factors to consider. Although Bedrock provides elastic resources, the economics of large-scale video processing still need to be evaluated. Whether the collaboration between TwelveLabs and AWS can find a balance between performance and cost will determine the pace of adoption for this technology.

One-line Summary

TwelveLabs Pegasus 1.5 launches on Amazon Bedrock, supporting two-hour video structuring and multimodal search.

Paragraph Summary

TwelveLabs' video-language model Pegasus 1.5 is now officially available on Amazon Bedrock. Its core capability is converting up to two hours of video into timestamped JSON metadata, and it adds video segmentation and multimodal prompting. Developers can use reference images to precisely locate target segments in videos, making it suitable for scenarios such as media retrieval, security monitoring, and e-commerce analysis. With Bedrock's managed infrastructure and pay-as-you-go model, the barrier to integrating video AI is significantly lowered. However, model accuracy is still limited by training data coverage and scene complexity, and the computational cost and latency of long video processing are also factors to weigh in actual deployment.