2026年9月1日
|
我们的全新视频分析智能体功能可将令牌消耗降低高达88%，成本降低高达66%，质量提升高达7%。
Rohan Doshi
Google DeepMind 高级产品经理
Mario Lučić
Google DeepMind 研究总监
收听文章
[[duration]]分钟
此内容由Google AI生成。生成式AI处于实验阶段。
今天，我们在最新模型中推出智能体视频理解：Gemini 3.7 Flash、3.6 Flash和3.5 Flash-Lite。这项新功能提高了准确性，同时大幅降低了视频分析的令牌使用量和成本。与结合代码执行与Gemini模型原生图像理解的智能体视觉类似，智能体视频理解利用Gemini的原生视频工具来提升性能，并解锁视频处理的新能力，如亚秒级时刻检索、更准确的异常检测、精确计数等。
该功能今日起可通过Gemini API在Google AI Studio和Gemini Enterprise Agent Platform中用于视频上传和YouTube视频。
基准测试
与当前“静态”处理（模型以固定帧率（默认1 FPS，可通过API调整）摄取视频）不同，智能体视频理解将模型的核心推理与原生视频工具配对，动态搜索、扫描和检查跨视觉帧、音频和转录文本的目标视频片段。在标准视频分析基准测试中，具备智能体视频理解的Gemini模型可将分析成本降低高达66%，令牌消耗降低高达88%，同时准确性提升高达7%。
这些效率提升在长视频（从10分钟的操作指南到90分钟的讲座和多小时录制）中尤为显著，而静态处理迫使开发者在高令牌成本或丢失关键细节的技术之间做出选择。
启用智能体视频理解可将令牌消耗降低高达88%，并使用Gemini 3.7 Flash将准确性提升高达7%。
虽然这些提升覆盖所有三个受支持的模型，但具备智能体理解的Gemini 3.7 Flash在整体质量上提供最佳表现，并在质量与成本效率之间实现最佳组合，使其在测试的视频理解模型中处于准确性-成本帕累托前沿。
使用智能体视频理解使Gemini 3.7 Flash在视频分析的准确性-成本帕累托前沿上占据一席之地。
工作原理
与静态处理（模型以固定帧率摄取媒体流）不同，智能体视频理解使Gemini能够以主动、目标导向的角色确定_观看什么_、以_什么速度_、通过_哪种模态_（帧、音频或转录文本），仅获取所需的时刻和信号。虽然开发人员以前可以手动完成此操作，但借助智能体视频理解，Gemini可以通过智能体循环完成，调用内部工具加载视频文件的相关部分，显著减少开发开销。
功能与用例
智能体视频理解改变了开发人员在各种高要求应用中处理长视频内容的方式。
- 亚秒级时刻检索：精确定位在1 FPS下容易错过的瞬间状态变化和紧凑剪切边界，使精确的自动化视频编辑成为可能。
- 长视频大海捞针式搜索：在不消耗数百万令牌的情况下回答跨多小时的复杂查询。
- 异常检测：以更高FPS重新采样感兴趣的时间窗口，以检查快速运动和细微视觉伪影。
- 动作与物体计数：准确跟踪重复的物理运动和随时间变化的独特物体。
实际结果
我们的许多早期访问合作伙伴在测试智能体视频理解时看到了强劲表现。以下是他们的评价：
开始使用
智能体视频理解可通过Google AI Studio和Gemini Enterprise Agent Platform中的Gemini API使用，覆盖Gemini 3.7 Flash、3.6 Flash和3.5 Flash-Lite。它使用标准Gemini API令牌定价，无额外功能费用。
要启用它，只需在API配置中将处理设置为“agentic”。阅读我们的开发者指南以深入了解该功能及如何开始使用。
from google import genai
client = genai.Client()
interaction = client.interactions.create(
model="gemini-3.7-flash",
input=[
{
"type": "video",
"uri": "https://youtu.be/7Z5Vy9JBANs",
"processing": "agentic"
},
{
"type": "text",
"text": "What are the 3 most important announcements in this keynote?",
},
],
)
print(interaction.output_text)
我们还将代理式视频理解带来的效率与质量提升，带给Google产品中的数十亿用户。该功能将很快在Gemini应用中向所有用户推出，覆盖Flash和Flash-Lite模型。未来几个月内，代理式视频理解还将为YouTube视频观看页面上的“询问YouTube”功能提供支持，利用Gemini基于视觉内容提供更高质量的答案。
_感谢以下人员对本工作的贡献：Sergi Caelles、Filip Pavetić、Ahmet Iscen、Suhas Yogin以及代理式视觉团队。
在您的收件箱中获取Google最新资讯
订阅我们的新闻通讯，获取产品更新、活动信息、特别优惠等更多内容。
您的信息将按照Google隐私政策使用。您可以随时选择退出。
Sep 01, 2026
|
Our new agentic feature for video analysis cuts token consumption by up to 88%, reduces costs by up to 66%, and boosts quality by up to 7%.
Rohan Doshi
Senior Product Manager, Google DeepMind
Mario Lučić
Research Director, Google DeepMind
Listen to article
[[duration]] minutes
This content is generated by Google AI. Generative AI is experimental
Today, we’re launching agentic video understanding across our latest models: Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite. This new capability improves accuracy while dramatically reducing token usage and costs for video analysis. Similar to agentic vision, which combines code execution with Gemini models’ native image understanding, agentic video understanding uses Gemini’s native video tools to improve performance and unlock new capabilities for video processing like sub-second moment retrieval, more accurate anomaly detection, precise counting and more.
The feature is available today for video uploads and YouTube videos via the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform.
Benchmarks
Unlike current ‘static’ processing, where the model ingests the video at a fixed frames-per-second rate (default 1 FPS, adjustable via API), agentic video understanding pairs the model’s core reasoning with native video tools to dynamically search, scan, and inspect target video segments across visual frames, audio, and transcripts. Across standard video analysis benchmarks, Gemini models with agentic video understanding reduce analysis costs by up to 66% and token consumption by up to 88%, while improving accuracy by up to 7%.
These efficiency gains are especially pronounced on long-form video (from 10-minute how-to guides to 90-minute lectures and multi-hour recordings), where static processing forces developers to choose between high token costs or techniques that drop critical details.
Activating agentic video understanding drops token consumption by up to 88% and boosts accuracy by up to 7% with Gemini 3.7 Flash.
While these gains span all three supported models, Gemini 3.7 Flash with agentic understanding offers the best possible quality overall and the best combination of quality and cost efficiency, putting it at the accuracy-to-cost pareto frontier among tested models for video understanding.
Using agentic video understanding places Gemini 3.7 Flash at the accuracy-to-cost pareto frontier for video analysis.
How it works
Instead of static processing where the model ingests media streams at a fixed frame rate, agentic video understanding enables Gemini to take an active, goal-directed role in determining what to watch, at what speed, and through which modality (frames, audio, or transcript), fetching only the moments and signals needed. While developers could previously do this manually, with agentic video understanding, Gemini can accomplish it through an agentic loop, invoking an internal tool to load the relevant part of the video file, significantly reducing development overheads.
Capabilities and use cases
Agentic video understanding transforms how developers can process long-form video content across a variety of demanding applications.
- Sub-second moment retrieval: Pinpoint split-second state changes and tight cut boundaries that are easily missed at 1 FPS, making precise automated video editing possible.
- Long-form needle-in-a-haystack search: Answer complex queries across multi-hour videos without consuming millions of tokens.
- Anomaly detection: Resample interesting time windows at higher FPS to inspect rapid motion and subtle visual artifacts.
- Counting action & object: Accurately track repeated physical movements and distinct objects over time.
Real-world results
Many of our early access partners saw strong performance while testing with agentic video understanding. Here’s what they have to say:
Getting started
Agentic video understanding is available via the Gemini API in Google AI Studio and Gemini Enterprise Agent Platform, launching across Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite. It uses standard Gemini API token pricing with no additional feature fee.
To enable it, simply set processing to "agentic" in the API configuration. Read our developer guide to get more insights into the feature and how to get started.
from google import genai
client = genai.Client()
interaction = client.interactions.create(
model="gemini-3.7-flash",
input=[
{
"type": "video",
"uri": "https://youtu.be/7Z5Vy9JBANs",
"processing": "agentic"
},
{
"type": "text",
"text": "What are the 3 most important announcements in this keynote?",
},
],
)
print(interaction.output_text)
We are also bringing the efficiency and quality improvements of agentic video understanding to billions of users across Google products. The feature will roll out to all users in the Gemini app across Flash and Flash-Lite models soon. And in the coming months, agentic video understanding will also power YouTube's ‘Ask YouTube’ feature on the video watch page, leveraging Gemini to deliver higher-quality answers grounded in the visuals.
Acknowledgement for their contribution to this work:Sergi Caelles, Filip Pavetić, Ahmet Iscen, Suhas Yogin, and the Agentic Vision team.
Get the latest news from Google in your inbox
Sign up for our newsletters with product updates, event information, special offers, and more.
Your information will be used in accordance with Google's privacy policy. You may opt out at any time.
本文内容采集自官方网站，排版和翻译可能与原页面存在差异。
阅读官方全文