Video looks like one input, but an AI cannot inspect it as one indivisible object. A two-hour recording is a sequence of changing images, sound, speech, and timestamps. Before a model can answer a question, some system has to decide which pieces of that sequence deserve space in its limited working context.
The surprising part is that a longer context window does not remove this decision. Each retained frame expands into many pieces the model must process, each second of audio adds more, and the relationships across time still have to be computed. Keeping more evidence can preserve a brief event, but it can also bury that event inside thousands of irrelevant moments.
Google's September 2026 agentic video mode now lets a model search a timeline and adjust frame rate, resolution, and modality around promising moments, making evidence selection a practical product choice rather than a hidden preprocessing detail.
So the useful question is: How can an AI answer a precise question about hours of video without placing every frame in context?
The compact answer is to treat long video as a budgeted search for evidence. Fixed-rate sampling buys predictable coverage. Change-aware sampling keeps moments when the picture moves or cuts. A temporal hierarchy narrows a long recording to a few candidate intervals. Query-adaptive rewatching then spends dense frames, higher resolution, or audio only where the question needs them.
We will follow one realistic request through all four techniques. A warehouse camera records aisle 7 for two hours. A worker enters at 10:00. A red forklift first touches shelf B at 10:04.3, then backs away. The question is not merely whether both objects appeared. It is: When did the forklift first touch the shelf after the worker entered?
That wording matters. The answer needs an identity, an order, and a brief physical relation. A transcript may confirm that a worker called for help. A single frame may show the forklift beside the shelf. Only a short sequence can establish the first contact.
A video becomes a budgeted evidence packet
A frame is one still image from the video timeline. In a common transformer-based path, a vision encoder splits that image into patches and converts the patches into numerical representations the language model can process. I will call those representations visual tokens. One frame is therefore not one token; it expands into a small field of spatial evidence.
Video adds a second requirement. The model must connect what appears in one frame with what changes in later frames. That temporal relationship distinguishes "forklift near shelf" from "forklift moves toward shelf, makes contact, then reverses." The model's attention mechanism can compare retained visual tokens across time, but it cannot reconstruct a moment that the input pipeline never kept.
Audio and transcripts form separate evidence streams. Audio preserves nonverbal signals such as a metallic impact. A transcript is cheaper to search and excellent for spoken names or instructions, but it omits silent motion. Timestamps align the streams so a clue in one can point to a region in another.
Now imagine sampling the warehouse video once each second. The system keeps frames at 10:04.0 and 10:05.0. The actual touch at 10:04.3 may fall between them. Raising the rate to ten frames per second makes the event much more likely to appear, but it multiplies the visual evidence for the entire two-hour recording.
This is the first useful insight: temporal detail is purchased with context. The family of techniques differs mainly in who decides where to spend that detail, and when.
