Turn any video, audio file, or image into facts a model can read.
One API call. AnalyzeMedia analyzes the file once and returns a small text document with every fact and its time: who is in it, what happens, what is said, what plays. A model reads that in under a second. Watching the video would cost it a minute and 60,000 tokens, every time.
Below: a real 3-minute clip and what the API returned for it. Move the pointer across the timeline.
Move the pointer across the timeline. Bars are spans, thin marks are instants; green is measured, blue is observed.
A is seen playing guitar and singing on the sidewalk.
- File
- One handheld take, 3:34, 1280 by 720, 50 MB
- People
- A, a young woman with a guitar, singing
- Camera
- Handheld, low jitter, zoom in to 1.56x
- Moments
- 1:34 and 2:49, money in the guitar case
- Passers-by
- At 0:08, 1:24, and 1:44
- Music
- Two songs, 112 BPM, C# major
How it works
Three calls with curl. The result is a .analyzemedia file, plain JSON, that you keep next to the original.
Send the file
Ask for an upload URL, then put the bytes to it. Up to 10 minutes and 300 MB.
POST /v1/uploads
{ "filename": "clip.mp4", "bytes": 50294174 }Say what it is
Start the job with the upload id and one of six types. The API never guesses the type from the content.
POST /v1/analyses
{ "upload_id": "up_…", "type": "video/footage" }Get the facts
Poll the job or give it a webhook. Then download the file and hand it to any model.
GET /v1/analyses/an_…/result → clip.mp4.analyzemedia
Every route, with request and response bodies, is in the route reference.
Why not let the model watch the video?
It can. It just costs a minute and about 60,000 tokens for a 3-minute clip, every time you ask, and it forgets the clip after each question. The following table compares the two approaches, first for the clip above and then for a library of 1,000 clips like it.
| Watch the video | Read the file | |
|---|---|---|
| One clip | Upload 50 MB. The model takes more than a minute and about 60,000 tokens. | The model reads about 4,000 tokens immediately. |
| 1,000 clips | Upload 50 GB. The model takes a day, for one question. | Search a folder of text files, or give the model the summaries. |
| The second question | The same cost again. The model kept nothing. | No cost. The file is unchanged. |
| A different model | Another upload, and possibly a different answer. | The same file and the same facts. |
The figures assume a 1280 by 720, 30 fps clip and a video model that uses about 300 tokens per second of video. The file is the 12.6 KB result above. Exact numbers vary by model.
The same questions, answered both ways
Each transcript asks once and shows two turns: a model that watched the video, then a model that read the file the API returned. Every second answer comes from the file, with the times it records.
Six file types, one price each
You declare the type when you start a job, and the API returns the fields that type defines. Every job runs the full analysis. There are no tiers.
| Type | What the API returns | Price |
|---|---|---|
| video/footage | One continuous take as recorded. Describes what the camera captured and how the camera moved. | 20¢ per minute |
| video/edit | A finished video. Describes the scenes, cuts, layouts, captions, overlays, and sound. | 20¢ per minute |
| audio/speech | A recording of people talking. Includes the transcript, the speakers, the silences, and the levels. | 6¢ per minute |
| audio/music | A track or a mix. Identifies the recording and describes its tempo, key, beats, and energy. | 2¢ per minute |
| image/scene | A photo or a video frame. Describes the composition, the people, and the objects, with bounding boxes. Has no time. | 3¢ per image |
| image/graphic | A screenshot, document, slide, or chart. Lists its parts in reading order, with bounding boxes. | 3¢ per image |
A failed job is never charged. The same file analyzed again as the same type is free. Pricing has the minimums, the credit packs, and the limits.
What people build with it
We built AnalyzeMedia to enable agentic video editing. An agent reads the files for a folder of clips and finds the usable segments, the people in frame, the moments when they look at the camera, and the silences. The agent then cuts the video without watching it. The same files support other uses.
- Agentic video editing. Give an agent the files instead of the footage. The agent picks the take, the moment, and the cut points from the files, and only touches the video to render the result.
- Competitor ads in ecommerce. Analyze a competitor's ads and ask questions across all of them: how fast they cut, what the captions say, which hooks appear in the first three seconds, and what music plays.
- Content research for creators. Analyze the videos that perform well in your niche and compare their structure: scene count, caption style, overlays, pacing, and where the speech sits.
- A searchable media library. Thousands of clips, photos, and recordings become a folder of text that a model can search by person, place, time, and spoken words.
- Subtitles, chapters, and alt text. A script can derive these from the file without running a model: captions from the timed words, chapters from the scenes, and alt text from an image's summary.
The output is an open format
The API returns the analyzemedia format, version 0.1: one JSON file that sits next to the original and describes it. The format is public. Any program can read a file, and any program that can measure and describe media can write one.
Every value says how it was found
Measured from the bytes, or observed by a model. The file records which, field by field. No field holds an opinion.
Bound to the original
The file carries the original's hashes, so a reader knows it describes this file and not a re-encode with the same name.
Documented field by field
What a sidecar is, the six types, and how to write one are in the docs.
Start with $5 of credit
Enough for 25 minutes of video. No card needed to try it. Or drop files on the Mac app and let it call the API for you.