Search tools

These tools find photos and video moments. They are the ones an assistant reaches for most.

search_photos

Find photos by a plain-language description. The assistant passes a description and gets back photos ranked by how well they match what is in the picture.

Inputs

Field	Type	Required	Notes
query	string	yes	A description, for example “sunset over water”
after	ISO date	no	Only photos taken on or after this date
before	ISO date	no	Only photos taken on or before this date
capture_kind	string	no	Limit by how the file was made: photograph, screenshot, export, or unknown
caption_has_text	boolean	no	true for only photos whose description contains transcribed text (signs, receipts, screens), false to exclude them
min_score	number (0 to 1)	no	Drop weak matches below this threshold
limit	integer	no	How many to return, default 20, up to 100

On the cloud surface, search_photos also accepts an optional country filter (a two-letter country code like US, or the country’s English name) to limit results to a country.

Add capture_kind: photograph when someone wants their real photos and not screenshots. Screenshots have very sharp synthetic edges, so they otherwise crowd into any results sorted by sharpness.

Returns a ranked list of photos. Each includes its file name, the date taken, camera details where available, a short description, and how the file was made (photograph, screenshot, export, or unknown). On the local connection, results also include the photo’s location on your device and a match score; the cloud connection keeps those out of what the assistant reads.

Tip

Short, concrete descriptions work best. “Red bicycle against a brick wall” beats “a nice photo of a bike.” The search reads what is in the picture, not the filename.

find_similar

Given one photo, find others that look like it. Useful for “more like this” once a photo has been found.

Inputs

Field	Type	Required	Notes
media_id	integer	yes	The photo to find matches for
limit	integer	no	How many to return, default 10
min_score	number (0 to 1)	no	Drop weak matches below this threshold

Returns a ranked list of visually similar photos, in the same shape as search_photos.

If the given photo exists but has not been processed yet, the tool says so rather than returning an empty result, so the assistant can explain the difference.

search_transcripts

Search the words spoken in your videos. Returns the moments where the words you are looking for were said, with timestamps that jump straight there.

Both connections

Available on the local desktop connection and, with Cloud AI Access, on the cloud connection too. The listening always happens on your own computer; the finished transcripts sync for the folders you share, so assistants on the web and your phone can search them as well.

Inputs

Field	Type	Required	Notes
query	string	yes	Words or a phrase to look for
after	ISO date	no	Only videos from on or after this date
before	ISO date	no	Only videos from on or before this date

Returns the matching videos and the moments inside them where the words occur, each with a timestamp.

The listening runs entirely on your computer: audio never leaves your device. With Cloud AI Access enabled, the text transcripts for the videos in your shared folders sync so they can be searched from anywhere; without it, transcripts stay on your device only.