What the Assistant can do

The Assistant edits the project in its own window. You ask in plain words, it makes the edits one at a time, and each one lands where you can see it. Almost all of them can be undone. This page goes through what you can ask for, job by job, and where the limits are.

Menu: View ▸ Assistant
Keys: ⌃⌘A
Where: Inspector ▸ Assistant tab
Needs: Your own Anthropic API key
Agent tools: get_projectrender_frameundoredo

The same tools as MCP

The Assistant works by calling tools, one for each job. They’re the same tools an outside AI agent uses when it connects to Frame Job over MCP, so anything here is something an agent can do too. There are a few exceptions, listed in What it won’t do, and some tools that write files ask you first.

Four tools are always at hand: reading the project, rendering a frame, undo and redo. It finds every other tool by searching for what it wants to do, so you never need to name one. A tool kept from it outright doesn’t turn up in that search at all.

Each job below lists its tools, because those are the names you’ll see in the tool call lines. All tools describes each one.

What you can ask for

Ask in your own words, for example “put a title over the first five seconds”. You don’t need to know which job a request falls under. See Chat with the Assistant for how to send one.

It can open, create and save projects, move the playhead, set the in and out range, change the selection, and enter or leave a nest. It can also make a new project from a range or a vertical version.

Saving asks first. Making a blank new project, a new project from a range or a vertical version asks first when it names where to save. Moving the playhead, setting the range and changing the selection aren’t edits, so they make no undo step.

A project it opens or creates gets a window of its own, and the Assistant stays with the project it started in.

Agent tools: open_projectnew_projectnew_project_from_rangemake_vertical_versionsave_projectset_playheadset_rangeset_selectionset_scope

Tracks, markers and the canvas

It can add, remove and reorder tracks, remove the empty ones, rename a track, hide, mute, solo or lock one, and set its matte and mix. It can add, move, rename and remove markers, make one a chapter, and set its colour, which you can’t do yourself. It can change the canvas’s size, frame rate and background.

Agent tools: add_trackremove_trackmove_trackset_trackremove_empty_tracksadd_markerset_markerremove_markerset_canvas

Media

It can import, relink and remove media, copy it into the project and set a file’s Camera Log. Copying into the project asks first.

It can also save where a file’s speech is, read the on-device analysis of a file, and attach or read a recording’s cursor file.

Agent tools: import_mediaadd_mediarelink_mediaremove_mediaset_media_colorset_media_speechconsolidate_mediaget_media_analysisset_media_cursorget_cursor_events

Placing clips

It can place media, text, shapes, backgrounds, adjustment clips and graphics. It can insert and overwrite, paste and duplicate, and put text behind a subject.

Agent tools: add_clipadd_text_clipadd_shape_clipadd_adjustment_clipadd_backgroundadd_blurred_backgroundtext_behind_subjectinsert_clipoverwrite_clippasteduplicateinsert_graphic

Editing clips

It can move, trim, split, slip and roll, rate-stretch, ripple-delete and lift, insert time and close gaps. It can detach audio and attach it again and make J-cuts and L-cuts.

For several cameras or long recordings it can split at shot changes, copy pieces to another track, switch a shot’s angle and switch by speaker.

Agent tools: move_cliptrim_starttrim_endrate_stretchsliprollsplitdeleteripple_deleteripple_delete_rangeripple_delete_rangeslift_rangeinsert_timeclose_gapdetach_audioattach_audiosplit_at_shotscopy_piecesswitch_angleswitch_by_speaker

Nests

It can nest clips or take a nest apart. An edit lands in whichever timeline the window is showing, unless the Assistant names a nest, and it can open a nest as you do. Markers belong to the timeline, never to a nest.

Agent tools: nestunnest

Speed

It can set a clip’s speed, reverse a clip, freeze a frame, add speed ramps, retime a range and speed up the typing in a screen recording.

Agent tools: set_speedset_reversedfreeze_frameadd_speed_rampretime_rangespeed_up_typing

Look and sound

It can change how a clip looks and sounds: position, scale, crop, colour, filters, volume, fades, text and its style, and shapes. It can rename or label a clip, turn its picture or sound off, and paste attributes.

It can also work with transitions, actions and zooms, including Auto Zoom, and with Remove Background, camera layouts and reframing.

Agent tools: set_videoset_audioset_textset_shapeset_clipset_enabledpaste_attributesanalyze_cutoutadd_transitionset_transitionset_transition_durationset_transition_styleadd_actionadd_zoom_actionauto_zoomswitch_layoutreframe_clipset_actionset_action_endremove_action

Captions and words

It can import captions, transcribe on this Mac, read a clip’s or the whole timeline’s timed words, re-lay word captions for the canvas, and export captions.

Transcribing runs on this Mac only, because the Assistant can’t send audio to the cloud. Exporting captions asks first.

Agent tools: import_captionstranscribeget_transcriptget_timeline_transcriptrelay_captionsexport_captions

Audio clean-up

It can find silences and filler words and cut them, normalise loudness, enhance a voice, match one clip’s sound to another and synchronise audio.

Agent tools: detect_silencedetect_fillersnormalize_clipenhance_voicematch_audiosync_audio

Graphics

It can write, change, place, render, rename, delete, restore, export and import graphics, import their fonts and pictures, and add shortcut overlays. Exporting a graphic asks first. See Graphics with an agent.

Agent tools: add_graphic_clipset_graphicwrite_graphicget_graphicgraphic_referencerender_graphicimport_graphic_fontimport_graphic_assetdelete_graphicrename_graphicrestore_graphiclist_graphicsinsert_graphicexport_graphicimport_graphicadd_shortcut_overlays

Export

It can export, check how an export is going, or cancel it. Exporting asks first, because it writes a movie and replaces any file already there.

Agent tools: exportexport_statuscancel_export

Cut words by asking

You can ask the Assistant to take out filler words, silences or a false start, for example “cut the false start at the beginning of this clip.” It cuts each request as one undo step, labelled Undo Ripple Delete rather than with the sheet’s name.

Its filler search reads only media that already has a cloud transcript made with filler words, and it can’t upload audio to make one. In other media the search finds nothing, and it reports which media has no such transcript. To start the paid transcription, press Transcribe in the Remove Filler Words sheet yourself.

It removes a false start only when you ask, because deciding what counts as one is a judgement of meaning. It reads the words and their times, says which it will cut, then cuts them. The words come from a transcript the file already has or from this Mac’s own speech recogniser, so no audio is uploaded. There’s no menu command for this.

See Cut false starts.

It looks at rendered frames

The Assistant checks its work by rendering a frame of the finished timeline, drawn the way an export draws it. It’s told to look before and after a visual edit. It renders at the playhead unless it picks a time, and it’s told to keep the image small.

Rendering writes a temporary picture file without asking. Rendering to a path it chose asks first.

Earlier renders don’t survive compaction, so it renders again when it needs to look. See Long conversations.

Each edit is an undo step

Every edit the Assistant makes is its own undo step on the project, the same as your own, and Edit ▸ Undo names it after the edit. A turn isn’t grouped, so a turn that made ten edits takes ten undos.

Graphics files are the exception. Writing, restoring, deleting or importing a graphic, and importing a font or picture for one, change files in the project and make no undo step. See Undo rules.

To take back its last change, press ⌘Z outside the message field. You can also ask the Assistant to undo or redo.

How it behaves

Short replies. A reply is a sentence or two, and it doesn’t repeat what the tool lines already show.
Captions only on request. It writes or restyles a caption only when you ask.
Medium effort, thinking on. The model always runs with these two Anthropic settings, and Frame Job’s Settings has no control for them. The faint working notes are its thinking updates.

Only its own window’s project

The Assistant acts only on the project in its own window, and each window has its own conversation. A tool call that names another project doesn’t run, and the transcript shows a failed: line: The assistant edits its own window's project, …; … is a different project.

If its project has closed, a tool call fails with “The assistant’s project is no longer open.”

Limits

It works only on the project in its own window, and each window has its own conversation.
It needs your own Anthropic API key. There’s no Frame Job account. See Set up the Assistant.
It can’t record, close its window’s project, or send audio to the cloud transcriber.
A tool call that arrives during a drag waits until you release the mouse. See While you are dragging.
Stop doesn’t cancel an export that’s already been queued. See Stop a turn.
With compaction off, a long conversation eventually fails until you Clear it.
Changing the key or the model ends every window’s conversation.