Will AI in DaVinci Resolve 20 Replace Editors?
The mention of AI in video editing is such a touchy subject. I mean, I don’t want to be replaced by the machine. The brand new DaVinci Resolve 20 beta is loaded with tons of new AI tools, but they seem to take care of the boring, time-consuming tasks so we can be more creative. This video and blog post will explain all the DaVinci Resolve 20 AI features, and we can decide what our creative future holds.
AI Music Editor
The new AI-powered Music Editor can be accessed in the audio tab of the inspector. You can enter a desired duration using standard time code entry rules and press the Retime button to shorten or extend the clip to that duration. This is an “intelligent edit” that works effectively with beat-driven music (pop, dance, etc.). It is not intended for free-form, ambient, or non-beat-driven material.
The Music Editor does not use time compression/expansion or alter the pitch of the music but instead uses editing techniques like a music editor might fit the material to the desired length for the video.
It will look for logical places to create transitions based on whether you’re extending length (by repeating section(s)) or removing material and attempting to use the original composition's ending or fade if it works well. If the ending is abrupt, it’s up to you to manually create a fade-out.
The Music Editor tool can also be decomposed to manually re-edit the audio clip outside of the special container created when it first performs the analysis.
AI Beat Detector
DaVinci Resolve Studio 20 can now automatically detect beats for a piece of music on the timeline and supports snapping to these beat marker locations. This can be especially useful when trying to make edits to the beat.
To detect beats, right click on the timeline audio clip and select Detect Music Beats. Once analyzed, the timeline clip displays beat indicator overlays that you can snap other timeline elements to.
You should turn snapping on and use the trim tool for adjusting clip duration to snap to the beat of the music.
NOTE: the Beat Detector will only work effectively with beat-driven music, and with 4/4 or 3/4 time signatures at this time. Also, since the Edit page is frame-based, the beat points will accurately indicate where the beats occur, but you may not be able to place them exactly on a beat due to the frame grid. You can alternatively move music cuts in the Fairlight page to sample locations.
AI IntelliScript
DaVinci Resolve 20 has a powerful new AI-powered IntelliScript tool that uses our transcription engine to match and generate a timeline using the original script. This feature matches the transcribed clips against the original script text file and places them on the timeline based on the best-selected takes. Any additional and alternative takes are automatically placed on additionally created tracks (and disabled). Once the timetable is set, you can review the best take on track one and preview alternative takes for specific regions.
Once you have transcribed your clips in the media pool, select all the relevant clips, right-click, and select Create New Timeline Using IntelliScript. When prompted with a file dialog, select a text file with the desired script and press open. The tool automatically matches for similar phrases rather than only exact phrase matches.
I found this tool to work incredibly fast, but it did miss a few of my scenes from a test shoot, and the edit points were very tight. I look forward to trying future iterations of the IntelliScript tool as the public beta of Resolve 20 progresses.
Magic Mask 2
Magic Mask v2 can now be found in the Magic Mask palette on the Color page. This new version has been entirely redesigned to provide dramatically better results than the previous version. It even allows you to paint in or paint out additional regions from the mask.
The additional paint-out tool works on a frame-by-frame basis and is helpful in removing any flicker caused by the magic mask. The paint tool works downstream of the magic mask creation. The + and - paint icons can be used to paint in and out additional sections manually in order to further refine the mask.
The new Magic Mask selects objects using points rather than strokes. Similar to the previous version, the mask can be refined with these additional "clicks." You can use the positive and negative clicks to include or exclude regions, respectively.
AI Audio Assistant
The Audio Assistant takes your unmixed audio tracks and creates a pro-level audio mix. It categorizes and organizes your tracks, evens out dialogue levels, adjusts the levels of music and SFX against dialogue, and masters the finished output to your desired delivery standard, such as YouTube.
This tool is located under the timeline dropdown from the top menu bar and did a great job. I found it did a rather harsh compression, but that could be because I chose the YouTube preset, which has a delivery spec setup for -14 LUFs, which is very loud.
AI Multicam SmartSwitch
DaVinci Resolve 20 supports a powerful new AI-powered multicam SmartSwitch tool. This tool analyzes the audio and video for lip movement to automatically switch multicam angles based on the active speaker.
It does take a little while to process, but it it could easily be run while you grab a coffee. Then, you sit back down to watch and refine if necessary so that you can very quickly publish your long-form podcast or have a first pass at a multicam edit for your documentary.
Here’s how it works:
Once you place a multicam clip on a timeline, open the multicam mode in the source viewer, select the multicam clip on the timeline, and select the new SmartSwitch icon. You can then configure the Multicam SmartSwitch tool based on your multicam editing preferences.
When the multicam clip has been edited to the timeline with a single mic source, you can also configure the tool to automatically switch the mic by enabling Switch Mic with Angle Switch.
AI Voice Convert
The feature is on the Edit page and has wide-ranging implications. It allows you to take a generated voice model that has been captured by Resolve, and “drive” that voice model using another voice recording (clip or clips), including any inflections, pitch variation, emotional color, etc.
It’s not like eleven labs where you type in a script and it generates an audio file for you, but it’s not too different. The main difference between the built-in DaVinci Resolve Voice Traning and generation is that you always convert an audio file that has been recorded. It takes audio and converts it instead of taking text and creating audio files.
Imagine replacing your voice if it had been recorded in a noisy environment, doing perfect ADR with the actor’s voice, or driving the narration of a non-scripted project if you don’t have a great voice sound but know what inflections you want.
AI Animated Subtitles
In DaVinci Resolve 20, you can add Fusion title templates to style and animate subtitle tracks.
To style a subtitle track, drag any Fusion title template from the effects library on the subtitle track header. The Fusion effects now override any track or per-caption styling applied to the track via the inspector, allowing you to create fine-tuned text looks from the underlying Fusion compositions.
Fusion title templates can now also access the word timing data of subtitles created using DaVinci Resolve's transcription engine. Add and customize Statement, Slide In, Word Highlight or other animated templates from the Effects Library, or create your own subtitle templates to highlight and animate the words currently being spoken.
This is a perfect implementation and is perfect for social media content.
AI Intellicut Audio
Intellicut works on the Fairlight page of DaVinci Resolve 20. It uses AI and speaker analysis to “checkerboard“ multiple speakers to their own track if the source is a single track. For example, if 2 actors are picked up by a single boom mic, they’ll be broken out to separate tracks, with the clips auto-cut. Once the operation is complete, it also mutes the original single track.
This is very useful because it allows you to apply different effects and gains to different voices at a track level.
Remove Silence
Remove silence is also on the Fairlight page of DaVinci Resolve 20 and in some ways acts like a gate or expander. In simple terms, the remove silence feature will delete sections of the audio below a set volume threshold.
Select one or more clips (or all clips on a track), right-click, and choose “Remove Silence.” A floating popover window appears, allowing you to adjust various parameters to remove silent portions (or near-silent, low-level material), leaving a series of clips with gaps where the undesired material was removed.
There are controls for threshold, padding to preserve the front or back of the area of interest, and minimum duration. There is also the ability to create fades.
As you adjust the controls, you can see what will result in a reddish highlight color.
Then, choose “Remove Silence” to finish.
AI Smooth Cut 2
The Smooth Cut transition in DaVinci Resolve 20 beta now automatically uses our AI-powered Speed Warp interpolation for extremely clean transitions in edits with jump cuts. Speed Warp AI is automatically enabled when you apply Smooth Cut transitions to edits. In situations where you would like the previous Smooth Cut behavior, uncheck the Speed Warp AI checkbox in the transition tab of the inspector.
For the first time since this sort of transition first appeared in all editing software a decade ago, I found this new version of Smooth Cut to be very useful. I used it to hide a jump cut in an interview. I could also see this as very helpful anytime we needed to stitch shots together that had some weird technical glitch on one or two frames.
AI SuperScale
AI SuperScale update with improved quality for 3x and 4x upscaling. SuperScale allows to uprez your older footage and integrate it better with modern high-res cameras. This is very similar to the very popular Topaz software.
AI Depth Map 2
DaVinci Resolve Studio 20 features a dramatically improved new Depth Map. The effects can be used in both better and faster modes. Adding the Resolve FX Depth Map to a clip automatically applies the new Depth Map version.
The depth map effect can be used to help blur out background to simulate shallow depth of field, or in my case I often use it to build color contrast into scenes with warming a foreground or cooling off a background.
