Refine Rendered Magic Mask 2 in Resolve 20
Previously, I demonstrated how to render the Magic Mask in place on the edit page by directing the blue mask output from the Magic Mask node to the color page's green RGB output, eliminating the need for re-tracking. Today, we'll refine the rendered Magic Mask in Fusion to address imperfect edges and add a personal touch.
Slap Comp
Prepare background images for quick combination in Fusion.
Decompose the ProRes file to original pixels and remove attributes to revert to native color.
Adjust Fusion workspace to a top-down layout and move the media out node to the bottom.
Import background images (Media in 2, 3, and 4) from the Media Pool.
Import the rendered Magic Mask shot from the Renders folder.
Merge the foreground (rendered Magic Mask) over the background by connecting its output to the background input of a merge node (yellow for background, green for foreground).
Create an alpha channel for the ProRes 422 render using the Bitmap tool.
Connect the output of the rendered Magic Mask to the yellow background input of the Bitmap node.
Set the Bitmap tool's channel to "luminance" to generate the alpha channel.
Connect the Bitmap tool's output to the merge node to create the initial slap comp.
Connect the merge output to media out and review on the edit page.
Core Mask and Edge Mask
Examine the mask (Bitmap tool) using the subview menu (V key) and color inspector to identify transparency (e.g., 0.99805 value).
The goal is to create a solid core matte while retaining intricate edge details.
Copy the current Bitmap tool and rename it "Core" or "Hard Mat."
In the inspector of the "Core" mask, adjust the "High" value until the alpha hits one, ensuring a solid core.
Keep "Clip Black" and "Clip White" checked when manipulating alpha channels.
Visualize subtle alpha values using gamma slamming (holding the view LUT button with color management enabled).
Choke in the "Low" value of the core mat to pull it inwards, ensuring a hard edge with no feathering.
Address transparency in areas like the shirt by firming up transparent spots.
Combine the soft edge mask and hard core mat using another merge node (Shift+Space, MRG).
Connect the soft edge mask to the yellow background input and the hard core mat to the green foreground input.
Future refinement of edges will be done using color curves.
Color Curves for Edge Refinement
Use color curves to correct and sculpt the edge of the Magic Mask to remove flicker.
Load the merge tool into the viewer and hit "A" for alpha.
Adjust the gamma slider to reveal extra "garbage" around the edge.
Add a "Color Curves" tool (Shift + Space, then type "curves" and select "Color Curves").
Connect the output of the merge tool to the yellow background of the curves tool.
Work on the alpha channel only by disabling red, green, and blue.
Pin a point at 0.5 (center gamma point) by setting both in and out to 0.5 to avoid over-adjusting.
Select a point on the curve and hit "F" to flatten, which sculpts the edge and removes flicker.
Dragging the number associated with the selected point allows for fine-tuning the edge.
The spline editor can also be used for more precise control by selecting the alpha, zooming to fit, and using "T" for transition to adjust ease in/out values.
Combining Masks with a Separate Merge
To combine the refined Magic Mask with another mask (e.g., a polygon mask for the background), use another "Merge" tool.
Take the output of the color curves tool (Magic Mask) and connect it to the background input of the new merge tool.
Create a new mask using the polygon tool to define the area you want to combine (e.g., a hill in the background).
Draw the polygon disconnected first to see the image you're working on, then close it (Shift + O).
Connect the output of the polygon mask to the foreground input (green) of the new merge tool (ensure it goes to the foreground, not the mask input).
Feather the polygon mask by selecting the polygon tool and adjusting the feathering control (temporarily hide controls with Command + K to see the blend better).
Right-click on "Shape Animation" for the polygon and select "Remove Polygon Polyline" if you don't intend to animate it. This allows for easier manipulation of individual points (Alt or Option drag).
Connect the output of this combined merge tool to the mask input of your main merge node to apply the combined mask.
Edge Mask Blur
This is a unique polish step. Observe the current composite; you'll notice a slight black edge, indicating it's not perfectly integrated.
Blending Technique: To blend better, blur only the edge of the composited image, not the un-composited image. This pulls pixels from both the foreground and background.
Applying Blur: Select the merged output, hit Shift+Space, and add a standard blur tool.
Initial Blur Result: The blur improves the blend, but blurs the entire image, which isn't ideal.
Custom Edge Mask Setup: Create a custom edge mask to limit the blur to the necessary blending area.
Creating the Edge Mask
Take the output of the main mask.
Blur this edge mask first (e.g., 0.5 blur). This is a crucial trick.
Use a curves tool (Color Curves, working only on Alpha) to precisely grab the semi-transparent edge.
Set a point at 0.5 input, output 1.
Drop the far-right output to 0.
Hit 'S' for smooth on all points. This allows for precise sculpting of the inner and outer edges.
Applying the Edge Mask to Blur
Connect the output of your custom edge mask to the blur's mask input.
This ensures only the edges are blurred.
Stencil the Mask with a Polygon Tool & Merge
There might be areas not needing blurring, such as the bottom part (the real foreground).
Use a merge node to combine masks.
Take the output of the curves (edge mask) into the merge.
Take the output of the polygon mask (the area to exclude) into the foreground of the merge.
Change the merge operator from "over" to "stencil" to cut out the unwanted area.
Final Blur Application: Apply the refined edge mask to the blur, so only specific regions (e.g., shoulder, head) are blurred, effectively integrating the foreground with the background and removing black edges.
Adding Film Grain
DaVinci Resolve's Film Look Creator has a great grain generator.
Since the original image is denoised and still images have no grain, adding film grain helps further integrate the composite.
Go to the Color page, add "Film Look Creator" to the node.
Set it to "Clean Slate."
In the inspector, enable "Grain" and select a type (e.g., 16mm for YouTube).
Resources:
Download the footage and learn more about combining masks by joining the "cutting club" at creativevideotips.com/cuttingclub.
