Compositing with Quartz Composer

There’s a lot of wonderful possibilities for real-time visual compositing with Quartz Composer. Most existing QC learning resources though, tend to emphasise the generative graphics capabilities of QC. For those with a post-production, animation, motion graphics or VJ background – QC’s composition potential can be difficult to unleash. This page is going to develop as both an introduction to compositing with QC, and a repository for compositing related examples. (It also presumes you’ve read up on some of the basics, and have installed QC). Any tips/refinements/suggestions most welcome.

– Jean Poole

(While this page is focussed around compositing with QC – resources for learning other aspects of QC are linked at the bottom.)

Composition Idea / Technique

Possible ways to achieve this in Quartz Composer?

Composite multiple video sources

– in specific positions / angles etc

QC_3Dcompositing

 

QC easily allows multiple images sources – simply add another billboard and attach an image to it, and it should appear. Each billboard can be repositioned and angled as preferred. When overlapping, the billboard with the higher number will be foregrounded, and the billboard with the lower number will be behind it..This kind of compositing also works for displaying video sources on the surfaces of structures such as cubes in QC.Key concepts to understand:

  • Compositing in the output window. ie. positioning billboards and sprites, sizing with QC units not pixels.
  • Compositing inside an image cord. ie. compositing multiple image sources together into single image output using blend mode patches, everything done in the pixel space of the inputs.
  • Dynamic dimensions of the output window and input sources, and the simple maths you’ll need to put things in specific places and have fit/fill/1:1 as you want them.

QC Patches to understand:

  • Billboard and it’s different sizing modes
  • Sprite and it’s 3D capabilities
  • Crop and Billboard vs. Image texturing properties and Sprite. (The latter is IMMEASURABLY faster but the controls are more abstracted)
  • Source Atop / Over / In and how they can be used for masking
  • 3D transform and how they can be nested
  • Mathematical Expression rather than Math as amongst other things it allows you to name your inputs and titles itself with what it’s doing. Sanity not spaghetti.

(compositing-example.qtz, with multiple image sources)

(Thanks to *sPark for tips for this section)

Pre-framing video into compositions

QC_compositing

 

This compositing is especially useful as a visual effect for use within apps such as VDMX / COGE – a QC patch with several carefully composited video inputs, will automatically and precisely position any video being played in VDMX/COGE, into the pre-composited position. Continue triggering or mixing any clips in VDMX/COGE and these will all appear within your pre-framed composition..Simple example uses might include framing an image within a TV screen in a room… or placing one billboard above the others, with an alpha channel ‘hole’ in it, and placing the video input behind that, for easy framing of your video clips..

QC_pre-framing-example-by-jp.qtz (500kb, includes notes, and example images shown on left.)

(How to prepare QC files for VDMX)

(How to prepare QC files for COGE)

(How to prepare QC files for Resolume)

Working With Layers from Photoshop

QC + COGE PSD :  video / coge forum thread.
CoGePSDLayers QCPlugin – present an image structure from a Photoshop PSD file’s layers, and a structure with the image resources – the image data “without the layer”.
(supports PSD files with 72dpi, rasterized layers only, without layer effects, masks, layer locking, layer opacity values and blend modes).


((To learn: structure patches, and how to maintain render sizes ))

Image Masking

mask_example

QC billboards + sprites can have a ‘mask image’ applied/connected to them, but this method  has limitations – it masks by multiplying the black of one layer over another, and  lets  the underneath layer show through the lighter areas. This means if your billboard /sprite is made smaller than fullscreen, your mask effectively becomes a black rectangle.mask-image-example.qtz ((Simple example using image of broken mirror as a mask image on 2 sprites, with gradient background. ))

Travel Mattes

– using alpha channels + track mattes in After Effects, to enable layer A to be composited on top of layer B, using layer C* as a compositing mask. (*eg the cross layer below )

travel-matte

This can be achieved in QC by placing the ‘Mask to Alpha patch’ in between an image mask and the Mask Image input on a billboard or sprite. ((Hat-tip to JimBob Hunter ))

QC_mask2alpha

Nesting + Pre-composing in AE

(Collecting parts of a composition as a group, then using that group as a single item within another composition…to apply FX to all, or easily create multiples etc )

Grouped hierarchies can be tackled a few ways in QC:Eg A 3D transformation patch can have a billboard displaying an image ( or multiple billboards), put inside it, which allows composition by group….The ‘Render in Image’ patch can have a range of items put inside it, and will enable QC to consider all of them as an overall image, which can then be composited as a whole, or have visual FX applied etc…Notes: This R.I.I. patch needs to have a ‘clear’ patch inside it, set as layer 1, and some parameters may need to be published so that they ‘emerge’ from within the patch, and can be seen in the level that the render in image patch is at..See also: ”Rendering Destination Dimensions” patch for maintaining image size relevant to your final output.Iterators / Replicate in Space… create multiple copies of an item within them…( tutorial )Challenge? how to adjust parameters individually for different objects / layers?.

Nesting_and_compositing.qtz (Annotated example by Minuek)

Replicate_In_Space.qtz  (Annotated example by Minuek).
((To learn: Structure related patches?? ))

Parenting

Parent-child / chain reaction relationships between objects / images / parameters

Relationships can be built by putting objects inside other patches, creating hierarchies like the above…Input splitters – can be used to send the same data from one parameter to another
Interpolators and maths patches can be used to make this data range better ‘suit’ what might be needed in another parameter….

((To learn: Structure related patches?? + separating iterations…??))

Null Objects

Invisible objects that allow different parameters to be linked and controlled easily, without placing any visual in the composition.

“Null objects are equivocated by the Input Splitter patch. Right-clicking a node will allow you to add an automatically typed Input Splitter. A little tip, if you plan on publishing the input, publish it before adding the input splitter, otherwise the default published name will be generic rather than the port’s name attribute.” (via Colin Roache in the comments)

Inverse Kinematics

(An animation technique related to parenting / chained relationships.)

InverseKinematics

Other software of note: Animata.
And related plugin for QC..
((To learn: ??))

Camera Path

– How to control the movement of a camera in QC?

The timelines patch allows movements to be more precisely controlled over time.((View settings (command + 2) then use bezier curves and graphs to map out movements over time. Many different timelines can be added. )).

See patch and video by Rybotron. (( Related discussion on VDMX forum… ))
((To learn: ??))

Camera Projection / Light transmission

having a 3D layer project onto others inside After Effects.)

(insert example explanatory image)

AE_camera_projectingAbove image, from AE PDF tutorial on projecting video inside AE.To investigate – whether QC can achieve projection like this.. ie can layers / surfaces have the AE equivalent of light transmission?((To learn: ??))

With the Above Concepts..

How to Create Custom Visual Compositing Effects in QC?

eg Parallax Effect

Layers moving at different speeds, relative to their distance from the horizon.

(insert example explanatory image)

eg Film-Strip Effect

– Utilising time delay of frames from any video…

See Film-strip patch from Joe Catchpole..The delay plug-in by Bangnoise
((To learn: Timeline? Queue? ??))

eg Radial Rotation

– with anchor point in corner

Put image inside ‘3D transformation’ patch.. And create the range of movement required. ( Eg select rotation, add a splitter, then set the min/max values for that splitter. )

Note: “Billboards don’t work in 3d space, so you need to work with sprites. Sprites are basically the same as a billboard, but you need to do some of the height and width calculation yourself. Example attached, of how to offset the anchor point, and how to size the sprite correctly based on different input images.” – skynoise example..

((To learn: How to easily adjust anchor point for rotation in QC? ??))

eg Radial Slices as visual effect:

qc-bb-radial

(image found via tumblr)

radial slices that juxtapose 2 x different:
– Videos or images…
– frames from the same video?
– versions of same video? ( Eg one with FX, one without )Possible ways to achieve this?
Create radial slices mask video for sending to mask image? Or for using in some kind of compositing combination??

Delay plug-in by Bangnoise “When the image texturing properties patch is used inside an Iterator, this can be used to split a image into multiple pieces. ”

(See Iterator tutorials @ Superflea )

((To learn: iterators / image texturing properties ??))

eg Delay Effect?

 

Delay patch?? Breaking into segments //
How to select a portion of an image within QC?
(Crop patch??) See example patch by ?? and delay plug-in by Bangnoise..- ?? “image texturing properties patch. When used inside an Iterator this can be used to split a image into multiple pieces.”.((To learn: image texturing properties patch / crop patch ??))

eg YOUR DESIRED / IMAGINED

VISUAL COMPOSITING EFFECT

 

 

The above tips are intended as a transitional guide to those approaching QC from a non-programmer background, and looking to achieve various cinematic composition / motion graphics / post-production / animation effects. To unleash the full power of QC, explore:.

.

Thanks to feedback and QC advice from: Minuek / *sPark / Mr.CoGe / Joe Catchpole

 

Quartz Composer tutorials at skynoise:

Part 1 (Intro and overview of resources available)
Part 2 (Making a split-screen effect)
Part 3: DIY Anchor Rotation FX for VDMX
Part 4: 3D Objects With Video Textures in VDMX
Part 5: Using Twitter Hashtags + RSS feeds in VDMX

skynoise.net

3 Comments

  1. Null objects are equivocated by the Input Splitter patch. Right-clicking a node will allow you to add an automatically typed Input Splitter. A little tip, if you plan on publishing the input, publish it before adding the input splitter, otherwise the default published name will be generic rather than the port’s name attribute.

  2. Pingback: Visual Compositing with Quartz Composer | Minuek

  3. Pingback: Motion Improv: How Visuals for Audego Album Were Made with Quartz Composer

Leave a Reply

Required fields are marked *.


This site uses Akismet to reduce spam. Learn how your comment data is processed.