Darwin Live Visuals Masterclass


THE ART OF LIVE VIDI-YO!
– Darwin Festival, Aug 2023
skynoise.net lnk.bio/jeanpoole

Example projects

Animated concert visuals for Ania Reynolds (piano album tour)
Hermitude concert visuals
Mexican Musicman music video
Curse Ov Dialect Animated music video

Live Cinema

What is ‘Live Cinema’?
Live Cinema as a performing art
The future of live cinema
The Light Surgeons (UK) – Super Everything
Regroup Collective (Syd)
Kid Koala – StoryVille Mosquito (CAN)

Projection Mapping

If you can corner pin, you can projection map.
For 3D – Perspective Matters. 3D tips via VVVV

Madmapper can use a live camera, to help with mapping onto organic shapes.
They call it their ‘spacial scanner’ feature. Simple to use – for example. (YT tute)

Examples: Icebook theatre (mapping video onto paper models)
Max Cooper (live concert mapped onto Acropolis)
VJ Suave (projections onto environments, and travelling through a city)
Installations by Yannick Jacquet (Legoman)
Installations by Romain Tardy  (including this botanic gardens projection in Mexico)
Shadow Spirit: First Peoples Exhibition – Melbourne – great example of blended projections.

Software:
Madmapper  (eg Madmapper 4 demo or Mapping a building or Spatial Scanner demo (how to use a camera to map an organic shape))
Mapping With Resolume (tutorial)
Mapping With QLab
Mapping With Millumin
Mapping With HeavyM
Other Mapping Software (Disguise, Touch Designer , Dataton Watchout , VPT (free) , Smode , VVVV , Mapmap (free) etc )

Lightform LightAct iOS + Android mapping apps:  DynaMapper

Other Useful Video Related Links

Inspirations

Lucky People Centre International (eng subs), (espanol subs) – Feature film with interesting AV layering.
Sculpture UK AV duo
Soda Jerk – collage artists extraordinaire. Latest feature film – Hello Dankness, and their earlier work.
RunWrake – animator and master at mixing media. RIP (site / YT videos)
Len Lye NZ – kinetic sculptor, scratch film artist and motion obsessive. (Free Radicals / A Colour Box / website.)
Regroup CollectiveCOIL show in Darwin + Darwin workshop
Jeff Noon’s Cobralingus – an interesting book and thought experiment about building imaginative processes / tools / workflows.
Brian Eno’s book/diary A Year With Swollen Appendices – full of interesting riffs – including it’s genesis, a note from a friend suggesting
“Why not consider you’ve already written your book, you just need to find it?”
(Esp relevant for visualists always creating new material, rather than reevaluating/ reharvesting/ remixing –
What if – your material is already on your hard drives, you just need to find it? (or the linking threads)?
Nathalie Miebach’s Sculptural storms

Interesting After Effects Software
Stardust – a particle engine, enabling graphics to be arranged + replicated in highly organised or chaotic ways
Newton – a physics engine, which animates graphics in scenes, according to defined rules (eg gravity / wind / springiness) etc
Paintlink – enables frame by frame painting over clips with Photoshop
Autofill – automated reveals along pathways of objects / images
Element 3D –  for exploring 3D objects inside AE
Saber – great for neon outlines
FXConsole – great for easy adding of FX to any layer.
Hundreds more boutique tools available at aescripts.com, covering lots of different niches / visual effects / AE problem solvers / time savers.

Alternatives to After Effects (for most aspects of it):
Motion (mac)
Da Vinci Resolve (free for mac / pc – see the Fusion section) Calvary  (mac/pc)
Figma (browser based motion graphic design)
RunwayML (browser based + machine-learning assisted. Monthly subscription, free version available.)
EB Synth – free software for auto-generating a range of stylised frames, based on an input. (see video)
(I used EB Synth to make the character animations in this music video)

What can we learn from Visual Music?

Center for Visual Music – history of visual music
The Visual Music Archive – a collection of inspirational works from the ever expanding field of Visual Music.
Synaesthesia: Music Notation – “Colour and music synaesthesia is a condition where individuals see colours in response to music stimuli.”

What can we learn from Dance Choreography?

The Choreographer’s Toolbox
– shape / space / timing / dynamics + form + structure
Dance_notation
http://bibliodyssey.blogspot.com.au/2006/07/visual-context-of-music.html 

What can we learn from Motion Graphics?

What is motion design? Motion Plus Design – have a nice 9 min video answer for this, and they are hoping to “found the world’s first Centre dedicated to the world of motion design”.

“To represent a dynamic study on a sheet of paper, we need graphic symbols of movement.”
Dziga Vertov, “We: The variant of the manifesto” (1920), in Kino-Eye: the writings of Dziga Vertov, ed. Annette
Michelson, trans. Kevin O’Brien (University of California Press, 1984), p.7.

A famous example of designing motion for the screen: Sergei Eisenstein’s sequence diagrams for his movies, Alexander Nevsky and Battleship Potemkin.

“Motion Graphic Scores use the ideas of Graphic Notation and reconfigure them regarding animation, time based media and the digital domain.” – Christian Fischer , What is a motion graphic score?.

Thinking about motion graphic design over time? See The Art of film titles (Esp. A Brief History of Title Design + their list of  film title analysis examples )

Audiovisual Academy – well structured series of videos looking at history of moving image, optics and photography, visual software, video hardware and LED equipment etc.

Related Reading

Understanding Comics – Scott McCloud
In the Blink of an Eye – Walter Murch ( Feature Doco –  SIGHT & SOUND: The Cinema of Walter Murch + The six criteria of film editing (6min) )
Motion Graphic Design, Applied History and Aesthetics, 2nd Edition By Jon Krasner
Live Visuals – History, Theory, Practice – Steve Gibson, Stefan Arisona, Donna Leishmann, Atau Manaka

Concert Visuals

Live Video Software
VDMX (mac) (See: The VDMX manual , tutorials, tutorial series of ‘Video fundamentals’ , and forum (ask questions + meet peers!))
Resolume (mac/pc)
ipad options? TouchViz, dJay Pro, Colorcode VJ

Video protocols for real-time sending video between apps – NDI, Syphon, Spout
(eg These enable video to be sent from VDMX/Resolume to/from mapping software, VFX software, game engines, utility apps etc – and do so efficiently, by piping the signal between the apps, using the graphics card)

Introduction to movie playback in VDMX
MIDI, OSC, DMX and other ways to control video
The Control Surface Plugin
Using a Control Surface to trigger clips in a Media Bin

Receiving MIDI Clock from Ableton Live in VDMX

Introduction To The Cue List PluginLinks to an external site.
Cue List Overview

Masking in VDMX5: Masking is the process of modifying an image or video’s alpha channel to alter how it composites with other images/videos, similar to using a stencil. This video demonstrates how to take the video being generated by one layer and apply it to another layer’s alpha channel, masking it
Luma Key and Masking Techniques in VDMX
Zooming Feedback
Using a VDMX Step Sequencer to trigger media clips
Sending DMX From a VDMX Color Picker
Using an iOS device as a live camera input in VDMX
Connecting an iOS Camera to VDMX Wirelessly Using AirBeam
Behind the Scenes of LCD Soundsystem’s Visuals at Coachella 2016

Other Useful Software
Test Card Auto generated Test Pattern (mac)
Monitor test Utility (PC)
Web-based test pattern generator
Neofinder Incredibly handy for locating files on various hard-drives (mac)

Hardware 
Programmer’s Guide to Video Systems  – explains why video systems have so many quirks! Display Technologies
In general, consider projectors and LED screens etc – as a second monitor.

Super helpful, very thorough reference articles by Blair Neal:
Survey of Alternative Displays (2022 Update)
A Creative Technology Taxonomy
How to make an installation stay up 4evr (MacOS)
Guide to Projectors for Interactive Installations
Guide to Cameras For Interactive Installations

Control systems / protocols
MIDI
OSC
Touch OSC (for ipad based control of parameters)
Timecode – used for synchronising AV, esp in large scale productions.

Live Cinema

What is ‘Live Cinema’?
Live Cinema as a performing art
The future of live cinema
The Light Surgeons (UK) – Super Everything
Regroup Collective (Syd)
Kid Koala – StoryVille Mosquito (CAN)

Projection Mapping

If you can corner pin, you can projection map.
For 3D – Perspective Matters. 3D tips via VVVV

Madmapper can use a live camera, to help with mapping onto organic shapes.
They call it their ‘spacial scanner’ feature. Simple to use – for example. (YT tute)

Examples: Icebook theatre (mapping video onto paper models)
Max Cooper (live concert mapped onto Acropolis)
VJ Suave (projections onto environments, and travelling through a city)
Installations by Yannick Jacquet (Legoman)
Installations by Romain Tardy  (including this botanic gardens projection in Mexico)
Shadow Spirit: First Peoples Exhibition – Melbourne – great example of blended projections.

Software:
Madmapper  (eg Madmapper 4 demo or Mapping a building or Spatial Scanner demo (how to use a camera to map an organic shape))
Mapping With Resolume (tutorial)
Mapping With QLab
Mapping With Millumin
Mapping With HeavyM
Other Mapping Software (Disguise, Touch Designer , Dataton Watchout , VPT (free) , Smode , VVVV , Mapmap (free) etc )

Lightform LightAct iOS + Android mapping apps:  DynaMapper

Other Useful Video Related Links

Inspirations

Lucky People Centre International (eng subs), (espanol subs) – Feature film with interesting AV layering.
Sculpture UK AV duo
Soda Jerk – collage artists extraordinaire. Latest feature film – Hello Dankness, and their earlier work.
RunWrake – animator and master at mixing media. RIP (site / YT videos)
Len Lye NZ – kinetic sculptor, scratch film artist and motion obsessive. (Free Radicals / A Colour Box / website.)
Regroup CollectiveCOIL show in Darwin + Darwin workshop
Jeff Noon’s Cobralingus – an interesting book and thought experiment about building imaginative processes / tools / workflows.
Brian Eno’s book/diary A Year With Swollen Appendices – full of interesting riffs – including it’s genesis, a note from a friend suggesting
“Why not consider you’ve already written your book, you just need to find it?”
(Esp relevant for visualists always creating new material, rather than reevaluating/ reharvesting/ remixing –
What if – your material is already on your hard drives, you just need to find it? (or the linking threads)?
Nathalie Miebach’s Sculptural storms

Interesting After Effects Software
Stardust – a particle engine, enabling graphics to be arranged + replicated in highly organised or chaotic ways
Newton – a physics engine, which animates graphics in scenes, according to defined rules (eg gravity / wind / springiness) etc
Paintlink – enables frame by frame painting over clips with Photoshop
Autofill – automated reveals along pathways of objects / images
Element 3D –  for exploring 3D objects inside AE
Saber – great for neon outlines
FXConsole – great for easy adding of FX to any layer.
Hundreds more boutique tools available at aescripts.com, covering lots of different niches / visual effects / AE problem solvers / time savers.

Alternatives to After Effects (for most aspects of it):
Motion (mac)
Da Vinci Resolve (free for mac / pc – see the Fusion section) Calvary  (mac/pc)
Figma (browser based motion graphic design)
RunwayML (browser based + machine-learning assisted. Monthly subscription, free version available.)
EB Synth – free software for auto-generating a range of stylised frames, based on an input. (see video)
(I used EB Synth to make the character animations in this music video)

What can we learn from Visual Music?

Center for Visual Music – history of visual music
The Visual Music Archive – a collection of inspirational works from the ever expanding field of Visual Music.
Synaesthesia: Music Notation – “Colour and music synaesthesia is a condition where individuals see colours in response to music stimuli.”

What can we learn from Dance Choreography?

The Choreographer’s Toolbox
– shape / space / timing / dynamics + form + structure
Dance_notation
http://bibliodyssey.blogspot.com.au/2006/07/visual-context-of-music.html 

What can we learn from Motion Graphics?

What is motion design? Motion Plus Design – have a nice 9 min video answer for this, and they are hoping to “found the world’s first Centre dedicated to the world of motion design”.

“To represent a dynamic study on a sheet of paper, we need graphic symbols of movement.”
Dziga Vertov, “We: The variant of the manifesto” (1920), in Kino-Eye: the writings of Dziga Vertov, ed. Annette
Michelson, trans. Kevin O’Brien (University of California Press, 1984), p.7.

A famous example of designing motion for the screen: Sergei Eisenstein’s sequence diagrams for his movies, Alexander Nevsky and Battleship Potemkin.

“Motion Graphic Scores use the ideas of Graphic Notation and reconfigure them regarding animation, time based media and the digital domain.” – Christian Fischer , What is a motion graphic score?.

Thinking about motion graphic design over time? See The Art of film titles (Esp. A Brief History of Title Design + their list of  film title analysis examples )

Audiovisual Academy – well structured series of videos looking at history of moving image, optics and photography, visual software, video hardware and LED equipment etc.

Related Reading

Understanding Comics – Scott McCloud
In the Blink of an Eye – Walter Murch ( Feature Doco –  SIGHT & SOUND: The Cinema of Walter Murch + The six criteria of film editing (6min) )
Motion Graphic Design, Applied History and Aesthetics, 2nd Edition By Jon Krasner
Live Visuals – History, Theory, Practice – Steve Gibson, Stefan Arisona, Donna Leishmann, Atau Manaka

[/vc_column_text][/vc_column][/vc_row]