A work in progress, aimed as a useful overview of live video possibilities – a round-up of current software, hardware and techniques, and where possible, leaning towards the DIY side. A gathering spot for various links that illuminate some of the creative potential / opportunities available with video projection.
[ Last Updated : May 8, 2012 for a guest class at RMIT. ]
Examples
Live Audiovisual Exploration?
The Light Surgeons : True Fictions performance trailer.
and show reel. DFUSE – Particle Performance.Nosaj Thing’s audiovisual show.
Mapping Video Onto Surfaces
555 Kubik, Etienne De Crecy, Lasercave, dataflux.
Projection Enhanced Theatre
The Ice Book - a performance blending animation, puppetry, and film to bring a pop-up book vividly to life in front of the audience.
Projection Mapping Stop Motion Animations
La Cena by VJ Suave / see also Homeless by VJ Suave (interview with VJ Suave) and Suburbia by Zeal - a clip was made using a combination of stop motion animation and projection mapping – the link includes a video describing his process of automating DSLR and projector synchronisation.
Planning & Preparation
Planning. Every hour spent planning will be savoured later when it has saved time shooting, capturing, processing, editing, rendering, exporting, loading into VJ software etc etc. Some useful tools/ideas?
Storyboarding? ( see also Hitchcock iphone app )
Understanding Comics by Scott McCloud is a great book for stimulating thoughts about how to pre-visualise and storyboard your ideas.
The Sour music video is an excellent example of how good planning can bring excellent results ( in this case, utilising the webcams of users all over the world ).
Preparing for VJ performance? Think about variations of clips, templates, transitions, layers, masks, looping points. For live flexibility, think about modular clips that allow improvisation, adaptibility. Clips that loop nicely. Clips that layer and composite well together. Clips that can mask areas of the screen. Clips that allow dynamic transitions.
Production
There are a variety of basic editing packages out there to chop, layer and splice your videos. Dedicated compositing, motion graphics and visual effects software will likely prove very useful for video creation – from editors such as imovie and windows movie maker through to Final Cut Pro, Adobe Premiere or free software from Avid, and compositing software such as After Effects. In the 3D realm, there are options like Blender ( free ), 3DsMax, Cinema 4D and Unity Pro ( a game engine ). Increasingly though, it is possible to generate and manipulate a lot of video material directly from within VJ software optimised to work via graphics cards ( rather than the CPU ).
Software for VJing, live audiovisuals, real-time cinema
Software needs : appropriate codecs, resolution, responsiveness.
VJ Forums Wikipedia’s VJ software list.
VDMX 5 from vidvox.net ( Highly modular, adaptible, powerful possibilities, good Quartz Composer integration. )
Modul8 ( well suited for motion graphics )
Resolume Avenue ( Strong Audiovisual features )
Isadora ( another modular contender )
Max / MSP + Jitter ( Very powerful, customisable graphic programming language environment – People are excited about the MAX4Live software which integrates the sequencing possibilities of Ableton Live, and allows audio performers to tightly synchronise their custom Jitter parameters )
Processing ( More accessible variety of the above )
VVVV ( PC based patching software )
Animata – Live animation software, controllable via Quartz Composer and OSC – eg control limb movements / animation components in real-time )
Momo : How to use Ableton Live to sequence live video
Hardware
Basic Technical needs : projectors, screens, cables, adaptors.
Capturing, Mixing, Controlling, Processing, Outputting.
Capturing : A smorgasbord of sources including mobile phones and even digital still cameras can now capture or create moving images, which can be easily repurposed with a variety of editing software such as imovie, final cut pro, adobe premiere or free software from avid.
HD camera considerations? Which codec does it record as? Quality?
Popular with VJs? Canon + Nikon DSLRs.
Digital still cameras? Many can shoot high speed, which makes for great slow motion footage. Underwater casings are also more prevalent and cheaper for digital still cameras.
The Microsoft Kinect camera can be used to capture 3D data about a scene, or can be used in conjunction with a video camera to record and combine 3D + visual imagery.
Mixing : Video signals can be mixed with devices like these: Edirol V4 / Edirol V8 / AVM 02 / VIXID
The DVJ allows DVDs to be scratched and speed controlled like CDs.
The Spark DVI mixer shows promise for mixing HD digital signals of 2 computers.
Controlling : Video software can be controlled by much more than a double clicked mouse. Using some sort of hardware controller is one of the biggest significant steps to making live video more instrument-like, responsive, versatile and fun. Mostly we’re talking about using knobs, buttons, sliders and touchscreens to control video sfotware parameters. Most real-time video software also allows a range of easy automated processes to control parameters – eg letting audio analysis functions use audio levels to control a parameter, or setting up a range of mathematical processes which can add randomness or ‘swing’ to your video parameters.
I use a Korg Nano controller for it’s compact size, and previously used the Novation Nocturn for its portability and on laptop screen interface.
The Monome is a minimalist controller…
( see also, the arduinome )
Popular dedicated video controllers : Codanova, Livid Instruments Ohm64
Quneo: an intriguing new controller offering multi-touch, pressure, velocity and location sensitivity.
Using Turntables to control video + data ( not just music! ) - Serato Video / Mix Emergency - Serato Scratch video and quartz composer
The Jazz Mutant Lemur sports a huge touchscreen. See the Lemur controller being used in live Audiovisual manner. ( Lemur now have an advanced ipad app that replicates this functionality. )
iphones / iPads and Android touchscreen devices can be used as midi / OSC controllers : touch OSC, mrmr, MSA remote.
Any USB connectable joysticks, dancemats or game controllers can be used to manipulate video parameters, using Software such as Junxion. The DIY electronics Arduino kit : “a tool for making computers that can sense and control more of the physical world than your desktop computer. Arduino can be used to develop interactive objects, taking inputs from a variety of switches or sensors, and controlling a variety of lights, motors, and other physical outputs.”
Processing
Hardware as live FX Units eg Circuit bending.
Outputting
Mobile projectors open up possibilities. Livestream-ing to the web. Net-jams. Pico projectors are tiny pocket projectors that enable all kinds of projection possibilities. eg the videohuahua project.
Connecting to multiple screens / projectors can be done using external devices such as the Matrox Triple Head to Go card, which will divide for example – a very wide video sent from a single computer such as 2400×600 – onto 3 separate screens of 800×600.
Video Mirror Units ( Austria ) allow video projections to move around, by controlling a motorised mirror at the very front of the projector. The mirror and motor are moved using DMX signals, the protocol used for stage lighting. A device called UDMX, allows this to happen through a computer via USB ( it’s essentially a DMX to USB convertor all wrapped up inside a cable ).
Projection Mapping:
Madmapper MadMapper is built around the idea of sharing video content between applications, using a Mac OS X based framework called Syphon. ( List of tutorials for connecting various software to Madmapper. )
VPT : a free multipurpose realtime projection software tool for Mac and Windows created by HC Gilje.
Millumin - Mac software for ‘creating and performing audiovisual shows’. Includes 4 corner mapping for layers, mesh warping for screens and multi-display edge blending.
Meshwarp Server – ‘a versatile Mapping Tool to create complex Video Projection Installations.’
IR Mapio – ‘a set of editor and plug-in of FreeFrameGL standard to arrange 2D mapping projections of arbitrary complexity’ – to be used within VJ apps such as VDMX, Resolume or Modul8.
We also have the ability to generate our own 2D and 3D objects for projection onto. Aside from options such as laser cutting and printing 3d shapes – there is a range of origami software available that prints paper with the fold marks needed to make the paper into a 3D shape.
ORI-REVO: - A Design Tool for 3D Origami of Revolution.
ORIPA – An Origami Pattern Editor.
Treemaker - a program for the design of origami bases
Pepakura Designer - make templates for paperwork models from 3D data files.
Techniques?
Editing? ( In the Blink of an Eye – Walter Murch )
Processes & workflows.
Graphic Design.
Composition.
Motion Graphics and rhythms.
Zen TV Masterclass ( By Cold Cut / ninjatune.net ) – on how to create audiovisual loops for live sequencing and performance.
(the teeming void): After the Screen: Array Aesthetics and Transmateriality ( A great essay that frames a few interesting ideas about a shift to projections away from screens. )
Post-Production
Software? Adobe After Effects, Motion and many, many other applications can go beyond the edit, and provide rich processing capacity. As well as these traditional tools, many VJ applications provide capacity for real-time exploration of effects and manipulation.
Budding visual artists looking for something like Photoshop with a timeline, usually find themselves at the door of After Effects soon enough. And while higher end software packages exist, it remains a sophisticated and powerful application for creating animation or motion graphics, for layering and compositing text, graphics and video, and for adding special effects or post-producing ( colour corrections etc ). There’s an abundance of tutorial sites online, but there’s a few resources that are particularly handy :
videocopilot.net – has a great range of free introductory video tutorials, and sells more advanced tutorials ( as well as software plugins ).
ae.tutsplus.com – Probably the most regularly updating AE blog out there.
aescripts.com – Useful scripts that can make several tasks easier ( auto-detect edit points and separate clips into individual layers? No problem. )
creativecow.net/after-effects-tutorials-podcast – The pioneering motion graphics site also has 150+ video podcast episodes to wander through, from easy through to brain-squintingly hard.
Arguments about resolution & production : codec quality vs flexibility for performance ( with smaller file sizes )?
Recording & remixing.
Processes & workflows.
Live Video Inspirations:
Cold Cut (UK) – “Audiovisual collage pioneers back in the day..”
E.B.N. – “More 90′s AV cut n paste..”
The Light Surgeons ( UK ) – “Next level live audiovisual choreography..”
UVA (UK) – “yeah those guys that do the Massive Attack visuals.. and lots more”
Lucky People Centre (SE) “Provocative feature length cinema made with a DJ sensibility”
Solu (Fi) – “She has a beautiful live AV aesthetic..”
Gangpol Und Mit (FR) – “Rad and great live audiovisual French duo..”
Runwrake(UK) – “Fantastic and inventive animation veteran..”
Adam Curtis (UK) – “Not live – but this BBC documentary maker has a very fluid collage and narration style.”
*spark (UK) and Vade (US) – “Both interesting live video artists in their own right, and expanders of live video possibilities for others.”
And sure, cinema magicians such as: Jacques Tati, Alejandro Jodorowsky, Michel Gondry, Chris Marker, Werner Herzog, Harmony Korine, Stacy Peralta, David Lynch, Spike Jonze, Walter Murch, Federico Fellini, Stanley Kubrick.
Pingback: skynoise » Nov Eyeball Snippets