I spent a few nights in a hospital basement last year, projecting video and controlling lights for The General Assembly – onto a room filled with paper strips, while audiences roamed between rooms for mini-sets. It was part of Melbourne Music Week and super fun – the video below shows it up nicely.
Will be doing projections for TGA again this saturday at The Toff In Town :
Melbourne, as the most Nathan Barley of Australian cities, so easily lampooned for its population of bushranger bearded baristas with half-baked app ideas, makes a strong argument for being Australia’s Portland. Perfectly placed then, for reviewing Lumen – new real-time visual software coded by Jason Grlicky in downtown Portland, which tries to add some contemporary twists to the quirky history of video synthesis.
What is Lumen?
A mac based app (needing OSX 10.8 or later) for ‘creating engaging visuals in real-time’… with a ‘semi-modular design that is both playable and deep.. the perfect way to get into video synthesis.’ In other words – it’s a software based video synthesiser, with all the noodling, head-scratching experiments and moments of delightful serendipity this implies. A visual synthesiser – that can build up images from scratch, then rhythmically modify and refine them over time. It has been thoughtfully put together though, so despite the range of possibilities – it’s also very quickly ‘playable’ – and always suggesting there’s plennnnttttyyyy of room to explore.
The Lumen Interface
While the underlying principles of hardware based video synthesisers are being milked here to good effect – a lot of the merits of Lumen are in the ways they’ve managed to make these principles easily accessible with well considered interface design. It has been divided into 3 sections – a preset browser (which also features a lovely X/Y pad for interpolating between various presets), a knob panel interface, and a patch panel interface. It’s a very skeuomorphic design, but it also cleverly takes the software to places where hardware couldn’t go (more on that later).
What should be evident in those screengrabs, is that experimentation is easy- and there’s a lot of depth to explore. The extensive reference material helps a lot with the latter. And as you can see, they can’t help but organise that beautifully on their site:
Lumen comes pre-loaded with 150+ presets, so it’s immediately satisfying upon launch, to be able to jump between patches and see what kind of scope and visual flavours are possible.
… and it’s easy to copy and remix presets, or export and swap them – eg on the Lumen slack channel.
Midi, OSC + Audioreactivity
Although all are planned, only midi exists in Lumen so far, but it’s beautifully integrated. With a midi controller (or a phone/tablet app sending OSC to a midi translating app on your computer) – Lumen really comes into it’s own, and the real-time responsiveness can be admired. Once various parameters are connected via midi control, those of course can effectively be made to be audioreactive, by sending signals from audioreactively controlled parameters in other software. Native integration will be nice when it arrives though.
Video Feedback, Software Style
Decent syphon integration of course opens a whole range of possibilities…. Lumen’s output can be easily piped into software like VDMX or COGE for use as a graphic source or texture, or mapping software like madmapper. At the moment there are some limitations with aspect ratios and output sizes, but that’s apparently being resolved in a near-future update.
With the ability to import video via syphon though, Lumen can reasonably considered as an external visual effects unit. Lumen can also take in camera feeds for processing, but it’s the ability to take in a custom video feed that can make it versatile – eg video clips created for certain visual ideas, or the output of a composition in a mapping program.
This screengrab below shows the signal going into Lumen from VDMX, and also out of lumen back into VDMX. Obviously, at some point this inevitably means feedback, and all the associated fun/horror.
macOS 10.8 or newer (Each license activates two computers)
There’s an army of lovers of abstracted visuals that are going to auto-love Lumen, but it has scope too for others looking for interesting ways to add visual textures, and play with real-time visual effects on video feeds. It could feasibly have an interesting place in a non-real-time video production pipeline too. Hopefully in a few years, we’ll be awash in a variety of real-time visual synthesis apps, but for now Lumen is a delightfully designed addition to the real-time video ecosystem.
Interview with Lumen creator, Jason Grlicky
– What inspired you to develop Lumen?
I’ve always loved synthesizers, but for most of my life that was limited to audio synths. As soon as I’d heard about video synthesis, I knew I had to try it for myself! The concept of performing with a true video instrument – one that encourages real-time improvisation and exploration – really appeals to me.
Unfortunately, video synths can be really expensive, so I couldn’t get my hands on one. Despite not being able to dive in (or probably because of it), my mind wouldn’t let it go. After a couple failed prototypes, one morning about I woke up with a technical idea for how I could emulate the analog video synthesis process in software. At that point, I knew that my path was set…
– When replicating analogue processes within software – what have been some limitations / happy surprises?
There have been so many happy accidents along the way. Each week during Lumen’s development, I discovered new techniques that I didn’t think would be possible with the instrument. There are several presets that I included which involve a slit-scan effect that only works because of the specific way I implemented feedback, for instance! My jaw dropped when I accidentally stumbled on that. I can’t wait to see what people discover next.
My favorite part about the process is that the laws of physics are just suggestions. Software gives me the freedom to deviate from the hardware way of doing things in order to make it as easy as possible for users. The way that Lumen handles oscillator sync is a great example of this.
Can you describe a bit more about that freedom to deviate from hardware – in how Lumen handles oscillator sync?
In a traditional video synth oscillator, you’ll see the option to sync either to the line rate or to the vertical refresh rate, which allows you to create vertical or horizontal non-moving lines. When making Lumen, I wanted to keep the feeling of control as smooth as possible, so I made oscillator sync a knob instead of a switch. As you turn it clockwise, the scrolling lines created by the oscillator slow down, then stop, then rotate to create static vertical lines. It’s a little thing, but ultimately allows for more versatile output and more seamless live performance than has ever been possible using hardware video synths.
Were there any other hardware limitations that you were eager to exploit the absence of within software?
At every turn I was looking for ways to push beyond what hardware allows without losing the spirit of the workflow. The built-in patch browser is probably the number-one example. Being able to instantly recall any synth settings allows you to experiment faster than with a hardware synth, and having a preset library makes it easier to use advanced patching techniques.
The Snapshots XY- Pad, Undo & Redo, and the Transform/K-Scope effects are all other examples of where we took Lumen beyond what hardware can do today. Honestly, I think we’re just scratching the surface with what a software video instrument can be.
How has syphon influenced software development for you?
I had an epiphany a couple years back where I took a much more holistic view of audio equipment. After using modular synths for long enough, I realized that on a certain level, the separation between individual pieces of studio equipment is totally artificial. Each different sound source, running through effects, processed in the mixer – all of that is just part of a larger system that works together to create a part of a song. This thinking led me to create my first app, Polymer, which is all about combining multiple synths in order to play them as a single instrument.
For me, Syphon and Spout represent the exact same modular philosophy – the freedom to blend the lines between individual video tools and to treat them as part of a larger system. Being able to tap into that larger system allowed me to create a really focused video instrument instead of having to make it do everything under the sun. Thanks to technologies like Syphon, the future of video tools is a very bright place!
What are some fun Lumen + Syphon workflows you enjoy – or enjoy seeing users play with?
My favorite workflow involves setting up Syphon feedback loops. You just send Lumen’s output to another VJ app like CoGe or VDMX, put some effects on it, then use that app’s output as a camera input in Lumen. It makes for some really unpredictable and delightful results, and that’s just from the simplest possible feedback loop!
What are some things you’re excited about on the Lumen roadmap ahead?
We have so many plans for things to add and refine. I’m particularly excited about improving the ways that Lumen connects with the outside world – be that via new video input types, control protocols, or interactions with other programs. We’re working on adding audio-reactivity right now, which is going to be a really fun when it ships. Just based on what we’ve seen in development so far, I expect it to add a whole new dimension to Lumen while keeping the workflow intuitive. It’s a difficult balance to strike, but that’s our mission – never to lose sight of the immediacy of control while adding new features.
I recently animated some vintage botanical illustrations for an interactive exhibition installation at The Royal Botanic Gardens, Sydney. It was fun to collaborate with Robert Jarvis ( zeal.co ) on this – who programmed the interactivity (incorporating childrens’ webcam photos into the various creatures and plant-life storylines), as well as with D.A. Calf ( dacalf.com ) who brought the world to life so well. And a special shout-out to Luke Dearnley and Sophie Daniel who produced it.
One of the video-art greats passed away recently – RIP Bill Etra, who leaves behind a huge legacy for his work at the intersections of art and technology. Below, Bill Etra demonstrates the functions of the Rutt/Etra Video Synthesizer. (1974)
“Bill Etra, an artist and inventor who, with a partner, created a video animation system in the early 1970s that helped make videotape a more protean and accessible medium for many avant-garde artists, died on Aug. 26 near his home in the Bronx. He was 69.
The cause was heart failure, said his wife, Rozalyn Rouse Etra. Mr. Etra had spinal stenosis for many years and was mostly bedridden when he died.
Mr. Etra and Steve Rutt created the Rutt/Etra video synthesizer, an analog device studded with knobs and dials that let a user mold video footage in real time and helped make video a more expressive art form. Among the artists who used it were Nam June Paik, regarded by many as the father of video art, and Woody and Steina Vasulka, who founded the Kitchen performance space in downtown Manhattan in 1971.”
“The dream was to create a compositional tool that would allow you to prepare visuals like a composer composes music,” Mr. Etra wrote. “I called it then and I call it now the ‘visual piano,’ because with the piano the composer can compose an entire symphony and be sure of what it will sound like. It was my belief then, and it is my belief now after 40 years of working towards this, that this will bring about a great change and great upwelling of creative work once it is accomplished.”
“Developed in 1972, the RUTT/ETRA Video Synthesizer was one of the first commercially available computerized video animation systems. It employed proprietary analog computer technology to perform real time three dimensional processing of the video image. In the first use of computer animation in a major Hollywood picture, Steve Rutt, working directly with Sidney Lumet, used the Rutt/Etra to create the animated graphic for the film’s “UBS” Television Network.”
Rutt-Etra-Izer is a WebGL emulation of the classic Rutt-Etra video synthesizer, by Felix Turner, which ‘replicates the Z-displacement, scanned-line look of the original, but does not attempt to replicate it’s full feature set’. The demo allows you to drag and drop your own images, manipulate them and save the output. Images are generated by scanning the pixels of the input image from top to bottom, with scan-line separated by the ‘Line Separation’ amount. For each line generated, the z-position of the vertices is dependent on the brightness of the pixels.
Am glad to finally upload that edit-medley – because creating a set of concert visuals for Hermitude was one of my favourite projects last year, seeing it from drawing-board and sketch paper, through to the stage screen. Hermitude had approached (having worked together on Dr.Seuss Meets Elefant Traks at Sydney’s Graphic Festival in 2012) – about developing video for their tour promoting Dark Night, Sweet Light – and wanted a visual set that suited their music, would work well within a hectic stage lighting environment, and was diverse but felt like a coherent, consistent show.
To suit Hermitude’s fun and festive sound and their dynamic live performances – I developed an overall visual style palette to enhance that, and mapped out a visual choreography for the show. And though I was excited about making some Hermitude clips of my own, it was also an exciting opportunity to collaborate with some talented animators, coders and cinematographers. It was fantastic to be able to work with these artists to craft the Hermitude set:
Neil Sanders – a Melbourne hand-drawn illustrator and animator extraordinaire, famous for his signature organic tumblr loops… Ori Toor – another hand-drawn abstraction loop specialist, beaming pixels to us from the Middle East. Colin E. White – moodily stylised New York animator. Brad Hammond – A Melbourne 3D Unity animation ninja + coder. (And shout-out to Kejiro Takahashi from Japan, for his ongoing publishing of Unity software addons… ) Stu Gibson – A Tasmanian surf + aerial cinematographer, who was very generous with his wild coastline footage (which I used to make the Bermuda Bay clip below)
It was also a pleasure to develop this visual set over time, because Luke ‘Dubs’ + Angus ‘El Gusto’ (aka Hermitude) are so down to earth and friendly, despite their relentless touring and acclaim, as are the whole Elefant Traks crew – especially their tireless manager (and collaborator) Urthboy and Luke Dearnley (Sub Bass Snarl), their wizardly tour manager (who designed a clever + efficient video rig featuring live cams – for routing and controlling their stage video feeds).
A lot of pixel-sweat across quite a few months, but …
.. so satisfying to see it all come together in the end.
I was lucky enough recently to catch a film-talk panel between director Joshua Oppenheimer and John Safran, at the Melbourne International Film festival. Having just seen the Look of Silence earlier that day, and already in awe of the brave and audacious film-making from the earlier companion film (The Act of Killing) – it was humbling and a privilege to hear about some of what went into the making of the film – and what some of its’ impacts have been since.
Given that Indonesia has not officially or publically discussed the mass killings that happened in 1965-66 (supposedly to get rid of a communist threat) – and that many of the perpetrators are entrenched in power today, it’s quite remarkable that these two films got made – prompted national discussions about them – and that the second film was given official recognition:
“On November 10, 2014, 2,000 people came to the official and public premiere of the film in Jakarta, and on December 10, 2014 – International Human Rights Day – there were 480 public screenings of the film across Indonesia. The screenings of the film in Indonesia has been sponsored by the National Human Rights Commission of Indonesia and the Jakarta Arts Council.” ( Via wikipedia)
Incredibly, after the first film – which featured the ‘surreal / defensive(?)’ boasting of one of the mass-killers – an Indonesian journalist saw the film, and persuaded their magazine to send out investigative journalists to document similar people in 60 different locations across Indonesia – and then published all of these in one go – alongside an in depth reaction to Oppenheimer’s film – which broke the silence, and allowed Indonesian media to move past the taboo of discussing these events.
Regardless of your awareness of this Indonesian mass killing, these are powerful films on many levels – well worth hunting down.
The film focuses on the perpetrators of the Indonesian killings of 1965–66 in the present day; ostensibly towards the communist community where almost a million people were killed.
Invited by Oppenheimer, Anwar recounts his experiences killing for the cameras, and makes scenes depicting their memories and feelings about the killings. The scenes are produced in the style of their favorite films: gangster,western, and musical.
The name “Anonymous” appears 49 times under 27 different crew positions in the credits. These crew members still fear revenge from the death-squad killers.
When the government of Indonesia was overthrown by the military in 1965, Anwar and his friends were promoted from small-time gangsters who sold movie theatre tickets on the black market to death squad leaders. They helped the army kill more than one million alleged communists, ethnic Chinese, and intellectuals in less than a year. As the executioner for the most notorious death squad in his city, Anwar himself killed hundreds of people with his own hands. Today, Anwar is revered as a founding father of a right-wing paramilitary organization that grew out of the death squads. The organization is so powerful that its leaders include government ministers, and they are happy to boast about everything from corruption and election rigging to acts of genocide.
The Act of Killing is about killers who have won, and the sort of society they have built.
In The Act of Killing, Anwar and his friends agree to tell us the story of the killings. But their idea of being in a movie is not to provide testimony for a documentary: they want to star in the kind of films they most love from their days scalping tickets at the cinemas. We seize this opportunity to expose how a regime that was founded on crimes against humanity, yet has never been held accountable, would project itself into history.
And so we challenge Anwar and his friends to develop fiction scenes about their experience of the killings, adapted to their favorite film genres – gangster, western, musical. They write the scripts. They play themselves. And they play their victims.
“Through Oppenheimer’s footage of perpetrators of the 1965 Indonesian genocide, a family of survivors discovers how their son was murdered, as well as the identities of the killers. The documentary focuses on the youngest son, an optometrist named Adi, who decides to break the suffocating spell of submission and terror by doing something unimaginable in a society where the murderers remain in power: he confronts the men who killed his brother and, while testing their eyesight, asks them to accept responsibility for their actions. This unprecedented film initiates and bears witness to the collapse of fifty years of silence.”
So Mexico morphed into MOFO… and now it’s late January 2015. Anyways. Here is some documentation for what happened in Hobart, the Mexican samples will have to wait a little longer.
I was in Hobart to do triple-Screen Video Projections at ‘ Faux Mo‘, which is the afterparty venue each night for the MOFO Festival, connected to the MONA gallery in Hobart, Tasmania. It tends to be eclectic – here’s the program.
Am super excited – it’ll be my first time in any of the Americas. From Nov 26 – Dec 22 I’ll be wandering through Mexico City, Oaxaca, Tijuana, as well as Cuernavaca, Metepec and a few other places in between.
Chancha Via Circuito – a favourite listen in recent years – has a new album out – Amansara (Wonderwheel Recordings). I first discovered his enchanting atmospheres and mixing on his wonderful ZZK Records mixtape (promoting his previous album Rio Arriba). His music seems to thrive best in mixtapes (see also Mixtape Cumbiero European Tour 2013 and a mixtape at Testpressing for new album ), reminding at times of early Future Sound of London and their wandering from soundscape to rhythm and back again. There’s a warmth to this music, and despite a slower tempo, there’s a momentum to it all as well. Recommendo!
Oh and a special shout-out too, for Paula Duro, who makes the enchanting artwork for Chancha (and featured in the backlayer of the collage above), as well as much of her own cool stuff. Check out her playful cosmic palette at flickr.
So I’ve been reading a lot lately. And swimming in words returns me to writing. Or at least – some words about books.
The infinite shelves at Goodreads are responsible for the bulk of the book orders above (want to swap recommendations?). I don’t know what took me so long to finally join Goodreads, I’d long been finding it tricky to get interesting book recommendations (particularly – good fiction – compared to say music or movies). Amazon has a decent catalogue, but I’ve found it unreliable for recommending new fiction of interest. And while I prefer the hand-curation of say – the Brainpickings bookshelf, the McSweeney’s Journal or DJ Rupture’s Mudd Up Book Club (includes a pretty great collection of sci-fi set in non-anglo cities), each of those are a pretty limited lense.
Anyways, I seem to have my reading for the next while sorted, which is also going to mean some more words here over time.
And if you’re not already aware of the second-hand booksellers below, this is where I found the bulk of the above:
Book Depository (my first choice – best range, generally cheapest overall to send to Australia)
BetterWorldBooks (good range – but seems deliberately deceptive in the way they offer ‘free postage ‘ – they show cheap prices after a search, but every single time, clicking on a book found in search, shows up as a much higher price, when you want to buy it.)
As real-time video software continues to evolve, we’re starting to see some really thoughtfully considered applications – such as Millumin, by Philippe Charaund, software dedicated to “create and perform audiovisual shows”. In part, Millumin is possible because of today’s easy re-routing of video between applications (thanks to software such as Syphon on mac and Spout on PC), which has enabled some developers to focus on specialty areas, and allowed others to provide ways of usefully integrating different parts of a video workflow.
Where does Millumin fit in?
While there are a lot of real-time video tools and specialities available, Millumin’s great strength is as over-arching software – and providing useful ways for co-ordinating and controlling other software ( eg triggering and manipulating clips inside VJ software, and recompositing, mapping and sequencing that video with Millumin, easily jumping between very complex compositions).
Millumin will especially be of interest to those seeking to sync media in tightly curated shows eg syncing video with important theatrical cues, conference cues, or a specific sequence of events in a music show. Aside from the time-based controls, it’s also a pretty effective piece of mapping software – which includes a built in capacity to edge blend between projectors.
In other words, Millumin provides good control over time (sequencing) and space ( compositing and mapping). It’s a unique recipe – while there are other apps that offer more advanced portions of what Millumin does – eg Vezer‘s sequencing and timeline options, or Madmapper‘s mapping controls – there’s nothing else that quite manages to do what Millumin does. QLab is probably it’s closest competitor, with the strengths and weaknesses of each meaning one or the other will suit your workflow better.
From their site guide: 1. Drag-and-drop ﬁles from Finder to the Dashboard, and click on the cells to play them 2. Use the Workspace toolbar to move, map, warp, mask … to rotate and scale the layers directly in the workspace. 3. Change blend mode, add effects, transitions and more from the Properties Panel 4. In the Library, manage your ﬁles, Syphon servers and inputs 5. Create a Composition, then organize your media in time with keyframes
– includes ability to play compositions within compositions ( like nested compositions in After Effects)
– Also like AE – includes adjustible keyframes – change opacity, position, scale or rotation change over time to specific values.
– cue points can be added.
– pause on cue points 6. Import this Composition into the Dashboard + switch between complex compositions easily. 7. The Magic Key is [SHIFT] : maintain it to multi-select and snap items
Control of time / sequencing: Millumin’s key-framable timelines will be warmly familiar to everyone who has used video editing software, and tends to find such functions missing within VJ software. Most VJ software will show a timeline / playhead for each clip – but much more rare is a capacity to place many clips along a timeline, and easily add cue points, and easy linear arrangements. Example nice touch? Drag and drop a clip onto a timeline, then drag the end of it to auto-loop as long as you need.
Room for improvement? There are lots of little user interface quirks that could be removed / better designed. Admittedly this is partly because Millumin reminds so much of video editing and compositing software – which brings a whole bunch of fine-tuned expectations – and sets an unfair benchmark – relatively new software made by one person could hardly be expected to match the resources and foundations of established editing and compositing software.
Control of space / compositing:
Video compositors will find it a pleasure to be able to create complex compositions, and nest and even animate these comps within other comps. In this respect Millumin is the closest thing to a real-time After Effects that exists. Sequencing and switching between various comps is trivial to implement…
.. and these ‘presets’ / ‘dashboard selections’ – can be triggered from other software using midi or OSC – eg the M1-m10 presets built into a VDMX control surface window below.
Millumin can also take in as many syphon inputs as can be thrown at it – which integrates it well with VDMX’s capacity to send out many. All of these can be composited differently in Millumin’s compositions, allowing for a huge amount of flexibility and convenience. (Snap below includes sequined ninja in oyster cave footage used at recent Dark Faux Mo festival in Hobart.)
Control of space / mapping:
Millumin features great controls for multiple outputs, and features multi-screen edge blending and feathering of masks:
Room for improvement? Being able to work better with multiple projectors that have different aspect ratios to each other.
As a standalone application, Millumin has a limited range of visual effects. On the other hand – deep syphon integration means easy piping in of video from other software, for sequencing or compositing, and quartz composer integration means being able to easily add customised QC elements, effects and compositions to any of that video piped in.
– Mac OSX 10.6 or later. (PC version in the pipeline)
– 599€ (VAT not included) = A license for Millumin on 2 different computers. for 2 computers . Educational and rental pricing available by negotiation.
Millumin is very thoughtfully crafted software, with a nicely expanding feature set. And while it’s missing refinement or lacking more detailed control in a few places, it continues to develop and evolve into a fantastic and versatile tool for live video, especially with multi-screen compositing.
– Lotech (NZ) for inspirational use of Millumin @ Splore – each VJ could send a signal into the machine running Millumin, which effectively let them play on a pre-mapped structure, and for ongoing feedback about Millumin over time.
– Jem the Misfit (NZ/Aus/Ger) – for highlighting how creatively Millumin could be used for compositing.