VIX: A New Kind of Video Editor
- /
- Software Video Editing VIX
- /
- No Comments
Today I would like to introduce you to VIX, my latest video editing application for OS X.
VIX is a different kind of video editor. What makes it different from the NLEs that you might be used to, for example Final Cut Pro, is that VIX is video editing driven by audio. The basic premise is that VIX will automatically edit a list of videos to the tempo of a song, but it goes much further than that. You have complete control over every edit and there is a fun interplay between the manual process and allowing VIX to automatically edit for you.
The following is a video created entirely with VIX. It’s a “behind the scenes” video for a photo shoot for the iPad fashion magazine Em Oi here in Vietnam. For a moderately skilled person in the use of Final Cut or Adobe Premiere, this might have taken anywhere from a day to three days. In VIX, it took about an hour:
Solving Problems with a Big Hammer
To give some history into how VIX came into being, I’ll have to explain my involvement with Em Oi.
For the last two years, I’ve been building a publishing platform for creating iPad and iPhone magazines. This platform is called Aspekt and it is very similar in concept to Adobe DPS or MagPlus, but with some huge differences in terms of usability and interaction. I’ll be talking a lot more about Aspekt and its history in future blog posts.
So, In order to “eat our dog food”, to really test every aspect of the platform, we needed to create our own magazine and run it through the gauntlet; from conception to execution to publication. We tossed around a lot of different ideas: travel magazine, food magazine, lifestyle magazine, etc. But, we had a limited budget and limited time, so we’d have to conceive of something that was relatively easy to execute, didn’t require a huge staff and could really showcase the media capability of the Aspekt platform. It would also have to be something that we could conceivably sell to recoup the expense of producing it. So after a lot of back and forth, we decided to go with the idea of taking fashion photographers and having them shoot “stories”. Typically, when a fashion photographer shoots an editorial, they are aiming to hit a two to three key shots. With Em Oi, we were giving them carte blanche to shoot and tell a story. We’d combine the photos, videos and other interactive elements into a feature and each issue of the magazine would be made up of five or six of these features. No articles, no text, just photos, videos, music and interactivity.
We ended up doing six different shoots, each shoot producing a mountain of video that required editing. Not only am I building the platform, I’m also the creative and design director. This is partially by necessity as designing and producing the issue requires an iteration on the technology during the design process. As I’m designing a feature, if Aspekt can’t do something I want it to do, I have to stop the design, put the technology hat on and make Aspekt do what it couldn’t do before. This is why “eating your own dog food” is such an invaluable exercise.
Anyways, to make a long story short, it was someone’s responsibility to take the hours of video and massage it into something beautiful. That job fell on my shoulders. The first video I edited took me about four or five days to produce. Considering that I had seven or eight videos to do, I was looking at a solid five weeks of work just for the video portion of the issue. If I had a bigger budget, and hadn’t made so many mistakes with the budget for the first issue (live and learn), I would have hired somebody to do it. But when you are bootstrapping, money is a finite resource and we’d pretty much extinguished the budget already.
I had suddenly remembered that a very long time ago, I had written some R&D hack job to learn more about opencv and training haar classifiers to identify objects in individual video frames. You would feed videos to this system, have it tag the temporal locations of these objects it was trained to recognize and then once I had built up a database of clips, I would enter in some tags, specify a song and have the system grab the tagged videos and automatically edit them together to the beats in the song. It was a pretty crazy system.
What if I could simplify it and simply give the system a list of videos, a song and then have it edit everything together? I ripped out the needed bits from the R&D hack and after some experimentation, it was kicking out some pretty cool videos. Nothing I’d publish, of course, but a good starting point. But to make this work, I’d have to update the code and put a user interface on it so I could interact with it and give guidance on how the edit should go. Considering I had five excruciating weeks of doing this by hand in Final Cut Pro, I decided to take the gamble and try to get a working version done in two or three weeks and then crank out the videos quickly afterwards.
Any project manager will tell you that most software engineers have horrible conceptions of time. Ask a software developer how long something will take and then multiply by 150% to get the most accurate estimate. But, I was able to get a usable tool done in two weeks. I am only highlighting this because its a career first, so I need to brag about it a little bit.
How VIX Works, How to Work VIX
The basic premise behind VIX is a pretty simple one: the software will analyze an audio file, attempting to locate the beats in it. These beats become your edit points (markers in VIX parlance). You supply VIX a list of videos and VIX will insert clips from those videos at each beat for the entire song. It does this editing semi-randomly. It will take that list of videos and pulling one from the list randomly to insert at an edit. It will then move onto the next edit, pulling out another video randomly from the list. Once the list is empty, it will refill the list and continue on. You have a lot of control over this process, including turning off the randomness, but more on that later.
Now, it’s not likely that this automatic video edit will produce your final polished video. Therefore, your next step is to go in and tweak these edits by deleting them, moving them around, adding new ones and changing the parameters VIX uses to do its automatic editing. It’s this iterating on parameters and tweaking edits that gets you closer to a final edit by roughing the sketch out and then filling in the fine details.
My typical workflow with VIX involves a few steps before I even launch VIX:
- Log the clips I want to use and perform some basic trimming on them
- Import the logged clips into After Effects for color grading and conforming
- Render to ProRes from After Effects
It’s at this point that I will ingest the rendered After Effects clips into VIX and start the editing process. I tend to start broad, playing the entire edit and only stopping to fix obvious issues. Next, I’ll focus on a particular section of the song and iterate over the edits until I’m satisfied. I will then move onto the next part of the song, lather, rinse, repeat.
Once I’m done, I typically just export from VIX as a QuickTime, but there are times will I will export the edit to Final Cut Pro to do things like cross fades and fade outs or more color grading and finishing. VIX supports exporting to After Effects and Final Cut Pro. You can get an edit into Adobe Premiere via After Effects as well.
Watch and Learn
Below are two videos that highlight how to use VIX. The first one is a basic introductory tutorial, while in the second one I actually go through the process of editing a video from start to finish.
Get The Beta
You can grab the beta version from here.
If all goes well, the first public release should be in the App Store by month’s end.