Machine Learning + Video
Fall 2018

Animating with Machine Learning

Artificial Intelligence isn’t sci-fi anymore – it’s here and now. AI is the advantage and even the necessity in so many industries and services. But I wanted to explore an area where AI is just beginning to make an impact: art.

Pictured Above: A still from a short film I made using style transfer

Style Transfer and Optical Flow

Style transfer is a process by which computers extract a style from one image and apply it to another. When used with what’s called optical flow, style transfer can be applied to moving video with a rotoscoping effect. This process is incredibly tedious to setup, computationally heavy, and still in developmental stages. But it makes you wonder about what’s next.

Pictured Above: A set of clips testing out different styles applied to different pieces of footage. The styles are: Robert Delaunay portrait (top left), Mondrian painting (top right), hundred-dollar bills (bottom left), The Great Wave (bottom right).

A Trip to Town

After some trials, I began a bigger project. I shot footage in Boston and used style transfer to glaze the city with trippy, cartoon visuals. It was a painstaking process that took weeks to render. Below is the result of this marriage between artist and computer.

Pictured Above: A culminating project using this style transfer technique and footage I shot in Boston.