Runway ML - Portrait Training Process
What if collective identity and perspective existed in a single intelligent agent?

Jibber Jabber

The MDP Ghost

Jibber Jabber is a fictional character created from the amalgamation of several data sets (audio, image and text) using machine learning software. The character symbolizes the collective "ghost" of the Media Design Practices Department and the Wind Tunnel studio space vacated at the end of the Spring 2022 semester. As the MDP ghost, Jibber Jabber roams and introduces the space to the incoming Transportation Design students and faculty.

Teammates: Mario Santanilla and Zhuoyu Li
Role: Co-creator
Timeline: 3 weeks
Advisors: Jenny Rodenhouse, Adit Dhanushkodi, Ben Hooker, Maxim Safioulline
Tools: Premiere Pro, Runway ML (text & image), Descript (voice), Soundraw (music), and Photoshop

Process

.

The project brief required us to use ourselves as a medium and investigate how can machine learning extend one another's attributes, mix, mash, and collaborate and formulate new in-between aesthetics from various machine learning software.

We trained different models with video recordings collected over the course of a week. From there, we exported those video recordings into individual images to create portraits of each of the team members for training the figure of Jibber Jabber and the MDP studio (the backdrop). In addition, we trained a voice model comprised of all three of the team member's voices. The script that the trained voice recited was trained from the historical information of the Wind Tunnel found online and MDP course syllabi.

A timelapse video communicating the entire process of the project including the different datasets, the software used and how everything fed into one another for the final piece.
Screenshot of the Descript software
The voice of Jibber Jabber was trained in Descript and based off voice prints from each team member.
Video Notes on White Board
After the respective software produced the outcome from the source material, we discussed how we might display in the content in a video for the final presentation.
Screenshot of Descript software
The script of Jibber Jabber was generated in Runway ML from the information on the history of the Wind Tunnel and MDP course descriptions.

Documentation

.

As part of project deliverable was the creation of a publicly accessible dataset as a way to participate in the public discourse of AI model development.
MDP is currently located in the Wind Tunnel space of ArtCenter College of Design's (ACCD) South Campus. This building was original commissioned by Consolidated Vultee, Douglas, Lockheed and North American, who formed the Southern California Cooperative Wind Tunnel and managed by Caltech. Throughout its history, it has had multiple roles and uses, most notably as a wind simulator for military aircraft. MDP arrived in the space in 2009, after ACCD purchased the space in 2004, and will move out at the end of Spring 2022 to make room for the Transportation Design Program. 

Reflection

.

The process of training a model was quite challenging due to the amount of data that was required and some of the legal limitations around identity, especially with regards to the voice. The final voice ended up sounding more like mine because I was required to read a legal disclaimer to initiate the training process, instead of a mix between all three of the team members like we had intended. In the end, I'm very satisfied with the outcome of the video as it provided us the opportunity to consider the materiality of AI.

In the end, we were able to look under the hood of AI and understand how it is built conceptually, learned how to collect and label data, used it to train a model to witness how the process works and what can go wrong with it, and explored how AI can be used for the purpose of visual creative expression.

Previous Project

Image of previous project, "Omni"OMNI

Next Project

Image of next project, "The Black Box"The Black Box