Jibber Jabber is a fictional character created from the amalgamation of several data sets (audio, image and text) using machine learning software. The character symbolizes the collective "ghost" of the Media Design Practices Department and the Wind Tunnel studio space vacated at the end of the Spring 2022 semester. As the MDP ghost, Jibber Jabber roams and introduces the space to the incoming Transportation Design students and faculty.
The project brief required us to use ourselves as a medium and investigate how can machine learning extend one another's attributes, mix, mash, and collaborate and formulate new in-between aesthetics from various machine learning software.
We trained different models with video recordings collected over the course of a week. From there, we exported those video recordings into individual images to create portraits of each of the team members for training the figure of Jibber Jabber and the MDP studio (the backdrop). In addition, we trained a voice model comprised of all three of the team member's voices. The script that the trained voice recited was trained from the historical information of the Wind Tunnel found online and MDP course syllabi.
The process of training a model was quite challenging due to the amount of data that was required and some of the legal limitations around identity, especially with regards to the voice. The final voice ended up sounding more like mine because I was required to read a legal disclaimer to initiate the training process, instead of a mix between all three of the team members like we had intended. In the end, I'm very satisfied with the outcome of the video as it provided us the opportunity to consider the materiality of AI.
In the end, we were able to look under the hood of AI and understand how it is built conceptually, learned how to collect and label data, used it to train a model to witness how the process works and what can go wrong with it, and explored how AI can be used for the purpose of visual creative expression.