ætherfon is an interaction design project, where users can create spheric sounds with hand tracking via webcam and artificial intelligence. The goal of the experiment is to playfully introduce the users to the topics of music and artificial intelligence and to give them room for artistic development in especially in times of isolation during the Covid-19 pandemic.​​​​​​​ Despite the technical structure, it was important to us to make the interface and functions as barrier-free as possible.

With the help of PoseNet, a machine learning model that allows estimation of human posture in real time, the hand movements of the users can be tracked. Through this motion capture, they are able to change the speed, volume, audio quality, and frequency of multiple sounds to create music. At the same time, the music is graphically visualized using p5.js, a JavaScript library, and gives the user additional visual feedback.​​​​​​​​​​​​​​ 

The project was developed with three fellow students (Tobias Braun, Charleen König and Moniek Wiese) at the Hamburg University of Applied Science/HAW Hamburg in 2021. The project was the winner of the interaction day 2021 in the student category, awarded by the Verdieck Stiftung and the students of the HAW Hamburg. 

→Services: Branding and interaction design concept