Brief
Create a sensor-based interface allowing users to interact with an audio-visual environment—either through physical computing (e.g. Arduino) or motion tracking via the Gesture and Media System at Northern Dance in the Ouseburn.
What I built
I built a VR experience where the player's position influenced the environment: an audio-visual game where effects distorted music, and the goal was to find positions in the room where the track sounded normal again. I used the HTC Vive and Unity — it was an early exploration of spatial interaction rather than a polished product.
Sync VR product video
The Idea
The idea was simple: vocals and instrumentals were separated, with distortions applied. The player searched for the point in the room where the music 'synced' and sounded normal.
I originally planned something simpler with Arduino and an ultrasonic sensor, but tracking position in 3D space pushed me towards VR—which opened up more room for experimentation.
Initial prototype
Once I had outlined the idea, I built a prototype focused on altering sound through position—straightforward to implement by mapping positional values to audio parameters such as pitch.
Making the playspace
In research I found that a common VR concern is becoming too immersed and walking into real-world obstacles. I mapped the physical room into the virtual space so users could move with more confidence.
Creating more levels
I added a second level focused on moving objects in the room rather than the player—two objects to reposition, each controlling half the audio mix.
Building the UI
To make it feel like a complete standalone experience, I built an in-game UI — one of the harder parts, as it required detecting the direction the VR controllers were facing.
