4 VR Impression Scenes of CMU
4 VR Impression Scene of CMU is a spatial audio design project for 360º video. This project consists of four different video scenes recorded with a Samsung 360º camera and each scene has its own spatial audio design, aiming to create different mood from the scene and affecting the audiences. The project mainly uses Facebook360 Spatial Workstation in Reaper for spatial editing and mastering, Logic Pro X for music composition and production, and YouTube for presenting.
The videos for this project are my own recording. All the music in this video are my original composition and production and their placement in the video are my original design as well, with the help of a few sound effect source found elsewhere.
The motivation was inspired by the fact that video games are now looking into the world of VR more than ever, following the revival of VR technologies. As a game designer and sound designer, I was intrigued by spatial audio design after I took a VR game design course from IDeATe and realized the importance and potential of sounds in VR. I went to a lecture on spatial audio design and composition in ETC given by an alumnus Larry Chang, and this project stems from that very lecture.
The composition is based on the following idea. The four scenes are deliberately videoed at different times of day or weather- a snowy afternoon, an unnerving night in the woods, a sunny afternoon in Spring, and a peaceful night just before summer.
1. A Snow Day in April:
The wind hurls the snows around the garden space behind the building of College of Fine Art, where sound of instrument practices often fills the place. Even thought it’s a day of snow, there are still students practicing bassoon and tuba somewhere in that building of music. As if wakened by the lonely sound of practice, more sounds come in to form a grand play- the Glockenspiel ringing around the corner of a nearby library, string instruments plucking and bowing to play as accompanies, native American flute and oboe come in towards the second half of the scene to echo with the tuba. This piece follows a sense of coldness and loneliness reflected by the weather but tries to invoke a sense of “the winter is almost over; the spring is at the door, so just wait a little bit longer”.
2. Skewed Light and Unnerving Dark:
The woods at the backside of CMU is immerged by complete darkness. Nervously looking around, there’s nothing visible but some scattered and skewed light points faintly shine from a distance which seems impossible to reach. Without notice, shadowy footsteps sound around you while crows cawing to foreshadow something unspeakable. The ambient, which is my composition, features string sections, mellow brass, Glockenspiel, and Tubular Bells, aiming to create a background music similar to horror movie background ambient music. The footsteps are designed to sound like something is walking around the listener but, instead of trying to jump out and approach, patiently lurking in the darkness to wait for a strike chance. The brass melody is a copycat of Stravinsky’s Rite of Spring- it uses a progression in a similar fashion of the opening of Stravinsky’s famous piece but in my own way, which is trying to stretch the note intervals as chaos and dissonant as possible to invoke the sense of uneasiness in the listener. At the end of this second scene, the existence lurking in the dark finally steps forward to take its chance to prey. The Glockenspiel strikes serves as the incoming music of the dark existence, and the breathing comes closer with every sound and finally the monster roars behind the listener for a closure.
“Haru” means “spring” in Japanese and it fits my third scene- a sunny afternoon at a wide lawn. Since it’s spring, everyone is cheered up, the music from the music hall carries a completely different mood. Starting with string section plucking, the sound which comes from the tennis court serves as a drum base and percussion. After the string section and a little pause, a clarinet comes in: the melody carries the cheering mood as one who’s eagerly looking forward to a bright day finally get a wish-granted. As the clarinet solo briefly concludes its phrase, a mixture of English horn, French horn, bassoon, marimba, Guzheng, and string accompany section come in to play a harmonized melody which is composed with a hint of Japanese traditional music. The instruments come from various angles and distances: the orchestra sections mainly come from the rooftop of the white music hall, while the other instruments come either from other building around the scene, or from the open air in between the campus buildings to create a sense of imaginary sound. As the music concludes, the scene fades dark and makes the transit into the night.
4. A Peaceful Night:
As the final scene of this project, the video is taken from the exact same spot as the Haru scene. Only the sound of crickets and a solo piano can be heard as if there’s someone invisible playing on the empty lawn under the neon light from the LED sign. Comparing to previous scenes, this part of music features obviously fewer instruments. The design intension was to invoke the sense of quietness and peacefulness. The inspiration of this composition came from the time when I was about to graduate and to say goodbye to so many friends from college. I want to use this piece to mark the finale of my undergraduate life and as a gift to my friends.
After the piano solo accompanied by bass and harp plucking, a gentle melody played by flute, English horn, and bassoon creates the sense of a mellow night and further develop the peacefulness to echo around the space. As for the finale of the entire project, everything quiets down to give way for a clear bell melody except for the plucking accompaniments. The simple melody calms down the mood of the listener and fades out with the video as a nice and quiet closure.
The software I used for this project includes Facebook360 Spatial Workstation, Reaper, Adobe Premiere Pro, Logic Pro X, Audacity, FFMPEG, and Gear 360 ActionDirector. The camera I used is a Samsung Gear 360 camera.
The sound design process was the following:
1. Based on the video, design the sound in the scene space and choose instruments that feels proper for the music;
2. Compose the music in Logic Pro X and arrange the sound effects in Logic Pro X to preview the effect and cue the timing; then, edit and master all the sounds (second time for sound effects which are found on YouTube);
3. Use FB360 Spatial Workstation in Reaper to place each sound in their desired places.