ReacTIVision Controller.

Project Proposal

Nicholas Alex Thom – 10566736.

I am interested in working with the reacTIVision open source computer vision framework with the view of creating a multi-touch surface controller that can be used by musicians as an interactive mixing console. The aim of this project is to create an alternative user controller, rather than a mouse and keyboard, to control various mixing functions, such as, volume, Equalization, compression etc. Until more recent times many electronic user interfaces had been focused around the keyboard, that mostly operate as a MIDI controller, leaving a substantial amount of instrumental and none instrumental musicians at a loss when faced with a computer (McGill University, 2003). The reacTable is a multi-user electro-acoustic instrument with a table top user interface, an instrument which has been designed for novice and advance musicians by combining physical representation with control, allowing for multiple users to share control of musical parameters by moving physical artefacts upon the table surface (Jorda, et al., 2014). Now days, there can be many benefits for the musician to use an alternative mixing controller system. Use within the studio environment would allow mixing engineers the freedom to intermingle with the artists, building a closer relationship to artists and understanding the creative processes, a skill which can also be transferred into a real-time live performance, allowing the additional creation of unique mixes and allow for multi-user participation (Selfridge and Reiss, 2011). The reacTable has two main variations for use. One which is designed for the advanced user, providing controls to a sophisticated and precisely controllable synthesizer, and the other, which has been designed to be more playful for entertainment and educational use (Kaltenbrunner, et al., 2006). The reacTIVision framework is a great way for me to explore using physical motion as control and apply this to the mixing process.

In this project I aim to use the open source reacTIVsion framework to create a reacTable style multi-user interface, which will be able to play back audio and manipulate various different data points, allowing user control of audio and audio effects. This type of device will be created to be used as a type of mixing controller for a musician/mix engineer to use and experiment with as a creative alternative to a computer mouse and keyboard. The programming will be done using to software Max MSP with the reacTIVision software. I currently plan to only use Max for audio and effects processing as well as, the data processing for the reacTIVision. However, I do plan to explore how this could be linked into a DAW such as Logic Pro and how this style of controller could be used in a DAW. The main concerns I have with a project like this is establishing how each piece of software will communicate with each other and if they will be reliable. Having already experimented with a webcam and reacTIVision at home and encountered errors adding more pieces of software in could likely make this project very complicated very quickly.

         

Bibliography

Jorda, S., Kaltenbrunner, M., Geiger, G., and Alonso, M., 2006. The reacTable: A Tangible Tabletop Musical Instrument and Collaborative Workbench. [pdf] Music Technology Group Pompeu University. Available at: https://www.researchgate.net/publication/247929547_The_reacTable_a_tangible_tabletop_musical_instrument_and_collaborative_workbench [13/03/2019].

Kaltenbrunner, M., Jorda, S., Geiger, G., and Alonso, M., 2006. The reacTable: A Collaborative Musical Instrument. [pdf] Music Technology Group Pompeu University. Available at: https://ieeexplore-ieee-org.plymouth.idm.oclc.org/stamp/stamp.jsp?tp=&arnumber=4092244 [13/03/2019].

McGill University, 2003. New Interfaces for Musical Expression. [pdf] McGill University. Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.309.7863&rep=rep1&type=pdf#page=82 [13/03/2019].

reacTIVision, 2019. reacTIVision 1.5.1. [Online] Available at: http://reactivision.sourceforge.net/ [13/03/2019].

Slefridge, R., and Reiss, J., 2011. Interactive Mixing Using Wii Controller. [pdf] Audio Engineering Society, Available at: http://www.eecs.qmul.ac.uk/~josh/documents/2011/SelfridgeReiss-2011-AES130.pdf [13/03/2019].