[pic]
Augmented Reality Musical Instrument
Name: Pengzhi Zhou
[pic]
Abstract
Playing a musical instrument has been proven to improve cognitive functions, help fight memory loss, reduce stress, lower blood pressure and help prevent depression in the developing brain. But if we want to play the physical instruments we have to come to specify place or just buy them home, and the problem as high price(such as piano) or discommodious that stops many people from getting approach to them. So if we can control the visual 3D instruments using our hands movement, it will be a facility to the people who like to play the varieties kinds of instruments. The aim of this project is to develop a system that allows a person to control a visual instrument using hand gestures. In order to track and recognise the hand gestures, the system must be able to detect and track the customer’s hand and recognise the hand gestures. And because our objective is to control the musical instruments, for lifelike the system will be produced in Augmented Reality environment.
Table of Contents:
1 Introduction 8
1.1 Aim and objectives 8
1.2 Techniques 8
2. Background Techniques Researched 9
2.1 The similar work 9
2.1.1 An Augmented Reality Based Learning Assistant for Electric Bass Guitar 9
2.1.2 Augmented Reality Interface for Electronic Music Performance 10
2.2 Motion detection 11
2.2.1 Image segment comparison 12
2.2.2 Image roping 14
2.3 Augmented Reality 16
2.3.1 The data flow in AR 18
2.3.2 Affine Representations 19
3 Applying the Techniques 23
3.1 Motion detection 23
3.1.1 Detect the motion 23
3.1.1.1 Convert to gray scale 24
3.1.1.2 Image comparisons 25
3.1.2 Hand detection 26
3.2 Augmented Reality 28
3.2.1 Control the light condition 29
3.2.2 Using two markers to create the virtual drum 31
3.2.3 Detect the virtual drum surface position 33
3.2.4 Occlusion problem 36...