FROM LIGHT TO SOUND: SIGNAL MODALITY TRANSLATION

FROM LIGHT TO SOUND: SIGNAL MODALITY
TRANSLATION
Jiaxuan Wang School of Electrical and Electronic Engineering, University of Manchester. UK Patrick Gaydecki School of Electrical and Electronic Engineering, University of Manchester, UK
1 INTRODUCTION
This paper describes a signal modality translation project the purpose of which is to represent acoustic signals from images or patterns of light. Signal modality translation is a rapidly emerging technology; it has been widely used to synthesize images from wave signals or acoustic signals. Conversely in this project. signals in the form of images and video are modulated into continuous acoustic sounds, by the method of real-time digital signal processing (DSP).
The hardware equipment of this project is a conventional CCD camera, signal processing core, and an audio output system. The software equipment is MATLAB software with the Image Acquisition
Toolbox.
The whole structure of the project comprises four process stages: the Image Acquisition System. the Feature Extraction System. the Sound Modulation System and the Sound Playback System. The output of this entire system is acoustic audio composed of various instrument library tones and is modulated according to the image features of the video input. Each audio outputis not only unique as the video content changes. but also features continuous tones with distinct combinations
of pitches and timbres.
     
 
 
     
  
 
 
    
 
 
Sound
Modulation
System
Feature
Extracting
System
Image
Aquisation
System
  
Figure 1: Four main stages in this project.
This system could bring benets to individuals with visual impairment, by aiding them in navigation and location. Moreover, it could be developed to applications in recognition systems. because visual
information in some circumstance is difcult to extrac