Multimodal Signal Processing with Application to Neuroprostheses and Biometrics
Time and Place
Auditorium (NSH 1305)
Combining information is a vital element of comprehension and understanding. Multimodal signal processing allows us to fuse information from multiple information signals and sources for benefit in medical diagnosis, cognitive neuroscience and biometrics. With current pressure within the healthcare system to deal with an increasingly aging population and with demands for increased security, multimodal signal processing will become more important in the future.
This talk will overview recent work in three areas neural engineering (multisensory integration and neuroprostheses) and biometrics. It will also point to other multimodal aspects in these and other areas which are expected to be important in the future.
Multimodal biometrics provide the most adaptive approach to person verification and identification. We review existing methods employed to carry out audio-visual integration, for speaker identification. Additionally, we examine classifier combination theory and its’ applications towards person recognition, such as the fusion of audio and face experts, or indeed, the fusion of several face and/or several audio experts. We will also highlight which fusion techniques are more suited towards speaker identification, a multi-class problem, and which are suited towards speaker verification/authentication. Results are provided on existing and newly acquired multimodal databases.
2. Neuroprostheses: Multisensory Integration:
Everyday tasks involve the seemingly automatic integration of information from multiple sensory modalities. For example, driving a car involves the synthesis of a number of sensory activities: visual (seeing the road), auditory (hearing the car engine and passing traffic), somatosensory (feeling the steering wheel) and motor (depressing the accelerator). The combination of inputs from different senses can function to reduce perceptual ambiguity and also enhance stimulus detection. Despite the fundamental role that sensory integration plays in performance and perception, how and when information from separate sensory modalities comes together in the human neocortex is an unsolved problem. We overview new analysis of multisensory (audio and visual) integration from a nonlinear dynamic viewpoint. It is hoped this approach will become useful in aiding the diagnosis and monitoring of schizophrenia.
3. Neuroprostheses: Monitoring and Analysis of Attention
The ability of humans to extract information from the environment and select a response, to maintain focus for prolonged periods of time on even monotonous tasks, and to identify thought processes and actions in accordance with our goals is facilitated by the attention system of the brain. Selective attention is examined with regard to the use of its characteristic electrophysiology in a Brain Computer Interface, for the severely disabled. The results of a preliminary dependent BCI design involving Steady-State Visual Evoked Potentials (SSVEPs) are first presented. In this BCI subjects shifted gaze towards one of two visual stimuli in a real-time gaming environment to make selections. An independent BCI involving SSVEPs and real-time biofeedback is developed: the Visual-Spatial Attention Control (V-SAC) BCI. Real-time experiments in which left/right spatial attention is classified by extracting (SSVEPs) will be described. In a further BCI study some critical issues of the use of SSVEPs in the V-SAC BCI are addressed through an offline analysis of high density EEG data. A comparison is made between using stimulation frequencies within and outside the alpha band and the effects behavioural performance on BCI accuracy is examined.
Richard Reilly received his B.E., M.Eng.Sc.
and Ph.D. Degrees all in Electronic Engineering, from
Dr Reilly was the 1999/2001 Silvanus
P. Thompson International Lecturer for the IEE. In 2004 he was awarded a US
Fulbright Award for research collaboration into multisensory
integration with the Nathan Kline Institute for Psychiatric Research,
Dr Reilly is an Associate Editor for IEEE Transactions on Multimedia and also a reviewer for IEEE Transactions on Biomedical Engineering, IEEE Transactions on Neural Systems and Rehabilitation Engineering, IEEE Transactions on Industrial Electronics, Journal of Applied Signal Processing, Signal Processing and IEE Proceedings Vision, Image & Signal Processing.
For appointments, please contact Louise Ditmore (firstname.lastname@example.org).