Malcolm Slaney
Dr. Malcolm Slaney is a Consulting Professor at Stanford CCRMA, where he has led the Hearing Seminar for more than 20 years, and an Affiliate Faculty in the Electrical Engineering Department at the University of Washington. He is a coauthor, with A. C. Kak, of the IEEE book "Principles of Computerized Tomographic Imaging." This book was republished by SIAM in their "Classics in Applied Mathematics" Series. He is coeditor, with Steven Greenberg, of the book "Computational Models of Auditory Function." Dr. Slaney has a long career in industry, including doing machine learning and auditory perception work at Google Research. Before Google, Dr. Slaney worked at Bell Laboratory, Schlumberger Palo Alto Research, Apple Computer, Interval Research, IBM's Almaden Research Center, Yahoo! Research, and Microsoft Research. For many years, he has led the auditory group at the Telluride Neuromorphic Cognition Engineering Workshop. Dr. Slaney's recent work is on understanding the role of attention in conversational speech and general audio perception.
Auditory perception has been the core of my research
over the years, most recently understanding perception by
decoding brain waves.
Much of my work uses techniques from AI, machine-learning,
deep neural networks and Bayesian learning to solve interesting problems.
I spent a few years investigating an algorithm
known as Locality Sensitive Hashing (LSH) that is used to
efficiently find nearest neighbors. I wanted to understand
how to make LSH more efficient. I wrote a tutorial
with Michael Casey and Christoph Rhodes. Then with
colleagues at Yahoo I wrote a "definitive" article about how
to choose the optimum parameters. Both the Matlab
(optimization) and Python (implementation) code is online
too.
I wrote a column for IEEE Multimedia
Magazine about my vision of the multimedia world. The
columns are online. 
I get to
work with lots of wonderful image data and some very smart
computer-vision people.
I've been working on finding similar
songs in large music databases with 
I've been working with several
talented 
There is now a new version of
the Auditory Toolbox. It contains 
My primary scientific goal is to
understand how our brains perceive sound. My role in this
research area is a modeler, I build models that explain the
neurophysiological and psychoacoustic data. Hopefully these
models will help other researchers understand the mechanisms
involved and result in better experiments. My latest work in
this area is titled "Connecting Correlograms to
Neurophysiology and Psychoacoustics" and was presented at
the 
The information in most auditory
models flows exclusively bottom-up, yet there is
increasing evidence that a great deluge of information is
flowing down from the cortex. A paper I wrote for the 
I have written
several papers describing how to convert auditory
representations into sounds. I have built models of the
cochlea and central auditory processing, which I hope both
explain auditory processing and will allow us to build
auditory sound separation tools. These papers describe the
process of converting sounds into cochleagrams and
correlograms, and then converting these representations
back into sounds. Unlike the printed versions of this
work, the web page includes audio file examples. It
includes better spectrogram inversion techniques, a
description of how to invert Lyon's passive cochlear
model, and a description of correlogram inversion. This
material was first presented as part of the Proceedings
of the ATR Workshop on "A Biological Framework for
Speech Perception and Production" published in
September 1994. A more refined version of this paper was
an invited talk at the 
Pattern Playback is
the term used by Frank Cooper to describe his successful
efforts to paint spectrogram on plastic and then convert
them into sound. I wrote of Pattern Playback techniques,
from Frank Cooper's efforts to my own efforts with
auditory model inversion, in a paper which was published
at the 1995 IEEE International Conference on Systems,
Man, and Cybernetics. My paper is titled "Pattern
Playback from 1950 to 1995". The image at the left shows a
portion of one of Cooper's spectrograms.
Chris Bregler, Michele Covell,
and I developed a technique we call Video Rewrite to
automatically synthesize video of talking heads. This
technology is cool because we use a purely data driven
approach (concatenative triphone video synthesis) to
create new video of a person speaking. Given new audio, we
concatenate the best sequence of lip images and morph them
into a background sequence. We can automatically create
sequences like the Kennedy and Johnson scenes in the movie
"Forrest Gump."
We studied how adults convey
affective messages to infants using prosody. We did not
attempt to recognize the words, let alone to distill more
nebulous concepts such as satire or irony. We analyzed
speech with low-level acoustic features and discriminated
approval, attentional bids, and prohibitions from adults
speaking to their infants. We built automatic classifiers
to create a system, Baby Ears, that performs the task that
comes so naturally to infants. The image on the left shows
one of the decision surfaces which classifies approval,
attention and prohibition utterances on the basis of their
pitch. 
Eric Scheirer and I worked on a
system for discriminating between speech and music in an
audio signal. This paper describes a large number of
features, how they can be combined into a statistical
framework, and the resulting performance on discriminating
signals found on radio stations. The results are better
then anybody else's results. (That comparison is not
necessarily valid since there are no common testing
databases. We did work hard to make our test set
representative.) This paper was published at the 1997
ICASSP in Munich. The image on the left shows clouds of
our data.
Work we've done to morph between
two sounds is described in a paper at the 1996 ICASSP.
This work is new because it extends previous audio
morphing work to include inharmonic sounds. This paper
uses results from Auditory Scene Analysis to represent,
match, warp, and then interpolate between two sounds. The
image on the left shows the smooth spectrogram, one of two
independent representations used when morphing audio
signals.
I have written Matlab m-functions
that read and write QuickTime movies. The WriteQTMovie code
is more general than previous solutions for creating movies
in Matlab. It runs on any platform that Matlab runs on. It
also lets you add sound to the movie. The ReadQTMovie code
reads and parses JPEG compressed moves.
I created a Hypercard stack to make it
easier for people with a Macintosh and CDROM drive to
interact with the Acoustical Society of America's 
I wrote a program for the Macintosh 660/AV
and 840/AV computers that uses the DSP (AT&T3210) to
monitor audio levels. VUMeters runs on any Macintosh with
the AT&T DSP chip. Source and binaries are included.
Bill Stafford and I wrote TCPPlay to allow us
to play sounds from a Unix machine over the network to the
Macintosh on our desks. This archive includes Macintosh
and Unix source code and the Macintosh application. There
are other network audio solutions, but this works well on
the Macintosh.
In a past life, I worked on
medical imaging. A book on tomographic imaging
(cross-sectional x-ray imaging) was published by IEEE Press:
Avinash C. Kak and Malcolm Slaney, Principles of
Computerized Tomographic Imaging, (New York : IEEE
Press, c1988). The software used to generate many of the
tomographic images in this book is available. The parallel
beam reconstruction on the left was generated with the
commands
