Course Requirements HCS / ACN 6389 Matlab problems & lab assignments (40%) Speech Perception Lab  Oral presentations (10%)  Term project paper (50%)  Dr. Peter Assmann Fall 2018 Dates for lab assignments Term project: important dates Lab assignment 1: due Sept 13 Sep 6: Submit project topics (title only) Lab assignment 2: due Oct 11 Sep 20: Submit project outline Lab assignment 3: due Nov 1 Sep 27: Present project idea in class Lab assignment 4: due Nov 29 Nov 29 / Dec 8: Project presentations • 3-4 page written reports (code + results) on Dec 13: Final project paper due lab assignments Homework assignment Matlab First-time Matlab users: Variables are defined as matrices   Complete the interactive MATLAB Tutorial Scalars: 1 x 1   https://www.mathworks.com/help/matlab/getting-  Vectors: Row (nx 1) and column (1 x m) started-with-matlab.html Matrices: nx m  Matlab Onramp (free 2-hour online Matlab course) Matlab online (requires login)  Speech signals are represented as vectors https://matlab.mathworks.com/  (i.e., as a row or column of numbers). 1 Starting with Matlab Starting with Matlab Command Window – enter commands at Get help!   the prompt >> >> help help Create some numeric variables:  help Display help text in Command Window. help, by itself, lists all primary help topics. Each primary topic >> x = 5; % 1x1 scalar corresponds to a folder name on the MATLAB search path. >> y = 1:12; % 1x12 vector >> doc Matlab >> z = rand ( 4, 4 ); % 4x4 matrix doc Displays help in Matlab’sBrowser Window. More detailed information is provided. Starting with Matlab Starting with Matlab Try this: Create character variables by enclosing   >> help plot text in single quotes: >> a = 'c'; % 1x1 character >> b = dir( '*.m'); % directory listing Starting with Matlab Starting with Matlab Create character variables by enclosing Colon (:) operator   text in single quotes: >> x = 1 : 10; % create vector [1,2,3,…,10] >> x = 1 : 1 : 10; % same as above >> a = 'c'; % 1x1 character >> x = 1 : 2 : 10; % 1,3,5,…,10 >> b = dir( '*.m'); % directory listing >> x = 0 : 0.5 : 2; % [0,0.5,1,1.5,2] The semicolon tells Matlab not to echo the contents of the variable on the screen. This is important for large vectors (e.g. 10 seconds of speech). >> help : 2 Starting with Matlab Starting with Matlab Concatenate variables [make a list] Concatenate variables [make a list]   >> a = [1 2 3 4 5]; % row vector >> a = [1; 2; 3; 4; 5]; % column vector >> b = [100 -5 0]; >> b = [100; -5; 0]; % uses semi-colon >> c = [a b]; >> c = [a; b]; >> disp(c) Numeric variables c = 1 2 3 4 5 100 -5 0 Starting with Matlab Starting with Matlab Concatenate variables (make a list) Concatenate variables (make a list)   Be careful not to concatenate a row vector  >> a = 'Matlab is fun! '; Character variables with a column vector! >> b = 'Speech perception is interesting.'; >> a = [1 2 3 4 5]; >> c = [a b]; >> b = [100; -5; 0]; >> disp(c) >> c = [a b]; c = Error: Dimensions of arrays being concatenated are not consistent. Matlab is fun! Speech perception is interesting. Starting with Matlab Starting with Matlab Workspace – repository for variables you Workspace – repository for variables you   have created or loaded. Variables may be have created or loaded. Variables may be viewed, manipulated, saved, or cleared. viewed, manipulated, saved, or cleared. List the variables in the workspace: List the variables in the workspace:   >> who >> whos Name Size Bytes Class Attributes Your variables are: a 1x10 80 double ans 1x10 20 char a ans b c b 1x4 32 double c 1x14 112 double 3 Starting with Matlab Starting with Matlab Clear the workspace when you start a new Save your results:   script. >> clear % Save all variables from the workspace to test.mat: >> save test.mat % Save selected variables from the workspace to test.mat: >> save('test.mat', 'a', 'b'); Starting with Matlab Exercise  Load Matlab data from a file: 1. Create a row vector awith integers from 1 to  100. % Load all variables from file test.matinto the workspace : 2. Create another row vector bwith integers  >> load test.mat from 5 to 95 in  steps of 5. 3. Concatenate aand band store it as a third  % Load selected variables from test.mat: vector, c. >> load('test.mat', 'a', 'b'); 4. Plot the vector c.  5. Put the commands into a script and save it in  a Matlab script, exercise1.m.  Starting Matlab Starting Matlab Matlab opens in a specific directory Use “cd <path>” to change to another  (folder) on your computer, allowing you to directory: load or save files without specifying a >> cd C:\Users\assmann\Documents path. Type “cd” to find the current directory  Be careful to distinguish forward slash (/) from backslash (\) Use the cd command to change to another --these differ depending on the operating system  directory >> cd ( 'your\new\directory\' ) 4 Digital representations of speech Digital representations of signals Speech waveform: representation of the  amplitude variations in the signal as a function  of time.  x 104 • Amplitude quantization 2 amplitude time •• ITs(n8aim 0mt0hpe0el esesa axrmamatmepp llpieislns e8/g,s t)kh H e z Amplitude 01 with 16-bitamplitude -1 quantization. sampling quantization -2 0 200 400 600 800 Time (ms) Digital representations of signals Vector representation of speech Sampling frequency (e.g. 44.1 kHz) >> [y, fs]=audioread('wheel.wav'); % load waveform  (cid:0) Nyquist frequency Effects of discrete-time sampling on bandwidth  audioread.mis a Matlab function to read data from  (cid:0)(cid:0) .wav files Quantization rate (16 bits)   Left side = output variables; right side = input variables (cid:0)16 bits =216 quantization steps  The variable ‘y’ contains the speech signal. (cid:0)Effects of discrete-level quantization on dynamic range  The variable ‘fs’ contains the sample rate in Hz. Exercise Exercise  Create a script, exercise2.mto load and plot a   Display the sample rate  waveform file.   Compute the size of the waveform vector y  Download (or record) a .wav file and put it in   Plot the waveform the current directory:  Play the waveform using the soundsccommand  https://www.utdallas.edu/~assmann/hcs6389/scripts_functions.html  Right click on wheel.wav >> [y, fs]=audioread('wheel.wav');  5 Vector representation of speech Annotating plots  Load the waveform and plot it:  >> [y,fs]=audioread('wheel.wav'); % load  >> axis( [ 0 400 -1 1 ] ); % set axis limits waveform  >> xlabel('Time (ms)'); % x-axis label  >> t=(1:length(y) ) ./ (fs/1000); % set up time  >> ylabel('Amplitude'); % y-axis label axis in milliseconds  >> title('Waveform plot'); % axis title  >> plot( t, y ); % use plot command  Time (x) axis in milliseconds (1000 ms = 1 second)  Amplitude (y) axis ~ sound pressure level quantized  to 16 bits and range set to ±1. Exercise Try this: >> axis tight >> axis auto  Use built-in Matlab functions to find various properties of the waveform: » length ( y ) % vector length » min ( y ) % minimum value » max ( y ) % maximum value » mean ( y ) % mean value » std ( y ) % standard deviation » plot ( y ) % plot waveform » sound ( y, fs ) % listen to waveform » soundsc( y, fs ) % “safe” version of sound 21 Fourier analysis and synthesis • D.P.W. Ellis (2009). An introduction to signal  processing for speech. In The Handbook of  Phonetic Science, , 2ndedition, edited by  Hardcastle, Laver, and Gibbon. chapter 22, pp.  757‐780, Blackwell. Ellis (2009, p. 12) 6 Spectral analysis Frequency domain representation  Amplitudespectrum: sound pressure levels  Why perform spectral analyses of speech? associated with different frequency  The ear+brain carry out a form of frequency analysis components of a signal  The relevant features of speech are more readily visible in the amplitude spectrum than in the raw Power or intensity waveform Amplitude or magnitude  BUT: the ear is not a Fourier analyzer. Log units and decibels (dB)  Fourier analysis provides amplitudeand phase spectrum; speech cues have been mainly associated Phasespectrum: relative phases associated  with the amplitude spectrum. However, the ear is not with different frequency components "phase-deaf"; many phase changes are clearly audible. Degrees or radians  Frequency selectivity is greatest for frequencies below 1 kHz and declines at higher frequencies Spectral analysis in Matlab  Amplitude spectrum of a vector: » X= fft (y); » help fft FFT Discrete Fourier transform. FFT(X) is the discrete Fourier transform (DFT) of vector X. If the length of X is a power of two, a fast radix-2 fast- Fourier transform algorithm is used. If the length of X is not a power of two, a slower non-power-of-two algorithm is employed. For matrices, the FFT operation is applied to each column. FFT(X,N) is the N-point FFT, padded with zeros if X has less than N points and truncated if it has more. Spectral analysis in Matlab Spectral analysis in Matlab Log magnitude (amplitude) spectrum: Log magnitude (amplitude) spectrum:   » X= fft (y); » plot(20*log10(abs(fft(y)))) » m = 20 * log10 ( abs ( X ) ); 140 » help abs 120 100 ABS Absolute value. 80 ABS(X) is the absolute value of the elements of X. When X is complex, ABS(X) is the complex 60 modulus (magnitude) of the elements of X. 400 200 400 600 800 1000 7 Plotting amplitude spectra Plotting amplitude spectra » help fp » [a,f]=fp(wave,n,rate,string,window); »[a,f]=fp(y,1000,rate,'speech','hann'); FP: function to compute & plot amplitude spectrum Usage: [a,f]=fp(wave,n,rate,string,window); speech 20 wave: input waveform 10 n: if unspecified, n=length(wave); padded with zeros if 0 rstanrteienc:g ess:a stmaitrplyel ef orar tger ianp hH (zd (edfeafualut lnt o1n0e0)00 Hz) mplitude (dB)--2100 A-30 windowoptions: 'hann', 'hamm', 'kais', or 'rect' -40 (default=hanning) [a,f]:FFT log magnitude, frequency -500 1 Frequen2cy (kHz) 3 4 Speech spectrograms Speech spectrogram  What is a speech spectrogram?  Display of amplitude spectrum at successive instants in  running amplitude spectra (codes amplitude time ("running spectra") changes in different frequency bands over time).  How can 3 dimensions be represented on a two- dimensional display?  Gray-scale spectrogram  Waterfall plots  Animation  Why are speech spectrograms useful?  Shows dynamic properties of speech  Includes frequency analysis “The watchdog” Speech spectrograms in Matlab waveform spectrogram » help specgram SPECGRAM Calculate spectrogram from signal. B = SPECGRAM(A,NFFT,Fs,WINDOW,NOVERLAP) calculates the spectrogram for the signal in vector A. SPECGRAM splits the signal into overlapping segments, windows each with the WINDOW vector and forms the columns of B with their zero-padded, length NFFT discrete Fourier transforms. 8 Speech spectrograms in Matlab » help sp sp: create gray-scale spectrogram Usage: h=sp(wave,rate,nfft,nsampf,nhop,pre,drng); wave: input waveform rate: sample rate in Hz (default 8000 Hz) nfft: FFT window length (default: 256 samples) nsampf: number of samples per frame (default: 60) nhop: number of samples to hop to next frame (default: 5 samples) pre: preemphasis factor (0-1) (default: 1) drng: dynamic range in dB (default: 80) title: title for graph (default: none) 9