Under the Hood

hnr.py

This function is inspired by the Speech Analysis repository at https://github.com/brookemosby/Speech_Analysis

surfboard.hnr.get_harmonics_to_noise_ratio(waveform, sample_rate, min_pitch=75.0, silence_threshold=0.1, periods_per_window=4.5)

Given a waveform, its sample rate, some conditions for voiced and unvoiced frames (including min pitch and silence threshold), and a “periods per window” argument, compute the harmonics to noise ratio. This is a good measure of voice quality and is an important metric in cognitively impaired patients. Compute the mean hnr_vector: harmonics to noise ratio.

Parameters:
  • waveform (np.array, [T, ]) – waveform signal
  • sample_rate (int > 0) – sampling rate of the waveform
  • min_pitch (float > 0) – minimum acceptable pitch. converts to maximum acceptable period.
  • silence_threshold (1 >= float >= 0) – needs to be in [0, 1]. Below this amplitude, does not consider frames.
  • periods_per_window (float > 0) – 4.5 is best for speech.
Returns:

Harmonics to noise ratio of the entire considered

waveform.
Return type:

float

jitters.py

This file contains all the functions needed to compute the jitters of a waveform.

surfboard.jitters.validate_frequencies(frequencies, p_floor, p_ceil, max_p_factor)

Given a sequence of frequencies, [f1, f2, …, fn], a minimum period, maximum period, and maximum period factor, first remove all frequencies computed as 0. Then, if periods are the inverse frequencies, this function returns True if the sequence of periods satisfies the conditions, otherwise returns False. In order to satisfy the maximum period factor, the periods have to satisfy pi / pi+1 < max_p_factor and pi+1 / pi < max_p_factor.

Parameters:
  • frequencies (sequence, eg list, of floats) – sequence of frequencies == 1 / period.
  • p_floor (float) – minimum acceptable period.
  • p_ceil (float) – maximum acceptable period.
  • max_p_factor (float) – value to use for the period factor principle
Returns:

True if the conditions are met, False otherwise.
Return type:

bool
surfboard.jitters.get_mean_period(frequencies, p_floor, p_ceil, max_p_factor)

Given a sequence of frequencies, passes these through the validation phase, then computes the mean of the remaining periods. Note period = 1/f.

Parameters:
  • frequencies (sequence, eg list, of floats) – sequence of frequencies
  • p_floor (float) – minimum acceptable period.
  • p_ceil (float) – maximum acceptable period.
  • max_p_factor (float) – value to use for the period factor principle
Returns:

The mean of the acceptable periods.
Return type:

float
surfboard.jitters.get_local_absolute_jitter(frequencies, p_floor, p_ceil, max_p_factor)

Given a sequence of frequencies, and some period conditions, compute the local absolute jitter, as per https://royalsocietypublishing.org/action/downloadSupplement?doi=10.1098%2Frsif.2010.0456&file=rsif20100456supp1.pdf

Parameters:
  • frequencies (sequence, eg list, of floats) – sequence of estimated frequencies
  • p_floor (float) – minimum acceptable period.
  • p_ceil (float) – maximum acceptable period.
  • max_p_factor (float) – value to use for the period factor principle
Returns:

the local absolute jitter.
Return type:

float
surfboard.jitters.get_local_jitter(frequencies, p_floor, p_ceil, max_p_factor)

Given a sequence of frequencies, and some period conditions, compute the local jitter, as per https://royalsocietypublishing.org/action/downloadSupplement?doi=10.1098%2Frsif.2010.0456&file=rsif20100456supp1.pdf

Parameters:
  • frequencies (sequence, eg list, of floats) – sequence of estimated frequencies
  • p_floor (float) – minimum acceptable period.
  • p_ceil (float) – maximum acceptable period.
  • max_p_factor (float) – value to use for the period factor principle
Returns:

the local jitter.
Return type:

float
surfboard.jitters.get_rap_jitter(frequencies, p_floor, p_ceil, max_p_factor)

Given a sequence of frequencies, and some period conditions, compute the rap jitter, as per https://royalsocietypublishing.org/action/downloadSupplement?doi=10.1098%2Frsif.2010.0456&file=rsif20100456supp1.pdf

Parameters:
  • frequencies (sequence, eg list, of floats) – sequence of estimated frequencies
  • p_floor (float) – minimum acceptable period.
  • p_ceil (float) – maximum acceptable period.
  • max_p_factor (float) – value to use for the period factor principle
Returns:

the rap jitter.
Return type:

float
surfboard.jitters.get_ppq5_jitter(frequencies, p_floor, p_ceil, max_p_factor)

Given a sequence of frequencies, and some period conditions, compute the ppq5 jitter, as per https://royalsocietypublishing.org/action/downloadSupplement?doi=10.1098%2Frsif.2010.0456&file=rsif20100456supp1.pdf

Parameters:
  • frequencies (sequence, eg list, of floats) – sequence of estimated frequencies
  • p_floor (float) – minimum acceptable period.
  • p_ceil (float) – maximum acceptable period.
  • max_p_factor (float) – value to use for the period factor principle
Returns:

the ppq5 jitter.
Return type:

float
surfboard.jitters.get_ddp_jitter(frequencies, p_floor, p_ceil, max_p_factor)

Given a sequence of frequencies, and some period conditions, compute the ddp jitter, as per http://www.fon.hum.uva.nl/praat/manual/PointProcess__Get_jitter__ddp____.html

Parameters:
  • frequencies (sequence, eg list, of floats) – sequence of estimated frequencies
  • p_floor (float) – minimum acceptable period.
  • p_ceil (float) – maximum acceptable period.
  • max_p_factor (float) – value to use for the period factor principle
Returns:

the ddp jitter.
Return type:

float
surfboard.jitters.get_jitters(f0_contour, p_floor=0.0001, p_ceil=0.02, max_p_factor=1.3)

Compute the jitters mathematically, according to certain conditions given by p_floor, p_ceil and max_p_factor.

Parameters:
  • f0_contour (np.array [T / hop_length, ]) – the fundamental frequency contour.
  • p_floor (float) – minimum acceptable period.
  • p_ceil (float) – maximum acceptable period.
  • max_p_factor (float) – value to use for the period factor principle
Returns:

Dictionary mapping strings to floats, with keys

”localJitter”, “localabsoluteJitter”, “rapJitter”, “ppq5Jitter”, “ddpJitter”
Return type:

dict

shimmers.py

This file contains all the functions needed to compute the shimmers of a waveform.

surfboard.shimmers.validate_amplitudes(amplitudes, frequencies, max_a_factor, p_floor, p_ceil, max_p_factor)

First check that frequencies corresponding to this set of amplitudes are valid. Then Returns True if this set of amplitudes is validated as per the maximum amplitude factor principle, i.e. if amplitudes = [a1, a2, … , an], this functions returns false if any two successive amplitudes alpha, beta satisfy alpha / beta > max_a_factor or beta / alpha > max_a_factor. False otherwise.

Parameters:
  • amplitudes (list) – ordered list of amplitudes to run by this principle.
  • frequencies (sequence, eg list, of floats) – sequence of frequencies == 1 / period.
  • max_a_factor (float) – the threshold to run the principle.
  • p_floor (float) – minimum acceptable period.
  • p_ceil (float) – maximum acceptable period.
  • max_p_factor (float) – value to use for the period factor principle
Returns:

True if this set of amplitudes satisifies the principle

and this set of frequencies satisfies the period condition, False otherwise.
Return type:

bool
surfboard.shimmers.get_local_shimmer(amplitudes, frequencies, max_a_factor, p_floor, p_ceil, max_p_factor)

Given a list of amplitudes, returns the localShimmer as per https://royalsocietypublishing.org/action/downloadSupplement?doi=10.1098%2Frsif.2010.0456&file=rsif20100456supp1.pdf

Parameters:
  • amplitudes (list of floats) – The list of peak amplitudes in each frame.
  • max_a_factor (float) – The maximum A factor to validate amplitudes. See validate_amplitudes().
Returns:

The local shimmer computed over this sequence of amplitudes.
Return type:

float
surfboard.shimmers.get_local_db_shimmer(amplitudes, frequencies, max_a_factor, p_floor, p_ceil, max_p_factor)

Given a list of amplitudes, returns the localdbShimmer as per https://royalsocietypublishing.org/action/downloadSupplement?doi=10.1098%2Frsif.2010.0456&file=rsif20100456supp1.pdf

Parameters:
  • amplitudes (list of floats) – The list of peak amplitudes in each frame.
  • max_a_factor (float) – The maximum A factor to validate amplitudes. See validate_amplitudes().
Returns:

The local DB shimmer computed over this sequence of amplitudes.
Return type:

float
surfboard.shimmers.get_apq_shimmer(amplitudes, frequencies, max_a_factor, p_floor, p_ceil, max_p_factor, apq_no)

Given a list of amplitudes, returns the apq{apq_no}Shimmer as per https://royalsocietypublishing.org/action/downloadSupplement?doi=10.1098%2Frsif.2010.0456&file=rsif20100456supp1.pdf

Parameters:
  • amplitudes (list of floats) – The list of peak amplitudes in each frame.
  • max_a_factor (float) – The maximum A factor to validate amplitudes. See validate_amplitudes().
  • apq_no (int) – an odd number which corresponds to the number of neighbors used to compute the shimmer.
Returns:

The apqShimmer computed over this sequence of amplitudes

with this APQ number.
Return type:

float
surfboard.shimmers.get_shimmers(waveform, sample_rate, f0_contour, max_a_factor=1.6, p_floor=0.0001, p_ceil=0.02, max_p_factor=1.3)

Compute five different types of shimmers using functions defined above.

Parameters:
  • waveform (np.array, [T, ]) – waveform over which to compute shimmers
  • sample_rate (int) – sampling rate of waveform.
  • f0_contour (np.array, [T / hop_length, ]) – the fundamental frequency contour.
  • max_a_factor (float) – value to use for amplitude factor principle
  • p_floor (float) – minimum acceptable period.
  • p_ceil (float) – maximum acceptable period.
  • max_p_factor (float) – value to use for the period factor principle
Returns:

Dictionary mapping strings to floats, with keys

”localShimmer”, “localdbShimmer”, “apq3Shimmer”, “apq5Shimmer”, “apq11Shimmer”
Return type:

dict

dfa.py

surfboard.dfa.get_deviation_for_dfa(signal, window_length)

Given a signal, compute the trend value for one window length, as per https://link.springer.com/article/10.1186/1475-925X-6-23 In order to get the overall DFA (detrended fluctuation analysis), compute this for a variety of window lengths, then plot that on a log-log graph, and get the slope.

Parameters:
  • signal (np.array, [T, ]) – waveform
  • window_length (int > 0) – L in the paper linked above. Length of windows for trend.
Returns:

average rmse for fitting lines on chunks of window lengths on the

cumulative sums of this signal.
Return type:

float
surfboard.dfa.get_dfa(signal, window_lengths)

Given a signal, compute the DFA (detrended fluctuation analysis) as per https://link.springer.com/article/10.1186/1475-925X-6-23 See paper equations (13) to (16) for more information.

spectrum.py

Spectrum features. The code in this file is inspired by audiocontentanalysis.org For more details, visit the pyACA package: https://github.com/alexanderlerch/pyACA

surfboard.spectrum.get_spectral_centroid(magnitude_spectrum, sample_rate)

Given the magnitude spectrum and the sample rate of the waveform from which it came, compute the spectral centroid.

Parameters:
  • magnitude_spectrum (np.array, [n_frequencies, T / hop_length]) – the spectrogram
  • sample_rate (int) – The sample rate of the waveform
Returns:

the spectral centroid sequence in Hz.
Return type:

np.array [1, T / hop_length]
surfboard.spectrum.get_spectral_slope(magnitude_spectrum, sample_rate)

Given the magnitude spectrum and the sample rate of the waveform from which it came, compute the spectral slope.

Parameters:
  • magnitude_spectrum (np.array, [n_frequencies, T / hop_length]) – the spectrogram
  • sample_rate (int) – The sample rate of the waveform
Returns:

the spectral slope sequence.
Return type:

np.array [1, T / hop_length]
surfboard.spectrum.get_spectral_flux(magnitude_spectrum, sample_rate)

Given the magnitude spectrum and the sample rate of the waveform from which it came, compute the spectral flux.

Parameters:
  • magnitude_spectrum (np.array, [n_frequencies, T / hop_length]) – the spectrogram
  • sample_rate (int) – The sample rate of the waveform
Returns:

the spectral flux sequence.
Return type:

np.array [1, T / hop_length]
surfboard.spectrum.get_spectral_spread(magnitude_spectrum, sample_rate)

Given the magnitude spectrum and the sample rate of the waveform from which it came, compute the spectral spread.

Parameters:
  • magnitude_spectrum (np.array, [n_frequencies, T / hop_length]) – the spectrogram
  • sample_rate (int) – The sample rate of the waveform
Returns:

the spectral spread (Hz).
Return type:

np.array [1, T / hop_length]
surfboard.spectrum.get_spectral_skewness(magnitude_spectrum, sample_rate)

Given the magnitude spectrum and the sample rate of the waveform from which it came, compute the spectral skewness.

Parameters:
  • magnitude_spectrum (np.array, [n_frequencies, T / hop_length]) – the spectrogram
  • sample_rate (int) – The sample rate of the waveform
Returns:

the spectral skewness.
Return type:

np.array [1, T / hop_length]
surfboard.spectrum.get_spectral_kurtosis(magnitude_spectrum, sample_rate)

Given the magnitude spectrum and the sample rate of the waveform from which it came, compute the spectral skewness.

Parameters:
  • magnitude_spectrum (np.array, [n_frequencies, T / hop_length]) – the spectrogram
  • sample_rate (int) – The sample rate of the waveform
Returns:

the spectral kurtosis.
Return type:

np.array [1, T / hop_length]

misc_components.py

This file contains components which do not fall under one category.

surfboard.misc_components.get_crest_factor(waveform, sample_rate, rms, frame_length_seconds=0.04, hop_length_seconds=0.01)

Get the crest factor of this waveform, on sliding windows. This value measures the local intensity of peaks in a waveform. Implemented as per: https://en.wikipedia.org/wiki/Crest_factor

Parameters:
  • waveform (np.array, [T, ]) – waveform over which to compute crest factor
  • sample_rate (int > 0) – number of samples per second in waveform
  • rms (np.array, [1, T / hop_length]) – energy values.
  • frame_length_seconds (float) – length of the sliding window, in seconds.
  • hop_length_seconds (float) – how much the window shifts for every timestep, in seconds.
Returns:

Crest factor for each frame.
Return type:

np.array, [1, T / hop_length]
surfboard.misc_components.get_f0(waveform, sample_rate, hop_length_seconds=0.01, method='swipe', f0_min=60, f0_max=300)

Compute the F0 contour using PYSPTK: https://github.com/r9y9/pysptk/.

Parameters:
  • waveform (np.array, [T, ]) – waveform over which to compute f0
  • sample_rate (int > 0) – number of samples per second in waveform
  • hop_length (int) – hop size argument in pysptk.swipe. Corresponds to hopsize in the window sliding of the computation of f0.
  • method (str) – is one of ‘swipe’ or ‘rapt’. Define which method to use for f0 calculation. See https://github.com/r9y9/pysptk
Returns:

Dictionary containing keys:
”contour” (np.array, [1, t1]): f0 contour of waveform. Contains unvoiced

frames.

”values” (np.array, [1, t2]): nonzero f0 values waveform. Note that this

discards all unvoiced frames. Use to compute mean, std, and other statistics.

”mean” (float): mean of the f0 contour. “std” (float): standard deviation of the f0 contour.
Return type:

dict
surfboard.misc_components.get_ppe(rat_f0)

Compute pitch period entropy. Here is a reference MATLAB implementation: https://github.com/Mak-Sim/Troparion/blob/5126f434b96e0c1a4a41fa99dd9148f3c959cfac/Perturbation_analysis/pitch_period_entropy.m Note that computing the PPE relies on the existence of voiced portions in the F0 trajectory.

Parameters:rat_f0 (np.array) – f0 voiced frames divided by f_min
Returns:The pitch period entropy, as per http://www.maxlittle.net/students/thesis_tsanas.pdf
Return type:float
surfboard.misc_components.get_shannon_entropy(sequence)

Given a sequence, compute the Shannon Entropy, defined in https://ijssst.info/Vol-16/No-4/data/8258a127.pdf

Parameters:sequence (np.array, [t, ]) – sequence over which to compute.
Returns:shannon entropy.
Return type:float
surfboard.misc_components.get_shannon_entropy_slidingwindow(waveform, sample_rate, frame_length_seconds=0.04, hop_length_seconds=0.01)

Same function as above, but decorated by the metric_slidingwindow decorator. See above for documentation on this.

Parameters:
  • waveform (np.array, [T, ]) – waveform over which to compute the shannon entropy array
  • sample_rate (int > 0) – number of samples per second in waveform
  • frame_length_seconds (float) – length of the sliding window, in seconds.
  • hop_length_seconds (float) – how much the window shifts for every timestep, in seconds.
Returns:

Shannon entropy over windows.
Return type:

np.array, [1, T/hop_length]
surfboard.misc_components.get_loudness(waveform, sample_rate)

Compute the loudness of waveform using the pyloudnorm package. See https://github.com/csteinmetz1/pyloudnorm for more details on potential arguments to the functions below.

Parameters:
  • waveform (np.array, [T, ]) – waveform to compute loudness on
  • sample_rate (int > 0) – sampling rate of waveform
Returns:

the loudness of self.waveform
Return type:

float
surfboard.misc_components.get_loudness_slidingwindow(waveform, sample_rate, frame_length_seconds=0.04, hop_length_seconds=0.01)

Same function as get_loudness, but decorated by the metric_slidingwindow decorator. See get_loudness documentation for this.

Parameters:
  • waveform (np.array, [T, ]) – waveform over which to compute the kurtosis array
  • sample_rate (int > 0) – number of samples per second in waveform
  • frame_length_seconds (float) – length of the sliding window, in seconds.
  • hop_length_seconds (float) – how much the window shifts for every timestep, in seconds.
Returns:

Frame level loudness
Return type:

np.array, [1, T / hop_length]
surfboard.misc_components.get_kurtosis_slidingwindow(waveform, sample_rate, frame_length_seconds=0.04, hop_length_seconds=0.01)

Same function as above, but decorated by the metric_slidingwindow decorator. See above documentation for this.

Parameters:
  • waveform (np.array, [T, ]) – waveform over which to compute the kurtosis array
  • sample_rate (int > 0) – number of samples per second in waveform
  • frame_length_seconds (float) – length of the sliding window, in seconds.
  • hop_length_seconds (float) – how much the window shifts for every timestep, in seconds.
Returns:

Kurtosis over windows
Return type:

np.array, [1, T/hop_length]
surfboard.misc_components.get_log_energy(matrix, time_axis=-1)

Compute the log energy of a matrix as per Abeyrante et al. 2013.

Parameters:
  • matrix (np.array) – matrix over which to compute. This has to be a 1 or 2-dimensional np.array
  • time_axis (int >= 0) – the axis in matrix which corresponds to time.
Returns:

The log energy of matrix, computed as per

the paper above.
Return type:

float
surfboard.misc_components.get_log_energy_slidingwindow(waveform, sample_rate, frame_length_seconds=0.04, hop_length_seconds=0.01)

Same function as above, but decorated by the metric_slidingwindow decorator. See above documentation for this.

Parameters:
  • waveform (np.array, [T, ]) – waveform over which to compute the log energy array
  • sample_rate (int > 0) – number of samples per second in waveform
  • frame_length_seconds (float) – length of the sliding window, in seconds.
  • hop_length_seconds (float) – how much the window shifts for every timestep, in seconds.
Returns:

log_energy over windows
Return type:

np.array, [1, T/hop_length]
surfboard.misc_components.get_bark_spectrogram(waveform, sample_rate, n_fft_seconds, hop_length_seconds)

Convert a spectrogram to a bark-band spectrogram.

Parameters:
  • waveform (np.array, [T, ]) – waveform over which to compute the bark spectrogram.
  • sample_rate (int > 0) – number of samples per second in waveform.
  • n_fft_seconds (float > 0) – length of the fft window, in seconds
Returns:

The original spectrogram

with bins converted into the Bark scale.
Return type:

np.array, [n_bark_bands, t]

utils.py

This file contains a variety of helper functions for the surfboard package.

surfboard.utils.metric_slidingwindow(frame_length, hop_length, truncate_end=False)

We use this decorator to decorate functions which take a sequence as an input and return a metric (float). For example the sum of a sequence. This decorator will enable us to quickly compute the metrics over a sliding window. Note the existence of the implicit decorator below which allows us to have arguments to the decorator.

Parameters:
  • frame_length (int) – The length of the sliding window
  • hop_length (int) – How much to slide the window every time
  • truncate_end (bool) – whether to drop frames which are shorter than frame_length (the end frames, typically)
Returns:

The function which computes the metric over sliding

windows.
Return type:

function
surfboard.utils.numseconds_to_numsamples(numseconds, sample_rate)

Convert a number of seconds a sample rate to the number of samples for n_fft, frame_length and hop_length computation. Find the closest power of 2 for efficient computations.

Parameters:
  • numseconds (float) – number of seconds that we want to convert
  • sample_rate (int) – how many samples per second
Returns:

closest power of 2 to int(numseconds * sample_rate)
Return type:

int
surfboard.utils.max_peak_amplitude(signal)

Returns the maximum absolute value of a signal.

Parameters:[T, ] (np.array) – a waveform
Returns:the maximum amplitude of this waveform, in absolute value
Return type:float
surfboard.utils.peak_amplitude_slidingwindow(signal, sample_rate, frame_length_seconds=0.04, hop_length_seconds=0.01)

Apply the metric_slidingwindow decorator to the the peak amplitude computation defined above, effectively computing frequency from fft over sliding windows.

Parameters:
  • signal (np.array [T,]) – waveform over which to compute.
  • sample_rate (int) – number of samples per second in the waveform
  • frame_length_seconds (float) – how many seconds in one frame. This value is defined in seconds instead of number of samples.
  • hop_length_seconds (float) – how many seconds frames shift each step. This value is defined in seconds instead of number of samples.
Returns:

peak amplitude on each window.
Return type:

np.array, [1, T / hop_length]
surfboard.utils.shifted_sequence(sequence, num_sequences)

Given a sequence (say a list) and an integer, returns a zipped iterator of sequence[:-num_sequences + 1], sequence[1:-num_sequences + 2], etc.

Parameters:
  • sequence (list or other iteratable) – the sequence over which to iterate in various orders
  • num_sequences (int) – the number of sequences over which we iterate. Also the number of elements which come out of the output at each call.
Returns:

zipped shifted sequences.
Return type:

iterator
surfboard.utils.lpc_to_lsf(lpc_polynomial)

This code is inspired by the following: https://uk.mathworks.com/help/dsp/ref/lpctolsflspconversion.html

Parameters:lpc_polynomial (list) – length n + 1 list of lpc coefficients. Requirements is that the polynomial is ordered so that lpc_polynomial[0] == 1
Returns:length n list of line spectral frequencies.
Return type:list
surfboard.utils.parse_component(component)

Parse the component coming from the .yaml file.

Parameters:component (str or dict) – Can be either a str, or a dictionary. Comes from the .yaml config file. If it is a string, simply return, since its the component name without arguments. Otherwise, parse.
Returns:
tuple containing:
str: name of the method to be called from sound.Waveform dict: arguments to be unpacked. None if no arguments to
compute.
Return type:tuple
surfboard.utils.example_audio_file(which_file)

Returns the path to one of sustained_a, sustained_o or sustained_e included with the Surfboard package.

Parameters:which_file (str) – One of ‘a’, ‘o’ or ‘e’
Returns:The path to the chosen file.
Return type:str
exception surfboard.utils.YamlFileException(message)