I'm currently developing an electron app, which I hope will be able to measure the pitch of guitar input on desktop.

My initial idea is one tone at a time so please let me know if FTT is appropriate.

Edit: per comments it seems that FTT is not great so I'm considering using Harmonic Product Spectrum for example

I don't have too much experience with node.js, but so far I've managed to fork the broken microphone package and tweak it a bit to be able to fetch a wav format data from sox.

This is the actual code that spawns the process and fetches the data (simplified, it actually has a startCapture method which spawns the recording process):

const spawn = require('child_process').spawn;
const PassThrough = require('stream').PassThrough;

const audio = new PassThrough;
const info = new PassThrough;

const recordingProcess = spawn('sox', ['-d', '-t', 'wav', '-p'])
recordingProcess.stdout.pipe(audio);
recordingProcess.stderr.pipe(info);

And in another js file, I listen for the data event:

mic.startCapture({format: 'wav'});
mic.audioStream.on('data', function(data) {
    /* data is Uint8Array[8192] */
});

Ok so I'm getting an array of data which seems to be a good start. I know I should be applying somehow a pitch detection algorithm to start the pitch analysis

Am I going in the right direction? What format should this data be in? How can I use this data for pitch detection?

I'm currently developing an electron app, which I hope will be able to measure the pitch of guitar input on desktop.

My initial idea is one tone at a time so please let me know if FTT is appropriate.

Edit: per comments it seems that FTT is not great so I'm considering using Harmonic Product Spectrum for example

I don't have too much experience with node.js, but so far I've managed to fork the broken microphone package and tweak it a bit to be able to fetch a wav format data from sox.

This is the actual code that spawns the process and fetches the data (simplified, it actually has a startCapture method which spawns the recording process):

const spawn = require('child_process').spawn;
const PassThrough = require('stream').PassThrough;

const audio = new PassThrough;
const info = new PassThrough;

const recordingProcess = spawn('sox', ['-d', '-t', 'wav', '-p'])
recordingProcess.stdout.pipe(audio);
recordingProcess.stderr.pipe(info);

And in another js file, I listen for the data event:

mic.startCapture({format: 'wav'});
mic.audioStream.on('data', function(data) {
    /* data is Uint8Array[8192] */
});

Ok so I'm getting an array of data which seems to be a good start. I know I should be applying somehow a pitch detection algorithm to start the pitch analysis

Am I going in the right direction? What format should this data be in? How can I use this data for pitch detection?

• javascript
• node.js
• audio
• fft

• Yes! I love the idea btw, I hope you make a tuner of it ;) Always wanted a command-line guitar tuner. – user3310334 Commented Dec 15, 2016 at 22:47
• 1 Using the FFT is not a great way to measure pitch, especially if you want sufficient accuracy for a tuner. There are much better pitch detection algorithms, e.g. Harmonic Product Spectrum. Note: there is subtle but important difference between pitch and frequency - you want to measure musical pitch, not frequency. – Paul R Commented Dec 15, 2016 at 23:04
• 4 Read the above link about pitch - pitch is a percept - when you have a complex sound like a musical instrument playing a single note, there are multiple components (usually fundamental + harmonics, but some of these may be missing) - pitch is related to the fundamental frequency, but the perceived pitch may be different from the physical frequency of the fundamental. Note that frequency really only applies to a single component, whereas pitch applies to the whole complex sound. Only for a pure sine wave would they be the same. – Paul R Commented Dec 15, 2016 at 23:13
• Note also that there are dozens of very similar questions on SO already, usually people trying to implement instrument tuner apps or similar, and misguidedly thinking that all they need is an FFT - you might want to search for the tags [fft], [dsp], [pitch], [frequency], etc, or just "FFT tuner". – Paul R Commented Dec 15, 2016 at 23:15

1 Answer 1

Since you're getting a buffer with WAV data, you can use the wav-decoder library to parse it, and then feed it to the pitchfinder library to obtain the frequency of the audio.

const Pitchfinder = require('pitchfinder')
const WavDecoder = require('wav-decoder')
const detectPitch = new Pitchfinder.YIN()

const frequency = detectPitch(WavDecoder.decode(data).channelData[0])

However, since you're using Electron, you can also just use the MediaStream Recording API in Chromium.

First of all, this will only work with Electron 1.7+, because it uses Chromium 58, the first version of Chromium to include a fix for a bug which prevented the AudioContext from decoding audio data from the MediaRecorder.

Also, for the purposes of this code, I'll be using ES7 async and await syntax, which should run just fine on Node.js 7.6+ and Electron 1.7+.

So let's assume your index.html for Electron looks like this:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8">
    <title>Frequency Finder</title>
  </head>
  <body>
    <h1>Tuner</h1>

    <div><label for="devices">Device:</label> <select id="devices"></select></div>

    <div>Pitch: <span id="pitch"></span></div>
    <div>Frequency: <span id="frequency"></span></div>

    <div><button id="record" disabled>Record</button></div>
  </body>

  <script>
    require('./renderer.js')
  </script>
</html>

Now let's get to work on the renderer script. First, let's set a few variables we'll be using:

const audioContext = new AudioContext()
const devicesSelect = document.querySelector('#devices')
const pitchText = document.querySelector('#pitch')
const frequencyText = document.querySelector('#frequency')
const recordButton = document.querySelector('#record')
let audioProcessor, mediaRecorder, sourceStream, recording

Alright, now onto the rest of the code. First, let's populate that <select> drop down in the Electron window with all the available audio input devices.

navigator.mediaDevices.enumerateDevices().then(devices => {
  const fragment = document.createDocumentFragment()
  devices.forEach(device => {
    if (device.kind === 'audioinput') {
      const option = document.createElement('option')
      option.textContent = device.label
      option.value = device.deviceId
      fragment.appendChild(option)
    }
  })
  devicesSelect.appendChild(fragment)

  // Run the event listener on the `<select>` element after the input devices
  // have been populated. This way the record button won't remain disabled at
  // start.
  devicesSelect.dispatchEvent(new Event('change'))
})

You'll notice at the end, we call an event that we've set on the <select> element in the Electron window. But, hold on, we never wrote that event handler! Let's add some code above the code we just wrote:

// Runs whenever a different audio input device is selected by the user.
devicesSelect.addEventListener('change', async e => {
  if (e.target.value) {
    if (recording) {
      stop()
    }

    // Retrieve the MediaStream for the selected audio input device.
    sourceStream = await navigator.mediaDevices.getUserMedia({
      audio: {
        deviceId: {
          exact: e.target.value
        }
      }
    })

    // Enable the record button if we have obtained a MediaStream.
    recordButton.disabled = !sourceStream
  }
})

Let's also actually write a handler for the record button, because at this moment it does nothing:

// Runs when the user clicks the record button.
recordButton.addEventListener('click', () => {
  if (recording) {
    stop()
  } else {
    record()
  }
})

Now we display audio devices, let the user select them, and have a record button... but we still have unimplemented functions - record() and stop().

Let's stop right here to make an architectural decision.

We can record the audio, grab the audio data, and analyse it to obtain its pitch, all in renderer.js. However, analysing the data for pitch is an expensive operation. Therefore, it would be good to have the ability to run that operation out-of-process.

Luckily, Electron 1.7 brings in support for web workers with a Node context. Creating a web worker will allow us to run the expensive operation in a different process, so it doesn't block the main process (and UI) while it is running.

So keeping this in mind, let's assume that we will create a web worker in audio-processor.js. We'll get to the implementation later, but we'll assume it accepts a message with an object, {sampleRate, audioData}, where sampleRate is the sample rate, and audioData is a Float32Array which we'll pass to pitchfinder.

Let's also assume that:

• If processing of the recording succeeded, the worker returns a message with an object {frequency, key, octave} - an example would be {frequency: 440.0, key: 'A', octave: 4}.
• If processing of the recording failed, the worker returns a message with null.

Let's write our record function:

function record () {
  recording = true
  recordButton.textContent = 'Stop recording'

  if (!audioProcessor) {
    audioProcessor = new Worker('audio-processor.js')

    audioProcessor.onmessage = e => {
      if (recording) {
        if (e.data) {
          pitchText.textContent = e.data.key + e.data.octave.toString()
          frequencyText.textContent = e.data.frequency.toFixed(2) + 'Hz'
        } else {
          pitchText.textContent = 'Unknown'
          frequencyText.textContent = ''
        }
      }
    }
  }

  mediaRecorder = new MediaRecorder(sourceStream)

  mediaRecorder.ondataavailable = async e => {
    if (e.data.size !== 0) {
      // Load the blob.
      const response = await fetch(URL.createObjectURL(data))
      const arrayBuffer = await response.arrayBuffer()
      // Decode the audio.
      const audioBuffer = await audioContext.decodeAudioData(arrayBuffer)
      const audioData = audioBuffer.getChannelData(0)
      // Send the audio data to the audio processing worker.
      audioProcessor.postMessage({
        sampleRate: audioBuffer.sampleRate,
        audioData
      })
    }
  }

  mediaRecorder.start()
}

Once we start recording with the MediaRecorder, we won't get our ondataavailable handler called until recording is stopped. This is a good time to write our stop function.

function stop () {
  recording = false
  mediaRecorder.stop()
  recordButton.textContent = 'Record'
}

Now all that's left is to create our worker in audio-processor.js. Let's go ahead and create it.

const Pitchfinder = require('pitchfinder')

// Conversion to pitch from frequency based on technique used at
// https://www.johndcook.com/music_hertz_bark.html

// Lookup array for note names.
const keys = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B']

function analyseAudioData ({sampleRate, audioData}) {
  const detectPitch = Pitchfinder.YIN({sampleRate})

  const frequency = detectPitch(audioData)
  if (frequency === null) {
    return null
  }

  // Convert the frequency to a musical pitch.

  // c = 440.0(2^-4.75)
  const c0 = 440.0 * Math.pow(2.0, -4.75)
  // h = round(12log2(f / c))
  const halfStepsBelowMiddleC = Math.round(12.0 * Math.log2(frequency / c0))
  // o = floor(h / 12)
  const octave = Math.floor(halfStepsBelowMiddleC / 12.0)
  const key = keys[Math.floor(halfStepsBelowMiddleC % 12)]

  return {frequency, key, octave}
}

// Analyse data sent to the worker.
onmessage = e => {
  postMessage(analyseAudioData(e.data))
}

Now, if you run this all together... it won't work! Why?

We need to update main.js (or whatever the name of your main script is) so that when the main Electron window gets created, Electron is told to provide Node support in the context of the web worker. Otherwise, that require('pitchfinder') doesn't do what we want it to do.

This is simple, we just need to add nodeIntegrationInWorker: true in the window's webPreferences object. For example:

mainWindow = new BrowserWindow({
  width: 800,
  height: 600,
  webPreferences: {
    nodeIntegrationInWorker: true
  }
})

Now, if you run what you've put together, you'll get a simple Electron app that lets you record a small section of audio, test its pitch, and then display that pitch to screen.

This will work best with small snippets of audio, as the longer the audio, the longer it takes to process.

If you want a more complete example that goes more in depth, such as the ability to listen and return pitch live instead of making the user click record and stop all the time, take a look at the electron-tuner app that I've made. Feel free to look through the source to see how things are done - I've done my best to make sure it is well commented.

Here's a screenshot of it:

Hopefully all this helps you in your efforts.