Web Audio

An emerging platform for Audio applications


FOSSASIA 2015

Web Audio

An emerging platform for Audio applications

How to make noise on the Web


FOSSASIA 2015

About Me..


Agenda

  • What is WebAudio??
  • Why WebAudio matters??
  • How to use WebAudio??
# What??

What?? - History

  • bgsound
  • <object> , <embed>
  • <audio>
  • Audio Data API
  • Web Audio API
  • ← Flash
  • ← HTML5

What?? - Philosophy

... include the capabilities found in modern game audio engines as well as some of the mixing, processing, and filtering tasks that are found in modern desktop audio production applications.

— W3C WebAudio Draft Spec

What?? - Status

What?? - Status

Specification FPWD LC CR PR Rec
Web Audio API Dec 2011 Q1 2014 Q3 2014 Q1 2015 Q1 2015

— http://www.w3.org/2011/audio/wiki/W3C_Audio_Publications_and_Milestones#Latest_Estimates

What?? - Status

http://caniuse.com/#search=audio-api

# Why??

Why Audio on the Web??


→ Distribution


Why Audio on the Web??


→ Collaboration


  • Real-time control (WebSockets)
  • Real-time collaboration (WebRTC)
  • Eg. SoundTrap

Why Audio on the Web??


→ Expression


  • Integration with visual elements
  • Integration with interactive elements
  • Eg. Allen the Alien

Why in the browser??


→ Open Standards


  • Flash sucked for doing real-time audio
  • Dependency on Adobe + Browser Vendor
  • No visbility/control over changes and features
# How??
## How?? - Basics

"signal routing graph paradigm"

How?? - Implementation


→ Javascript API

							
	// Create Context and Nodex
	var audioCtx = new AudioContext();
	var oscillator = audioCtx.createOscillator();
	var filter = context.createBiquadFilter();

	// Set parameter values
	oscillator.detune.value = -400;
	filter.type = filter.LOWPASS;
	filter.frequency.value = 5000;

	// Connect nodes
	oscillator.connect(filter);
	filter.connect(audioCtx.destination);
							
						

How?? - Implementation


→ Audio Processing done at a lower level (C++/ASM)

— Chrome Source

How?? - Implementation


→ Realtime (Callback Based)

							
	var sp = context.createScriptProcessor();
	sp.onprocess = function (processEvent){
		/* fill your own buffer */
		var out = autoTune(processEvent.inputBuffers);
		processEvent.outputBuffers = out;
	}
	sp.connect(context.destination);
							
						
## How?? - Nodes - 3 Types of Nodes (Source, Effect, Destination). - Each node exposes Parameters. - Connect Nodes in all sort of configuration.
## How?? - Sources - Oscillator - AudioBufferSource - MediaElementAudioSource - MediaStreamAudioSource
## How?? - Effects - Gain - BiquadFilter - Delay - Analyser (FFT) - StereoPanner - Panner (3D-Panner) - Convolver - WaveShaper - DynamicsCompressor
## How?? - Destination - Destination (loudspeaker) - Offline Destination (pre-render) - MediaStreamAudioDestination

Upcoming Changes


ScriptProcessor → AudioWorker


ScriptProcessor inside a WebWorker

How?? - Parameters


→ Automation (synchronous automated value change)

							
 playRate.setValueAtTime(2, 5);
 frequency.linearRampToValueAtTime(400, 2);
 delayTime.exponentialRampToValueAtTime(3.1, 10);
 pan.setTargetAtTime(-1, 7.2);
 gain.setValueCurveAtTime([0, 1, 0.7, 0.7, 0.5, 0.2, 0], 0, 0.5)
							
						

How?? - Parameters

→ Connect Nodes to Parameters

							
	var osc = context.createOscillator();
	var gain = context.createGain();

	// Connect oscillator to gain
	osc.connect(gain.gain);

	// Connect buffer to gain and gain to output
	buffer.connect(gain);
	gain.connect(context.destination);
							
						

AM : Message : Carrier :
FM : Message : Carrier :
## How?? - Libraries

Tone.js

— Tone.js

SoundModels

— SoundModels

BabylonJS

— BabylonJS

## How?? - Tools

Firefox WebAudio Inspector

RecorderJS

— Recorderjs

Braid

— Braid

Demos

## Questions? _http://chinpen.net/talks_ @ntt
## Bonus Resources - Web Audio Book : http://chimera.labs.oreilly.com/books/1234000001552 - HTML5 Audio : http://www.html5audio.org - Web Audio in Nodejs?? https://github.com/sebpiq/node-web-audio-api - Open Music http://openmusic.github.io/ - Web Audio Weekly Mailing List http://blog.chrislowis.co.uk/waw.html