There are no theoretical limitations to the performance of the computer as a source of musical sounds, in contrast to the performance of ordinary instruments.

— Max Mathews
The Digital Computer as a Musical Instrument (1963)

Max Vernon Mathews (1926–2011)

American pioneer of computer music at Bell Laboratories:

• Developed MUSIC I (1957), first widely-used program for computer sound generation

• Created GROOVE (1970), first real-time interactive music synthesis system

• Created the musical accompaniment for Daisy Bell (1961)

→ Max/MSP software named in his honor

Daisy Bell (1961)

The first computer-synthesized song, created on an IBM 704 at Bell Labs:

• Demonstrated music synthesis potential
• Milestone in computer music history

• Inspired HAL 9000 singing in 2001: A Space Odyssey (1968)

→ The MUSIC-N family influenced later computer-music environments; Max/MSP was named in Mathews’s honor.

Computer music

Computer as universal instrument without physical limitations

1. Composition
Algorithmic generation of musical structures and notation

2. Sound synthesis
Creating sounds from mathematical descriptions

3. Sound control
Real-time manipulation through performance interfaces

The computer as instrument

Discrete numbers represent sound pressure samples, enabling digital representation and processing.

John Chowning (*1934)

Chowning's research at Stanford AI Lab

Acquired MUSIC IV from Max Mathews, studied:

• Computer programming and hardware
• Psychoacoustics (spatial perception, localization)
• Acoustics (physics of sound and reverberation)
• Mathematics (Doppler shift for movement simulation)

→ Fascination with composing for loudspeakers as spatial instruments

Compositional approach

• Spatial compositions with sounds possessing "internal dynamism"
• Liberation of musical sound from physical constraints through computer synthesis

Constraint: Non-real-time processing required 10+ minutes to compute a few seconds of audio

Spatial music through quadraphonic synthesis

System: 4-channel DAC creating illusory acoustic environments

Psychoacoustic parameters controlled:

• Movement: Doppler shift simulates sound trajectory
• Distance: Direct-to-reverberant signal ratio
• Azimuth/Position: Energy distribution across speakers

Discovery of FM synthesis

Accident through experimentation (1967):

• Jean-Claude Risset mentioned additive synthesis from his projects with trumpet sounds
• Chowning experimented with simple two-oscillator system
• Discovered FM's ability to create complex timbres efficiently

First commercially successful digital synth

• Simple algorithm (two oscillators) generates rich, controllable spectra
• Analysis-by-synthesis approach to trumpet timbre
• Both harmonic and inharmonic spectra possible

→ Patent licensing of the technology by Stanford University to Yamaha (~20 million dollars)

Founding of CCRMA (1975)

Center for Computer Research in Music and Acoustics

• First university center dedicated to computer music
• Home to innovations in spatialization, FM, physical modeling
• Continues as leading research facility today

Important audio concepts for FM

• Timbre and Spectrum

  • Distribution of frequency components

• Partials and harmonics

  • Building blocks of complex sounds

• Fourier transform

  • Analyzing sounds into sine wave components

Timbre and spectrum

Timbre (perceptual):
The sonic quality that distinguishes instruments playing the same pitch

Spectrum (technical):
The distribution of frequency components and their amplitudes

Listen to the same pitch (C4 ≈ 261.6 Hz):

Trumpet: ▶ Play     Clarinet: ▶ Play

→ Same pitch, different timbre

Comparing clarinet and trumpet at C4

Understanding spectra with a sawtooth wave

• Each peak in the spectrum represents one sine wave (partial or harmonics)
• Harmonic series 260, 520, 780, 1040... Hz

▶ Play sawtooth wave C4 ≈ 260 Hz

Pure tone (sine wave)

A sine wave is a single frequency component, the fundamental building block of all sounds

Complex tones (example: sawtooth)

Musical instrument sounds and basic waveforms (except sine) contain many sine waves (click for graphing calculator)

Fourier transform

Fourier transform converts time domain (waveform) to frequency domain (spectrum):

• Reveals the individual sine wave components of any sound
• Shows which frequencies are how strong

→ Proves that complex sounds are sums of sine waves

Additive synthesis

Concept: Build complex sounds by adding sine waves together

Challenges:

• Requires many oscillators (one per partial)
• Controlling timbre changes over time is complex
• Computationally expensive

→ Additive synthesis: conceptually simple, practically expensive

Subtractive synthesis

Alternative approach: Generate rich harmonic content, then filter out unwanted frequencies

Process:

• Oscillator generates complex waveform (sawtooth, square)
• Filter removes unwanted frequency components
• Envelope controls amplitude over time

→ More efficient than additive synthesis, but offers less spectral control

Efficiency of FM synthesis

• Uses two oscillators (carrier + modulator)
• Generates rich, controllable spectra through frequency modulation
• Simple algorithm creates both harmonic and inharmonic sounds

→ Achieves rich timbral control with minimal parameters

Origins of frequency modulation

Mainly developed by radio broadcasting engineer Edwin Armstrong (1890 - 1954) for transmitting high-fidelity sound over broadcast radio (since the late 1930)

• FM Radio - demodulation

Simple frequency modulation (FM)

In FM synthesis, the instantaneous frequency of a carrier oscillator (C) is varied according to the output of a modulator oscillator (M).

Carrier frequency (C) - Sets the perceived pitch

Modulator frequency (M) - Determines harmonic/inharmonic character

Modulation depth (D) - Controls spectral brightness/richness

Effects of FM synthesis

• vibrato-effect for modulator frequencies at sub-audio level (below 30 Hz)
• rich spectra with increasing modulator amplitude and frequency

FM synthesis parameters

The resulting frequency components are determined by:

• Ratio between Carrier and Modulator
• Modulation depth (D)

→ These two parameters control the entire spectral output.

click for graphing calculator

Detuning the oscillators

• If the carrier is detuned, the entire harmonic spectrum shifts up or down by that same amount.
• Detuning the modulator compresses or expands the separation between the sidebands

FM synthesis formula

Basic formula:

Parameters:

• - Carrier amplitude
• (or ) - Carrier frequency (perceived pitch)
• (or ) - Modulator frequency (sideband spacing)
• - Modulation depth (frequency deviation or amount of modulation)

Derived FM parameters

Harmonicity ratio:

• Determines harmonic (rational) or inharmonic (irrational) spectrum

Modulation index:

• Controls the number of significant frequency components (sidebands)

Sidebands (spectral components)

New frequency components appear in pairs symmetrically around the carrier frequency and define the timbre of the sound:

• is an integer that determines the order of the sidebands
• carrier
• modulator

Calculating the sidebands

Reflected sidebands and interference

Lower sidebands extending below 0 Hz reflect at zero with a 180° phase shift, potentially interfering with positive-frequency components.

• Coincident series (e.g., 1:1, 2:3 ratio): reflected sidebands land on existing positive frequencies

→ regular harmonic spacing with interference

• Non-coincident series (e.g., 3:5 ratio): reflected sidebands fall between positive frequencies

→ irregular spacing, no interference

Coincident vs. non-coincident series

Control over a sound's "brightness"

Modulation index

Number of significant (perceivable) frequency components increases with :

→ Increasing the modulation index creates more sidebands with greater amplitudes, redistributing energy across the spectrum and increasing spectral richness.

Harmonicity ratio and spectral character

The ratio between M and C determines the harmonicity of the resulting spectrum.

Harmonicity ratio: (modulator ÷ carrier)

→ If is rational (), the spectrum contains only harmonic frequencies

→ If is irrational, the spectrum is inharmonic

Listening examples: FM spectra

1. Simple (C=260, H=1.0, I=0.5) ▶ Play
2. Rich harmonic (C=260, H=1.0, I=5.0) ▶ Play
3. Complex rational (C=260, H=1.3, I=6.0) ▶ Play
4. Odd harmonics (C=260, H=2.0, I=3.0) ▶ Play
5. 2:3 ratio (C=260, H=1.5, I=3.0) ▶ Play
6. Inharmonic √2 (C=260, H=1.414, I=3.0) ▶ Play
7. Sub-harmonic (C=650, H=0.25, I=2.1) ▶ Play
8. Irrational π (C=100, H=3.14159, I=4.0) ▶ Play
9. Extreme (C=100, H=1.3, I=9.0) ▶ Play

Calculating sideband amplitudes

Sideband amplitudes are determined by mathematical scaling factors known as Bessel functions of the first kind:

• The amplitude of the -th sideband is calculated as , where is the sideband order and is the modulation index.
• Total average power of the signal remains constant, only the spectral distribution of that energy changes.

→ Bessel functions act as a mathematical "lookup table"

Dynamic timbres

Coupling an envelope to both the carrier amplitude and modulator level creates realistic, brass-like dynamic changes in both loudness and brightness

Multiple modulators

Parallel modulators (M1→C, M2→C):

• Sideband series add together
  C ± k·M₁ and C ± k·M₂

Cascaded modulators (M1→M2→C):

• Each sideband from M1→M2 acts as a sine wave modulator for the carrier
• Result: sidebands of sidebands (exponential spectral growth)

Feedback in FM

Feedback routes an operator's output back to its own input:

• Generates spectral complexity without additional oscillators
• Creates additional sideband frequencies beyond standard FM pairs

Phase modulation (PM)

PM is the derivative of FM. It varies the phase angle rather than frequency, but produces identical sidebands to FM.

Digital FM synthesis uses PM because:

• More stable with feedback loops
• Easier to implement digitally
• Same audible result as true FM

→ All digital FM synthesizers (DX7, etc.) actually use PM

Yamaha DX7 digital synthesizer

▶ Audio example: DX7 electric piano preset

Yamaha DX7 (1983)

The Yamaha DX7 brought Chowning's academic research to consumer market and defined 1980s sound (pop, new wave, film scores)

• First affordable digital synthesizer using FM
• Over 2 million units sold (best-selling synth ever)
• 16-note polyphony
• 61-note keyboard with velocity sensitivity
• 32 algorithms (6 sine wave operators each)

→ Stanford earned ~$20 million from Yamaha patent

Other FM synthesizers

Current hardware and software FM synthesizers:

• Arturia DX7 V
• Elektron Digitone II
• Korg opsix SE
• Korg Volca FM
• Native Instruments FM8

FM synthesis in Max/MSP and Pure Data

Max originated in Miller Puckette’s work at IRCAM; Pure Data followed as a related open-source environment in 1996.

• osc~ (Pure Data) or cycle~ (Max): carrier and modulator
• *~: scales the modulator and sets the frequency deviation
• +~: adds the modulation signal to the carrier frequency

Max tutorial: simpleFM~ ↗

Turenas (1972) (quadraphonic version)

Title: Anagram of "Natures"

• All sounds created through FM synthesis
• No recordings or traditional instruments
• Quadraphonic spatialization with Doppler shifts
• Movement through virtual acoustic space
• Demonstrates FM's ability to create complex, organic timbres

→ Established FM as legitimate compositional tool

FM Synthesis and artistic research

• Discovery emerged from compositional practice (sound spatialization)
• Required teaching himself advanced mathematics, programming, and signal processing
• Six years of systematic research to make it musically controllable
• Continuous artistic application used in his own compositions and widely adopted by other artists
• A major patent licensing success for Stanford with significant impact on the music industry

→ Artistic inquiry and scientific understanding enabled genuine innovation

Determining the fundamental frequency

For rational harmonicity ratios (in lowest terms), the fundamental frequency can be calculated:

Example: , Hz

• Fundamental Hz

→ All sideband frequencies are integer multiples of 26 Hz.

Works cited

• Chowning, John. “Turenas: The Realization of a Dream.” Journées d’Informatique Musicale, Université de Saint-Étienne, 2011.
• Chowning, John. “The Synthesis of Complex Audio Spectra by Means of Frequency Modulation.” Computer Music Journal, vol. 1, no. 2, Apr. 1977, pp. 46–54.
• Chowning, John. “Fifty Years of Computer Music: Ideas of the Past Speak to the Future.” Lecture Notes in Computer Science, pp. 1–10, https://doi.org/10.1007/978-3-540-85035-9_1.
• Mathews, M. V. “The Digital Computer as a Musical Instrument.” Science, vol. 142, no. 3592, Nov. 1963, pp. 553–557, https://doi.org/10.1126/science.142.3592.553.

• Truax, Barry. “Organizational Techniques for C:M Ratios in Frequency Modulation.” Computer Music Journal, vol. 1, no. 4, Nov. 1977, pp. 39–45.
• Roads, Curtis, and Max Mathews. “Interview with Max Mathews.” Computer Music Journal, vol. 4, no. 4, 1980, p. 15, https://doi.org/10.2307/3679463.

Web resources

• Brockhaus, Immanuel. “Yamaha DX7 Piano.” Cult Sounds, https://www.cult-sounds.com/yamahaDX7Piano.html. Accessed 7 Dec. 2023.

• Cycling ’74. “MSP Synthesis Tutorial 5: Frequency Modulation.” Max 8 Documentation, https://docs.cycling74.com/max8/tutorials/06_synthesischapter05. Accessed 8 Dec. 2023.

• “Discovering Digital FM—John Chowning Remembers.” Yamaha Synth, Yamaha Corporation, https://yamahasynth.com/learn/synth-programming/fm101-discovering-digital-fm-john-chowning-remembers/. Accessed 20 Dec. 2025.

• Smith, Julius O. “Frequency Modulation (FM) Synthesis.” Center for Computer Research in Music and Acoustics, Stanford University, https://ccrma.stanford.edu/~jos/sasp/Frequency_Modulation_FM_Synthesis.html. Accessed 20 Dec. 2025.

Third-party materials

These notices cover third-party assets reproduced locally in this slide deck. Concise rights information appears at the point of use; complete attribution, source and licence information is provided here.

Excluded from the course CC BY 4.0 licence

• HAL 9000 eye — original illustration based on 2001: A Space Odyssey (1968); excluded from the course CC BY 4.0 licence.

• The computer as instrument — diagram by Lorenz Schwarz, closely redrawn after Max V. Mathews, “The Digital Computer as a Musical Instrument,” Science, vol. 142, no. 3592, 1963, pp. 553–557, https://doi.org/10.1126/science.142.3592.553

Third-party materials (continued)

Separately licensed material

• John Chowning — photograph provided by John Chowning, CC BY-SA 3.0, via Wikimedia Commons
• Yamaha DX7 composite — photographs by Leo-setä, iixorbiusii, and Georgfotoart; composite by Pittigrilli; Wikimedia Commons, CC BY 4.0

Third-party materials (continued)

Permitted material

• Trumpet and clarinet samples — University of Iowa Electronic Music Studios, Lawrence Fritts; used under the source’s unrestricted-use terms

Public-domain material

• US Patent 3,665,105 (1972), drawing page 2 — inventor John M. Chowning; public domain