Some Thoughts on Gestured Sound

Introduction

On May 21st 14:45, I recorded myself rambling about some of the ideas I was developing related to Gesture, after writing down an outline about the topic. The hope was at some point was to turn this into a blog post. This might yet happen down the line.

The writings below have been transcribed from this recording, several months later. Some editing was done in this process, including using zetdoc to chunk things out and tag them with labels, so each thought can be individually referenced and used in other places in the future.

Transcribed And Curated Thoughts

Three sections: Development of idea. Implementation of idea. Extensions of the implementation. [guufkuajo]

Gesture refers to one or more control signals modulating the parameters of a sound object. [gwpwpssrs]

Lines shaping the sound of a sound object over time. [gdkferrkq]

simple mechanism is the basis for gesture. [gihrwuafi]

Realizations: when gesture is thought of as a signal or topology of signal flow, fun things happen. Events that feed back into conductor and influence it (temporal weight). Phasor (rephasor) combined together to skew it and that creates localized temporal skewing. Combinations of GSGs; mix/match. Articulated signal generator: envelope generator with GSGs for mare articulations to happen in signals. [gkajaohfd]

Don't need a GSG to make a gesture. Sample and hold can be used with phasor to make gesture. [giqqkhhsi]

Gesture is an overloaded term. means a lot of things to a lot of different people. I mean something very specifically. [ghwehsfar]

Imagine a performed musical idea as a set of gestures that shape a particular sound. [gfiliikhl]

"traditional" musician performing some kind of instrument: they move their limbs muscles to intentionally produce the kind of sound they have in mind. Physical gesture. physical sound. [gisreqliq]

Next phase: electronic instruments that be human performed. physical gesture performing synthesized sound. Roughly the same level of dexterity. Not using of laws of physical acoustics to make sound, so things can be more ergonomic. More potential of sounds. It's a little more abstract. [gqakqeqsa]

Synthesized sound is way simpler in sonic complexity compared to acoustically produced sound. [gdjlodsdr]

Do we need to build more complex sounds to make better music? No! Consider Clara Rockmore and the Theremin. The Swan. It's a masterful virtuosic performance. It's mostly just frequency control. Amplitude is more slow-moving expression. [gpqidwaup]

Theremin is a very high resolution controller. Sensitive to any movement. Exploited by Clara. Basically just controlling the frequency of humble oscillator. Not a Stradivarius. But the interface is compelling. Allows her to use natural dexterity and physical musicality. [ghdedsljf]

Inspiring notion: A compelling performance from well controlled frequency control. You don't need to invent complex sounds, just create really meaningful control mechanisms Overall, simplifies things. [gaqwalrdk]

What if the computer could be the performer? Thus... Synthesized Gesture performing Synthesized sound. [gripfleke]

To some meaningful degree, capture the nuance of what is performed as some kind of procedurally generated continuous signal. Map it to something such as the frequency of oscillator. [goheuwded]

Gesture, in this context, is a specialized control signal generator. [geedpluss]

Composing with control signals and curves: this is thinking with Gesture. [gjkilafqa]

Gesture addresses the question: what are interesting ways to structure music in a computer? [geurjukwp]

Structures such as MIDI model music as a series of discrete events. The stuff underneath those events are hardly considered. [guwqwswap]

Gesture begins to address the stuff underneath the "notes": the Performance Layer. [grjkqslpf]

Notes, though displayed as discrete units, will continuously flow from one to the other in performance. For some instruments this is very clear, like string instruments and singing. [gujheasqk]

Consideration of monophonic instruments on their own terms. Lyricism. Computer generated lyrical performance. [giikjkpdr]

Define mathematical structures that crudely imitate the lyricism that human performers do naturally. [gdeflrdhj]

Gesture in a computer music context is a set of synchronized coordinated control signals that modulate a sound object. [gshkjuqik]

Notes don't really exist when there isn't a human performer. Notes are notes. They are a shorthand interpretted by humans, and turned into physical that manifest into perceptual events of notes. These notes, now sound waves propagating through the air, are a little bit different than the notes on the page. We can call the semantic differences "interpretation". [gspepqihs]

Performance layer: representation of notes is different than how they are performed. This already happens in electronic music, but here it is more formally considered. [guqeipldl]

Transcribing A Solo: different ways to write it down, depending on how you hearing it. Can be equally valid. [gukaqjple]

Gestalts: perceptual movements and audio cues. These work together to create events our brain will lump together as a note. [gjrakhplh]

How to implement Gestures to produce Gestalts? Breakpoint line generators and automation curves work well! We have 'em already. But... how to use them outside of DAW? [gasfooijk]

First attempt: libline. problem: drift. Roundoff and truncation created loss of precision which causes drift. Lines are in completely different place. [gldwssara]

Prop: similar problem. Marched to beat of it's own drum (clock source). Eventually drifted out of sync with other elements around it. [gauopisfo]

Solution: you need a centralized clock source. How to make this work for a line generator though? Phasors! Continuous signals that can embed timing information. You can now at every sample where you are in a period. Very easy to detect when a new period begins. [grhadswlh]

It is pretty trivial to "resynthesize" a new phasor based on the input of another phasor. Wait a sample, get delta, optional scale. Scaling value makes phasors at speeds relative to input phasor, like a clock subdivider. [grafrffsi]

Could use phasor as interpolation coefficient for interpolation in line generator. [gedepouqj]

Problem: rephasor still has drift. [gfdplqhqs]

First attempt to fix rephasor drift: Group things into phrases and then reset the clock at the end. Phrases would be specified in whole integer beats. Structured things in hierarchical "ramp tree". [gkdfwelsh]

Second attempt: correction is built in as a feedback loop. Look at where it is vs where it is supposed to be. Method requires two samples of delay, but it's a much more generalized rephasor now. No more more phrases, which were clunky. Lots more flexibility for control. [gfjdfoohq]

Rephasor with automatic correction accidentally re-invented phase-locked loop. [gsiiupkeh]

Gesture Sequencing: the means of programming a Gesture Signal Generator. [gdiridpal]

GestVM: use virtual machine to do sequencing. [gwfjsqphh]

Why a VM? Generative Music Structures. Wanted a way to build gestures with shared information and communication. GestVM is re-entrant. Concurrent gestures. [gdwrouhaa]

Realizations from Gesture implementation: blurred line between notes (as seen in music theory) and signal. Leads to some interesting outcomes. Blending of Signal Theory and Music Theory. [gfhlqjsuj]

Gesture Signal Generator: Input Signal of conductor, output is gesture. What if you fed back into conductor? State of sequence can influence the tempo of the conductor, causing the global tempo change. [gqlrusrje]

Tempo automation: traditionally there's a global tempo track. Drop notes in, and things slow down depending on where they are. In new setup, the notes themselves are given weight. Weight "bends" the main tempo. Allows one to sequence passages that slow down and speed up automatically. Many notes with different weights can be played at once and things will sort themselves out. Good for isorhythmic composition techniques. [glisqrkuh]

Temporal Skewing: another way of warping time. two rephasors working together. One rephasor creates a single ramp, that is proptional to N ramps. A "monoramp". Monoramp is then put through a skewing function that adds exponential curve. Put that through another rephasor, that undoes the slowing operation of the previous rephasor. The rephasor logic doesn't care about slope of ramp. It only needs to be positive. Result of all this: time warping that is localized and relative to the bounds of the slower ramp. Not time will be destroyed or created. [gsfusshad]

Articulated Gesture Signal Generator: Gesture plus an envelope generator. It's a way to add articulation. Percussive, transients, etc. Made up of 3 gesture signal generators (gate, signal a, signal b), and shaping parameters for the envelope (attack/release) that can also be GSGs. [gaqluarik]

Musical Notes are more than just numerical sequences. [gkfdhdjdu]

Gestures makes lyrical performance more of a possibility. [glkldipqo]

Realism not a goal: more about transferring headspace when I play music on my bass to work with a computer. [gkiodwhqu]

Anthropomorphic: sounds with personality. gesture could be a way of doing that. That's where I'm headed. Sounds that almost feel like creatures. [ghlulfuio]