Gesture

Messages tagged with the label gesture.

[6a6c494d] 2023-01-29-12-51: ink: review page 2022p1weight.jpg

[2ed6a8c8] 2022-11-26-06-29: Gesture Paths: a construct that can be used to represent Gesture.

[a283189f] 2022-11-26-06-29: The goal of the Gesture Path is to have a high level construct that can be converted to Uxn code in GestVM.

[37175300] 2022-11-26-06-29: A Gesture Path is a "Path" in the CS sense. Start value, end value, and a sequence of scalars connected together by things called trajectories. A, B, C.

[8fd7c0cd] 2022-11-26-06-29: Trajectories are kind of like duration, but they are more accurately speeds in the context of GestVM (specifically, rate multipliers for the input conductor).

[4b80474f] 2022-11-26-06-29: A Gesture Path is very similar to how a breakpoint line generator would be described.

[bdf81b8e] 2022-11-26-06-29: Special START and END markers that exist outside of the path: START can be the end of a previous path or empty; END path can be the start of the next path or empty.

[ab26ab55] 2022-11-26-06-29: An empty START value inidicates the beginning of the gesture, which means the gesture needs to be initialized to the value at the start of the path. An empty END value means the end of Gesture, so hold onto the last value of the path.

[ec2ca7d2] 2022-11-26-06-29: Non-linear gesture path: path that has a branch point. Which branch it chooses it determined by some kind of external state (user input, random number generator).

[619be10f] 2022-11-26-06-29: Thinking about anthrosound: external states could be like pheromones. Sound object critters could "smell" the state and instinctively know where to go.

[c4a96b2d] 2022-11-26-06-29: Non linear paths could be built up of smaller linear paths.

[40c2e7fe] 2022-11-26-06-29: They will hopefully turn out to be a useful construct for building up complex Gesture systems.

[65272cbc] 2022-11-25-15-31: Current headspace: building a notation system for Gesture. A way of representing what are essentially breakpoint lines in a pretty way.

[2bcb4e9f] 2022-11-25-15-31: These are not sounds, these are lines that could ideally control sounds. They are related.

[90d8188c] 2022-11-25-15-31: There is a difference between notating for gesture and notating for sound. Ideally, the notation system should be for gestures that lead to beautiful sound. But that isn't always the case.

[2cd6b50f] 2022-11-25-15-31: Look at written language: this a much closer symbolic represntation of sound, since they usually have a correspondence to phonemes. So, symbols for phonemes.

[59fc140e] 2022-11-25-15-31: With Gesture, one can get quite stuck thinking about just the lines, but it does need to get to sound eventually. The grand plan is to use Gesture to build sound objects with personality and character. Having musical counterpoint be the result of these objects socializing with one another.

[dd53921a] 2022-11-25-15-31: It's possible more notation will be needed for the gestlings series, as lines may be too low level.

[1c4cd9f6] 2022-11-24-07-06: Gesture is a system that produces sound structures, inspired by the human voice and lyricism. Perhaps someday, it will push beyond the voice and lyricism and produce novel structures on its own terms.

[2fa2c58a] 2022-11-20-10-27: not a lot of active work done here in a while. All my attention has been placed into a new project called (gestlings), which aim to be studies in Sonic Gesture.

[68c0fc1c] 2022-07-18-13-13: Where this is all headed: construction of a notation system for synthesized gesture.

[e2e3a9ce] 2022-07-18-12-58: Some motions imply a kind of "swooshing" motion with the entire hand. It's a physical gesture inputting in a synthesized gesture!

[8bd27e91] 2022-07-18-10-52: Keyboards and mouse driven systems will ultimately yield the same kinds of ideas. Let's make new systems and amuse ourselves.

[bc7fd4b2] 2022-07-18-10-52: Smaller latency creates a tighter creative feedback loop, which hopefully should create a better flow state.

[9fcf3086] 2022-07-18-10-52: We are limited to our limbs for notating gesture.

[9f8ca7e6] 2022-07-18-10-52: In a way, ergonomics influences the possible notation systems that can be used.

[5af50be3] 2022-07-18-10-52: The way we carry our body and perform actions with our limbs carries certain meaning. It influences how we think.

[b2288656] 2022-07-18-10-52: Weird input systems yield unusual actions with our limbs yield new creative perspectives and headspaces.

[23931930] 2022-07-18-10-52: Ergonomics is about efficiently moving the body to do a particular task. In this case, the task would be producing or notating gesture.

[4f132990] 2022-07-18-10-52: Rework classic definition of Gesture: efficient movement of the limbs to notate a gesture, to be synthesized and performed later.

[7dca32d4] 2022-07-18-10-52: Gesture notation system should be: efficient, natural, fast. This produces a smooth creative feedback loop.

[89b59e25] 2022-07-18-10-52: Movement of the limbs: these are typically our hands and fingers working a peripheral (keyboard, mouse, controller, etc) connected to a computer. System interprets these actions and in turn performs a gesture. Assuming a realtime system.

[f4202093] 2022-07-18-10-52: The amount of time between the movement of the limbs and the synthesized gesture performed by the computer is the latency of the creative feedback loop.

[96abc39f] 2022-07-17-14-17: Synthesized gesture: a technical foundation for things like lyricism, singing, asemic speech, speeched music.

[5c973180] 2022-07-17-14-17: A line of thinking between synthesized gestured and cute singing cartoon characters.

[09e032cc] 2022-07-16-15-16: Concern: what if the natural affordances of the Grid are too different from Gesture? Simply the wrong tool for the job.

[34bd5d68] 2022-07-16-10-58: Input addresses how to physically write gestures for a computer to interpret. Physiologically: how can I get my limbs moving in a way that my computer will understand? Press, turn, etc. Can I do it in an ergonomic, comfortable, intuitive way?

[1f0ef6bb] 2022-07-16-10-58: Display: once a gesture is represented inside of a computer, how can it be displayed in a human readable way?

[7b4e222a] 2022-07-16-10-58: Gesture Signal Generators are line generators. The conventional way to show lines is with plots. Plots have diminishing returns for readability after a certain size, and when there are multiple different lines happening at once.

[34ba568b] 2022-07-16-10-58: Performance: how does the computer take in structured gesture and perform it? These gesture systems and algorithms are in place, and is currently the most clear.

[eca494ab] 2022-07-16-10-58: Considerations for Gesture: Input, Display, Performance.

[ba7283a5] 2022-07-16-10-58: This may be similar to MVC?

[f9b3493d] 2022-07-16-10-58: It's not enough to develop algorithms for Gesture. One also has to construct gestures, and this is done using some kind of input mechanism.

[20a6945e] 2022-07-16-10-58: How I'm currently inputting gesture: my keyboard and a text editor. How I'd like to do it: using things like my Grid and Arc.

[f8f803f1] 2022-07-10-18-52: Birdsong is very gesture oriented. Very hard to notate using traditional western notation.

[02721c78] 2022-07-09-15-24: Another way of framing gesture: Nonverbal action with ones body to express some abstract idea. In a way, this is what musicians do when they are playing their instruments.

[bfbd7720] 2022-07-09-15-24: Physical Gesture Being Applied to Physical Sound: Musicians using gestures produced from their body to manipulate some kind of acoustic musical instrument.

[31998449] 2022-07-09-15-24: When I play my upright bass, my hands work closely with the laws of physics in the universe to get it to produce a meaningful sound. We are at the whims of Mother Nature.

[f4f42055] 2022-07-09-15-24: When instruments are electronic instead of acoustic, the performers are less physically constrained by the laws of physics to produce a meaningful sound (there's still the physics of electricity, which will be overlooked for now). This is synthesized sound.

[cea59827] 2022-07-09-15-24: Physical Gesture Being Applied to Synthesized Sound: musicians using gestures produced from their body to manipulate an interface for an electronic instrument (such as the Theremin).

[ff9c3873] 2022-07-09-15-24: Despite it's limited dimensionality, the Theremin is a very nuanced instrument, whose interface capable of very expressive music. Ability to capture physical gesture quite accurately. Leverages our fine motor skills.

[a151e2fd] 2022-07-09-15-24: Synthesized Gesture Applied To Synthesized Sound: Write computer programs that produce sound, as well as programs that produce gestures that control that sound. Aim is to imitate the continuous gestures produced by a physical human (in a very simplified way).

[9425d3ba] 2022-07-09-15-24: With synthesized gesture, there is still a human interacting with the comptuer to make things go. Let's assume they are still musicians. What are they doing with their limbs instead of using them to directly manipulate a musical instrument?

[c5d8194b] 2022-07-09-15-24: Cannonical definition of Gesture -> physical gesture, physical sound -> physical gesture, synthesized sound -> synthesized gesture, synthesized sound.

[d6e34c5e] 2022-07-09-15-24: Cannonical Gesture: movement of the limbs or action to convey some sort of meaning.

[99190556] 2022-07-07-16-01: Before "Gesture" was a term in my computer music vocabulary, I was exploring gesture-aligned concepts in my software. Things like libline (2015-2016) was motivated by the desire for better control signal generators in Sporth.

[6ac3cd98] 2022-07-07-16-01: Gesture has been an incremental idea over several years.

[622cb7d0] 2022-07-07-16-01: I was writing about a "Gesture Sequencer" a year before I even wrote any code for it. I don't even think I was thinking about things like phasors and rephasors controlling it at the time either.

[b612ca61] 2022-07-07-16-01: Before the Gesture Sequencer was an idea, circa 2019, I was at a choral concert, thinking about how the voices interacted in the acoustic space. Which to me is the guiding principal behind what I think of as Gesture in music.

[6b1bba61] 2022-07-07-16-01: The term "Gesture" I stole from ICLC conference circa 2016 or 2017. In that context, they were using it in the context of thinking about live coding and gesture.

[b2b6a0fc] 2022-07-03-15-37: Gesture Beyond Approximating Physical Gesture.

[51e6a75c] 2022-07-03-15-37: What we are establishing: synthesized gesture being applied to synthesized sound. AKA fancy control signal generators controlling sound objects.

[a7183216] 2022-07-03-15-37: Techniques for synthesized gesture are a kind of structured DSP. Approximating physical gesture and human performances are a starting point. Structures and algorithms can be explored on their own terms.

[ad1d16ad] 2022-07-03-15-37: Coordinated Gestures: structures used to deal with coordinating and synchronizing many gestures at once.

[0a9f28fb] 2022-07-03-15-37: N-order Gestures: Gestures controlling other gestures.

[e67d7eb6] 2022-07-03-15-37: Gestures and Conductor Signals: A kind of N-order signal. When a gesture is programmed to be a conductor signal for another gesture. This can enable composition using conductors with asymmetric ramps. Technically this should work.

[5a7036bf] 2022-07-03-15-37: Gestures and Interactivity: how to make Gestures controllable in realtime?

[ce27de87] 2022-06-30-14-25: brain dump: (thoughts_on_gestured_sound)

[6324252c] 2022-06-30-14-14: Musical Notes are more than just numerical sequences.

[7672df8e] 2022-06-30-14-14: Gestures makes lyrical performance more of a possibility.

[6de2948c] 2022-06-30-14-14: Realism not a goal: more about transferring headspace when I play music on my bass to work with a computer.

[47c73cde] 2022-06-30-14-14: Anthropomorphic: sounds with personality. gesture could be a way of doing that. That's where I'm headed. Sounds that almost feel like creatures.

[29bec400] 2022-06-30-14-14: Why a VM? Generative Music Structures. Wanted a way to build gestures with shared information and communication. GestVM is re-entrant. Concurrent gestures.

[347851c5] 2022-06-30-14-14: Realizations from Gesture implementation: blurred line between notes (as seen in music theory) and signal. Leads to some interesting outcomes. Blending of Signal Theory and Music Theory.

[87bc1b5a] 2022-06-30-14-14: Gesture Signal Generator: Input Signal of conductor, output is gesture. What if you fed back into conductor? State of sequence can influence the tempo of the conductor, causing the global tempo change.

[7d18b6c4] 2022-06-30-14-14: Tempo automation: traditionally there's a global tempo track. Drop notes in, and things slow down depending on where they are. In new setup, the notes themselves are given weight. Weight "bends" the main tempo. Allows one to sequence passages that slow down and speed up automatically. Many notes with different weights can be played at once and things will sort themselves out. Good for isorhythmic composition techniques.

[13c11402] 2022-06-30-14-14: Temporal Skewing: another way of warping time. two rephasors working together. One rephasor creates a single ramp, that is proptional to N ramps. A "monoramp". Monoramp is then put through a skewing function that adds exponential curve. Put that through another rephasor, that undoes the slowing operation of the previous rephasor. The rephasor logic doesn't care about slope of ramp. It only needs to be positive. Result of all this: time warping that is localized and relative to the bounds of the slower ramp. Not time will be destroyed or created.

[087c0bd6] 2022-06-30-14-14: Articulated Gesture Signal Generator: Gesture plus an envelope generator. It's a way to add articulation. Percussive, transients, etc. Made up of 3 gesture signal generators (gate, signal a, signal b), and shaping parameters for the envelope (attack/release) that can also be GSGs.

[b4021974] 2022-06-30-14-14: Solution: you need a centralized clock source. How to make this work for a line generator though? Phasors! Continuous signals that can embed timing information. You can now at every sample where you are in a period. Very easy to detect when a new period begins.

[b03b331d] 2022-06-30-14-14: It is pretty trivial to "resynthesize" a new phasor based on the input of another phasor. Wait a sample, get delta, optional scale. Scaling value makes phasors at speeds relative to input phasor, like a clock subdivider.

[a2afec85] 2022-06-30-14-14: Could use phasor as interpolation coefficient for interpolation in line generator.

[32f78481] 2022-06-30-14-14: Problem: rephasor still has drift.

[6239a714] 2022-06-30-14-14: First attempt to fix rephasor drift: Group things into phrases and then reset the clock at the end. Phrases would be specified in whole integer beats. Structured things in hierarchical "ramp tree".

[3523ee48] 2022-06-30-14-14: Second attempt: correction is built in as a feedback loop. Look at where it is vs where it is supposed to be. Method requires two samples of delay, but it's a much more generalized rephasor now. No more more phrases, which were clunky. Lots more flexibility for control.

[1ddcf6a4] 2022-06-30-14-14: Rephasor with automatic correction accidentally re-invented phase-locked loop.

[2dbd2f07] 2022-06-30-14-14: Gesture Sequencing: the means of programming a Gesture Signal Generator.

[93518f44] 2022-06-30-14-14: GestVM: use virtual machine to do sequencing.

[5b064f74] 2022-06-30-14-14: Gestalts: perceptual movements and audio cues. These work together to create events our brain will lump together as a note.

[013eed56] 2022-06-30-14-14: How to implement Gestures to produce Gestalts? Breakpoint line generators and automation curves work well! We have 'em already. But... how to use them outside of DAW?

[729110b0] 2022-06-30-14-14: First attempt: libline. problem: drift. Roundoff and truncation created loss of precision which causes drift. Lines are in completely different place.

[0cefd13e] 2022-06-30-14-14: Prop: similar problem. Marched to beat of it's own drum (clock source). Eventually drifted out of sync with other elements around it.

[1465c8d6] 2022-06-30-14-14: Gesture in a computer music context is a set of synchronized coordinated control signals that modulate a sound object.

[1faf8d41] 2022-06-30-14-14: Notes don't really exist when there isn't a human performer. Notes are notes. They are a shorthand interpretted by humans, and turned into physical that manifest into perceptual events of notes. These notes, now sound waves propagating through the air, are a little bit different than the notes on the page. We can call the semantic differences "interpretation".

[c8adf727] 2022-06-30-14-14: Performance layer: representation of notes is different than how they are performed. This already happens in electronic music, but here it is more formally considered.

[c6085f7a] 2022-06-30-14-14: Transcribing A Solo: different ways to write it down, depending on how you hearing it. Can be equally valid.

[2a37b245] 2022-06-30-14-14: Define mathematical structures that crudely imitate the lyricism that human performers do naturally.

[56d70380] 2022-06-30-14-14: Composing with control signals and curves: this is thinking with Gesture.

[acb5c69f] 2022-06-30-14-14: Gesture addresses the question: what are interesting ways to structure music in a computer?

[c989190f] 2022-06-30-14-14: Structures such as MIDI model music as a series of discrete events. The stuff underneath those events are hardly considered.

[b56817f3] 2022-06-30-14-14: Gesture begins to address the stuff underneath the "notes": the Performance Layer.

[c54a0186] 2022-06-30-14-14: Notes, though displayed as discrete units, will continuously flow from one to the other in performance. For some instruments this is very clear, like string instruments and singing.

[dd656f2b] 2022-06-30-14-14: Consideration of monophonic instruments on their own terms. Lyricism. Computer generated lyrical performance.

[42a21753] 2022-06-30-14-14: Theremin is a very high resolution controller. Sensitive to any movement. Exploited by Clara. Basically just controlling the frequency of humble oscillator. Not a Stradivarius. But the interface is compelling. Allows her to use natural dexterity and physical musicality.

[08907b26] 2022-06-30-14-14: Inspiring notion: A compelling performance from well controlled frequency control. You don't need to invent complex sounds, just create really meaningful control mechanisms Overall, simplifies things.

[bdf37a6a] 2022-06-30-14-14: What if the computer could be the performer? Thus... Synthesized Gesture performing Synthesized sound.

[e4ac92a2] 2022-06-30-14-14: To some meaningful degree, capture the nuance of what is performed as some kind of procedurally generated continuous signal. Map it to something such as the frequency of oscillator.

[aa2f7c11] 2022-06-30-14-14: Gesture, in this context, is a specialized control signal generator.

[49a4130b] 2022-06-30-14-14: Gesture is an overloaded term. means a lot of things to a lot of different people. I mean something very specifically.

[3d7dd647] 2022-06-30-14-14: Imagine a performed musical idea as a set of gestures that shape a particular sound.

[d1ba87d8] 2022-06-30-14-14: "traditional" musician performing some kind of instrument: they move their limbs muscles to intentionally produce the kind of sound they have in mind. Physical gesture. physical sound.

[8068a810] 2022-06-30-14-14: Next phase: electronic instruments that be human performed. physical gesture performing synthesized sound. Roughly the same level of dexterity. Not using of laws of physical acoustics to make sound, so things can be more ergonomic. More potential of sounds. It's a little more abstract.

[257e212b] 2022-06-30-14-14: Synthesized sound is way simpler in sonic complexity compared to acoustically produced sound.

[f8d290cf] 2022-06-30-14-14: Do we need to build more complex sounds to make better music? No! Consider Clara Rockmore and the Theremin. The Swan. It's a masterful virtuosic performance. It's mostly just frequency control. Amplitude is more slow-moving expression.

[cc36c05e] 2022-06-30-14-14: Three sections: Development of idea. Implementation of idea. Extensions of the implementation.

[9f9f11b1] 2022-06-30-14-14: Gesture refers to one or more control signals modulating the parameters of a sound object.

[263abb68] 2022-06-30-14-14: Lines shaping the sound of a sound object over time.

[d4b9c03d] 2022-06-30-14-14: simple mechanism is the basis for gesture.

[6050e432] 2022-06-30-14-14: Realizations: when gesture is thought of as a signal or topology of signal flow, fun things happen. Events that feed back into conductor and influence it (temporal weight). Phasor (rephasor) combined together to skew it and that creates localized temporal skewing. Combinations of GSGs; mix/match. Articulated signal generator: envelope generator with GSGs for mare articulations to happen in signals.

[d886441d] 2022-06-30-14-14: Don't need a GSG to make a gesture. Sample and hold can be used with phasor to make gesture.

[e7e35083] 2022-06-03-20-32: "Diagrams, gestures and formulae in music" by G. Mazolla and M. Andreatta. This was mentioned to me while I was rambling about Gesture Mastodon. I don't understand the math, or really what this paper is about, but it looks intetesting. https://isidore.science/document/10670/1.rmwpgg