Simple Path

The first Gestling!

1. Top Level Map

Below is the top level "map" of the program. Chunks of code have been abstracted out as named code blocks, denoted by "<<" and ">>".

This should provide a gist of what is happening in the program.

<<procedures>>

patch =

<<sndkit_patch>>

program_words = {}


<<generate_macros>>

-- I forget what this is called
table.insert(program_words, "|0100")

mklabel(program_words, "mel")


<<vertex_shorthand_for_gesture_path>>

path =

<<gesture_path_as_lua_structure>>

mkpath(program_words, path)
mkjump(program_words, "mel")

program_tal = table.concat(program_words, " ")
compile_tal(program_tal)
lil(patch)

2. Overview

Now that it's possible to build GestVM programs inside of Lua, I am now focusing on the next immediate need, which is to build some abstractions that will help automate the building of these programs.

3. TAL as a lua table of tokens, or "words"

At the lowest level, Lua evaluates an Uxn program as a string. A sensible abstraction on top of this would be a lua table of Uxn words represented as strings.

4. Building Tables Via Procedures

A lua table could then be populated using a set of procedures.

mkmacro creates TAL macro. Since we aren't writing TAL code directly, this isn't strictly needed. But this original code was based off of handwritten TAL, so they stay.

mkmacro(program_words, "NUM", {"#24", "DEO"})
mkmacro(program_words, "DEN", {"#25", "DEO"})
mkmacro(program_words, "NEXT", {"#26", "DEO"})
mkmacro(program_words, "NOTE", {"#33", "ADD", "NEXT"})
mkmacro(program_words, "BHVR", {"#27", "DEO"})

mklabel creates a TAL label, used for jumps and the like.

mknum turns a lua value into an Uxn value that gets pused onto the stack. '4' would be '#04', 12 would be '#0c', 16 would be '#10', etc.

mknote adds a "note", which is really a matter of setting the next value. A break is also added, which is a subtle yet important part of the program logic that makes it work with GestVM.

mkdur sets the duration (rate?) scaling factor. This is represented as a fraction num/den. a value of 1/2 would give it a time half the rate of the conductor signal (a half note?). A value of 2/1 makes it twice the rate of a conudctor (an eigth note?).

mkbehavior adds an instruction that changes the so-called "behavior" of the Gesture. In other words, when we travel from point A to B, how do we ge there? Linear, exponetial, and step are some of the options.

function mkmacro(words, name, program)
    table.insert(words, "%" .. name)
    table.insert(words, "{")
    for _,v in pairs(program)
    do
        table.insert(words, v)
    end
    table.insert(words, "}")
end

function mklabel(words, label)
    table.insert(words, "@" .. label)
end

function mknum(words, val)
    table.insert(words, "#" .. string.format("%02x", val))
end

function mknote(words, note)
    table.insert(words, "#" .. string.format("%02x", note))
    table.insert(words, "NOTE")
    table.insert(words, "BRK")
end

function mkdur(words, num, den)
    mknum(words, num)
    table.insert(words, "NUM")
    mknum(words, den)
    table.insert(words, "DEN")
end

function mkbehavior(words, id)
    mknum(words, id)
    table.insert(words, "BHVR")
end

function mkjump(words, label)
    table.insert(words, ";" .. label)
    table.insert(words, "JMP2")
end

function compile_tal(tal)
    lil([[
    gmemnew mem
    gestvmnew gvm
    ]])

    gestvm_compile("mem", program_tal)
    lil("gmemcpy [grab mem] [grab gvm]")
end

5. Construct "path" structure using Lua syntax

Finally, the sequence of procedures could be abstracted further into a construct for Gesture known as a "path", similar in concept to a path discussed in the context of graph theory in computer science.

The data constructed by path can be parsed and converetd into TAL code via lua procedures using the mkpathprocedure.

function mkpath(words, path)
    for _, v in pairs(path)
    do
        if v.note ~= nil then
            mknote(program_words, v.note)
        end

        if v.dur ~= nil then
            mkdur(program_words, v.dur[1], v.dur[2])
        end

        if v.bhvr ~= nil then
            mkbehavior(program_words, v.bhvr)
        end
    end
end

At this point, there is now a reasonably concise way to describe a simple path using Lua syntax.

{
    v(7, {2,1}, 2),
    v(5),
    v(7),
    v(0, {2,5}),

    v(7, {2,1}, 2),
    v(10),
    v(9),
    v(5, {2,3}),

    v(3, {1, 1}, 3)
}

For brevity, a functioned called v is used as a shorthand to produce vertices in the gesture path. A vertex contains the note, the duration of that note (represented as rate scaling amount, whose fractional value is represented a tuple), and the behavior.

v = function (note, dur, behavior)
    x = {}

    x.note = note
    x.dur = dur
    x.bhvr = behavior

    return x
end

To my eyes, this is a notation a notation that is a fairly good compromise between readability and ease of input (it's text and there's not a lot of type).

6. Sndkit Patch

The generated gesture program can be used inside of a sndkit patch to control the melody of a sound.

Here is the sndkit patch, stored as a lua string.

[[
phasor [expr 96 / 60] 0

hold zz
regset zz 0

gestvmnode [grab gvm] [gmemsym [grab mem] mel] [regget 0]

mtof zz
blsaw zz
mul zz 0.5

butlp zz 800

dup
dup
verbity zz zz 0.1 0.1 0.1
drop
mul zz [dblin -15]
dcblocker zz
add zz zz

unhold [regget 0]

wavouts zz simple_path.wav

computes 10
]]

7. Notation

Here is the path represented using Gesture Notation.

8. Future

Try to build more abstractions for notation.

Explore non-linear gesture paths, and coordinated gesture path networks.

Make more interesting sounds that lend themselves more to being anthropomorphic. Things that talk and breathe. Vocal synthesis and things that somewhat resemble the voice in their own way.

Work on developing an Organynth sound: sounds that are unquestionably synthetic in origin, but also contain organic qualaties and behavior. This is a problem that is a mix of finding the right DSP (I believe physical models for things like the human voice have potential), and developing the right way to control these algorithms (this is where Gesture comes in).