Gesture Morphemes

1. Overview

A morpheme, in the context of gesture, is a construct that's used to construct Coordinated Gesture Paths (CGP). CGP are gesture paths that run parallel to one another, and are tightly related somehow, to the point where modifying one path could require modifying all the other paths. Using morphemes, CGPs can be constructed simultaneously, one slice at a time.

The term "morpheme" is a term borrowed from linguistics used to describe an atomic unit in language. It's pretty close to what is being done in this context, and of all the common "phemes" used in linguistics (phoneme, grapheme, morpheme), the best fit for Gesture.

2. Structure

A morpheme is made up of a set of gesture paths, where every path has an identical duration. Keeping the duration the same is important to the design of the morpheme, as it is what allows morphemes to be put together in a sequence without having to worry about the underlying gestures going out of sync.

3. Proportional Duration

To ensure that paths encapsulated in a morpheme always have the same duration, a proportional duration system is used. Rather than use the rate multiplier fractional value represented as two integer values, a proportional duration is represented as a single positive non-zero integer. This unit indicates how long a particular segment is relative to the path. So, if a gesture path had a segment with value of 1, a segment with a value of 3 would be 3 times longer.

Proportional durations are only relative to the path they belong to. Different paths can have different proportions, allowing for interesting polyrhythms to occur.

4. Timescaling

By default, a morpheme takes up one beat, however its duration can be stretched and squashed to fit inside beats bars, or units smaller than beats.

5. Tangled Code

local M = {}

<<morpheme>>
return M

6. Implementation

The Morpheme structure is implemented in lua code, using lua tables as it's structure.

6.1. Description of Morpheme Function

The morpheme(m, r) function is used to convert morpheme structures into gesture paths.

The morpheme structure m should be a table key-value pairs, where the key is the name of the gesture, and the value is a gesture path using proportional notation.

The r parameter is the time rescaling factor, expressed as a fractional value using a lua table {numerator, denominator}. A value of {2, 1} compresses the entire morpheme to be half of a beat (2x speed). A value of {1,2} will stretch the morpheme to take up 2 beats (0.5x speed).

As an example:

morpheme({
    a={{60, 3, 3}, {67, 1, 3}, {58, 2, 3}},
    b={{63, 1, 3}, {65, 1, 3}, {63, 1, 3}, {62, 1, 3}}
}, {1, 3})

Should yield:

{
    a={
        {60, {2, 3}, 3},
        {67, {2, 1}}, 3},
        {58, {1, 1}, 3}
    },
    b={
        {63, {4,3}, 3},
        {65, {4,3}, 3},
        {63, {4,3}, 3},
        {62, {4,3}, 3}
    }
}

6.2. Converting Proportional Durations to Rate Multipliers

To convert proportional durations to actual durations (expressed,as rate multipliers in Gesture), find the least common multiple for all the path lengths, rescale the paths so their lengths match, then convert to rate multiplier values using the LCM and the morpheme scaling factor. Note that duration is a unit describes the overall time, while length describes the sum of all the ratio units for a path when duration is expressed in proportional terms.

6.3. Lua Code for Morpheme Function

<<lcm>>
function M.morpheme(m, r)
    -- Get lengths of each gesture path
    local lengths = {}

    for k, v in pairs(m) do
        lengths[k] = 0
        for _,x in pairs(v) do
            lengths[k] = lengths[k] + x[2]
        end
    end

    -- Find LCM of path lengths

    length_lcm = 0

    for _, v in pairs(lengths) do
        if length_lcm ~= 0 then
            length_lcm = lcm(length_lcm, v)
        else
            length_lcm = v
        end

    end

    out = {}

    -- rescale paths

    for k, v in pairs(m) do
        local s = length_lcm / lengths[k]
        local row = {}
        for i=1,#v do
            row[i] = {v[i][1], v[i][2]*s, v[i][3]}
        end
        out[k] = row
    end

    -- convert to rate multiplier

    for k,v in pairs(out) do
        -- out[k][2] = {length_lcm, out[k][2]}
        for i=1,#v do
            -- apply scaling value r and obtain multipler

            local num = length_lcm * r[1]
            local den = v[i][2] * r[2]

            -- simplify multiplier fractions, if possible

            local div = gcd(num, den)
            if div == 0 then div = 1 end

            num = num / div
            den = den / div

            -- make sure multiplier values are in range

            if (num > 255 or den > 255) then
                error(string.format(
                    "%s[%d]: multiplier (%d, %d) out of range",
                    k, i, num, den))
            end
            v[i][2] = {num, den}
        end
    end


    return out
end

<<helpers>>

6.4. LCM and GCD algorithms

Least common multiple algorithm, taken from Rosetta Code.

function gcd(m, n)
    while n ~= 0 do
        local q = m
        m = n
        n = q % n
    end
    return m
end

function lcm(m, n)
    return (m ~= 0 and n ~= 0) and
        m * n / gcd(m, n) or 0
end

7. Morpheme Helpers

7.1. Append

The append(path, mp, r, m) function converts a morpheme to a set of gesture path structures and appends that structure to a table. path is the object containing the loaded path library. mp is the table of morphemes (converted to paths), which can be initialized to be an empty table.

The r value is the rate scaling factor, and m is the morpheme structure itself.

The idea with this utility is that a structure composed of morphemes is built up using this append operation.

The appender function returns a function that abstracts away the path variable, which should make for more readable code.

The core of the append function is the append operation itself.

function append_op(path, m, mp)
    for pname, p in pairs(m) do
        if mp[pname] == nil then
            mp[pname] = {}
        end
        for k, v in pairs(p) do
            table.insert(mp[pname], path.vertex(v))
        end
    end
end

function M.append(path, mp, r, m)
    append_op(path, M.morpheme(m, r), mp)
end

function M.appender(path)
    return function(mp, r, m)
        M.append(path, mp, r, m)
    end
end

7.2. Compile

The compile function will compile a set of paths into a table of Uxntal words. tal and path are lua objects from the tal and path libraries, respectively. words should be a table to put the Uxntal words (before they are compiled into Uxn bytecode). mp contains the table of gesture paths, presumably populated by the append operation defined previously.

function M.compile(tal, path, words, mp, head, lookup)
    head = head or {}
    for label, p in pairs(mp) do
        tal.label(words, label)
        if head[label] ~= nil then
            head[label](words)
        end
        lookup = lookup or nil
        path.path(tal, words, p, lookup)
        tal.jump(words, label)
    end
end

function M.compile_noloop(tal, path, words, mp, head, lookup, multilut)
    head = head or {}
    multilut = multilut or {}
    for label, p in pairs(mp) do
        tal.label(words, label)
        if head[label] ~= nil then
            head[label](words)
        end
        local lut = multilut[label] or lookup
        path.path(tal, words, p, lut)
        tal.jump(words, "hold")
    end
end

7.3. Articulate

The articulate function takes in a sequence of morphemes and duration scaling factors, and then returns a table of words that could be compiled using the compile_wordsfound in the Tal Lua module.

function M.articulate(path, tal, words, seq, head)
    local mp = {}

    for _,s in pairs(seq) do
        M.append(path, mp, s[2], s[1])
    end


    M.compile(tal, path, words, mp, head)
end

8. Working with Larger Morphemes

8.1. Attributes and Sets

In practice, it is typical for morphemes to become large and many. The operations and constructs below aim to make it easier to work with these heftier morphemes.

A morpheme, as stated previously, is a collection of named paths that run parallel to one another. These named paths can be thought of as attributes to a sound, and a morpheme can be thought of as a set of attributes. Morphemes that are tied together in a sequence (using something like mseq, for example), must all contain the identical combination of attributes. In other words, morphemes must all share an identical set of attributes.

8.2. Templates

The template function produces an abstraction that makes it easy to create variations from an existing morpheme:

x = {
    "a": path1,
    "b": path2,
    "c": path3,
}

xt = template(x)

y = x {
    "b": path4
}

This syntactic sugar works by taking advantage of a Lua quirk. In some situations, parantheses are optional for functions. If a function takes only a table as an argument, the curly braces of that table can be used instead of parentheses.

function M.template(m)
    return function(p)
        local o = {}
        for k,v in pairs(m) do
            o[k] = p[k] or v
        end
        return o
    end
end

8.3. Subsets and Partial Morphemes

A handful of attributes from a morpheme, known as a subset, can be extracted into a smaller morpheme, known as a partial morpheme. Partial morphemes can be turned into templates an managed as smaller components, then can be combined with other partial morphemes to quickly build full morphemes.

The subset function takes in a morpheme and a table of attributes, and returns a partial morpheme with those attributes. This will be a strict operation: attributes that don't exist in the morpheme will be treated as errors.

function M.subset(m, a)
    local o = {}

    for _,v in pairs(a) do
        if m[v] == nil then
            error("Attribute '" .. v .. "' doesn't exist.")
        end
        o[a] = m[a]
    end

    return o
end

8.4. Merge

A merge operation is a kind of join operation for morphemes A and B that combines their attributes into one morpheme. If there are any attributes that intersect between A and B, the attributes in B will be chosen.

function M.merge(A, B)
    local o = {}

    for k,v in pairs(A) do
        o[k] = v
    end

    for k,v in pairs(B) do
        o[k] = v
    end

    return o
end

9. Loading/Saving Morphemes

Similar to how it's done in path, using the asset component. Make sure that the asset component is initialized.

Note that the load function will require the path component, in order to properly convert the data structure to a path that lua can read.

function M.morpheme_to_data(path, morpheme)
    local morpheme_data = {}

    for k,v in pairs(morpheme) do
        morpheme_data[k] = path.path_to_data(v)
    end

    return morpheme_data
end

function M.save(asset, path, morpheme, filename)
    asset:save(M.morpheme_to_data(path, morpheme), filename)
end

function M.load(asset, path, filename)
    local morpheme_data = asset:load(filename)

    local morpheme = {}

    for k, v in pairs(morpheme_data) do
        morpheme[k] = path.data_to_path(v)
    end
    -- local gpath = {}
    -- for _,v in pairs(path_data) do
    --     table.insert(gpath, Path.vertex(v))
    -- end

    return morpheme
end