How Truly is built

4Recommendation engine

All functions are in docs/app/music.js and are pure: the same inputs and random source give the same path.

4.1 Start point

Each answer ℓ ∈ {1, …, 5} selects a quantile level q_ℓ ∈ (0.08, 0.25, 0.50, 0.75, 0.92):

Placing answers by quantile of the catalogue, rather than by a fixed mapping to the annotation scale, keeps every answer inside the region where recordings exist.

4.2 Optional nudges

The camera result is a tone τ and moves valence only. The voice result (v̂, â) is in standard deviations from everyday speech (7.6) and passes through c(x) = max(−1, min(1, x / 1.5)):

4.3 Goal

The default direction is steady when mood ≥ 4, calmer when mood ≤ 2 and energy ≥ 4, and brighter otherwise. The listener can always override it.

4.4 Planned points

Paths have n = 4, 6 or 9 pieces; "slow down" uses round(1.5 n) and keeps the first piece. The heading h is the unit vector from p₀ to p_g in standardised units. Starting near the listener's state and moving gradually follows the iso principle (Heiderscheit and Madson, 2015; Starcke and von Georgi, 2024).

4.5 Cost of a candidate

For candidate s at planned point p_i, with previous piece r and Δ the standardised step from r to s:

[·] is 1 when the condition holds, and λ_s ∈ [−1, 1] is the clipped difference between full plays and early skips, divided by 3 (section 9). Terms involving r are zero for the first piece. cost() returns every term separately; the chart below plots them for the path above.

Cost terms of the cheapest candidates, per step

• Closeness
• Step
• Backward
• Tempo
• No preferred instrument

4.6 Selection

buildPath(catalogue, points, keep, prefs, rng):
    path ← keep
    for each planned point p_i not covered by keep:
        candidates ← pieces not in path, performer not in path, not ruled out
        K ← the 8 candidates with the lowest C(s)
        pool ← members of K with rank < 4 and C ≤ C_min + 0.25
        choose from pool with weights 1 / (1 + rank)²    # 1, 1/4, 1/9, 1/16
        append the choice to path
    return path sorted by (v_s/σ_V, a_s/σ_A) · h          # forward order

Sampling among near-ties varies paths between sessions without choosing poor matches. The final sort guarantees monotone progress along the heading, even when a later point selected a piece slightly behind an earlier one. replaceSong repeats the choice for one index, first restricted to candidates whose projection on h lies between the neighbours' projections, then without that bound, and sorts again.

4.7 Explanations

For piece i > 0, the standardised differences from piece i − 1 are compared with ±0.15: arousal gives "calmer" or "livelier", valence "brighter" or "more reflective". The first piece "matches the mood and energy you described"; the last piece of a moving path "is where the path settles". A note is added for a favourite, a matching preferred instrument, or λ_s ≥ 2/3.

4.8 Path quality

These follow the smoothness and evenness criteria used for mood playlists (Donnelly and Gaur, 2022). Over all 25 answer pairs, 3 directions, 3 lengths and 5 seeds (1,125 paths) with the current catalogue:

The tests fail if any path exceeds a largest step of 2.2, an average step of 1.2 or a tempo change of 45 bpm.

7Voice pipeline

7.1 Capture

getUserMedia is called with echoCancellation: true, noiseSuppression: false and autoGainControl: false, so the signal keeps its natural dynamics. A ScriptProcessorNode with 4,096-frame buffers collects 5 f_s samples at the device rate, and the stream stops immediately afterwards. An OfflineAudioContext resamples to 16 kHz and the first 80,000 samples are kept.

7.2 Loudness normalisation

Applied identically in training (voice_teacher.py) and in the browser (voice.js). For a signal x split into 400-sample frames with RMS values e_t:

The output is γx, zero-padded or cut to 80,000 samples. If the loudest frame is below 0.002 the clip is rejected as silence. Normalising over the sounding frames only makes the level independent of pauses, and removes distance to the microphone as a source of variation.

7.3 Model

The spectrogram is computed inside the network, so the ONNX graph takes raw samples.

The DFT filters, with the periodic Hann window w[n] = 0.5 − 0.5 cos(2πn / 400), are:

Writing the short-time Fourier transform as a strided convolution uses only standard ONNX operators (Conv, Mul, Add, MatMul, Log), which every ONNX Runtime backend supports. Per-clip standardisation of the log-mel image removes any level offset left after 7.2.

The front end, run in this page

A JavaScript copy of 7.2 and of the DFT, mel and log stages, applied to five seconds of a recording from the collection. It matches the model's own filter bank to 2 × 10⁻⁹ and its log-mel output to 2 × 10⁻⁵. No microphone is used; the recording is downloaded only when you press the button.

Recording Compute the log-mel input

7.4 Teacher

Empathic-Insight-Voice-Small provides one regression head per attribute on BUD-E-Whisper encoder states. The audio is padded to Whisper's 30-second window, giving 1,500 × 768 hidden states; each head flattens them and applies 1,152,000 → 64 → 64 → 32 → 16 → 1 with ReLU. Only the Valence and Arousal heads are downloaded and run; no other attribute is ever computed. With the encoder, the teacher has about 235 M parameters, 560 times the student.

7.5 Training by distillation

7.6 Readout

The output z is in standard deviations of the training targets. The training mix leans towards expressive acted speech, so its mean sits above everyday speech; readout.json re-centres on the teacher's mean m over natural YODAS speech, so 0 means "as lively, or as bright, as ordinary speech":

The engine then applies the bounded nudge of 4.2. Arousal carries most of the weight because the teacher's valence depends largely on lexical content, which Truly never uses.

7.7 Evaluation