LipSync

A node that picks up the sound playing in the venue with a microphone, matches it against a fingerprint database of tracks analyzed in advance, and reports what track is playing right now, how far into it, and at what speed. It has no input ports; everything is operated from the node UI.

The order of use is pick a folder → Analyze → Listen. Analyze scans the specified folder recursively, including subfolders, and builds the fingerprint database. If the database is still empty, or while it is loading, pressing Listen does not start anything (the status area says so).

Outputs

• Track Path — file path of the matched track
• Position — playback position in seconds
• NormalizedPosition — 0-1. Only produced for tracks whose length is recorded in the database
• Playback Speed — playback speed estimated from how the position moves
• Confidence — how certain the match is
• Is Matched / Profile ID — whether a match is active, and the profile used for the match

Settings

• Capture (s) — length of one microphone capture. Default 5 seconds, slider 2-10
• Interval (s) — interval between match attempts. Default 2 seconds, slider 0.5-5
• Profile — band weighting used when extracting fingerprints. Choose from Auto (tries every profile) and the five options Balanced, Male Vocal, Female Vocal, Vocaloid, and High Pitch / Bright

The fingerprint database can be several MB of JSON, so it is loaded asynchronously off the main thread. Listen cannot be started while it is loading.