Developers

MIDI SysEx Broadcast

As our stage performances become more automated, it becomes challenging to align our disparate systems to know which song is being played. This means that we might need to configure our pedal board, laptop, keyboard, and other software to be aware of each change. At OnSong we believe that the value of software comes when it works together with other software. That's why we are taking a bold step to invent a protocol that can allow MIDI enabled apps and hardware to listen for changes and react in their own way.

How It Works

OnSong can be configured to broadcast information about the currently selected song using a proprietary system exclusive message. But don't worry, these messages are easy to decode and allow OnSong to transmit information that third-party hardware or software can use to keep performances in sync.

Any device that can be connected via MIDI to an iPad, iPhone, or Mac running OnSong can listen for these messages. For instance, you could link all computers and devices on stage to a network MIDI session so that all software can listen for these messages.

These System Exclusive messages, or "SysEx" is data sent over the MIDI protocol as a 7-bit data stream that is not specific to a channel. We will take a look at this data stream in hexadecimal "nibbles" to help understand how to process the stream.

Parsing Song Data

Songs are comprised of metadata such as the song title and key. For the sake of brevity, each of these name-value metadata pairs are delimited by a carriage return, or "0D". A common way to handle this then is to take the data and decode it into ASCII text, split the string by carriage returns "\r" and start processing each metadata component.

For each pair, the type of the metadata tag is indicated by the first ASCII character that is encountered. For instance, the song title's tag is a capital "S". This means that if you encounter "SWelcome to OnSong", the title of the song is simply "Welcome to OnSong" where "S" declares the type of metadata that follows. The value of the metadata pair is always a string. This means that numeric values are delivered as strings and not numeric values encoded in data. This allows the entire message to be processed as a string once received.

The order of these tags is not guaranteed, but generally we want the shorter and more commonly used tags to come first in the stream to allow for faster processing. You'll see that there are ways to request more metadata tag to be included in each message with special SysEx messages.

Breaking It Down

First, all SysEx messages begin with an "F0" nibble to indicate that the data that follows is a system exclusive message. The message is likewise terminated with an "F7" nibble. The contents in between carry the message. To avoid conflict, each manufacturer is assigned a Manufacturer ID. That means that directly following the initial SysEx nibble is a series of three other hexadecimal pairs, "00 02 24". This is the OnSong-specific manufacturer identifier. If you're planning to listen for OnSong SysEx messages, you'll want to look for that sequence at the start of the data stream.

The most common message used by OnSong SysEx is the song broadcast. The nibble directly after the manufacturer prefix is a carriage return which has a hexadecimal value of "0D". Once this is encountered, the rest of the stream up until the terminating "F7" nibble is ASCII encoded string data. This means that you can take the bytes of data and decode them into a string for further processing.

Let's take a look at an example. You may receive a MIDI SysEx message that looks like the following in hexadecimal:

F00002240D4B410D6B430D5438350D74342F340D5357656C636F6D6520546F204F6E536F6E67F7

Let's take a look at what this means. The F0 (in gray) in the beginning indicates that this is a system exclusive message. The F7 at the end (in gray) indicates that the message is finished. The system exclusive message is typically processed in packets and nibbles so once you begin reading it, you must then listen for the F7 as the termination.

The OnSong SysEx manufacturer identifier is a series of three bytes and is displayed in teal. This should be confirmed in order to process the rest of the data using this protocol.

The rest of the message can be read as ASCII-encoded text. The carriage return "0D" displayed in red, indicates a piece of metadata about the song is about to be encountered. After the carriage return is a tag shown in orange that is a single character indicating the type of value that comes next. The rest up until the next carriage return "0D" or termination "F7" is an ASCII-encoded string data that carries the value.

If we convert the data from the end of the SysEx manufacturer identifier to the terminating F7, you get the transposed key, original key, tempo, time signature, and song title in plain text. Note that a carriage return is encountered first and before each metadata pair. Only data that is available is returned even if the tag is requested:


KA
kC
T85
t4/4
SWelcome To OnSong

Available Tags

The most commonly needed tags are sent by default and in a particular order. The list below shows all available tags, with the default ones listed first.

K or Key *

This is the transposed key of the song, if transposition is enabled.

k - Original Key *

This is the original key of the song.

N - Song Number *

This is the song number, if available.

c - CCLI Number *

This is the song's CCLI number, if available.

D - Duration *

The duration of the song in seconds.

T - Tempo *

The tempo of the song in beats per minute (BPM).

t - Time Signature *

The time signature of the song as expressed as the number of beats over the duration like 3/4.

S - Song Title *

The title of the song.

A - Artist

The artist that performed the song.

O - Capo

The capo applied the song, if enabled.

C - Copyright

The copyright information for the song.

B - Books

The books in which the song belongs as delimited by the tab character.

E - Keywords

The keywords for the song.

F - Flow

The flow of the song such as the song's order.

I - Unique Identifier

The unique identifier used by OnSong.

L - Content

The full textual content of the song.

l - Lyrics

The lyrics of the song by removing chords.

Querying Tags and Requesting More Metadata

Only the tags denoted with an asterisk in the list above are returned by default. However, if your application requires other tags, you can request them by sending MIDI SysEx messages using the OnSong protocol.

First, you can query which tags are currently being supported using a question mark "?", or "3F" in hexadecimal, after the manufacturer identifier. When converted to a hexadecimal data stream, the system exclusive message looks like this:

F00002243FF7

This will return a list of tags prefaced by an exclamation mark "?" operator, or "21" in hexadecimal, that indicates the following data is a string containing characters related to the tags to be returned. This system exclusive message would return "!KkNcDTtS" after the prefix and looks like this:

F0000224214B6B4E6344547453F7

To add a tag that is missing, you can use the plus "+" operator, or "2B" in hexadecimal. For instance, you can request to listen to copyright and artist by appending "+CA" after the prefix, but before the termination:

F00002242B4341F7

You can just as easily remove tags using the minus "-" operator, or "2D" in hexadecimal. To remove the copyright and artist tags we've just added, send this:

F00002242D4341F7

While not recommended, you can also just set the tags you want using the equals "=" operator, or "3D" in hexadecimal. This could effect how other apps operate. To listen for everything "KkNcDTtSAOCBEFIlL", you could send the following:

F00002243D4B6B4E6344547453414F43424546496C4CF7

It is important to note that your application should not assume that metadata will be in any particular order because you can alter the order of metadata returned in this manner.

Requesting Song Information

While MIDI broadcast will send a system exclusive message each time a song changes, you may want to request the current song's information such as when you first start listening for broadcast messages. To do this, you form an OnSong SysEx and send it. OnSong will hear this and respond with the standard broadcast with the currently selected songs.

To do this, you send the typical SysEx starting and ending bytes along with the manufacturer ID and followed by an "@" symbol or "40" in hexadecimal. Here's an example:

F000022440F7

Alternatively, you can request OnSong to send specific keys by appending those keys after the "@" symbol. For instance, to request lyrics and flow, you could do this in hexadecimal where 6C is a lowercase "L" and 46 is an uppercase "F":

F0000224406C46F7

You can also request everything by sending an asterisk after the at symbol like this in hexadecimal:

F0000224402AF7

Performance

Note, if you are writing an application that requires very low latency/overhead, you can process the stream as it arrives and extract this information in a "just in time" method by evaluating each "nibble" and when each carriage return "OD" is found, look at the following nibble to determine if you require the value, only processing the value once encountered.

Registering for Broadcast SysEx

Typically, OnSong users would need to manually enable MIDI SysEx broadcast using a setting. However you can register your app with OnSong to enable MIDI SysEx broadcast messages. To do this, you will form a URL in your app and open it. This will cause the OnSong app to open and the broadcast settings to register your app. Here is an example on how to register an app for broadcast MIDI:

onsong://midi/broadcast/enable

This example uses the "onsong" URL scheme with the MIDI host. It tells OnSong that we are effecting the broadcast status of MIDI by enabling the source app. Even with broadcasting disabled in settings, OnSong will send MIDI SysEx.

When you are done listening, you can remove your app from the broadcasting apps list. To do this you would create a URL like the following and open it in your app:

onsong://midi/broadcast/disable

If you are not using an app, you can also pass your unique app identifier in the URL as the last parameter:

onsong://midi/broadcast/enable/com.domain.name
onsong://midi/broadcast/disable/com.domain.name

Additionally, you can use OnSong system exclusive messages to enable broadcast. Do to this, you use a greater than or less than symbol after the manufacturer identifier to enable or disable broadcast. After these symbols, you must provide an identifier for your service. To enable MIDI broadcast for "My App" you would send in hexadecimal:

F00002243E4D7920417070F7

Note that the data after the symbol shown in red is ASCII-encoded text for "My App". This prompt the user to allow access. To disabled access, you would send the same message with a less than symbol like this:

F00002243C4D7920417070F7