Follow is a program that displays a musical score and shows a cursor in the score that follows what you play. The program listens to a midi interface to read the notes that you play. When you have no midi interface you can use the build in keyboard on the screen. You can also use an audio-pitch recognizer that outputs a midi stream.
You can select a staff that you want to play and the program will play all other staves while following you on the selected staff. So you hear your self accompanied.
The program can also play an audio file (that is synchronized to the score) and it checks the notes you play to the actual notes in the score. The cursor is colored red if your notes are wrong. Follow can import timing data from abcweb to synchronize any audio file to the score.
The program also measures the timing of your performance. It can mark notes that you played too late or too early and also notes that you played with a wrong duration. It also computes a score that reaches 100% when all notes are played correct, on time and with the right duration. For all note errors, timing errors and duration errors a penalty is subtracted from the score.
Follow can also be used in a concert setup, where multiple devices show different parts of the same score. A master device determines the tempo and all other (slave) devices follow. The master can either use the play back feature or actually follow a midi instrument played by the director (or solist). (see the synchronization feature of Follow)
The metronome symbol flashes red when the metronome is running.
When you click this symbol playback starts with the metronome on (flash + click sound).
When you click this symbol while the metronome is running it stops the metronome.
With the input field to the right of the metronome you can change the score tempo. For instance, when you want to play slower. If an audio file is playing, the tempo of the audio is also changed. | |
Performance Bar | The bar displays your timing errors (see follow mode). The number in the bar indicates the difference in milliseconds between your note and the score time. A negative number means that you played to early, a positive number means you played too late. The colored bar represents the average absolute error of your last 10 notes. An orange or red color warns you when the average becomes too much. |
Start | Starts play back (in the selected follow mode). Normally a count-in is displayed
before play back really starts. The count-in can be disabled by checking menu item
No count-in in the Timing menu.
The computer can also play the score with the internal synthesizer or on the
connected MIDI device (see Output port).
The notes you play yourself are always echoed on the selected output port unless
you uncheck the Midi echo option. When the internal sythesizer is selected in combination with "mute own staff", the computer plays all staves, except the one you play your self. |
◉ | Microphone symbol. Its color is brown when the microphone is on. Dark-grey when the microphone is off. The symbol is clickable and toggles the microphone on/off. It is a shortcut for the use microphone checkbox in the microphone dialog. |
Volume Slider | Sets the volume of the internal synthesizer. Useful to balance the synthesizer and the audio player if an audio file is used. |
When an audio file is present, you can enable/disable sound, by clicking this icon. | |
Score | Here you can see how well you are playing. The score value is updated while you are playing
and increases until 100% when you make no errors and play all notes at the right time and with
the right duration.
The timing and duration checks can be dis/enabled with checkboxes in the menu. When these are not checked (default) only wrong notes are subtracted from the score. The score value is computed as follows: score = (correct notes - 0.5 * wrong notes - 0.5 * late notes - 0.5 duration errors) / total number of notes Each term in the error calculation is shown with the same background color as used for marking the offending notes in the score. The factor 0.5 for wrong notes, late notes and duration errors can be changed with URL options. |
Resizing | Dragging (touch-hold-move) downwards in the timing bar area resizes the bar. |
Touch/Click area | The on-screen keyboard (see menu to enable) has a sensitve area above the keys,
stretching along the whole keyboard. When you click/touch-and-hold in this area it lights up green and you can shift or resize the keyboard. Dragging upwards enlarges the keys vertically (makes them longer) and dragging downwards makes them shorter. Dragging sidewards shifts the keyboard to the left or right. |
Green +/- buttons | A click/touch-and-hold on these buttons to make the keys broader (+) or smaller (-). |
Score file | Browse the local file system and select a score file. The file can either contain MusicXML or ABC notation. The selected file is opened and the score rendered on the screen. In addition you can also select an audio file (.mp3 or .ogg) or a preload file. |
Use Dropbox | Change the score file button into the Dropbox Chooser button to browse files in your dropbox. |
Full Screen | Switches the browser to full screen display (if supported). |
Check mode | Notes you play are followed in the score. Which notes you have to play is determined by one of the three
check modes:
|
Follow mode | The aim of this program is to display a cursor in the score that follows what you play on a midi instrument.
You can select one of three modes to determine how the notes you play are followed and checked.
|
Synthesizer | This selection control can be used to activate the internal synthesizer
or to select a midi output port. A midi output port is only available when the browser
supports midi and the computer has a midi interface (connected to a midi instrument or
sound module). When the option "No sound" is selected the internal synthesizer
is disabled and no output is sent to MIDI. The audio player can still produce sound
when an audio file is loaded. The default selection is "Internal synth". |
Keyboard | Shows a keyboard where you can play the required notes. It is useful if you have no midi keyboard available. The keys can be moved and scaled by dragging (horizontally and vertially) in a small area just above the keyboard. There are also two +/- buttons. When you press-and-hold on these buttons, the keyboard is resized (scaled) in horizontal direction, i.e. the keys become larger/smaller. |
Show notes | With this option enabled you see the notes you must play highlighted on the keyboard with a red background. (The notes selected by the cursor are shown as depressed on the keyboard). |
Bluetooth LE | Opens a dialog window that shows Bluetooth LE MIDI devices (keyboards, sensors). You can pair with one or more of those devices. After paring, the received MIDI messages are processed normally by follow. When the Midi echo checkbox is on, the Bluetooth MIDI messages are also echoed on the normal MIDI output port. |
Save Preload | Pops up the save dialog where you can save a preload file. This file contains the score and most of the settings you entered either with the menu or with url parameters. |
Advanced menu | Shows the advanced menu items (in File menu and Timing menu). The items are shaded with a light blue color. Normally these items are hidden. |
Help | Shows this file. |
Show marks | Shows permanent marks in the score. These marks are set when the player makes an error, plays a note too late or too early and when a note is held down too short or too long. (timing, duration and error marks). At the end of a performance one can see which errors are made more precisely by checking the advanced menu item Error types. |
Show score calc. | Show the score calculation. The following calculation is shown in the menu bar:
good_notes - wrong_notes - late_notes - short_notes = score% of total_number_of_notes
When the option is unchecked only the score percentage is shown. |
Line Cursor | Shows the line cursor. This cursor moves linearly with (playback) time and is shown in addition to the normal note cursor. |
Hide Cursor | Makes the note cursor invisible. |
Check timing | Enables checking if you play the notes at the right time. An error is raised when you play
a note more than 150 milliseconds too late (or too soon). In addition a second error is raised
when you play a note later than 1000 millisiconds. Each error counts for 0.5 in the score calculation.
When an error is raised the offending note is marked in the score.
Both threshold values (150 msec and 1000 msec) can be changed with URL parameters lat1=nn&lat2=yy. See also URL parameter pnltim. |
Check duration | Enables checking if you play notes with the right the duration. A duration error is raised when you play a note 30% longer or shorter than the score duration. In addition the absolute time of the error should be more than 200 milliseconds. This second requirement is to avoid frequent errors in fast notes. When an error is raised the offending note is marked in the score. Both threshold values (30% and 200 msec) can be changed with URL parameters kort and minkort. |
Show Timing | This button shows a plot with all timing and duration errors of your performance. |
Metronome | When this option is enabled the metronome continues ticking after the count-in has finished. Otherwise the metronome stops after the count-in. This option is automatically selected when you start the score by clicking on the metronome icon. |
No count-in | Skip the count-in after clicking the start button. |
Midi echo | When playing a midi instrument the notes from the instrument are echoed back on the midi ouput port. This may not be desirable when the instrument already produces sound from itself. In that case the midi echo can be switched off. Notes from the accompaniment are always sent to the output port, unless the option Mute others has been checked. |
Test port | Plays three notes on the synthesizer port for testing the MIDI interface or synthesizer. |
Microphone | The Settings button opens the microphone dialog. |
Keyboard transparency | This makes the keyboard transparent and it overlays the score. May be useful for small devices. |
Error types | When checked the error types are shown next to the permanent marks in the score: numbers
appear at the end of a play-through or when the stop button is pressed. The meaning of the numbers is:
|
Exact | With this option checked, the cursor moves at the exact time of the next note(s).
It is the same as (and takes precedence over) the previous option, with this difference
that playing notes too soon or too late will not change the tempo. The cursor keeps moving in
the exact score tempo, and ignores any tempo changes by the player. With the Next note delay option checked, the cursor also moves at the score tempo, but adjusts the tempo (in adaptive modes) and relocates the current time to the actual played notes (in all modes). This option disables (because overrides) the Next note delay option. When playback mode is chosen, this option is disabled, because playback is always exact. |
Next note delay | After you play the selected notes the cursor moves to the next time position in the score.
When this option is checked (default) the cursor moves at the precise time when the next note
has to be played. When unchecked the cursor already moves at half the duration of the present
(sounding) note(s). Thus, it arrives at the next note before you have to play it. When you want to play in adaptive mode, this option is better be unchecked. Becaus the cursor arrives sooner at the next note, it allows you to speed up playing, which, in turn, increases the metronome tempo (in adaptive mode). |
Play one note before start | When the count-on is finished the computer starts playing the accompaniment one note before the cursor. So you do not start playing immeriately after the count-in, but the accompaniment does, one note before the selection, where you have to fall in. |
Use Mic. | Asks permission to use the microphone. When the microphone is working, a pitch detector converts the audio signal into notes, which are followed in the score. Only monophone melodies can be followed. To get access to the microphone the program uses the WebRTC API which is, at the time of writing, only available in Chrome and FireFox on the PC and Android platforms. |
Gain | The gain slider sets the volume level of the microphone. Decibel values of the currently received signal are shown by the organge volume bar (dB). The target sound should be stronger than the level indicated by the blue line in the volume bar. This level corresponds to the Min. Level slider (see below). The peaks of the current microphone signal are shown in the graph by a grey wave line. |
Sensitivity | The sensitivity slider sets the height of the dashed blue horizontal line in the graph.
When the green wave line drops below the dashed blue line a note is recognized.
Higher values of the sensitivity result in more (or earlier) recognition of pitches. Values above 5 are
not practical, because even without sound notes will be detected (in the background noise).
The optimal value is between 1 and 4. The green wave line represents the Average Squared Mean Difference Function (ASMDF) of the microphone signal. This function approaches zero when the input signal is periodical and the x-coordinate equals the period. The x-coordinate of the recognized period is shown with a small red vertical line in the graph. |
Min. Level | The minimum level is a treshhold for the microphone signal. Only signal levels above this threshold are considered for pitch detection. The value should be between -40 dB and 0 dB. The current value of the minimum level is shown in the microphone level bar by a vertical blue line. |
Cents bar | The cents bar displays the deviation of the recognized pitch from the closest musical note. Notes are supposed to have the twelve-tone equal temperament. Your instrument should be well tuned for the pitch detector to produce notes consistent with the score. |
Bass guitar | Normally the lowest note that can be detected is an F#3. When the bass checkbox is marked, the lowest possible note becomes an F#2. However, this doubles the internal sample buffer and makes detection four times slower. Therefore it should only be used when detection of such low notes is really needed. |
A link to follow.html can have several URL-parameters, which are explained in the table below. An example of a link with some parameters is:
http://wim.vree.org/js/follow.html?d:75mnyv0xsmye3mq/gadma_2.xml&syn&tmp=120&keyb&ksh&play
It plays a MusicXML file from Dropbox on the internal synthesizer and shows the played notes in the score and on the keyboard.
score file | The score file has to be a URL poining to a score file with exension .xml or .abc, for instance:
http://some.domain.org/follow.html?path/to/score_file.xmlThe file can also reside in your dropbox, where it has to be shared publicly. In this case you use the dropbox identifier prefixed with d: For example, dropbox tells me that a shared file in my dropbox has this link: https://www.dropbox.com/s/75mnyv0xsmye3mq/gadma_2.xml?dl=0The relevant part of this link is 75mnyv0xsmye3mq/gadma_2.xml, which you use as follows: http://wim.vree.org/js/follow.html?d:75mnyv0xsmye3mq/gadma_2.xmlNote the link is prefixed with d: Try it out by clicking the link. |
audio file | The file name can also be a URL pointing to an audio file (.mp3 or .ogg). You can have both a score file and
an audio file. For example:
http://wim.vree.org/js/follow.html?../js3/test/bwv99k.abc&../svgParse/bwv99_msc.mp3You can also use the dropbox syntax for the audio file. |
preload file | The third possibility for a file name is a preload file (see save). For instance:
http://wim.vree.org/js/follow.html?d:0q8p1zvfjn9ijw2/pre_ave.jsYou do not need any further URL parameters as all settings are included in the preload file. |
stf= | With the stf=staff_number parameter you specify which staff should be played by you. The notes in this staff are highlighted during your play. The first staff has number 1. |
extr | Sets the Extract staff option to Single. Can be used in combination with stf= to show only a single part from an multi part score. |
extrg | Sets the Extract staff option to Grand. Can be used in combination with stf= to show only a single grand staff from an multi part score. The Check mode remains unchanged. If you want all notes in the grand staff highlighted you have to add the URL option mod=3. |
pw= | The pw=xx parameter sets the page width to xx cm. (see Page width) |
fmd= | The fmd=follow_mode chooses one of the four follow modes. The value of follow_mode is 1, 2, 3, 4 or 5. (see menu) |
mod= | With the mod=check_mode parameter you can choose one of the three check modes. The value of check_mode is 1, 2, 3 or 4. (see menu) |
tmp= | The tmp=tempo sets the play back tempo. |
nodel | The Next note delay checkbox is unmarked. After (correctly) playing the highlighted notes the cursor jumps immediately to the next notes, without delay. Normally there is a dealy of half the (played) note length. |
metro | The metro parameter marks the metronome checkbox in the menu, so that the metronome continues when the count-in terminates. Normally the metronome stops ticking after the count-in. |
keyb | When the keyb parameter is present the keyboard will be shown. |
ksh | The ksh parameter marks the show notes checkbox, so that notes highlighed by the cursor are also highlighted on the keyboard. |
opa= | This parameter sets the opacity of the keyboard. opa should be in the range 1-9, where opa=1 is opaque and opa=9 is maximal transparent. |
nosnd | The nosnd parameter disables the internal synthesizer and MIDI output. It is the same as selecting the option "no sound" in the menu (Output port selector) |
noecho | Do not sent the MIDI messages of the keyboard to the output port (default: internal synthesizer). This option is needed when the keyboard already produces sound of its own. |
nocnt | Do not show and play the count-in before play back is started (start button pressed, central C played or metronome icon clicked) |
nomenu | Hides the menu. |
nobar | Hides complete menu bar at the top of the window, but not the menu button (see above for hiding that too) |
nocur | Hides the cursor. |
line= | Sets the position of the dotted line. line=1 is at the top and line=100 is at the bottom of the page. |
ip | When this parameter is present the device tries to connect to a Web Socket server at port 8091. The ip address
of this server is taken from the host part of the URL. The required socket server and a web server at port 8090
are both started when you run the program abcSrv.js with nodejs. When connecion is successful the menu button becomes green. Positioning of the cursor is now syncrhonized with a master device (see parameter ipm). |
ipm | When present this parameter makes the device the master of a group after connecting to a Web Socket server at port 8091. When connecion is successful the menu button becomes red. The master sends its cursor position (when it changes) to all slaves. The slaves synchronize their cursor movement to the master. |
hrz= | Switches to horizontal scrolling (side scroll mode) when hrz > 0. The default value is 50 and it determines the ac(de)celeration of the side scrolling speed. For fast moving scores a higher value (200) may be better. |
scl= | Sets the scale of the score when in side scroll mode (0 < scale < 2.0) |
shrnk= | Sets the note spacing when in side scroll mode. Note spacing is maximal when shrnk = 0. Higher values make the score shrink (it is the ABC %%maxshrink parameter) |
vol= | Sets the volume of the internal synthesizer (slider in the timing bar). |
pnlerr= | Sets the penalty factor for wrong notes in the score calculation (between 0 and 1, default 0.5). |
pnltim= | Sets the penalty factor for late (or early) notes in the score calculation (between 0 and 1, default 0,5). A note is considered too late if it is played more than LATE1 milliseconds too late (or too early). If played more than LATE2 milliseconds too late the penalty becomes 2. The default values are LATE1=150 and LATE2=1000, and can be set with URL parameters lat1=nn&lat2=yy. (see Check timing) |
pnlkrt= | Sets the penalty factor for notes that are played with incorrect duration, i.e. are held down not long enough or too long. (default value is 0.5) |
kort= | Sets the margin above which a duration error is raised. The value is a percentage (default 30) of the note duration. A smaller margin will require more accurate playing. (see Check duration) |
minkort= | Sets a minimum absolute duration error. Smaller errors will be ignored. Default 200 milliseconds. This threshold avoids frequent errors in fast notes. (see Check duration). |
mute | Enables (marks) the "Mute own staff" option in the menu. |
rstmute | Enables (marks) the "Mute rest" option in the menu. |
nosm | Disable smooth scrolling. |
ctrls | Show the controls of the audio player. Normally the audio player is hidden. Only works when an audio file is present. |
scal | Show the score calulation in the menu bar. (see Show score calc) |
tmng | Also include (and show) the timing penalty in the score calulation. Without this option no timing penalty is subtracted from the score. |
tmng2 | Enable checking of note durations. Notes played with incorrect duration will be marked in the music and a penalty value (see option pnlkrt) will be subtracted from the score. |
lncur | Show a line cursor at the current real time. In addition to the normal note cursor this line cursor shows playback time continuously, linearly moving from note to note. |
oneclr | Only use the blue color for the line cursor (one color). Without this option the line cusor receives the same color as the note cursor (i.e. green if you play the right note, red if you play the wrong note). |
stopat= | Specifies a midi number (key) that will function as start/stop button. Pressing this key has the same effect as clicking the start/stop button in the menu bar. Usually a very low key number is used (24). |
hinfo= | Sets the (initial) height of the menu bar (default 36 pixels) |
mat= | Specifies the opacity (matheid) of all cursors (default 0.5). |
kleur= | Specifies a color for each voice cursor by a comma separated list of RGB hex values.
Default: kleur=#f9f,#3cf,#c99,#f66,#fc0,#cc0,#ccc The line cursor takes the color of the first voice. If there are more voices than colors this list repeats. |
nolyr | Hide all lyrics in the score. |
hide2cur | The line cursor gets the visibility of the note cursor. (i.e. is also hidden when nocur is given). |
tmo= | Sets the timeout in milliseconds for the audio, when the cursor is waiting for the user to play the next note. The default value is 200 msec, which means that the audio will stop if the user fails to play a note within 200 milliseconds after the note is highlighted by the cursor. |
metmrk | Shows all permanent marks in the score. (timing, duration and error marks) See also Show marks and Error types. |
ftyp | Shows all error numbers next to the permanent marks at the end of a play through or when the stop button is pressed. The numbers are explained in Error types. |
selected staff --- Player ----- mute ---+ +--- no sound (playback mode) | | | | other staves --- Player ---- rstmute ---+-- portsel --+--- internal synthesizer (constant mode) | | (playback mode) | | | | Keyboard (MIDI in) --------+--- echo ---+ +--- external synthesizer (MIDI out) (or on-screen keys) | (or simulated play) | +--- Timing checks & cursor colors | ||||||||||||||||||||
The Player is the part of Follow that translates notes in
the score to MIDI events. These events are generated in real time and send to the output device so that
you can hear them. The output device is selected (designated) by option portsel.
With this option you can choose between the internal
synthesizer or an external synthesizer which should be connected to the MIDI output port. Or you can
just discard all events, so you have no sound from the player (but you can still have sound
from an audio file).
The keyboard can be a real keyboard connected to MIDI input, or the on-screen keyboard. The MIDI events of the keyboard are always processed by the timing checks of Follow to color the cursor and count the errors. Normally keyboard events are passed on to the port selector and from there to the synthesizer, so that you can hear what you are playing. But the events can be blocked by setting option echo to false. This may be desirable when the instrument already produces its own sound. The MIDI events from the Player can also be inhibited by option mute and rstmute. The first one blocks the events from the selected staff, i.e. the staff that you are supposed to play. The second one blocks the events from all other staves. These are supposed to be your accompaniment, but you might not want to hear them. | ||||||||||||||||||||
Correspondence between options, menu items and related URL parameters |
Multiple devices can follow the same score in precise synchronization. Each device can focus (extract) a different part of the score. In such a group of devices there is one master who determines the tempo, either by just using the play back feature or by actually playing on a midi device. The other devices are slaves and just follow the master. To achieve synchronization Follow uses a special server program called AbcSrv.js.
AbcSrv.js is included in the distribution zip file. It is a javascript program
that implements a simple http server and a special Web Socket server. The normal http server runs on port 8090 and the web socket server
on port 8091.
To run abcSrv.js you need to have Node installed. Node is a javascript
engine with unrestricted access to the computer's (networking) capabilities.
When you have Node installed you can start abcSrv.js in a terminal window (command line prompt) with:
# nodejs abcSrv.jsThe command for Node (nodejs in the example) may be different on your platform (for instance, just node). When started abcSrv.js prints the following output in the terminal window:
websocket server listening on port: 8091, address: 192.168.xx.xx http server listening in port: 8090, adress: 192.168.xx.xx web server root directory: /path/to/where/abcSrv.js/is/located/
Now you can check that the server works by just typing 192.168.xx.xx:8090 in a browser on another device connected to the same (WIFI) network. It should display a listing of files, one of which is follow.html. Then you can open follow.html as slave or as master, using:
http://192.168.xx.xx:8090/follow.html?some_preload.js&ip (for all slave devices) http://192.168.xx.xx:8090/follow.html?some_preload.js&ipm (for the master device)
In stead of a preload file, as in the example above, you can also just use a score file, or even no file at all and open a score file later with the menu button.
When a device succesfully connects to abcSrv.js the menu button becomes green. For the master device the menu button becomes red. This gives some feedback to indicate that all is workiing well and that henceforth all play/pause/jump actions will be issued by the master of the group. The slave devices cannot initiate any actions, they only follow the master.
You should have all files of the distribution (including abcSrv.js) extracted to one directory, which becomes the server root directory. All preload files you make or score files you want to use should also be in this directory.
On a normal (i.e. not jailbroken) iPad, abcSrv.js has been tested using the app Touch Code Pro from iTunes. This app can load and run abcSrv.js, turning the iPad into a simple http server and websocket server. While Touch Code Pro is running you can still use the browser to also open Follow, so you do not loose a device.