
-----------------------------  !SysExy (0.37)  -----------------------------

                         Copyright : Somascape 1999

                               Author : Lenny

----------------------------------------------------------------------------

0.0  Contents
     ========

     1.0  Overview
     1.1    System requirements
     1.2    Software status
     2.0  MIDI communications
     3.0  Iconbar
     4.0  The Panel window
     5.0  The Options window
     6.0  Identifying System Exclusive messages
     7.0  Creating System Exclusive data using a text editor
     8.0  Embedding System Exclusive data within MIDIfiles
     9.0  Contacting the author

----------------------------------------------------------------------------

1.0  Overview
     ========

This document describes the features and operation of the !SysExy
application, for use on a computer running RISC OS.

System Exclusive messages are a specific type of MIDI message.  They have
a broad range of use and fall into two main categories - Universal and
manufacturer specific.  Universal messages (Real Time and Non Real Time)
affect all (applicable) devices within a MIDI system, whereas manufacturer
specific messages are targeted towards specific devices.

Thus Universal messages are involved with system setup type operations,
whereas manufacturer specific messages are involved with setting up
individual devices.

!SysExy has been written as a MIDISupport plugin and enables MIDI system
exclusive messages to be captured over MIDI, and stored as files on disc
for later re-transmission over MIDI.  It functions therefore as a MIDI
Data Recorder (MDR).

It uses its own filetype (&770) for system exclusive data.  These SysEx
files are capable of containing multiple packets (in this document I use
the term 'packet' to mean 'system exclusive message').

  Eg.  The Roland R8 drum machine has a system exclusive command to
       request a dump of the R8's entire setup memory.  This dump
       comprises 30 packets which collectively make up the setup memory
       contents.
       These 30 packets can be captured and saved as a single multi-packet
       SysEx file, thus providing a self-contained record of that
       particular R8 setup.  Similarly with the R8's sequence (drum
       pattern) memory, ie a SysEx file containing the R8's sequence dump
       could be saved for each song project (so that you can always go
       back to a project at some later date).

That last point is !SysExy's main function - to capture 'snapshots' of a
MIDI device's memory contents, and to enable creation of a disc-based
library of such snapshots.

The program allows a maximum of 1000 system exclusive messages or (by
default) 64k of data to be held in memory (whichever limit is reached
first).  Sorry - no dynamic memory management in this respect, though the
64k limit can be increased (see section 2.0).

System exclusive messages can also be exported as MIDIFiles, enabling (for
example) synthesiser voice definitions which were originally captured with
!SysExy, to be loaded into a sequencer (eg !MelIDI or !MidiWorks) and thus
embedded within the song which requires those particular voices.  This
feature is mainly of use if you like to define your own instrument sounds,
rather than relying on the presets provided in your synthesiser.

A number of manufacturers make system exclusive data available as
downloadable MIDI files from their web sites.  These MIDI files can be
loaded into !SysExy, whereupon any system exclusive data is extracted.

You may also come across .syx or .mdx files.  !SysExy can load these as
they are in fact both identical to the SysEx file format that !SysExy
uses.  See the separate FileFormat text file.

It is also possible to use a text editor (eg !Zap) to create a textual
representation (a sequence of hex pairs) of a system exclusive message,
which can be loaded into !SysExy.

On systems with the 'new boot' structure (or the !Choices pseudo app), the
Choices scheme will be used for user preferences.

Interactive Help is supported, so run !Help (or !BubbleHlp, etc) when
first trying out the program.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

1.1  System requirements
     ===================

!SysExy requires the presence of ESP's MIDI Support software, including
!MidiMan (its front end application).  For use with external MIDI devices,
you will also need a MIDI expansion card and its associated MIDISupport
driver.  On a 26-bit OS, version 0.30 or later of the MIDISupport module
is required, whereas on a 32-bit OS version 0.34 or later is required
(along with !MidiMan 0.32 or later).  Contact Liquid Silicon
(www.liquid-silicon.com) if you need to upgrade.

!SysExy itself is 26/32-bit neutral, with no special SharedCLibrary
requirements as it has been linked with StubsG.  It runs on my StrongARM'd
RiscPC with RISC OS 4.02 through to RISC OS 4.37 (Select 3).  Although it
will run on any machine equipped with RISC OS 3.10 or later, the latest
(ie 32-bit) version of MIDI Support now requires RISC OS 3.50 or later.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

1.2  Software status
     ===============

This software is not PD ...

 ... It is Freeware.

That is, I retain the copyright, though it may be freely distributed.

Please, refer to the file 'Licence' within the '<SysExy$Dir>.Docs'
directory.

----------------------------------------------------------------------------

2.0  MIDI communications
     ===================

!SysExy makes use of a software MIDI driver (named 'SysExy') to both send
and receive MIDI data.  This driver needs to be connected into the MIDI
Support system.  This is done using !MidiMan :

 * Whilst !SysExy is running, open !MidiMan's Routing window and use the
   Source popup menu for the SysExy driver and select the required source
   driver(s).  Similarly, use the driver's Destination popup menu to
   select the required destination driver(s).

 * If you then use !MidiMan's Save Configuration facility, these
   'connections' will automatically be made whenever !SysExy is
   subsequently run (provided that !MidiMan is running at the time).

If you wish to exchange data with an external MIDI device, then you would
connect SysExy's driver to the relevant hardware MIDI driver for your
setup.  Eg AKA16_1 (if using an Acorn AKA16 MIDI card) or DMI50PortA_1 /
DMI50PortB_1 (if using a DMI50 card), etc. for both send and receive.

However, you could just as easily exchange data with another MIDI Support
aware application by connecting SysExy to its driver.

Once set up in this way, the setting of the Listen toggle in the Panel
window determines whether the MIDI input is actually monitored for the
arrival of system exclusive data.  When ticked, arriving messages are
captured and stored in memory.

The program will beep upon receiving a system exclusive message that would
exceed either limit of 1000 messages or (by default) 64k of data.  The 64k
limit can be increased if need be, by editing the 'UserOpt.MsgDataL'
parameter in the Messages file.  If you've got plenty of RAM, I recommend
a setting of 512k, which should cover most situations.

----------------------------------------------------------------------------

3.0  Iconbar
     =======

* Click SELECT on the iconbar icon to Open/Surface the Panel window.

* Click ADJUST on the iconbar icon to Open/Close the Panel window.

  Use the CTRL key with SELECT/ADJUST to open/surface/close the Options
  window.

  Use the SHIFT key with SELECT/ADJUST to open/surface/close both windows.

* Click MENU on the iconbar icon to open the iconbar menu.

  This contains 5 items : Info, Help, Options..., Clear and Quit.

  Info and Quit do the usual, Help runs this file, Options... opens the
  User Options window, and Clear removes from memory all currently loaded
  system exclusive messages.

  Note that no warning is given about losing unsaved data upon selecting
  Quit.

----------------------------------------------------------------------------

4.0  The Panel window
     ================

All of the main functions of the program are accessed directly from this
window.  There is no menu for this window, and hence no popup save window.

Of the two immediately visible sections, the top is split into 3 framed
areas, whilst the bottom section contains the main action buttons.

The top section :

 * The top area indicates the number of packets currently in memory, and
   provides a switch enabling the reception of MIDI data to be toggled on
   and off (when Listen is ticked, any system exclusive messages received
   by the SysExy MIDI driver will be captured.  (See section 2.0, 'MIDI
   Communications'.)

 * The middle area provides information about a single packet : its length
   (decimal and hex), its ID, and up to 10 bytes of the message.

   In the case of System Exclusive messages that are longer than 10 bytes,
   the message can be scrolled through using the bump icons next to the
   bytes list.  The number immediately to the right shows the offset into
   the message (of the leftmost displayed byte).

 * The bottom area (Cluster) enables a group of consecutive packets to be
   treated en masse.

Note that holding down the SHIFT key whilst clicking on any of the bump
icons (packet selection, byte offset and setting the clustered range) will
speed up the rate of change of the associated parameter.

The bottom (action button) section :

 * 2 file icons (for saving MIDI and SysEx files, respectively).

   A packet (or cluster) can be saved to disc either as a SysEx file or as
   a MIDIFile (for exporting system exclusive messages into sequencers) by
   dragging the appropriate file icon to the required filer window.

 * A writeable icon for entering the filename.

 * Save : To save packet(s) as a SysEx file (similarly if the RETURN key
   is pressed whilst the caret is in the filename icon).

 * Remove : To remove packet(s) from memory.

 * Tx : To transmit packet(s) over MIDI.

Note that each of these last three functions along with drag saving take
account of the 'Cluster' setting.  If Cluster is ticked then the range of
packets specified in that section will be saved/removed/transmitted. 
Otherwise the single packet shown in the middle area will be
saved/removed/transmitted.

There is a third section which appears only whilst transmitting MIDI data. 
This section gives information about the packet currently being
transmitted, and provides a single button Abort, which can be used to
interrupt the transmission of a clustered group of packets (interruption
occurs between packets, not in the midst of one).

Note that the action buttons other than Abort are disabled, and MIDI
reception is temporarily suspended, whilst transmission is in progress. 
Thus when transmitting packets, any packets returned in response by a
remote MIDI device will be ignored.

Closing the Panel window does not lose any data currently held in memory.

----------------------------------------------------------------------------

5.0  The Options window
     ==================

This window is accessed via the iconbar menu, and allows the user to
specify their preferred settings.  If these are saved, they will be used
as the default settings in subsequent sessions.

The upper section contains two settings, the effect of which will not be
apparent until the next time !SysExy is run (provided the options are
saved, that is).

 * Open Panel : If ticked, the Panel window will be opened upon running
   !SysExy.

 * Listen : This specifies the initial setting of the Listen switch in the
   Panel window.

The lower section contains settings which will take immediate effect.

 * Load action : Retain / Tx.  These determine what happens when loading a
   SysEx (or suitable Text) file.  Files dragged to the iconbar or Panel
   window can be treated transiently and immediately transmitted over
   MIDI, or retained in memory (for inspection, grouping or un-grouping),
   as determined by this option.

   Note that it is possible to set both options (ie Retain and Tx).

 * Open Panel upon load : If ticked, the Panel window will be opened when
   loading a SysEx (or suitable Text) file (provided that Load Action
   'Retain' is also selected).

   If the Load Action is set to Tx only, and a SysEx file is dragged to
   the iconbar whilst the Panel window is closed, then the Panel will be
   opened just for the duration of the transmission process.

 * Open Panel upon Rx : If ticked, the Panel window will be opened when a
   packet is received via MIDI (this also requires that the Listen switch
   in the Panel window is ticked, and that appropriate MIDI 'connections'
   have been set up using !MidiMan).

 * MIDIfile : When exporting packets as a MIDIfile, this option specifies
   whether multiple packets should be placed sequentially within a single
   track, or in separate tracks.

 * Export type : This is somewhat esoteric and is included for
   completeness more than anything else.  The Standard setting should
   generally be used.

   The MIDI specification describes two methods of embedding a system
   exclusive message within a MIDIFile : a) 'standard' and b) 'escape'.
     a) F0 <len> <remaining bytes, inc F7>
     b) F7 <len> <entire system exclusive message: F0 - F7>

   where <len> is a variable length value providing a count of the bytes
   which follow, up to and including the F7.

   Eg the message "F0 43 12 00 07 F7" would be coded :
     a) F0 05 43 12 00 07 F7
     b) F7 06 F0 43 12 00 07 F7

 * Inter-packet delay : This affects transmission of multiple packets.
   Some MIDI hardware cannot cope with receiving a quick succession of
   system exclusive messages, and needs to be given enough time to process
   the incoming MIDI data.  This option allows the user to specify a delay
   (in centiseconds) between adjacent packets.

In the absence of a preferences file, !SysExy will start up using its
internal defaults :

 * Open Panel : Yes
 * Listen     : No

 * Load action : Retain
 * Open Panel upon Load : Yes
 * Open Panel upon Rx   : Yes
 * MIDIfile : All packets in one track
 * Export type : Standard
 * Inter-packet delay : 20 cs

----------------------------------------------------------------------------

6.0  Identifying System Exclusive messages
     =====================================

This aspect of the program is user extendable, by way of the files present
in the '<SysExy$Dir>.IDs' directory.

The file '!_Manus' lists manufacturer ID codes.  This file MUST be
present.  The supplied copy contains just a short list of the more common
manufacturers, though you can extend this (up to 64 manufacturers).  This
file enables !SysExy to make basic identification of system exclusive
messages - at the level of manufacturer/universal type.

The remaining (optional) files (one per manufacturer) must (if present)
have the same filename as the name specified in the '!_Manus' file.

These files enable !SysExy to make a more precise identification of system
exclusive messages.  The contents are a series of lines each comprising a
pattern of bytes to be matched and a textual description of the packet's
usage.  Comments (lines beginning with a '#') and blank lines are ignored.

Each valid line has the general form :

  <comparison pattern>: <reported text>

ie any text preceding the first ':' is the comparison pattern.

The <comparison pattern> has the following format/syntax :

 * The first element specifies the byte (within the system exclusive
   message) from which to start the comparison.
   Eg.  B2=  specifies to start the comparison from the second byte.

 * There then follows a list of /consecutive/ comma (or space) separated
   hex digit pairs (maximum of 16 hex pairs).
   Note that specifying 'xx' means ignore/skip this byte.
   If one of the hex digits is specified as 'n' then this nibble will be
   masked out in any comparison.  Eg.  2n will match 20..2F.

The <reported text> should be 31 chars max.  Any spaces between the ':'
and the first non space character of the reported text are ignored.

Once the manufacturer has been determined (using the data in the !_Manus
file), the appropriate manufacturer file is searched /until/ a match is
found (if any), so make sure any 'catch-all' entries are listed after more
specific entries, eg (from the Roland file) :

  B4=28:    R8
  B4=28,4E: R8: H/S Pkt (ERR)
  B4=28,4F: R8: H/S Pkt (RJC)

is wrong as the first line would match all Roland messages with 40 (&28)
as the fourth byte.  The search process would therefore never reach the
following lines.  The following is correct :

  B4=28,4E: R8: H/S Pkt (ERR)
  B4=28,4F: R8: H/S Pkt (RJC)
  B4=28:    R8

Refer to the supplied files for example usage.


For Zap users :
~~~~~~~~~~~~~
You may like to add the following line

&500		&FFF	\#\*.IDs.\o\*		+Messages

to your '!ZapUser.Config.TypesLow' file.

This will apply syntax colouring to the files in the '<SysExy$Dir>.IDs'
directory.

----------------------------------------------------------------------------

7.0  Creating System Exclusive data using a text editor
     ==================================================

There are some situations which require the use of a system exclusive
message which it is not possible to capture from a MIDI device.  Eg many
devices will respond to a 'dump request' type message though may not ever
issue such a message themselves.

!SysExy is able to interpret a suitably formatted Text file which
describes a system exclusive message, thus any message can be created with
relative ease using a text editor.

The contents of the Text file should follow this description :

 * The data is represented as 2-digit hex characters separated by spaces,
   commas or linefeeds.

 * Comments can be introduced within the Text file with a '|' character.
   If it is not the first character on the line, it must be immediately
   preceded by a separator (space or comma), ie it must not be butted up
   against the data.  The remainder of the line is treated as a comment.

 * Additional spaces within the Text file (for aligning comments, etc) are
   allowed, and ignored appropriately.

The file should describe just a single system exclusive message - multiple
packets are not allowed.

Upon loading such a file into !SysExy, it is checked to see that it
contains valid data, ie that the interpreted data starts with &F0 and ends
with &F7.

Some example Text files are supplied in the 'Extra' directory.

----------------------------------------------------------------------------

8.0  Embedding System Exclusive data within MIDIfiles
     ================================================

It is often desirable to embed system exclusive data within a MIDIfile,
for example to ensure that the appropriate voice definitions are set up
for a particular sequence (song).

This does of course require the use of a sequencer which can cope with
system exclusive data, such as !MelIDI or !MidiWorks.

However, even with sequencers that do not directly cater for system
exclusive data, !SysExy can be 'manually' used alongside to transfer the
required system exclusive data prior to actually playing the sequence.

When exporting multiple packets as a MIDIfile, !SysExy will place the
individual packets either in a single track (at 4 bar intervals) or in
separate tracks (as determined by the MIDIfile option).

You should take care not to exceed your sequencer's number of tracks
capability when exporting multiple packets with the 'one packet per track'
option set.  For example, attempting to load a 20 packet MIDIfile (saved
using one packet per track) into !Serenade will lose the last four
packets, as !Serenade is a 16 track sequencer.

Apart from that proviso, it is fairly arbitrary which method you use, as
you will invariably want to move the packets around (which track and
position within that track) using the sequencer's copy/move facilities.

----------------------------------------------------------------------------

9.0  Contacting the author
     =====================

For bug reports, suggestions, sighs of adulation, derisive comments or
submission of gratuities, please contact me (Lenny) :

  email: riscos@somascape.org   (Use the button in the Info window.)

----------------------------------------------------------------------------
