This project has moved. For the latest updates, please go here.

SysExUtil problem

Topics: Samples
Jun 25, 2013 at 6:47 AM
I am attempting to use your SysExUtil sample to transmit a SysEx file to a device to edit its patch settings. Shorter messages seem to work well (a 6 byte "device connected" message returns the correct message and so does a 10 byte "fetch patch" message). However, I cannot get a longer message to behave properly (send a 650 byte "patch edit" message with no response on the device). I confirmed that sending the same 650 byte SysEx file through another tool (MIDI-OX) works fine. Please let me know if you see a problem with a 650 byte message right away or if you need more information. Thank you
-Sal
Coordinator
Jun 25, 2013 at 7:25 AM
You could install a virtual Midi port (like LoopBE1) and use that to route the outgoing midi sysex data to MidiOx.
SysExUtil -> LoopBE1 -> MidiOx

That way you can see if the output is correct or not.

It would also help to see the SysEx you are trying to send.
Jun 25, 2013 at 8:25 AM
I made a setup as suggested and the output seems to be correct. An interesting thing happened though as I "opened" MIDI-OX's output port to the device, on the MIDI-OX monitor it read "Opened MIDI output". I then went to SysExUtil and sent the SysEx file as normal (both in and out on SysExUtil are configured to go straight to the device) and then it worked and the device responded. However, once I closed that port on MIDI-OX, the problem resumed and the device would not respond.

Here is the SysEx file I am attempting to send.
-SalImage
Coordinator
Jun 25, 2013 at 6:08 PM
That is very odd -about having MidiOx opening the midi port. It sounds like you have a multi-client midi driver - or the first one to open the port would block other applications from opening it.

The data looks ok. It is one continues sysex message (F0 -> start and F7 is end of sysex), Sometimes devices use several sysex messages and that might sometimes cause problems when the next message comes too fast and the device is still busy processing the first block.

I also thought that perhaps the buffers were too small but the SysExUtil uses 1k buffers, ten of them.

Do you have any other midi ports available? Perhaps try those?
Would you be willing to try the test with the LoopBE1 virtual Midi driver?
Jun 25, 2013 at 7:03 PM
Ahhahh I found out what was happening. I thought it strange that MIDI-OX was able to open the port when SysExUtil already had it open, but I found that in the last line in the try finally statement in the PlayCommandHandler the port is being closed every time the Play button is pressed:
            try
            {
                this.appData.SysExSender.SendAll(this.appData.SelectedContentItems);
            }
            finally
            {
                this.appData.SysExSender.Close();
            }
I just commented out that close and then it started working. Please let me know if it okay leaving it like this or if there is anything else.

Thank you,
-Sal
Coordinator
Jun 25, 2013 at 8:31 PM
Nice Catch! Thanx.

I will try to implement that differently.

There may be problem with just commenting it out. The port will not be closed and will be (re)opened when sending the next command...
Coordinator
Jul 3, 2013 at 8:40 AM
Hi Sal,

I have checked in some new code. Wasn't able to test it yet but I think it should be ok. Can you give it a try?

Thanx,
Marc
Aug 7, 2013 at 4:10 AM
Hi Marc,
I tried the code and it works good thank you.

Im attempting to send out up to 200 MIDI messages in a row, each one excepting a return message. I believe the issue I'm having is sending such a large amount in a row out causes the send port to stop sending messages erroneously. Its as if it gets jammed for some reason that I do not know of. One solution I tried was increasing the number of initialized buffers:

this.outPort.BufferManager.Initialize(200, 1024);

Which did not fix the problem. Any help will greatly appreciated.

Thanks,
Sal
Coordinator
Aug 7, 2013 at 4:13 PM
I suspect that the device you are sending it to is overwhelmed..? You could try increasing the time you wait between sending sys-ex messages. That device needs time to process the incoming messages.

I would really suggest with this sort of problems to dump all the output into MIDI OX via a virtual Midi port (like LoopBE1). That way you can first identify if your code (or mine ;-) works correct and then send it to a real device.

The number of buffets does not need to be 200. It needs to be the number of buffers that are in use at the same time. 10 is usually fine. Do make sure that the length is appropriate for the max-length of the messages you are trying to send (max 64k). 1k sysex messages are typically considered large. Usually its no more than a couple of hundred bytes.

When you call the BufferManager.RetrieveBuffer() and it returns null, it means you have run out of buffers (all buffers are in use) - that is the time that you increase the number of buffers you allocate.

Could post some of your code on how exactly your sending the sys-ex messages? That way I might be able to give more specific feedback.
Hope it helps.
Aug 8, 2013 at 8:37 PM
Edited Aug 8, 2013 at 8:48 PM
Hello Marc,
I verified using LoopBE1 that the output from the code is working correctly. I also implemented the same approach in a Mac app ive been working on using CoreMIDI and it works good, the device does not get overloaded. The approach im using is to send a sysex message, and not until i receive the return message do i send the next one, and go up to 200. I used this exact method using CoreMIDI in the mac app, and it was able to fetch all 200.

Here is the code. I modified your SysexUtil program to perform this approach. First when I press the play button I execute the sequence of the 200 messages using FetchAll
protected override void Execute(object sender, ExecutedRoutedEventArgs e)
        {
            var portId = this.appData.MidiOutPorts.IndexOf(this.appData.SelectedMidiOutPort);
            //new
            if (_currentPortId != portId)
            {
                this.appData.SysExSender.Close();
                this.appData.SysExSender.Open(portId);
                _currentPortId = portId;
            }

            //this.appData.SysExSender.SendAll(this.appData.SelectedContentItems);
            _pc.FetchAll();
            base.Execute(sender, e);
        }
In FetchAll, this is where i setup up the event for when to send the next message as well as start by sending the first message:
public void FetchAll()
        {
            this.onFetchFinished += new FetchHandler(FetchNext);
            Fetch(new object[] { (byte)0x00, (byte)0x00 });
        }
All Fetch does is arrange the message appropriately and sends it out using your SysEX sender:
public void Fetch(object o)
        {
            // SEND Fetch message
            try
            {
                object[] os = (object[])o;
                byte banknumber = (byte)os[0];
                byte location = (byte)os[1];

                byte[] fetchMessage = new byte[10];
                fetchMessage = new byte[] { 0xF0, 0x00, 0x01, 0x55, 0x12, 0x01, 0x63, banknumber, location, 0xF7 };

                MidiSysExBuffer buffer = new MidiSysExBuffer(1024);
                buffer.Stream.Write(fetchMessage, 0, fetchMessage.Length);
                this.appData.SysExSender.Send(buffer);
            }
            catch (Exception ex)
            {
                Console.WriteLine("Fetch command catch: "+ex.Message);
            }
        }
and Fetch Next just calls fetch but increments accordingly:
void FetchNext(object myObject, FetchEventArgs myArgs)
{
            try
            {
               if(Global_location < 200)
               {
                    Global_location++;
                    string hexloc = Global_location.ToString("X");
                    byte byteloc = Convert.ToByte(hexloc, 16);
                    string hexbank = Global_bankNumber.ToString("X");
                    byte bytebank = Convert.ToByte(hexbank, 16);
   
                    Fetch(new object[] { (byte)bytebank, (byte)byteloc });
                  }
                else
                {
                   Global_bankNumber = 0;
                   Global_location = 0;
                 }
            }
           catch (Exception ex)
            {
                Console.WriteLine("Fetch next catch: "+ex.Message);
            }
}
When I receive the data:
 public void LongData(MidiBufferStream buffer, long timestamp)
        {
            Trace.WriteLine("Receiving buffer: " + buffer.BytesRecorded);

            var sysExBuffer = MidiSysExBuffer.From(buffer);

            ScheduleAddBuffer(sysExBuffer);
            PedalClass.HandleDataReceived(sysExBuffer); 
        }
I send the received buffer to a custom method that handles determining if it was received correctly and triggers the onFetchFinished event that allows to send the next message in FetchNext:
public static void HandleDataReceived(MidiSysExBuffer buffer)
        {
            switch (buffer.Stream.Length)
            {
                case 650: // fetch preset back
                    {
                        try
                        {
                            byte[] receivedBytes = new byte[650];
                            buffer.Stream.Read(receivedBytes, 0, 650);

                            byte[] nameBytes = new byte[16];
                            nameBytes = receivedBytes.ToList().GetRange(632, 16).ToArray();
                            string name = System.Text.ASCIIEncoding.UTF8.GetString(nameBytes);
                            name = name.Trim();

                            Preset preset = new Preset(0, -1, 0, name, "test" + name + ".syx", "test", "test");
                            FetchEventArgs args = new FetchEventArgs(preset);
                            staticThis.onFetchFinished(staticThis, args); 
                        }
                        catch (Exception ex)
                        {
                            Console.WriteLine("Fetch back catch: " + ex.Message);
                        }
                        break;
                    }
                default:
                    {
                        Console.WriteLine("DEFAULT HIT");
                        break;
                    }
            }
        }
So the problem I'm having is the messages stop sending before they should be finished. Please let me know any issues you might see on how I send or receive or use your program.

Thank you,
Sal
Coordinator
Aug 9, 2013 at 6:26 AM
I cannot see anything wrong with your code.

How did you check with LoopBE1 that the code worked fine? I mean you'd have to send 200 messages back in that test. Or did you change your code so no messages needed to be sent?

Could you place some Trace.WriteLine statements in you code so you get one line for sending a message (with bank and location) and one line for receiving a message. Then run it a couple of times and see if it always fails at the same point in the sequence?

What device is it, by the way?
Aug 23, 2013 at 2:40 AM
Hello,
I tried to cut down to the bare minimum and took your original SysexUtil program, and in the Play command replaced this code in execute:
this.appData.SysExSender.SendAll(this.appData.SelectedContentItems;
with
for (int i = 0; i < 200; i++)
            {
                this.appData.SysExSender.SendAll(this.appData.SelectedContentItems);
                Console.WriteLine("Send"+i.ToString());
            }
then in the receive I just added this to see how many are receiving:
public void LongData(MidiBufferStream buffer, long timestamp)
        {
            Trace.WriteLine("Receiving buffer: " + buffer.BytesRecorded);

            var sysExBuffer = MidiSysExBuffer.From(buffer);

            ScheduleAddBuffer(sysExBuffer);
            Console.WriteLine("receivng:"+count.ToString());
            count++;
        }
then in the program, I just select a fetch message (so we are just fetching the same data over and over 200 times) and I see the console is showing that we are sending 200, but not until that sending is finished do I receive a consistent 27 messages back at the end, when there should be 200 received back.

It would make sense that right when I send something I should see something received relatively close to the time I sent it, but in my case I do not receive anything until the end.
Thank you,
Sal
Coordinator
Aug 23, 2013 at 7:26 AM
Question is now: does the device send its responses - or do they get lost and if so - does the Midi driver lose them or is it our software (MIDI.NET)?

Can you route the input to MIDI OX for instance. That way you can check if the device is actually sending its responses. If so - could you insert a Thread.Sleep of like half a second to make sure the Midi input has enough time to process and copy the message...?

You could also try installing a new(er) Midi Driver for the port/interface you're using...?

We'll get there...