Multi Port MIDI Support

[spessasus]

Related discussion

I've started this discussion: #1360

Since #1352 was closed and not really a feature request, this is a sort of "official" feature request for this.

Describe the solution you'd like

Fluidsynth should recognize the 0x21 meta message.

Describe alternatives you've considered

N/A

Additional context

https://spessasus.github.io/spessasynth_core/extra/about-multi-port/

[pedrolcl]

Fluidsynth should recognize the 0x21 meta message.

Here are my two cents about this.

About 30 years ago, I was an user of the Cakewalk sequencer for MS-DOS and Windows. This software still exists today, by the way. They introduced this meta-event. This is why my stash still has a good amount of WRK files, and why I support the WRK format in some of my programs, including this meta-event. Thanks for reading all this off-topic trivia.

Not so long ago, the MIDI Technical Fanatic's Brainwashing Center was the most complete technical reference web site about MIDI before the MMA decided to publish all the specifications online. This is what they say about the obsolete meta-events:

The following Meta-Events are considered obsolete and should not be used. (The MMA would like you to know that they never endorsed their use, although since certain versions of CakeWalk utilized them, you may find existing MIDI files with these events). Use the Device (Port) Name Meta-Event instead of the MIDI Port Meta-Event.

[...]

MIDI Port

FF 21 01 pp

This optional event which normally occurs at the beginning of an MTrk (ie, before any non-zero delta-times and before any midi events) specifies out of which MIDI Port (ie, buss) the MIDI events in the MTrk go. The data byte pp, is the port number, where 0 would be the first MIDI buss in the system.

The MIDI spec has a limit of 16 MIDI channels per MIDI input/output (ie, port, buss, jack, or whatever terminology you use to describe the hardware for a single MIDI input/output). The MIDI channel number for a given event is encoded into the lowest 4 bits of the event's Status byte. Therefore, the channel number is always 0 to 15. Many MIDI interfaces have multiple MIDI input/output busses in order to work around limitations in the MIDI bandwidth (ie, allow the MIDI data to be sent/received more efficiently to/from several external modules), and to give the musician more than 16 MIDI Channels. Also, some sequencers support more than one MIDI interface used for simultaneous input/output. Unfortunately, there is no way to encode more than 16 MIDI channels into a MIDI status byte, so a method was needed to identify events that would be output on, for example, channel 1 of the second MIDI port versus channel 1 of the first MIDI port. This MetaEvent allows a sequencer to identify which MTrk events get sent out of which MIDI port. The MIDI events following a MIDI Port MetaEvent get sent out that specified port. It is acceptable to have more than one Port event in a given track, if that track needs to output to another port at some point in the track.

So I am not against this meta-event, but I also would understand that FluidSynth strictly support the standard only.

[hikari-no-yume]

The addition of the “device name” meta event seems to have come quite late; the spec on midi.org says it was approved in 1998/1999, but two-port devices had been around for at least half a decade by that point, e.g. the Roland SC-88 from 1994. So there is probably much more content out there using the pre-standard approach than the standard one, especially considering that by around 1999 General MIDI had reached its peak, and the standard approach doesn't work in all applications. Also, and this is just my subjective opinion, named devices does not seem as straightforward or practical a system for the common use-case of multiple ports on a single module. I think supporting the pre-standard approach is important.

[derselbst]

Thanks for the context, Pedro!

I think supporting the pre-standard approach is important.

I would like to argue against this. This MIDI event apparently was never part of any standard and was already deprecated back in 1999. Adding support to fluidsynth's MIDI player today - 25 years later - is just wrong, IMO. I'm not supporting the argument "There are so many legacy MIDI files out there". There are also a bunch of Nintendo64 MIDI files out there, which are using a special MIDI track looping convention, but I would never consider adding support for that to fluidsynth. What's important for me is that fluidsynth's API is strong enough to allow for being customized in such a way that corner-cases like those can be implemented by downstream applications. Which holds true, in this case.

[spessasus]

I would like to argue against this.

Why? First of all, there's not a single downside (in my eyes) to adding support for these. It would not break any existing MIDI files which worked previously, it would not change the sound behavior or anything like that.

This MIDI event apparently was never part of any standard and was already deprecated back in 1999. Adding support to fluidsynth's MIDI player today - 25 years later - is just wrong, IMO.

Even if it was "deprecated" back in 1999, take a look at this MIDI file

This is a MIDI arrangement of an music from a game called Antinomy of Common Flowers which came out in 2017.

It does use the 0x21 meta event. And not to mention that:

The MIDI itself is from 2019 (!)

So I don't think that this event is really deprecated if people still use it.

PS: Here's another one
The copyright says it's from 2016, and it uses channel 1 on port 0 for patch Electric Piano 2 and channel 1 on port 1 for patch Acoustic Bass. (It also uses 5 drum channels, but that's something different, not related).

So, in conclusion I think fluidsynth should support this event as it's still widely used today and adding it would not hurt the player in any way.

[derselbst]

adding it would not hurt the player in any way.

This is not about hurting or breaking existing functionality. It's not about the complexity of implementation, either. It's about where to draw a red line that says what's in scope and what's out of scope for fluidsynth. And it is my opinion, that this unofficial MIDI event is out of scope for fluidsynth.

I am not generally arguing against multi-port MIDI support. The MMA has an official event for this, which could be adopted. In addition, such an implementation should ensure it also works well with the audio drivers. I'm specifically thinking about the case audio.jack.multi=1. If this is done well, I would not mind if such an implementation adds a compatibility layer that silently translates these 0x21 cakewalk-events to the official MMA ones. I won't be the one implementing this though - I'm still feeling that such an implementation should be best implemented in a downstream MIDI player app that uses fluidsynth, rather than making fluidsynth's quirky MIDI player even more quirky.

That said, I will not accept a PR that just adds the "20%-magic-channel-indexing-logic" for supporting an obsolete MIDI event, sry.

[spessasus]

It's about where to draw a red line that says what's in scope and what's out of scope for fluidsynth.

Well, then that red line is very messy:

1. DLS Support. From the bottom of README:

For these reasons I've decided it would be easiest if the project stayed very focused on its goal (a Soundfont synthesizer), stayed small (ideally one file) and didn't dependent on external code.

DLS is not SF2, but it's supported, although the support is, well... very limited. So why bother supporting DLS at all then? Either support it properly or don't support it at all IMO.

2. GS support. Firstly, it's unofficial, so the argument that fluid should only support official MMA stuff is already invalid. Not to mention that the only supported GS sysEx is the Drum Part. Why would that be supported and not the rest of the GS messages if fluid decided to support GS? Same as DLS: either support it properly, or not at all...
3. XG support. It's also unofficial. The only thing it supports is the bank selection logic, same thing as GS: Why bother if the rest of the system is not supported at all? (talking parameters, program and bank selects via SysEx, effects, etc.) Why not take the fluidsynth's API is strong enough to allow for being customized in such a way that corner-cases like those can be implemented by downstream applications approach there and let the apps that use it handle XG?

I'm honestly really confused about the project's goal and the "red line". What is it, where does it draw?

• If it's a synthesizer, why does it have a MIDI player built in?
• If it's a soundfont synth, why does it support DLS?
• If it's a MIDI player, why is this widely used event not supported?
• If it already supports unofficial events (GS, XG SysExes), why not support 0x21?

It's really inconsistent with what the README says (a soundfont synthesizer).

PS: About this:

The MMA has an official event for this, which could be adopted.

I've not encountered a single MIDI file that uses this event (Device Name is what I'm talking about.) unlike the 0x21. Can someone please provide a .mid file that uses the official event for reference?

Because of the ambiguity of this event: the port is a string! For example MIDI Out or something like that. But it doesn't say that it has to be "MIDI Out X". So what if the port is named "My Port"? How would fluid interpret this? Does it mean port 1, 2, 3 or 4? Or maybe 0? Remember, fluidsynth is the synthesizer so it can't just tell the OS "please send these messages to a port called like this if exists". Meanwhile 0x21 is clear: 0 means 0, 1 means 1 and so on...

But that's just my opinion. I'm not the one in charge here, and if it's decided that this message exceeds the fluidsynth's scope, then so be it.

[hikari-no-yume]

If I made one, would you be willing to accept a PR that supports both the standard and non-standard way to do multi-port MIDI, using the same mechanism? They are both loosely specified and have unclear interpretations, but supporting simple 32-part cases should be straightforward.

[hikari-no-yume]

Though I think it's also worth saying that there's also a lot of multi-port files out there that don't contain any meta events for indicating ports, they would require some heuristic to play back correctly. It's a messy topic. I'm not sure if you can even warn about this easily…

[derselbst]

It's about where to draw a red line that says what's in scope and what's out of scope for fluidsynth.

Well, then that red line is very messy:

@spessasus I don't think I have argued this "red line" is straight and clear. It has always been a case by case decision. Yes, things have grown historically and they have changed over time. E.g. the section of the README you were citing is more than 20 years old, and explains the initial motivation by the original author Peter Hanappe back then - you're taking it out of context.

GS and XG support existed before I took over maintaining this project, just like the MIDI player did. It's a legacy that I won't judge about.

DLS support was done during my time, first and foremost because there was (at least one) user request (#320), and because the synthesis models are quite compatible, and because I personally considered it useful (and I still do). I appreciate your effort to track down bugs. Yet, I'll not rip off existing DLS functionality just because someone found a bug. If you are serious with your "all or nothing" attitude, you should agree with my previous comment, that it's better to implement a fully featured multi-port MIDI support, rather than only supporting a long deprecated MIDI event, just because there are a few recently created MIDI files around.

Pls. understand that I refrain to answer the rest of your questions, due to off topic.

If I made one, would you be willing to accept a PR that supports both the standard and non-standard way to do multi-port MIDI, using the same mechanism?

@hikari-no-yume Yes. As stated previously, adding an implementation that recognizes the standard multi-port way indicated by MMA, while staying backward compatible with the CakeWalk approach would be a reasonable implementation for me. If required, you may consider using fluidsynth's hashtable when implementing the string based multi port events... just in case you find it useful for internal bookkeeping.

Though I think it's also worth saying that there's also a lot of multi-port files out there that don't contain any meta events for indicating ports, they would require some heuristic to play back correctly

That also matches with my experience. However, I'm asking you to not go that far in this context.

[spessasus]

It's about where to draw a red line that says what's in scope and what's out of scope for fluidsynth.

Well, then that red line is very messy:

@spessasus I don't think I have argued this "red line" is straight and clear. It has always been a case by case decision. Yes, things have grown historically and they have changed over time. E.g. the section of the README you were citing is more than 20 years old, and explains the initial motivation by the original author Peter Hanappe back then - you're taking it out of context.

GS and XG support existed before I took over maintaining this project, just like the MIDI player did. It's a legacy that I won't judge about.

DLS support was done during my time, first and foremost because there was (at least one) user request (#320), and because the synthesis models are quite compatible, and because I personally considered it useful (and I still do). I appreciate your effort to track down bugs. Yet, I'll not rip off existing DLS functionality just because someone found a bug. If you are serious with your "all or nothing" attitude, you should agree with my previous comment, that it's better to implement a fully featured multi-port MIDI support, rather than only supporting a long deprecated MIDI event, just because there are a few recently created MIDI files around.

Pls. understand that I refrain to answer the rest of your questions, due to off topic.

If I made one, would you be willing to accept a PR that supports both the standard and non-standard way to do multi-port MIDI, using the same mechanism?

@hikari-no-yume Yes. As stated previously, adding an implementation that recognizes the standard multi-port way indicated by MMA, while staying backward compatible with the CakeWalk approach would be a reasonable implementation for me. If required, you may consider using fluidsynth's hashtable when implementing the string based multi port events... just in case you find it useful for internal bookkeeping.

Though I think it's also worth saying that there's also a lot of multi-port files out there that don't contain any meta events for indicating ports, they would require some heuristic to play back correctly

That also matches with my experience. However, I'm asking you to not go that far in this context.

Hi @derselbst, I didn't know that readme was not updated in over 20 years (maybe it's time to update it, btw? ;-) so sorry for misinterpreting it as the current state of fluid. I'm not gonna raise the topic of fluid's "red line" again.

And yes, I agree that implementing MMA's version with a translation layer for 0x21 is the best of both worlds, though my question still stands:

Because of the ambiguity of this event: the port is a string! For example MIDI Out or something like that. But it doesn't say that it has to be "MIDI Out ". So what if the port is named "My Port"? How would fluid interpret this? Does it mean port 1, 2, 3 or 4? Or maybe 0? Remember, fluidsynth is the synthesizer so it can't just tell the OS "please send these messages to a port called like this if exists". It has to know which channels to use for the track that requests such port.

But I'm sure this is solvable.

Anyways, since I've posted the test MIDI files in discussion but it's inactive, I'm posting the test MultiPort (the cakewalk ones) MIDIs here to help with development:

Test MultiPort Files

Playing them through Falcosoft MidiPlayer6 or my synthesizer is the correct behavior (or at least it sounds correctly, all instruments make sense and BASS (which FSMP6 uses) claims correct support for 0x21)

[derselbst]

Hi derselbst, I didn't know that readme was not updated in over 20 years (maybe it's time to update it, btw?

The section you've cited from is called "Historical background" - which is 20 yo. The rest is quite up to date.

Because of the ambiguity of this event: the port is a string! For example MIDI Out or something like that. But it doesn't say that it has to be "MIDI Out ". So what if the port is named "My Port"? How would fluid interpret this? Does it mean port 1, 2, 3 or 4? Or maybe 0? Remember, fluidsynth is the synthesizer so it can't just tell the OS "please send these messages to a port called like this if exists". It has to know which channels to use for the track that requests such port.

But I'm sure this is solvable.

The MIDI player has no knowledge about whether a MIDI driver exists, and if so, what its port names are. So, I would recommend to not attempt introducing such a dependency.

I would assume the portnames to follow a consistent scheme within a single MIDI file. So, if the author decides to use "MIDI Out", then it would be "MIDI Out 1", "MIDI Out 2", "MIDI Out 3", etc. Same if the author decides to stick with "My Port x". I do not consider a mixture of arbitrary portnames with in the same MIDI file to be a valid use-case.

We cannot be sure that the numbering is subsequent, though. E.g. it could be that "MIDI Out 2" is missing. Just like 0x21 events could use ports 0 and 2, but lacking 1. In both cases, the MIDI channels of the first port would need to be dispatched to synth channels 0-15, those of the next port dispatched to synth channels 16-31, etc. Because of that, I was previously suggesting to use a hash map. It needs to be sorted though.

[syntaro]

Hello,

Im using FluidSyth as 16 x 4 ports. You just need dispatch.
and It works for me.
but I have question abot this code.
If audiostream : synth = 1 : 1 ? Is it must?

typedef struct {
    fluid_settings_t *settings;
    fluid_synth_t *synth;
    fluid_audio_driver_t *audio;

    int soundfont_id;
}  __attribute__ ((aligned(32)))  fluid_handle_t;

jint jfluid_open(JNIEnv *env, jobject obj, jstring fontFile, jboolean lowlatency) {
    int id = findEmptyId();

    if (obj != nullptr && obj != _javaObj) {
        _javaObj = obj;
    }

    if (id < 0) {
        return id;
    }

    fluid_handle_t *handle = getFluid(id);

    if (handle == nullptr) {
        return -1;
    }
    if (handle->audio != nullptr) {
        delete_fluid_audio_driver(handle->audio);
        handle->audio = nullptr;
        std::this_thread::sleep_for(std::chrono::milliseconds(400));
    }

    if (handle->settings == nullptr) {
        handle->settings = new_fluid_settings();

        fluid_settings_setstr(handle->settings, "audio.driver", "oboe");
        fluid_settings_setstr(handle->settings, "audio.oboe.sample-rate-conversion-quality",
                              "Fastest");
        if (lowlatency) {
            fluid_settings_setstr(handle->settings, "audio.oboe.sharing-mode", "Exclusive");
            fluid_settings_setint(handle->settings, "audio.periods", 2);
            fluid_settings_setint(handle->settings, "audio.period-size", 64);
            fluid_settings_setstr(handle->settings, "audio.oboe.performance-mode", "LowLatency");
        } else {
            fluid_settings_setstr(handle->settings, "audio.oboe.sharing-mode", "Shared");
            fluid_settings_setint(handle->settings, "audio.periods", 16);
            fluid_settings_setint(handle->settings, "audio.period-size", 64);
            fluid_settings_setstr(handle->settings, "audio.oboe.performance-mode", "None");
        }
    }


    if (handle->synth == nullptr) {
        handle->synth = new_fluid_synth(handle->settings);
    }


    if (handle->synth == nullptr) {
        return -1;
    }

    while (true) {
        int count = fluid_synth_sfcount(handle->synth);
        if (count == 0) {
            break;
        }
        fluid_sfont_t *font = fluid_synth_get_sfont(handle->synth, count - 1);

        if (font == nullptr) {
            break;
        }
        fluid_synth_remove_sfont(handle->synth, font);

    }
    if (handle->soundfont_id >= 0) {
        fluid_synth_sfunload(handle->synth, handle->soundfont_id, 0);
        handle->soundfont_id = -1;
    }
    MBString file(env, fontFile);
    if (fluid_is_soundfont(file.mb_str())) {
        handle->soundfont_id = fluid_synth_sfload(handle->synth, file.mb_str(), 0);
    } else {
        return -1;
    }


    if (handle->soundfont_id == FLUID_FAILED) {
        return -1;
    }


    if (handle->audio == nullptr) {
        handle->audio = new_fluid_audio_driver(handle->settings, handle->synth);
        fluid_synth_set_gain(handle->synth, 0.7);
    }

    return id;
}


void jfluid_close(JNIEnv *env, jobject obj, jint id) {
    fluid_handle_t *handle = getFluid(id);
    if (handle != nullptr) {
        if (handle->soundfont_id >= 0) {
            fluid_synth_sfunload(handle->synth, handle->soundfont_id, 1);
            handle->soundfont_id = -1;
        }
        if (handle->audio != nullptr) {
            delete_fluid_audio_driver(handle->audio);
            handle->audio = nullptr;
        }
        if (handle->synth != nullptr) {
            delete_fluid_synth(handle->synth);
            handle->synth = nullptr;
        }
        if (handle->settings != nullptr) {
            delete_fluid_settings(handle->settings);
            handle->settings = nullptr;
        }
    }
}