development.xml: copy MAV_COMMAND_REQUEST_OPERATOR_CONTROL from mavlink/mavlink/master

[peterbarker]

This is a PR to discuss implementation of multi-operator control.

This message has already been merged in mavlink/mavlink/master

There was a stab at implementing the message.

I'm not sure the messages are fit for purpose.

Key points:

• there are some resources which should only ever be accessed by one mavlink node at a time:
  • only one GCS should be able to use streamed position control commands at a time (e.g. SET_POSITION_TARGET_GLOBAL_INT)
    • this doesn't necessarily need to be part of this permissions system, it could be done with timeout
  • only one GCS should be able to stream in FOLLOW_TARGET
    • this doesn't necessarily need to be part of this permissions system, it could be done with timeout
  • only one GCS should be able to stream in RC_CHANNEL_OVERRIDES / MANUAL_CONTROL
    • this doesn't necessarily need to be part of this permissions system, it could be done with timeout
  • only one GCS should be able to set streamrates / per-message-rates on a link
    • this doesn't necessarily need to be part of this permissions system, it could be done with timeout
      • this could still lead to fighting amongst GCSs as a valid implementation is to watch the rate and only adjust it if you don't like what you see; two GCSs doing this can oscillate control
• armed/disarmed behavioural changes
  • Peter and tridge have been involved in operations where we wanted to ignore all input from non-blessed GCSs.
  • a clean story for the operators to know that devices just "listening in" (to provide observability on the operation) can't affect the vehicle while it is armed
• need some way to make sure someone always has access to the vehicle
• tridge suggested a simple "steal control" which always works
• Leonard wants the primary pilot to always be able to assume control (RC input in this case)
• Leonard wants an absolute chain of custody for the control of the aircraft regardless of how the thing is being controlled
  • e.g. even guided-mode commands
• companion computers on the aircraft doing pos/vel commands
  • how do you stop the crazy cc from doing what it is doing?
  • does the CC need to request control for SET_ATTITUDE_TARGET?
  • operator still needs to take control back!

Some user stories

1. Canberra BVLOS operations (@peterbarker @tridge @MichelleRos )

• we typically run two laptops which must always be able to command the vehicle
  • only one of those laptops should be able to command the vehicle via RC_CHANNEL_OVERRIDES or MANUAL_CONTROL at a time
    • only one of these laptops should be able to manipulate the streamrates on the vehicle
• a H16 hand control unit also acts as a GCS.
  • This unit should be able to download parameters
  • this unit should not be able to manipulate fences, rally points and esp. not manipulate the mission
  • this unit should be allowed to set streamrates on its link to the vehicle

2. FreeSpace Operations shipops operations (@lthall)

• many RPi-based GCS all of which may want to control the vehicle via RC
  • all of them share the same GCS link to the flight controller (Silvus network)
  • all of them should be able to take control of RC for the vehicle
  • they should not necessarily be allowed to change the mission/rally/fence!
  • parameter sets are not necessarily allowed!
• other GCSs may be used to set up missions etc.
• precision landing relies on FOLLOW_TARGET coming from another mavlink node

3. Freespace Operations multi-GCS multi-vehicle craning operations

• FOLLOW_TARGET used and should only be used from one vehicle
• otherwise much as above without the precision landing
• need to be very careful that users don't take control of vehicles accidentally!

[peterbarker]

@lthall this was the spit-balling stuff I was working on: peterbarker@18003cf

[Georacer]

I'd like to submit another user story, for BVLOS operations.

There are 2 GCSs, the operator of the first is on Landing Site A, there is another operator at LSB. LSA and LSB are far from each other.

Each operator has overview of only his own LS, hence he's the only one responsible for operations around that LS.

LSA is initially in control of the vehicle. Takes off plans a route to LSB. At some mid-way point he transfers command to the LSB operator. Then it's exclusively up to the LSB operator to handle the operation.

Take-over can still be requested and granted at any point of the operation.

[Davidsastresas]

I'd like to submit another user story, for BVLOS operations.

@Georacer That is the exact use case @hamishwillee and I had on mind when we started working on this almost 2 years ago.

I don't think companion computers should be on this protocol, they should have the same system ID of the vehicle, so messages accepted/rejected by vehicle due to this protocol should propagate naturally to the CC.

Very happy to see this moving forward.

By the way, support for QGC was already added some time ago, more info in mavlink/qgroundcontrol#12410

[hamishwillee]

I don't think companion computers should be on this protocol, they should have the same system ID of the vehicle, so messages accepted/rejected by vehicle due to this protocol should propagate naturally to the CC.

A CC is not an operator, and attempting to consider it one makes the problem unsolvable. For a start, who's really in charge?

The proposal deals with everything else in the list. There is only ever one thing in charge or nothing in charge, and handover is either automatic or requires permissions. The thing in charge is the only external system that the autopilot and other components will accept state changing commands or messages from - where it is up to each component to decide the set of things it considers risky to object form any other system.
Any component on the same system is trusted including CCs.

The second proposal we made allows multiple things in charge but only one thing that can send movement controls.

[Georacer]

The second proposal we made allows multiple things in charge but only one thing that can send movement controls.

@hamishwillee did this 2nd proposal become part (and got merged) in the upstream mavlink PR? Do you want to try to add support for it in ArduPilot starting from this PR here?

[hamishwillee]

The second proposal we made allows multiple things in charge but only one thing that can send movement controls.

@hamishwillee did this 2nd proposal become part (and got merged) in the upstream mavlink PR?

Yes, it merged mavlink#2313. The updates allows support for both models, with slightly different behaviour in each mode.

Do you want to try to add support for it in ArduPilot starting from this PR here?

@Davidsastresas Is planning on working on this once there is more clear direction from ArduPilot on whether it is acceptable. @peterbarker Has clients doing similar things. I am sure they would both like to coordinate.

What this is waiting on is deeper evaluation of the use cases and whether the proposal meets them. My understanding is that is what @peterbarker is trying to achieve with this PR.

[TharakaUJ]

Hi! I'd like to add following user story.

Educational/competition operations

Organizers provide the drone platform while participants are allowed to run their own scripts on a companion computer connected via MAVLink.

Participant scripts should be allowed to:

• receive telemetry,
• send task-related MAVLink messages,
• send vision/pose/navigation-related data,

Participant scripts should not be allowed to:

• force arm/disarm the vehicle,
• modify parameters,
• upload/modify missions, fences, or rally points,
• use RC_CHANNEL_OVERRIDE / MANUAL_CONTROL,
• unexpectedly take control of the vehicle.

Ideally this restriction could be applied per MAVLink link/node rather than globally.

[Davidsastresas]

@TharakaUJ I am sorry but I think that use case is totally out of the scope of what we are discussing here, it is very niche and we are not planning on bringing companion computers to this, they use the same system ID of the vehicle and are considered "part of vehicle" from this protocol's point of view.

I think such situation could probably be handled in a much better way in custom code in those companion computers used in the competition.

[TharakaUJ]

@Davidsastresas I see, that makes sense. Thanks for the clarification and pointers!

Out of curiosity though, if the competition participants connected as separate GCS/MAVLink nodes instead of applications running inside the companion computer, would that kind of use case fit better into the scope of this discussion?

[Davidsastresas]

@TharakaUJ not really... Such a custom mavlink filtering as you describe in your first comment is really not in the scope of what we are discussing.

I really think your use case is better suited for custom software in your CC.

[TharakaUJ]

@Davidsastresas Got it, thank you for the clarification!

[timtuxworth]

I really like where this is going. From my perspective the proposal is ready for testing, there is only so much we can do with a theoretical discussion, we need to try it.

[hamishwillee]

I really like where this is going. From my perspective the proposal is ready for testing, there is only so much we can do with a theoretical discussion, we need to try it.

I could not agree more. I think it is solid. The handover of control stuff has already been verified in the version 1 PR - its testable.

Since that went in ArduPilot added support for multi-GCS in charge, and the protocol proposal was adjusted to compensate. These are the things that most need testing:

• There are effectively single and multi GCS in charge modes. The integration needs to test that it is very clear to the flight stack which mode it is in.
• When in multi-GCS modes the status message include an array of the GCS that are in control, but this is necessarily limited in size. ArduPilot defines a range of GCS that might be in control. We need people to understand the limitation and "accept" that this is the right solution (or propose another).

More generally there will always be discussions about what services should be restricted to the GCS in charge. We've worded it such that telemetry is available to all, but that commands and command-like messages should be reserved. Anything really that can change the flight state should be restricted.
But from a user perspective it is important to know, so it would be useful for ArduPilot to define a policy on what is blocked, and ideally a list that shows any exceptions to that policy.

[lthall]

I think one of the key things coming out of this discussion is that we are still treating “control” as a single thing, when in practice there are several different classes of control with different requirements.

At a high level I see:

• Operational / flight control
• Payload control
• Link/resource control

I also don’t think configuration control can really be separated from flight control. Changing parameters, accepted input sources, failsafe behaviour, mode configuration, etc. can absolutely and materially alter aircraft behaviour. Operationally those are inseparable.

The other important distinction is that some resources are naturally exclusive while others are naturally shared.

For example:

• RC override / MANUAL_CONTROL
• SET_ATTITUDE_TARGET
• active guided position streams

are fundamentally exclusive resources. Multiple simultaneous writers create unstable arbitration and “fighting” between controllers.

On the other hand:

• telemetry
• parameter reads
• mission downloads
• log downloads

are naturally shared.

Then there are resources that are technically shareable but operationally conflict-prone:

• parameter writes
• mission uploads/edits
• fence/rally updates

FOLLOW_TARGET is a good example of why control should not be defined purely by message type.

Multiple systems may be sending FOLLOW_TARGET messages at the same time. In ArduPilot, the selected target is determined by configuration, for example by setting FOLL_SYSID, and the aircraft only uses that source when the pilot/operator enters Follow mode.

So the authority is not ownership of the FOLLOW_TARGET message stream itself. The authority is the ability to configure which source is accepted, and the ability to place the aircraft into a mode that consumes that source for motion control.

That means the pilot/operator in command owns the chain of custody by:

• selecting or authorising the target source,
• selecting Follow mode,
• and retaining the ability to leave that mode or reclaim control.

I think this points toward a model that looks more like:

• one primary authority holder
• delegated capabilities
• exclusive vs shared resource ownership

rather than a single global “vehicle owner”.

The primary authority holder retains ultimate authority over the vehicle and may always reclaim complete control.

If the primary authority holder disconnects, the system should not become uncontrollable simply because the preferred primary GCS is unavailable.

A better model may be that the primary authority holder is preferred, but not required to be continuously connected.

If the primary GCS link is lost, the current connected authorised GCS, or the next authorised GCS to connect, should be able to act as the primary authority holder until the original primary returns.

This would be a temporary acting-primary role, not a permanent transfer of ownership.

When the original primary reconnects, it should be able to reclaim primary authority, either automatically or through an explicit handback policy depending on the operation.

That gives us:

• continuity of control during link loss
• no authority deadlock
• a clear recovery path
• a way for the original primary to resume control

Payload control is probably another area where we should avoid making the core authority model too detailed.

There may be many payloads on a vehicle, and MAVLink should probably not try to define detailed ownership semantics for every possible payload type. A camera, gimbal, winch, release hook, sensor package, sprayer, or custom payload may all have different operational requirements.

One practical approach is to treat payload control as a delegated capability group.

Payload delegation could support two modes:

• Specific payload delegation: payload authority is granted to a specific GCS/component.
• Any payload delegation: payload authority is open to any authorised external controller.

This keeps the model flexible without requiring detailed per-payload ownership in the core protocol.

For higher-risk payloads, the primary authority holder may delegate payload control only to a specific controller.

For lower-risk payloads, such as cameras or sensors, the system may allow any authorised external controller to use the payload functions.

In both cases, the primary authority holder retains the ability to revoke payload authority, reclaim complete control, and reject further payload commands.

So the model becomes:

• primary authority controls overall vehicle authority
• delegated controllers may be granted payload authority as a class
• payload-specific arbitration is implementation-specific
• primary authority can revoke delegation and reject external input when required

To me this feels closer to how these systems are actually operated in practice.

[Georacer]

@lthall while I find your approach and vision ideal, I don't see how this can be served by the existing mavlink messages. The two new mavlink messages are https://mavlink.io/en/messages/development.html#CONTROL_STATUS and https://mavlink.io/en/messages/development.html#MAV_CMD_REQUEST_OPERATOR_CONTROL.

I think this is the framework we have to work with in the context of this PR; and what it allows for is only a transfer of full-access control. Nothing in between.

What you describe (again, I wish we can implement it somehow) isn't possible with those two messages.
An exception is MAVLink-enabled payload components, which could manage their own owning GCS. But that's not the concern of the FC.

So the question is: Are we satisfied with what those two new messages offer us? Do we want to implement them?
Or do we prefer to skip them, think of new messages, and push for upstream mavlink to adopt them instead/in parallel?

[hamishwillee]

@Georacer Respectfully I disagree. Until a shared design is agreed by the MAVLink stakeholders and whatever messages we come up with moves into common.xml we should be open to all options. Once things are in common they are frozen, and it becomes hard to change them, so we need to make sure we address as many use cases as we can.

That said, perfect is the enemy of good, and I'd prefer not to iterate for-never.

@lthall Sounds ideal as long as:

• we can still support an exclusive model as needed by @Davidsastresas
• we can still support some kind of multiple GCS-in-control model as currently supported by ArduPilot
• Components can tell who their owner is. A key feature of this design is not just handover of control, but that MAVLink components know who they should accept commands from.
• If we're going to extend management of capabilities to subcomponent/payload level that isn't necessarily a MAVLink component we need to clearly define how those things can advertise their ownership.
• The models are implementable and we have stakeholders willing to implement them.

Intuitively I think that such a model naturally supports both use cases above, and more. It might be overkill in terms of implementation though - i.e. the extra needs might not be worth the complexity.

Do you have a MAVLink interface in mind?

[Davidsastresas]

@hamishwillee I'm also open to modifying the protocol if needed, but only if there's real commitment from everyone involved to follow through.

I don't mean to be rude, I'm genuinely happy to see this moving forward. But Hamish and I have been working on this for over a year, tagging several AP devs at each iteration and going through many rounds of revision with no objections raised.
Then we hit the new situation described by Peter and Tridge — needing to support two GCSs with the exact same authority — which broke assumptions in the earlier design. We adapted the protocol to handle that, and it was merged upstream (mavlink#2313). QGC support is already implemented (#12410). The ArduPilot firmware side has a draft PR (#29252) that stalled waiting for exactly this kind of discussion.

It's worth noting that the current protocol was designed with the assumption that operators maintain good direct communication — voice comms, coordinated procedures — and are well synchronized operationally. Security was intentionally left out of scope, as that belongs in a higher layer of abstraction, or to a next revision of this protocol. The protocol handles authority and handover, not trust or authentication.

@lthall I think your vision of primary authority with delegated capabilities and resource-level ownership is the right long-term direction. But adding that layer of granularity now, realistically, means months of additional protocol design and coordinated implementation across MAVLink, ArduPilot, and GCS software.
I'd rather we implement and test what we have and then evolve toward something more sophisticated as you describe.

As @timtuxworth said, there's only so much we can do with theoretical discussion. We need to fly this.

[lthall]

@lthall I think your vision of primary authority with delegated capabilities and resource-level ownership is the right long-term direction. But adding that layer of granularity now, realistically, means months of additional protocol design and coordinated implementation across MAVLink, ArduPilot, and GCS software.
I'd rather we implement and test what we have and then evolve toward something more sophisticated as you describe.

The problem with that approach is we put our selves in a position where we need a new message to support what I believe is the minimum viable product. I suspect that if a primary isn't set in my vision it could fall back to the system you outlined were basically who ever has control is the primary.

I understand the frustration of not getting engagement from the community. That happens with limited bandwidth. I have been waiting for that critical mass interest level (and industry maturity) since #20373.

[Davidsastresas]

@lthall What you proposed in ArduPilot/ardupilot#20373 is a nice initiative, but it's a draft. What we have here is finished work — protocol design merged upstream (mavlink#2158, mavlink#2313), GCS implementation done and tested (mavlink/qgroundcontrol#12410). You can see a video of it working with multiple QGC instances and SITL there.

What frustrates me isn't community engagement. It's that we notified AP devs many times ( in public channels like discord dev channels or directly mentioning in github threads ) across every iteration, and nobody raised objections. In fact, the opposite, the feedback was positive from a lot of people. And now, with a working implementation ready, we're discussing a significantly expanded scope.

But that's fine, I understand the situation evolved that way and I'm ready to move on.

Let's all agree on a minimum viable product and move forward.

[lthall]

@lthall What you proposed in ArduPilot/ardupilot#20373 is a nice initiative, but it's a draft. What we have here is finished work

I would not even describe this as "a nice initiative", it was an attempt to get engagement on an area I was aware was lacking for many years at that time. My point is I have been thinking about this and looking for the critical mass to form for years and I am very happy it is here.

I also appreciate the work you have put in and I understand the frustration of having something working and so close to the finish line to then have a bunch of people, that have not been running with you, say they want you to do two more laps. I am sorry I was not exposed to your work earlier.

[peterbarker]

Participant scripts should be allowed to:
* receive telemetry,
* send task-related MAVLink messages,
* send vision/pose/navigation-related data,

For this use case I would suggest adding a proxy which the user's scripts must use which has a whitelist of commands and messages which are passed through from the student's scripts. I really think a fine-grained and relatively-easily-changed mechanism would be better there.

Perhaps take ArduPilot's "output" mechanism and add filtering in there. We already support filtering commands coming in from "outputs" when the vehicle is armed, for example.

[pompecukor]

@lthall What you proposed in ArduPilot/ardupilot#20373 is a nice initiative, but it's a draft. What we have here is finished work — protocol design merged upstream (mavlink#2158, mavlink#2313), GCS implementation done and tested (mavlink/qgroundcontrol#12410). You can see a video of it working with multiple QGC instances and SITL there.

What frustrates me isn't community engagement. It's that we notified AP devs many times ( in public channels like discord dev channels or directly mentioning in github threads ) across every iteration, and nobody raised objections. In fact, the opposite, the feedback was positive from a lot of people. And now, with a working implementation ready, we're discussing a significantly expanded scope.

But that's fine, I understand the situation evolved that way and I'm ready to move on.

Let's all agree on a minimum viable product and move forward.

@Davidsastresas @lthall @peterbarker @tridge

We and I think many other have been waiting a lot for this, since even before David started to work on it. We were so happy when they did and following with keen interest. For our base use-case it would be very beneficial as is and would be so sad to have to wait several months for release just to take it to another level that Leo is looking for. Of course it is cool and the use case is sound, but I think there is a strong case in making that a V2 or Pro version, and not throttle the current one as it stands. That way we can all benefit from the current iteration and there will be no time pressure for the V2. Everybody wins.

[adamk808]

My use case, we have two drones (in the future we would like 3,4,5...),
Lead drone has Starlink, 900mhz radio and RC comms.
Follow drone has 900mhz radio and RC comms.
The 900mhz radio is for transmitting position etc between the two aircraft, we don't receive data at the base station with 900mhz.
Starlink is the primary comms, RC's with mavlink are backups that cut in and out of range, if primary comms is lost, a failsafe RTL is enacted, we can then take control with the RC.

When we fly we currently have two laptops set up with a pilot monitoring each drone, but the second laptop has no control of the follow drone. The pilot with the starlink connection, needs to select the follow drone, then take control, which in an emergency isn't ideal.

We would like each laptop to have control of each drone, without needing a Starlink on the second drone. This is probably possible currently, but I haven't got that far in development of follow mode yet.