Multi-seat vs no seats policy in the compositor
Description
Environment
Activity
Marius Vlad July 5, 2022 at 9:24 AM(edited)
Checked also multiple-seat (logical seats) an we can have multiple active/activated applications by correctly specifying WL_SEAT in udev rules.
Multiple ID_SEATS require physical multiple video outputs, which means means that inputs visible to one display unit will be invisible to the other, so means multiple compositor instances, each on each seat so I've ruled that out, but it is possible.
So with this, I think we've covered all the entries. I'll just follow-up with upstream and see if what we'd need to backport for the fake seat use case.
Marius Vlad July 4, 2022 at 9:04 AM
Soem further update, added fake seat support in to upstream @ https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/798/diffs?commit_id=1639b9c6995185383f13d0e2e26fee06b7de5dde
Marius Vlad June 30, 2022 at 12:59 PM
This means we need to de-couple activation from keyboard being present.
Quoting myself here, this has been re-done to ensure it works in environments without a keyboard: https://gerrit.automotivelinux.org/gerrit/c/src/agl-compositor/+/27595,
3) How should we handle the situation where to have no seat?
I'm looking into adding a fake-seat here.
Marius Vlad March 4, 2022 at 8:35 PM(edited)
> Any idea what the upstream community thinking is on how things will work with multi-head and these seat requirements?
This RFC was an idea I had to try to gather some real use case scenarios that I can also work/present upstream, attempt to discuss them, and then come up with some possible solutions. We've discussed this a bit a few days ago, and for the time being it seems a possible solution is to have/create a fake seat, even though there's none available (which coincidentally in AGL happens already on some the platforms). Faking seat might be the easy solution for no seat scenario. While for multiple seats we basically pick the first one available (kiosk-shell isn't really targeting multiple seats) and ignore the others. With multiple outputs we'd assign focused surface to the last started application (thus basically bouncing from an output to another, depending on the order of starting/closing of the app).
An alternative to that would be to have an association between and output and seat, which happens in the configuration file, just like we would have with applications where we place applications on different outputs. Instead of hanging of the focused surface out of a seat, we have it on an output, due to the fact that for kiosk-shell applications are all fullscreen, and visually it looks like an outputs are independent of each other (which technically are not). Think it is easy to see that for a traditional desktop this wouldn't be the case, where we have windows that overlap and which need to change focus based on pointer/touch. I need to explore to see if this option would be viable here, as with shell, we could have panels that spawn across outputs, though atm I don't see how would work as well here (we could have panels and other applications, although not necessary the case, we should consider it). I need to experiment a bit, maybe we can ignore the panels somehow and just consider the regular apps.

Scott Murray March 4, 2022 at 6:51 PM
Any idea what the upstream community thinking is on how things will work with multi-head and these seat requirements? Automotive isn't the only area where this seems likely to be a thing, e.g. multiple display kiosk boxes or the like. And I'm curious how onscreen keyboards would play into this? Not at all?
Some terminology context wrt to seats:
A seat is a collection of input and output devices (keyboard, mice, monitors)
the DRM-backend which is a native back-end requires having a seat present
the seat here is a logical seat, where we group together different inputs and assign them to different outputs
Some terminology context wrt to "activation" (activation is a misnomer and requires some further explanation):
activation of a surface: the process of making that surface currently displayed, receive input
xdg-shell activation of a surface: described better here: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/main/stable/xdg-shell/xdg-shell.xml#L811. It is basically a way to inform the user, visually that the current surface has received focus. Note, not to be confused with the first item above
and finally, activation of an application: currently focused application is now displayed to the user. For instance, from the agl-shell protocol. Better explained by the following image:
In the compositor, when performing the activation of an application we should also incorporate the other types, described above: the input focus activation + xdg-shell activation.
But, at this moment, the compositor handles the first and the second items only if there's keyboard present, being inside of handler that is being triggered when the keyboard changes focus. Obviously, if there's no keyboard, there's also no input focus activation nor there's that xdg-shell activation happening. It is worth mentioning that this hasn't been such an issue because we use windows without decorations (so xdg-shell activation won't matter), and because we use touch input, rather than a keyboard. With the introduction of the weston-terminal which uses decorations it is obvious that won't receive nor input nor we see client decorations activated.
This means we need to de-couple activation from keyboard being present.
The changes from https://gerrit.automotivelinux.org/gerrit/c/src/agl-compositor/+/27216 do just that but they move a bit forward than that, they attempt to also fix a situation where we might not have a seat at all.
Coming back to the original issue of problem: not being to be able to handle focus input + xdg-shell activation without a keyboard, was fixed by just adding it inside compositor activation of application, such that relying on a keyboard being present is no longer a requirement.
In order to know which surface to activate/deactivate we hang it off the seat, because naturally when there are multiple seats we assume that you have multiple outputs so you attach inputs to one seat paired with an output and attach others inputs to another seat paired with another output, as it is happens in traditional desktop environments.
So, ultimately this basically means that we assume we have at least a seat being present in the system. Assumption which from testing out r-car3 and rpi4 seem to hold right, but that is because of either we have power knobs which are treated by udev as input devices (the case of r-car3 we actually have a physical switch to turn the board on) while for rpi4 the display driver seem to be categorized as input device. With at least a input device being created we also have a seat, albeit a "fake" seat present. This also means that ini configuration is from required-input=false is also moot here due to this.
Normally not having anything hooked up in the system, wouldn't create a seat, or you can just specify --seat=0, if you believe the system won't need it (some sort of simple displays of data).
Coming back from the patches in https://gerrit.automotivelinux.org/gerrit/c/src/agl-compositor/+/27216 they also incorporate a fix to avoid having seat altogether in the system, but in the same it exposes another issue which I haven't really thought on updating them: what happens if we have multiple seats or with mutiple outputs?
Because we no longer need a seat, we basically make the focus surface global for all possible outputs, but also causes conflict when there are multiple seats: which seat would actually get focus?
So basically, in a more generic matter, these are the following cases to consider
single seat, single output
single seat, multiple outputs
no seat, single output
no seat, multiple outputs
multiple seats, single output
multiple seats, multiple outputs
For IC most likely we have no seat, single output but maybe also single seat single output (we have some sort of knob that controls the information or switch something). For IVI, multiple outputs could become a norm, so I guess w could have either single seat multiple outputs or single seat multiple outputs, with each touch input device controlling an output basically.
So some further questions to thinkg about:
1) The changes from https://gerrit.automotivelinux.org/gerrit/c/src/agl-compositor/+/27216 initially tried to fix a totally different issue wrt to requiring to have a keyboard. Should I just bring only those changes back and leave this multi-seats vs no seat policy off the table for the moment? This is actually a bug fix imo.
2) How we handle the situation where we have multiple seats?
3) How should we handle the situation where to have no seat?