Improve on Stumpwm's floating window support

[sdilts]

Stumpwm's support for floating windows is very-much hacked onto its design. How can Mahogany better support this feature?

[jcguu95]

Stumpwm did accumulate some legacy code. I see little value in having the separation of dynamic-group and floating-group as in stumpwm. That makes things too rigid, and eliminate the possibility of having dropdown windows in dynamic groups.

In my opinion, it should have been written in the following simpler design.

1. A window is either tiled or floating. If a window is tiled, it must belong to a frame. If a window is floating, it belongs to no frame.
2. Any tiled window behaves as a dynamic-window in stumpwm. In particular, when selected, it always gets tiled fully to its frame.
3. Any floating window, when not hidden, should be drawn on top of all current frames.

[sdilts]

@jcguu95 , this is close to what I was thinking, except that frames would still be present so views could be grouped together. The idea would be to have the ability to place frame trees at arbitrary coordinates instead of being locked to the dimensions of an output. So a view must still be associated with a frame, but a frame is either tiled or floating. I'm not married to the concept though, as creating and destroying floating frames could be too complicated for a relatively small benefit.

However, I'm fairly confident that the concept of a floating layer might be a good way to always render floating views on top and fix a few other issues with floating windows.

Mahogany has the concepts of layers, with views in upper layers being drawn over those of lower layers. We could deal with floating frames by placing them in one or more layers that when active, are drawn above the tiled layer. Raising a floating frame could cause all the other frames in that layer to be rendered, but I'm not sure if that would be useful.

[jcguu95]

Layers sound nice.

Collecting floating views in a frame is also nice. It may allow relevant drop-down views to come and go altogether. It's also good to have a uniform structure: group > frame > view. (How does 'layer' fit into this chain, by the way?)

But I wonder why tile/float should be a property of a frame. Namely, why should all views in a frame be either tiled or floating?

[sdilts]

The order would be group > layer > frame > view. We currently have one layer right now, the tiling layer.

Frames are actually n-ary trees, with each view being placed in the view-frames that form the trees' leaves. The floating property would be a property of the entire tree, and not of the individual frames. A way to model it could be that the root of the frame tree is floating, and its children are tiled in relation to the root.

The current frame definitions probably need some minor tweaking to get this right. There are some mentions of floating frames in the code, but they can be mostly ignored.

[jcguu95]

Is my following understanding correct? In your new definition, a frame is a tree. Each node's children is an ordered list of nodes. Each leaf of that tree is also called a view-frame. The root of that tree is an output-node.

If so, I'd say the terminology is a bit confusing. You are calling the tree a frame, and some of its nodes (view-)frames too. I would suggest you call the tree as frame-tree.. which means a tree of frames.

---

This is clean: group > layer > frame(-tree) > view!

Currently, there doesn't seem to be much code about layer in the codebase. In your design,

1. Will a group have an ordered list of layers?
2. Will a layer have an unordered list of frame-trees (per my suggested terminology), and two different frame-trees should not overlap on the 2D plane?

[sdilts]

Your summation of the frame-tree structure is accurate, with a few notes. A frame is something that has a width, height, and position. If this were Java, I'd declare the class named frame abstract, as you should be creating view-frames, or tree-frames, and never plain frame objects. That's why I've been using it as a generic term.

There are two other concrete types in the mess. output-node and tree-container. The output nodes are places in the tree that can be associated with an output, as that's where the tree roots are for tiled frames. The tree-container is there to have all of the output nodes for a group under one object.

This would need to be reworked a bit to support non-tiled frames. I think making the tree-container class work as a root without the output nodes would be good enough. That way, each layer could be represented by a single tree-container object. Renaming tree-container to something related to layer might make things more clear.

Maybe I should draw a picture.

---

The layer concept is a relatively new idea that I thought of when working on #113. The bones of it are in the hrt_scene.h header file. Any group related layers would go in layers attached to the hrt_group struct, which then get attached to the normal layer in the hrt_scene_root struct; the rest are used for the layer shell protocol. There current isn't anything in the lisp code that is layers related.

[sdilts]

To actually answer your questions:

Will a group have an ordered list of layers?

Yes. Exactly one of these layers would be the tiled layer, with one or more floating layers. I'd be tempted to just start out with just one tiled layer to see how well it works in practice.

Will a layer have an unordered list of frame-trees (per my suggested terminology), and two different frame-trees should not overlap on the 2D plane?

The unordered list part sounds right. Since floating frames should be able to overlap, I don't think that part is correct, unless you had something else in mind?

[jcguu95]

Since floating frames should be able to overlap, I don't think that part is correct, unless you had something else in mind?

Floating frames in different layers could overlap; that's fine. I'd be confused if floating frames in the same layer could overlap. In this later case, and with the 'unordered list part' being correct, how would you decide which one is visually shown on the top?

---

In any case, it would be nice to have a very clear object hierarchy specification (pictures are nice) first before we make any substantial change to the code. Thanks for your design!

[sdilts]

In this later case, and with the 'unordered list part' being correct, how would you decide which one is visually shown on the top?

Shoot, I forgot that we use a separate data structure for identifying where windows are located from the wlr_scene that defines the rendering order; in a pure wlroots only environment, you'd use wlr_scene_node_raise_to_top to define the order, but that's not how we are using it.

I'd honestly just go with what is easier. Famous last words, but switching between the two doesn't seem too hard if things are abstracted correctly.

[sdilts]

Here are some options:

In this option, we would have the ability to have multiple floating frames in a floating layer:

As slightly simpler option, there could only one floating frame per layer. This would simplify the rendering order and make it easier to have multiple floating layers, but could make managing the floating tree frames a bit harder. If the compromise of only having one frame tree per layer is acceptable, it's the option I'd try first.