Adapt Stumpwm's `defcommand` to Mahogany

[sdilts]

Now that we are able to detect and execute keybindings, we need a way to define and execute commands. In stumpwm, this is done via the defcommand macro.

Here are the requirements for right now:

• We need to be able to lookup the command via a string or its symbol name.
• Whatever function that gets generated by the defcommand macro needs to accept the seat and key sequence that was used to trigger the command as keyword arguments. If the user doesn't want to use this info, it should not affect how they write the command and they shouldn't need to specify those arguments when they write the function signature.
• Any commands registered via defcommand should have docstrings and other documentation easily available so we can build a help command.
• The implementation shouldn't expose any non-trivial lexical variables as a part of its interface so it's easier to refactor without breaking a public API.

In addition, there needs to be a way to interactively query the users for the parameters needed to run the command. It should also be possible to run the command programmatically so that commands can be reused.

[szos]

One thing that i think could be nice to have both for QoL and accessibility purposes is localization. I.e. have a command directly referenceable via its symbol, but have a translation table mapping arbitrary string names to a command. Likewise for the various names that may be prompted for.

If i remember right a string table is what stumpwm does internally for commands, but there isnt an easy way to define a localization file for stumpwm, and there is no possibility for e.g. accepting izquierda instead of left when prompting for a direction.

[jcguu95]

How about emacs' like which-key implementation?
justbur/emacs-which-key: Emacs package that displays available keybindings in popup

By using existing dialog-like software we don't have to implement the drawing in mahogany.

[sdilts]

This issue is specifically about the macro that defines commands, not how users execute them. See #33 and #51, and maybe even #85. Having an equivalent to which-key is a great idea, and the implementation of how commands are defined shouldn't block it from being implemented.

I'll update the issue text to make this more clear.

[yana-mim]

I've written out a potential starting point for this issue. It may be non-applicable (parts of it certainly are because they are sbcl specific, but mahogany already uses cffi so wrapping e.g. unix-pipe portably shouldnt be too much strain), but I thought it may be of interest. This example uses foot with fzf to prompt the user for input but leaves things open so other input methods can be used. Foot/fzf was used because things like rofi dont function on mahogany yet and mahogany doesnt have its own input bar like stumpwm does.

This example uses an (unfinished) interactive command system I've written which is available here: https://git.sr.ht/~yana/testing-commands

Here is the gist: https://gist.github.com/yana-mim/8e8721e67044da0239183948d80b6561

of note is that this requires making commands run outside the main thread, in order to map the foot window to receive input.

video demo:

This shows killing the current foot terminal so no windows are open, typing the key binding C-t M-x to run execute-extended-command, selecting run shell command, and entering the string foot to launch a foot terminal, which then opens.

foot-mx-example.mp4

[sdilts]

@yana-mim , that's fantastic! From a quick look, I like what I see.

If I'm understanding the gist correctly, could what you are doing with the pipes and file descriptors be handled by uiop:run-program's input and output arguments? I'll admit that I haven't seen something like that code before, and I had to read it a few times to understand it.

[yana-mim]

@sdilts Im glad you like it! The interactive command system isn't finished but is close to an initial minimal-stable version.

If I'm understanding the gist correctly, could what you are doing with the pipes and file descriptors be handled by uiop:run-program's input and output arguments? I'll admit that I haven't seen something like that code before, and I had to read it a few times to understand it.

I dont think the pipes and file descriptors can be handled by the input and output arguments, but I'm not sure.

For posterity's sake and to save other readers from taking the time to understand it (it is convoluted shell weirdness):

The whole program invoked is foot -- sh -c "fzf --print-query < /proc/4321/fd/1234 | tail -1 > /proc/4321/fd/1235". This opens a foot terminal window, tells it to run sh, and tells sh to take a shell line to run instead of operating interactively. The shell line sh is running is the portion in doublequotes (fzf ...). This needs to obtain input somehow to generate completions, and I don't think it can take it from the foot terminals input. To my understanding launch-program will bind fd 0 of the program run to its input argument. Perhaps we can get that fd to fzf somehow, but we would need to ensure that the fd is never read from by foot. The bigger issue is the output; to my recollection if the output is not a special file descriptor then the terminal and/or shell (I don't quite remember which) wont operate quite right. Though perhaps I'm misremembering.

Regardless, this was intended as a proof of concept, not a final solution. Ideally we would get the necessary wayland protocols implemented to run rofi or similar programs, instead of popping up a whole terminal emulator just to get input from the user (though of course, if someone prefers that they should be able to do so, its part of why having general input methods was desirable for me).

[yana-mim]

Given what I've found from playing around with this attempt at command invocation, this is a proposal for one way of handling the running of commands. This proposal divides the concept of "run a command" into three parts: invocation (e.g. via key press), interactive argument acquisition, and execution. This is not directly related to this issue as described in the initial comment, but seemed related enough to merit being put here instead of in its own issue.

Considerations: Mahogany's main thread should be kept free as much as possible, to allow events to be processed. Thus commands should be executed in their own threads. However commands should be executed sequentially, so some kind of
scheduling (either explicit or implicit) is needed.

Proposal:

• Have an execution worker thread that dequeues from a command execution queue, runs a thread with the command to execute, joins that thread (waits for it to finish), and loops back to the beginning.
• Have an invocation worker thread which dequeues from a command invocation queue, interactively obtains the commands arguments, enqueues the command with arguments to the command execution queue, and loops back to the beginning.
• Have a function add-command-to-queue which may be called programmatically by the user which enqueues a command at either the beginning or the end of the command invocation queue.
• Have a function invoke-command-from-main-thread which is used by the key press handler to enqueue commands at the end of the command invocation queue

This proposal allows for several things:

• If a commands execution takes a long time, all following commands may have their arguments interactively obtained.
• If a commands execution signals an error, the execution worker thread wont stop working. Alternatively, instead of spinning up a new thread, a top-level error handler for the worker thread could be established which catches all errors and messages the user that the command in question signaled an unhandled error. The error handler approach would also give the benefit of allowing an interactive debugger to be opened when an unhandled error is signaled, preventing the execution of another command until that error is handled.

Alternatively, command argument acquisition and execution could occur in the same thread, which would prevent arguments from being acquired until the previous command has finished running, and reduce the number of queues needed to one.

This proposal requires a thread safe queue.

A psuedocode example:

(defun execution-worker-thread-function ()
  (loop
    (let* ((to-execute
             (with-queue-locked-for-dequeue (*command-execution-queue*)
               (when (queue-empty-p *command-execution-queue*)
                 ;; Unlock the queue for enqueueing, become awoken when the
                 ;; enqueueing thread signals that something has been enqueued
                 ;; to the thread
                 (queue-wait-for-new-item *command-execution-queue*))
               (dequeue *command-execution-queue*)))
           (thread
             (bt2:make-thread (execution-thunk to-execute)
                              :name (execution-name to-execute)
                              :initial-bindings
                              (cons (cons *current-key-sequence*
                                          (execution-key-sequence to-execute))
                                    (cons *current-seat*
                                          (execution-seat to-execute))
                                    bt2:*default-special-bindings*))))
      (bt2:join-thread thread))))

(defun invocation-worker-thread-function ()
  (loop
    (let* ((to-invoke
             (with-queue-locked-for-dequeue (*command-invocation-queue*)
               (when (queue-empty-p *command-invocation-queue*)
                 (queue-wait-for-new-item *command-invocation-queue*))
               (dequeue *command-invocation-queue*)))
           (command (invocation-command to-invoke))
           (arguments (interactively-obtain-arguments command))
           (to-execute (list (format nil "Execution thread for ~A"
                                     (command-name command))
                             (lambda ()
                               (call-command-interactively-with-arguments
                                command arguments))
                             (invocation-key-sequence to-invoke)
                             (invocation-seat to-invoke))))
      (with-queue-locked-for-enqueue (*command-execution-queue*)
        (enqueue to-execute *command-execution-queue*)))))

(defun add-command-to-queue (command first?
                             &key key-sequence (seat *current-seat*))
  (with-queue-locked-for-enqueue (*command-invocation-queue*)
    (if first?
        (enqueue-at-beginning (list command key-sequence seat)
                              *command-invocation-queue*)
        (enqueue (list command key-sequence seat) *command-invocation-queue*))))

(defun invoke-command-from-main-thread (command key-sequence seat)
  (add-command-to-queue command nil :key-sequence key-sequence :seat seat))

[sdilts]

@yana-mim, #113 is at the point where rofi will display and receive mouse input, but it's pretty janky. It might be enough to play around with what you are working on with the command system though.

[sdilts]

It looks like we can break up running a command into two steps:

1. The interactive arguments, if any, need to be gathered.
2. The command needs to be ran.

This seems pretty obvious, but I think it's important for ensuring that we can run commands interactively as well as on there own in a script. Here are two approaches that might work:

• Make the arguments to the command functions optional, and use some mechanism inside the function to determine if the arguments should be pulled interactively.
• Gather the arguments before calling the function, so that the command function can be normal and called other places.

I'm more in favor of gathering the arguments before calling the function as it will lead to a better experience when calling the functions non-interactively. @yana-mim, how does this line up with the library you are working on?

[yana-mim]

I'm more in favor of gathering the arguments before calling the function as it will lead to a better experience when calling the functions non-interactively. @yana-mim, how does this line up with the library you are working on?

Thats how the library I'm working on handles interactive arguments; commands work like normal generic functions with interactive arguments only being obtained through a call to call-command-interactively.