An Alternate Completing Read

To resuscitate my blog, I thought I would join this month’s Emacs Carnival, with a suggested topic of Completion. Let’s talk about completion in my favorite subject, Emacs Lisp, as I love to show how simple and fun Lisp can be. This essay assumes you know the basics (and if not, we have great resources for you).

Interactive

Other than editing a buffer, the next most common user interface in Emacs happens in the mini-buffer at the bottom of the screen. Mickey Petersen has a great essay on completion in the mini-buffer, so let me dive into the Lisp-specific parts. You may know about the interactive macro that both puts a Lisp function on the magical M-x list of commands, but also interactively prompts the user for input on each of the function’s parameters. Let’s try writing a function that will prompt “the user” (and since Emacs is personal, I actually mean, you) for values for the parameters:

(defun completion-experiment-1 (a-string a-number a-file)
  "Prints interactive queries in the mini-buffer after completion."
  (interactive "sString: \nnNumber: \nfFile: ")
  (message "You entered: %s - %f - %s"
           a-string a-number a-file))

If you have never played around with interactive Emacs functions before, let me give you three approaches for running that function (begin by copying the code into your operating system’s clipboard), then:

Type M-S-; (the Option key with a colon character), and paste it (C-y) and press Return to evaluate it.
Create a new buffer, or switch to the Scratch buffer with M-x and typing: scratch-buffer. Next paste (C-y) and type C-x C-e to evaluate it.
If you are reading this essay inside Emacs with EWW, move the cursor after the last paren, and type C-x C-e to evaluate it.

At this point, you can run this function by typing M-x and entering completion-experiment-1. It will prompt you for three values. Let’s tease apart that cryptic parameter to interactive. We separate each prompt with a \n character (the character return), and we should have enough prompts to match the parameters to the function. For each segment, the first character defines the input type. For instance, n is for a number and f is for a file (see the help for interactive for all options).

Prompting for Values

Notice the number prompt insisted the input be recognizable as a number, so while trying to type whale resulted in an error, it would accept, -2.45e6 for the exponential way to enter negative two and a half million. However, the string prompt asked for a string, but did little validation.

The interactions in the magic prompt in interactive come from helper functions. Let’s try this:

(defun completion-experiment-2 ()
  "Prompt and print three values."
  (interactive)
  (let ((a-string (read-string "A String: "))
        (a-number (read-number "A Number: "))
        (a-file   (read-file-name "A File: ")))
    (message "You entered: %s - %f - %s" a-string a-number a-file))

This function acts like our initial function, but we control the prompts outside of the interactive helper.

Prompting with Choices

Instead of asking for any string with read-string, we could limit the user to select from a list of choices using completing-read. The following code creates a list of three choices and uses that as the list of options:

(let ((choices (list "First" "Second" "Third")))
  (completing-read "Choose: " choices))

Evaluating that code could look like this in the mini buffer area:

I use the word could deliberately, as packages like ivy, helm or vertico affect the user interface details. Add a pattern for fuzzy matching like orderless, to make selecting an item from the list easier:

Interactive and Completing Read

I know what you’re thinking: Wouldn’t it be nice if the interactive macro had a way to integrate with the completing-read function.

instead of passing in a string parameter, interative can take a list of functions, where each element in the list is used to populate the parameters. For instance:

(defun completion-experiment-1 (name fruit)
  "Select a fruit for the salad."
  (interactive (list (read-string "Your name: ")
                     (completing-read "Fruit: " '(apple orange banana))))
  (message "%s added %s to the salad." name fruit))

The completing-read function can take other types of lists, like associative lists which is a list of lists (recall that '(…) is almost a shortcut for (list …)¹ when you look at this example):

(let ((choices '(("First"  first-choice)
                 ("Second" second-choice)
                 ("Third"  third-choice))))
  (completing-read "Choose: " choices))

While completing-read accepts such a beastie, it only displays, and returns, the first element.

Completing Read and the Second Value

In some of my Lisp programs, I have a list of choices that make sense for my code, but would like to display more user-friendly version in my completing-read call. For example, I have a list of hostnames I want to select from, but I want the result to be the IP address:

(list ("Glamdring" . "192.168.5.12")
      ("Orcrist"   . "192.168.5.10")
      ("Sting"     . "192.168.5.220")
      ("Gungnir"   . "192.168.5.25"))

in the example above, I would like to select the string, Orcrist, but have the call return the string, 192.168.5.10.

When we use a list of tuples (Lisp calls these pairs, cons-cells), we use alist-get with the key from the result of calling completing-read to get the value or second part. We also need to pass in the function, equal to compare strings:

(let* ((choices '(("First"  . first-choice)
                 ("Second" . second-choice)
                 ("Third"  . third-choice)))
       (choice (completing-read "Choose: " choices)))
  (alist-get choice choices nil nil 'equal))

Replacing Completing Read

Adding this code to every function that needs it isn’t bad, but I want a helper function, alt-completing-=read that I could add to the interactive macro. For instance, this is what I would like to see:

(defvar favorite-hosts '(("Glamdring" . "192.168.5.12")
                         ("Orcrist"   . "192.168.5.10")
                         ("Sting"     . "192.168.5.220")
                         ("Gungnir"   . "192.168.5.25")))

(defun favorite-ssh (ip-address)
  "Start a SSH session to a given HOSTNAME."
  (interactive (list (alt-completing-read "Host: " favorite-hosts)))
  (message "Rockin' and rollin' to address, %s" ip-address))

Which could look like this in the mini-buffer:

Within the code, the interactive function would set the ip-address variable to 192.168.5.20.

I also want this new alt-completing-read function to be a drop-in replacement for completing-read, so the function should accept the same parameters, and pass them into the completing-read function.

(defun alt-completing-read (prompt collection &optional predicate
                                   require-match initial-input
                                   hist def inherit-input-method)
  (let ((choice
          (completing-read prompt collection predicate require-match
                           initial-input hist def inherit-input-method)))
    ;; ...
    choice))

I want this alt-completing-read function to take different types of collections, including:

Associative lists, e.g.: ((choice-1 result-1) (choice-2 result-2) …)
Associative lists of tuples, e.g.: ((choice-1 . result-1) (choice-2 . result-2) …)
Property lists, e.g.: (choice-1 result-1 choice-2 result-2 …)
Hash tables of key/values

While we have a predicate for figuring out a hash table, hash-table-p, we don’t have functions to distinguish between associative and property lists, so I will need these predicate functions:

(defun assoc-list-p (obj)
  "Return t if OBJ is an _associative list_.
Where OBJ is both a list, and its first element is a cons-cell."
  (and (listp obj) (consp (car obj))))

(defun prop-list-p (obj)
  "Return t if OBJ is a _property list_.
Where OBJ is both a list, and its first element is not a list."
  (and (listp obj) (not (listp (car obj)))))

These functions aren’t general and valid in all cases, so I may not want to pollute the namespace with them. Instead, I might use the flet function from the Common Lisp package to create functions that are internal and only available inside the alt-completing-read function. This will look like:

;; This `cl-flet' creates internal helper functions, predicates
;; to decide if 'obj' is an associative list or a property list:
(cl-flet ((assoc-list-p (obj) (and (listp obj) (consp (car obj))))
          (prop-list-p (obj)  (and (listp obj) (not (listp (car obj))))))

Another caveat. The completing-read function doesn’t accept a property list, so if the collection parameter is a property list, I need to convert it:

(when (prop-list-p collection)
  (setq collection (seq-partition collection 2)))

Alright, so if alt-completing-read takes a collection, and returns the first element in the data structure, we need to have different ways to pull the second element from it. When choice is the result from completing-read and collection is our parameter, we can use cond as a switching statement, like:

(cond
 ((assoc-list-p collection) (alist-get choice collection def nil predicate))
 ((prop-list-p collection)  (plist-get choice collection predicate))
 ((hash-table-p collection) (gethash choice collection))
 (t                         choice))

Let’s stitch these ideas into the fully functional … er, function:

(defun alt-completing-read (prompt collection &optional predicate
                                   require-match initial-input
                                   hist def inherit-input-method)
  "Calls `completing-read' but returns the value from COLLECTION.

Simple wrapper around the `completing-read' function that assumes
the collection is either an alist, a plist, or a hash-table, and
returns the _value_ of the choice, not the selected choice. For
instance, give a variable of choices like:

    (defvar favorite-hosts '((\"Glamdring\" . \"192.168.5.12\")
                             (\"Orcrist\"   . \"192.168.5.10\")
                             (\"Sting\"     . \"192.168.5.220\")
                             (\"Gungnir\"   . \"192.168.5.25\")))

We can use this function to `interactive' without needing to call
`alist-get' afterwards:

    (defun favorite-ssh (hostname)
      \"Start a SSH session to a given HOSTNAME.\"
      (interactive (list (alt-completing-read \"Host: \" favorite-hosts)))
      (message \"Rockin' and rollin' to %s\" hostname))"

  ;; This `cl-flet' creates internal helper functions, predicates
  ;; to decide if 'obj' is an associative list or a property list:
  (cl-flet ((assoc-list-p (obj) (and (listp obj) (consp (car obj))))
            (prop-list-p (obj)  (and (listp obj) (not (listp (car obj))))))

    ;; completing-read can't handle property lists, so we convert it
    ;; to an associative list by making a list of two-element lists:
    (when (prop-list-p collection)
      (setq collection (seq-partition collection 2)))

    (let* ((choice
            (completing-read prompt collection predicate require-match
                             initial-input hist def inherit-input-method))
           ;; Note that `choice' will always be a string, so we need to use string-equal
           (results (cond
                     ((assoc-list-p collection) (alist-get choice collection def nil #'string-equal))
                     ((prop-list-p collection)  (plist-get choice collection #'string-equal))
                     ((hash-table-p collection) (gethash choice collection))
                     (t                         choice))))
      (message "Choice: %s / Result: %s" choice results)
      (if (listp results) (car results) results))))

As mentioned before, Emacs supplies two types of associative lists, one where the elements are two-element lists, and the other are tuples joined with cons. And alist-get behaves slightly differently. For instance:

(alist-get 'two '((one . 1) (two . 2) (three . 3)))

Returns 2 as a single element, but:

(alist-get 'two '((one 1) (two 2) (three 3)))

Returns a list with the element 2. To address this, at the end of our alt-completing-read function, we check if results is a list and if so, return its first element.

Testing the Alternate Completing Read

Try it out:

(defvar favorite-hosts '(("Glamdring" . "192.168.5.12")
                         ("Orcrist"   . "192.168.5.10")
                         ("Sting"     . "192.168.5.220")
                         ("Gungnir"   . "192.168.5.25")))

(alt-completing-read "Host: " favorite-hosts)

Where selecting “Sting” returns “192.168.5.220”.

Love to write some unit tests to verify this, but since alt-completing-read calls the interactive function, completing-read, I need to mock this function. In Emacs Lisp, we do that with our old friend, flet:

(cl-flet ((completing-read (prompt collection &rest args)
            ;; Instead of asking the user, just return a value.
            "Second"))
  (completing-read "Prompt"
                   '(("First"  first-choice) ("Second"  second-choice) ("Third"  third-choice))))

This code doesn’t prompt the user, but simply returns the string, "Second".

Let’s write some tests to verify to show the possibilities:

(ert-deftest alt-completing-read-test ()
  (flet ((completing-read (prompt collection &rest args)
                          ;; Instead of asking the user, just return a value.
                          "Second"))

    (lexical-let ((data-set '(("Associative List, form 1:" .
                               (("First" . first-choice) ("Second" . second-choice) ("Third" . third-choice)))

                              ("Associative List, form 2:" .
                               (("First"  first-choice) ("Second"  second-choice) ("Third"  third-choice)))

                              ("Property List:" .
                               ("First" first-choice "Second" second-choice "Third" third-choice))

                              ("Hash Table:" .
                               #s(hash-table size 3 test equal data ("First" first-choice "Second" second-choice "Third" third-choice))))))
      (dolist (data data-set)
        (let ((prompt (car data))
              (choices (cdr data)))
          (should (equal (alt-completing-read prompt choices) 'second-choice)))))))

Whew.

Summary

This deep dive into Emacs’ completion features in Lisp functions began with a brief review of the interactive macro populates an interactive function’s parameters, and how completing-read can be use to select an option from a list.

We ended with us writing an alt-completing-read function you can use when you want to have the user…er, you, select from a list, but have the result from that list a different, a matching value, like selecting a host, but returning it’s IP address, or selecting a USB driver by name, but returning its ID value.

Footnotes:

While more pendantic than I wanted for this essay, but the list way to make a list is evaluated. For instance:

(let ((a 1)
      (b 2)
      (c 3))
  (list a b c))

Returns a list of (1 2 3) as the a and b and c are evaluated. The quote version of making a list, however, is not evaluated. For instance:

'(a b c)

Returns a list of the symbols, (a b c). Want both? Use the backtick, for instance:

(let ((b 2))
  `(a ,b c))

Returns a list of (a 2 c) where the b is evaluated.