Getting Boxes Done, the Code

As I mentioned before, I pretend ideas are on a virtual index card, and process them as I move them around organizational boxes. After creating a flowchart, I simplified it into more of a workflow:

What are the red letters, you ask? Those are the key bindings I use to move the virtual index cards from my inbox to other boxes (files or directories). You’ll see where I specify these and their implementation later.

Perhaps an example of using this workflow would be helpful.

I may notice that I’m running low on personal supplies, and quickly jot a note in Orgzly:

This puts a line in my Inbox:

* Buy toothpaste and eye drops

The next morning, as I’m reviewing the entries, I hit t to refile (move) the entry into my Task List. After I’ve been shopping, and later review the task list, I hit a to archive it, or c to archive it the completed file, which for me, is that day’s journal entry (however, you could specify a file like, accomplishments-yay-me.org). Completed tasks in org are typically archived and I discuss the details of this later on.

How about another example? The other day while noticing the large number of uncommitted text files (why yes, my org directories are under git control), I thought of an idea. I quickly kick off an Org capture and put another entry in my Inbox:

* Can I do a git commit automatically (at least, more regularly)?

When reviewing my Inbox the next day, I realize, while a good idea, I have no clue how to accomplish it. Since it isn’t ready, I refile it to my incubate file (by typing i). Returning to my incubating entries I hatch a plan (which in this example involves a little web searching). With a plan, the wish becomes a project, I update the entry:

* TODO Git commit on Sleep
  SCHEDULED: <2018-12-28 Fri>

  Certain directories under git-control often go weeks or months between
  commits, as "projects" containing text notes don't really have a *change* to
  commit. What about a checkpoint? Perhaps closing laptop or system sleeps, we
  commit all files.

  * On a Mac, checkout [[https://www.bernhard-baehr.de/][SleepWatcher]]
  * On Linux, checkout [[https://launchpad.net/ubuntu/+source/pm-utils][pm-utils]]

  Goal should be to have a script that both programs can run, e.g.

  #+BEGIN_SRC shell
      ~/bin/sleepwatcher -d -s ~/bin/on-sleep
  #+END_SRC

Note the TODO label and scheduled date, as these make it show on my agenda.

As a real project, I refile it from incubate to tasks using t, or if it’s a larger, involved project, to its own file in projects using m P. Completion of a project moves it again, but since this project contains notes I may have to reference later, I shuffle the entire file to my technical directory.

Why so many steps and so many files? In a large project management system, tasks are typically associated with states in a database where views present tasks in different states. I’m working with text files, so tasks in different states live in different files. Normally, this would be painful, but not with org-mode. Single key sequences shuffle the text to different files, allowing me to view state by simply viewing a file.

If this concept is clear, the remaining sections include my procedures (and Emacs functions) for processing ideas from inception to completion using the examples described above.

See all those destinations in the workflow illustration above? Let’s create constant variables, where the functions can use them later:

(defvar org-default-projects-dir   "~/projects"                     "Primary GTD directory")
(defvar org-default-technical-dir  "~/technical"                    "Directory of shareable notes")
(defvar org-default-personal-dir   "~/personal"                     "Directory of un-shareable, personal notes")
(defvar org-default-completed-dir  "~/projects/trophies"            "Directory of completed project files")
(defvar org-default-inbox-file     "~/projects/breathe.org"         "New stuff collects in this file")
(defvar org-default-tasks-file     "~/projects/tasks.org"           "Tasks, TODOs and little projects")
(defvar org-default-incubate-file  "~/projects/incubate.org"        "Ideas simmering on back burner")
(defvar org-default-completed-file nil                              "Ideas simmering on back burner")
(defvar org-default-notes-file     "~/personal/general-notes.org"   "Non-actionable, personal notes")
(defvar org-default-media-file     "~/projects/media.org"           "White papers and links to other things to check out")

Why yes, if you steal my code, you’ll want to change those values.

Oh, and since some of these are pretty cool and could have items that would show up on an agenda, we may want to add them to org-agenda-files automatically:

(add-to-list 'org-agenda-files org-default-inbox-file)
(add-to-list 'org-agenda-files org-default-tasks-file)

Or, you can use code like this to add all files in the projects directory as potential agenda files:

(setq org-agenda-files (list org-default-projects-dir))
(setq org-agenda-file-regexp "^[a-z0-9-_]+.org")

However, let’s begin by keeping it simple:

(dolist (agenda-file (list org-default-inbox-file
                          org-default-tasks-file
                          org-default-incubate-file
                          org-default-notes-file))
  (add-to-list 'org-agenda-files (expand-file-name agenda-file)))

Everything begins as an entry in the Inbox (org-default-inbox-file) that I call breathe.org (seemed like a good filename to avoid feeling overwhelmed), so I need a flexible capturing entry to make it easy to create entries in that file:

(add-to-list 'org-capture-templates
             `("t" "Task Entry"        entry
               (file ,org-default-inbox-file)
               "* %?\n:PROPERTIES:\n:CREATED:%U\n:END:\n\n%i\n\nFrom: %a"
               :empty-lines 1))

Calling org-capture (which most people bind to C-c c, but Spacemacs has as SPC C c) might look like the following screenshot:

Note: With Orgzly (or other mobile org-specific application) configured to use this file as its default destination for all new ideas or todos, your mobile device will match your system.

As a temporary holding spot for all incoming tasks, notes and ideas, the goal is to keep the Inbox empty. Every day, I review the collected entries, and adjust and move them to appropriate locations.

I have created a little helper system for processing these ideas through the workflow shown in the flowchart above. Here is a screenshot of it in action where I am currently determining what to do with an entry captured earlier. The menu at the bottom of the screen matches my workflow shown earlier:

Note: The screenshot above may not match the code you’ll see below. Since this essay is really just a literate programming approach to generating the code I actually use, I’ll be changing the code, but may not re-render the images. Sorry.

While Emacs has many options for calling functions, I’ve chosen to use the Hydra project to present a focused collection of keybindings to functions. Originally I made a minor mode, but found Hydra’s hint screen and repetitive key calls easier (once I start the hydra, I can simply type i multiple times to refile multiple entries to the incubation chamber).

Here is the definition for the Hydra, and you’ll notice that after the hint display, it is just a list of keybindings to functions:

(defhydra hydra-org-refiler (org-mode-map "C-c s" :hint nil)
    "
  ^Navigate^      ^Refile^       ^Move^           ^Update^        ^Go To^        ^Dired^
  ^^^^^^^^^^---------------------------------------------------------------------------------------
  _k_: ↑ previous _t_:   tasks     _m X_: projects  _T_: todo task  _g t_: tasks    _g X_: projects
  _j_: ↓ next     _i_/_I_: incubate  _m P_: personal  _S_: schedule   _g i_: incubate _g P_: personal
  _c_: archive    _p_:   personal  _m T_: technical _D_: deadline   _g x_: inbox    _g T_: technical
  _d_: delete     _r_:   refile                   _R_: rename     _g n_: notes    _g C_: completed
  "
    ("<up>" org-previous-visible-heading)
    ("<down>" org-next-visible-heading)
    ("k" org-previous-visible-heading)
    ("j" org-next-visible-heading)
    ("c" org-archive-subtree-as-completed)
    ("d" org-cut-subtree)
    ("t" org-refile-to-task)
    ("i" org-refile-to-incubate)
    ("I" org-refile-to-another-incubator)
    ("p" org-refile-to-personal-notes)
    ("r" org-refile)
    ("m X" org-refile-to-projects-dir)
    ("m P" org-refile-to-personal-dir)
    ("m T" org-refile-to-technical-dir)
    ("T" org-todo)
    ("S" org-schedule)
    ("D" org-deadline)
    ("R" org-rename-header)
    ("g t" (find-file-other-window org-default-tasks-file))
    ("g i" (find-file-other-window org-default-incubate-file))
    ("g x" (find-file-other-window org-default-inbox-file))
    ("g c" (find-file-other-window org-default-completed-file))
    ("g n" (find-file-other-window org-default-notes-file))
    ("g X" (dired org-default-projects-dir))
    ("g P" (dired org-default-personal-dir))
    ("g T" (dired org-default-technical-dir))
    ("g C" (dired org-default-completed-dir))
    ("[\t]" (org-cycle))
    ("s" (org-save-all-org-buffers) "save")
    ("<tab>" (org-cycle) "toggle")
    ("q" nil "quit"))

We just need to bind something like C-c s to call hydra-org-refiler/body. Or, under Spacemacs with a leader key:

(spacemacs/set-leader-keys-for-major-mode 'org-mode "r" 'hydra-org-refiler/body)

Or Doom’s leader key:

(map! :map org-mode-map :localleader
      :desc "boxes" "B" 'hydra-org-refiler/body)

Now I have a system for calling all these mover and shaker functions. Some functions, like org-schedule, come with Org, while others, I’ll need to write.

While reviewing the collected ideas in my Inbox, I often need to tidy them before moving them around. Add a TODO label to each task with T in my hydra, as well as schedule a date with an S (as a task without due date is just a wish). Before I move the subtree, I may need to change the header’s text (which I added to the hydra with an R key):

(defun org-rename-header (label)
  "Rename the current section's header to LABEL, and moves the
point to the end of the line."
  (interactive (list
                (read-string "Header: "
                             (substring-no-properties (org-get-heading t t t t)))))
  (org-back-to-heading)
  (replace-string (org-get-heading t t t t) label))

Completed tasks are typically archived, using org-archive-subtree. This is a brilliant feature for keeping your notes relevant (as long as this feature doesn’t surprise you by an accidental fat-fingering of the keyboard):

To archive to a separate file like this, set the variable, org-archive-location to "%s_archive".

However, in my world tasks marked as completed originate from multiple file locations; I would rather move these finished tasks to my journal. I use org-journal with a file for each day; for instance, 20180113 (without the .org extension…see my journal helper functions if you are curious).

This function archives a subtree to today’s journal entry (marking the task completed in the process):

(defun org-archive-subtree-as-completed ()
  "Archives the current subtree to today's current journal entry."
  (interactive)
  ;; According to the docs for `org-archive-subtree', the state should be
  ;; automatically marked as DONE, but I don't notice it, so let's force:
  (when (not (equal "DONE" (org-get-todo-state)))
    (org-todo "DONE"))

  (let* ((org-archive-file (or org-default-completed-file
                               (todays-journal-entry)))
         (org-archive-location (format "%s::" org-archive-file)))
     (org-archive-subtree)))

The following function returns the filename of today’s journal entry (which I can use as a refile destination):

(defun todays-journal-entry ()
  "Return the full pathname to the day's journal entry file.
Granted, this assumes each journal's file entry to be formatted
with year/month/day, as in `20190104' for January 4th.

Note: `org-journal-dir' variable must be set to the directory
where all good journal entries live, e.g. ~/journal."
  (let* ((daily-name   (format-time-string "%Y%m%d"))
         (file-name    (concat org-journal-dir daily-name)))
    (expand-file-name file-name)))

We now have a Hydra that easily kicks off functions that help keep my task inbox manageable. Each morning, I would like to start an environment when my Inbox is loaded and ready for refiling. The following function does that by calling functions I would normally do manually:

(defun org-boxes-workflow ()
  "Load the default tasks file and start our hydra on the first task shown."
  (interactive)
  (let ((org-startup-folded nil))
    (find-file org-default-inbox-file)
    (delete-other-windows)
    (ignore-errors
      (ha/org-agenda))
    (delete-other-windows)
    (split-window-right-and-focus)
    (pop-to-buffer (get-file-buffer org-default-inbox-file))
    (goto-char (point-min))
    (org-next-visible-heading 1)
    (hydra-org-refiler/body)))

This assumes that I have a special agenda display:

(defun ha/org-agenda ()
  "Displays my favorite agenda perspective."
  (interactive)
  (org-agenda nil "a")
  (get-buffer "*Org Agenda*")
  (execute-kbd-macro (kbd "A t")))

I hope you can grab and modify this code to suit your own needs and whims. I may attempt to make the code more general and useful to others, but at the moment, I’d like to see if this code has any legs.

Let me know if you have any improvements you’d make.