Open Thoughts https://judah.freedomland.xyz// en Wed, 27 Dec 2023 00:00:00 -0500 Wed, 27 Dec 2023 22:58:08 -0500 weblorg 0.1.0 (https://emacs.love/weblorg) Judah Sotomayor https://judah.freedomland.xyz//media/img/8bitme.png Blog Author https://judah.freedomland.xyz// In Pursuit of an Efficient Org-Agenda https://judah.freedomland.xyz//posts/efficient-org-agenda.html author@mail.com (Blog Author) https://judah.freedomland.xyz//posts/efficient-org-agenda.html Wed, 27 Dec 2023 00:00:00 -0500

Table of Contents

After beginning my Emacs and Org-Mode adventures I quickly realized the insufficiency of the standard agenda. It is wonderful at producing a list, but I am quickly approaching over 1000 notes. The agenda tooling is incapable of searching this many files efficiently, and it took over thirty seconds to generate an agenda.

This was intolerable. I had two options:

  1. Place all my TODO items into a single file.
  2. Narrow the number of files the agenda mechanism needs to search

I find the first option undesireable for reasons I mentioned in my post about zettelkasten tools. I like to have my todo items mixed with the context where they were born. A student of the Getting Things Done methodology might ask, "Doesn't this violate the central todo-list principle?" Yes, it does. Org-agenda allows me to have my cake and eat it too. I can create a centralized todo-list out of all my todo items, and then immediately jump into the context of my next task.

It took several iterations of configuration to reach a seamless workflow. If you'd like to jump straight to the final setup, here's the link.

1. First Iteration

The first piece of tooling I used came from this post and this Gist. Essentially it creates a function vulpea-project-files that can easily query for a tag, and then sets the org-agenda files to all the files containing this tag.

Coupled with a helper-function that adds the tag hastodos to any file containing a TODO entry, it functioned well.

(defun vulpea-project-files ()
  "Return a list of note files containing 'hastodos' tag." ;
  (seq-uniq
   (seq-map
    #'car
    (org-roam-db-query
     [:select [nodes:file]
      :from tags
      :left-join nodes
      :on (= tags:node-id nodes:id)
      :where (like tag (quote "%\"hastodos\"%"))]))))
(setq org-agenda-files (vulpea-project-files))

This solution is great because it is pure elisp. Anywhere you're running Emacs it should function just fine. It also works well with one of my needs: transparent file encryption. Emacs has extensions for both gpg and age encryption that allow files to be transparently encrypted and decrypted. This solution can make use of that where a standard grep could not.

2. Second Iteration

Eventually, I grew tired of seeing the entire solution in my config file. All told, it is about 170 lines. This is a lot for a small utility!

I decided to try using ripgrep to fix this:

(defun set-org-agenda-files-ripgrep ()
    (setq org-agenda-files (split-string (shell-command-to-string "rg -torg -l TODO /home/user/org"))))

This worked nicely. It takes just a few lines, runs just as fast–probably faster–than pure elisp, and is much easier to read and understand.

The only difficulty is encryption. ripgrep does not operate on open emacs buffers, instead it operates on what is saved to disk. Naturally, this means that it only sees the encrypted files.

3. Final

The solution to file encryption is ripgrep preprocessing. I owe this solution to this reddit post.

Ripgrep can run a command (or shell script!) to files before processing. While the original poster was using gpg to encrypt files, just a few modifications allowed me to use age.

#!/usr/bin/env zsh
case "$1" in
*.age)
    # The -s flag ensures that the file is non-empty.
    if [ -s "$1" ]; then
        exec /usr/bin/age --decrypt -i ~/.age/personal $1
    else
        exec cat
    fi
    ;;
*)
    ;;
esac

This script operates on all .age files, decrypting them to stdout. When using a preprocessor, ripgrep will simply search the output of the command.

I often have a mix of files in my notes directory, so I use filetypes to restrict ripgrep. This means I have to add a new filetype to allow .age files through the filter.

rg --type-add 'aorg:*.org.age' \
    -torg -taorg \
    --pre ~/age-preprocessor.zsh --pre-glob '*.age' -l TODO /home/user/org

Finally, we have our completed function:

(defun set-org-agenda-files-ripgrep ()
    (setq org-agenda-files (split-string (shell-command-to-string "rg --type-add \'aorg:*.org.age\' -torg -taorg --pre ~/age-preprocessor.zsh --pre-glob \'*.age\' -l TODO /home/user/org "))))

Hope this helps!

]]>
Reflections on a year of Zettelkasten: Tooling https://judah.freedomland.xyz//posts/reflections-zettelkasten-tooling.html author@mail.com (Blog Author) https://judah.freedomland.xyz//posts/reflections-zettelkasten-tooling.html Fri, 22 Dec 2023 00:00:00 -0500 Back in 2021 I began a zettelkasten using guides from zettelkasten.de. It has grown to over 723 notes in less than two full years.

image

I'll cover this topic in several posts. This first post will discuss the tools I've used to grow and maintain my second brain.

I have shifted between note-taking systems multiple times between now and the genesis of my Zettelkasten. Different strengths and weaknesses have left me uncomfortable in every system, up until the last: Emacs. Before I get to that, I'll cover the pieces of the system, what's changed, and what hasn't.

Text editing

Text editing is at the heart of the zettelkasten. A proper text editor will support links, backlinks, note search, and tagging. It acts as the gateway into your notes, the control panel for your interaction and editing.

Vimwiki

The first version was the amazing vimwiki, a system simple and elegant. It provided most of the features needed for a zettelkasten, and supported standard markdown. During this period I grew comfortable with Vim, which I consider the premier editing experience.

I began to use Taskwarrior, an amazing command-line task management tool. Taskwiki is another amazing vim extension that maps tasks between vimwiki and taskwarrior, allowing you to create "viewports" with various Taskwarrior filters. The combination of Vimwiki and Taskwarrior is exceptionally comfortable, and taught me my first important lesson: tasks and data are inseparable. It is impossible to provide sufficient information to do a task if that task is separated from its context, its data and metadata.

Eventually I grew tired of the limitations of editing text in a terminal. Vim doesn't support graphical things very well, and I wanted to view images inside my notes. While various extensions solve this problem, and adjacent problems, I had at this time noticed dendron.

Dendron

Dendron features a mixed hierarchical/networked note setup, which (according to the creators) scales well with heavy use. While I still have not reached the ~10k note count that the authors asserted Dendron would survive, the structure appealed to me because at the time I was working with several sorts of knowledge that a hierarchy represents well–things like programming language concepts in multiple languages. Dendron also features a note-graph, which represents both the hierarchy of notes and the links between notes. This feature was most attractive to me, as inside Vim I was having trouble visualizing my entire zettelkasten.

Overall, the featureset provided enough value that I made the change. I adapted all of my vimwiki links (or most!) to Dendron's format, and began to implement some of Dendron's more advanced features in my daily use. I installed VSCode and a Vim emulation plugin, and away I went.

The first snag I ran into was the limitations of Vim emulation plugins, so I switched that plugin to vscode-neovim, which embeds neovim into VSCode. I learned a value here: closer to Vim is more comfortable. Vim truly allows you to "edit text at the speed of thought". Its modal paradigm turns the keyboard into a uniquely fluid control panel, optimized for text editing. Extensions often miss features or have subtle bugs. The closer I can get to actual Vim, the more comfortable I am when it is time to edit.

Vim (V2)

After Vim emulation, VSCode+Dendron's most significant problem was its lack of serious todo functionality. Taskwarrior and its sister program Timewarrior had previously handled my todo lists, while Vimwiki handled documentation and project management. Dendron had no such extension. VSCode's slow startup time wore at me, and I decided I didn't appreciate all of the higher-order features of Dendron. Additionally, around this time development on Dendron ceased, and while the project leader declared it good enough to manage his personal projects, several rough edges and missing features took away from its initial promise.

I switched to Vim again for a short time, and tried to find plugins to piece together some of Dendron's features while avoiding the pain points. Around this time I discovered another innovation, "Emacs Org-mode." This video by Harry Schwartz from thoughtbot showed up in my feed. Reluctantly, I watched it. After seeing the power of Org-mode I decided to give it a try.

Org-mode

This entire post will probably devolve into a shill for Emacs and org-mode at this point, but bear with me as I explain my use-case and needs first.

So, here we go again. Another transition. Each of the last has cost me more than a few hours of troubleshooting and problems and configuration. Theoretically, I was using markdown the entire time, but the unique link formats from Vimwiki and Dendron made transitions difficult.

Now I'm preparing to switch to another format, one with completely incompatible links, formatting, and functionality.

In fact, all of my links broke. All my code-blocks broke. All my LaTeX math snippets broke.

It took me weeks to reach the same level of productivity. But Org-mode's features are amazing, and it was well worth the transition.

The ability to write and evaluate code, mixed in with all my notes, is amazing.

print("Hello!")
print( 2 + 2 )
Hello!
4

Emacs' power and scriptability far outstrips everything that Vim had to offer, and Evil mode satisfies my desire for Vim's editing model.

Emacs also handles task management. The default org-mode todo manager is fine, but with the addition of edna for better dependency management and d12frosted's little hack for todo-items in hundreds of files, it becomes much easier to manage projects spread across a zettelkasten of knowledge. org-roam is the final piece of the puzzle, integrating the best of ID-based linking and backlink features into emacs.

TODO Write ten zettels

TODO Rewrite d12frosted's absurd code into a simple ripgrep script

DONE Finish up this post

Org-mode provides far too many features to cover here, but suffice it to say I am very happy with this setup. I have complete freedom to do anything I need to do, all in plain-text, all with an easily-hackable and infinitely customizeable interface.

It has many issues, and I'll put out another blog post some day detailing the problems I've encountered so far, along with what I have as far as mitigations. Despite the issues, I heartily recommend Emacs to anyone looking for the ultimate endgame in text-editing capability.

Other editors are great. VSCode is easy to use, Vim is wicked fast, and Neovim is great. Emacs is superior in extensibility, it is superior in user-focus, and it is ultimately the most comfortable editor I've ever used. If you think you can handle the learning curve, I invite you to dive in.

At the least, don't waste as much time as I did trying to find the perfect setup.

Misc. Software

Version Control

I began using git around halfway through my wiki process. I wasn't very familiar with it when I began, but I've come to value it as a part of my daily workflow. It certainly makes data recovery easy, and coupled with a personal gitea instance it allows me to back up and access my projects from anywhere.

Document Production

Half of org-mode's appeal is perfectly typset documents with LaTeX. I have a full texlive install on my writing machine, along with an extensive configuration for APA, MLA, and Turabian. I also have Citar set up with org-cite and Zotero to make citations a breeze. So far, manual formatting has been a thing of the past.

Encryption

I use GPG and age to encrypt sensitive documents such as my journal files. It works fairly well with org-roam, which supports transparent file encryption.

]]>
RSS Feed https://judah.freedomland.xyz//posts/rss-feed.html author@mail.com (Blog Author) https://judah.freedomland.xyz//posts/rss-feed.html Thu, 21 Dec 2023 00:00:00 -0500 Rather than an email-based subscription service, I'll be using RSS. It's an amazing subscription mechanism used by a lot of blogs and podcasts. All you need to subscribe is an RSS client, which you can pass my link.

]]>
My SSH Key https://judah.freedomland.xyz//posts/my-ssh-key.html author@mail.com (Blog Author) https://judah.freedomland.xyz//posts/my-ssh-key.html Wed, 08 Nov 2023 00:00:00 -0500 In the interest of integrity, here is my public ssh key. This is the key I use to sign all of my commits. If any project purports to be from me, and is not signed by this key, there is a strong chance it is disingenuous.

Key:


ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII0YkBVeeBhoCm/+6mCteO7Ikv528ZDmg/tYtWc6O1qP

Fingerprint:


SHA256:9Dq4ppxhfAjbX+7HLXEt+ROMiIojI6kqQgUyFUJb9lI

Directions

If you're cloning a project off a major git platform, you can check that the latest commit states "Verified" or "Signed". Then you can check that my key fingerprint matches with the one above.

If you've gotten ahold of source code some other way, run git verify-commit HEAD to ensure that the latest commit has been signed.

Example

git verify-commit HEAD

Good "git" signature for development@freedomland.xyz with ED25519 key SHA256:9Dq4ppxhfAjbX+7HLXEt+ROMiIojI6kqQgUyFUJb9lI

Website repo

This link directs to my website's source code. You can check that the latest commits are signed with my key.

]]>
QubesOS Signing Key https://judah.freedomland.xyz//posts/qubesos-signing-key.html author@mail.com (Blog Author) https://judah.freedomland.xyz//posts/qubesos-signing-key.html

Double check your keys!

Do not blindly trust me. Find another website or another source for the key.

Find a photograph or other item that is difficult to forge.

This is the signature for the Qubes master signing key. I provide it here as another copy of a currently known-good key. The QMSK is the root of trust for all QubesOS development. Qubes release keys and developer keys are all signed with the QMSK.

Please see the Qubes website for more information.


427F 11FD 0FAA 4B08 0123
F0IC DDFA 1A3E 3687 9494

]]>