Arcology DB Creates a Queryable Cache of Knowledge
I organize my personal Knowledge Base in org-roam and org-roam creates a SQLite cache to achieve its performance and dynamicity. I extend this further, adding caching of page keywords, tracking the upstream code but for now a fork of it. I would love to reintegrate this code some time. This elisp code is based on Jethro Kuan's org-roam code and shares copyright with him.
Copyright © 2020 Jethro Kuan <email@example.com> Copyright © 2020 Ryan Rix <firstname.lastname@example.org>
Right now, it's assumed that this code will run on the server which hosts the Elixir service, but that is not a hard requirement – it could be generated locally and then the
sqlite3 file pushed to the server with Syncthing or so.
Rebuilding the Arcology DB within Elixir
The org-roam parser runs in Emacs and the Arcology does not. This is squared away by starting the Emacs parser using a shell-command.
This code is cool because it has three different types of string escaping in it. Good luck making sense of that!
defmodule Arcology.DbCommands do def build_command(), do: _build_command(false) def build_command(full_build: fb), do: _build_command(fb) defp _build_command(full_build) do arc_dir = Application.get_env(:arcology, :env)[:arcology_directory] org_roam_dir = Application.get_env(:arcology, :env)[:org_roam_source] db_location = Application.get_env(:arcology, :env)[:database] lisp_dir = to_string(:code.priv_dir(:arcology)) <> "/lisp" build_arg = case full_build do true -> "t" false -> "nil" end [ ~s<pushd >, lisp_dir, ~s<; emacs -Q --batch >, ~s<--eval '(setq org-roam-source-directory ">, org_roam_dir, ~s<")' >, ~s<--eval '(setq org-roam-directory ">, arc_dir, ~s<")' >, ~s<--eval '(setq org-roam-db-location (expand-file-name ">, db_location, ~s<" ">, File.cwd!, ~s<"))' >, ~s<-l arcology-batch.el >, ~s<--eval '(org-roam-db-build-cache >, build_arg ,~s<)' > ] |> Enum.join() end end
These tasks use that build command and send it to
The first rebuilds the DB from scratch:
defmodule Mix.Tasks.Arcology.Build do use Mix.Task @shortdoc "Create the arcology database" @impl Mix.Task def run(_) do Mix.Tasks.Arcology.build_command(full_build: true) |> Mix.shell().cmd() end end
Update an existing DB:
defmodule Mix.Tasks.Arcology.Update do use Mix.Task @shortdoc "Update the arcology database" @impl Mix.Task def run(_) do Mix.Tasks.Arcology.build_command(full_build: false) |> Mix.shell().cmd() end end
These mix tasks start Emacs with no configuration loaded before jamming its own configuration in,
arcology-batch sets up a batch builder
This relies on code from my Complete Computing Environment, i'm not sure what to do about that for now, but I don't think it matters so much. This elisp is a "batch" file to be run by the
mix tasks, loading the required libraries and then configuring org-roam minimally.
org-roam-source-directory, and optionally
org-roam-directory is specified in Arcology configuration and shoved in using the Mix Task above.
(unless (boundp 'org-roam-directory) (setq org-roam-directory (file-truename "~/org/"))) (load-file (expand-file-name "cce/packaging.el" org-roam-directory)) (add-to-list 'load-path default-directory) (add-to-list 'load-path org-roam-source-directory) (use-package dash) (use-package f) (use-package s) (use-package emacsql) (use-package emacsql-sqlite3) (require 'subr-x) (require 'cl) (require 'org-roam) (require 'org-roam-keywords) (setq org-roam-cached-keywords '("ARCOLOGY_RSS" "ARCOLOGY_KEY" "ARCOLOGY_TWITTER" "ARCOLOGY_FACEBOOK" "ARCOLOGY_OLD_PERMALINK"))