123 lines
4.4 KiB
Org Mode
123 lines
4.4 KiB
Org Mode
#+TITLE: Arcology DB Creates a Queryable Cache of Knowledge
|
|
#+ROAM_ALIAS: "Arcology DB" "arcology-db"
|
|
#+ARCOLOGY_KEY: arcology/arcology-db
|
|
|
|
I organize my personal [[file:../knowledge_base.org][Knowledge Base]] in [[file:../cce/org-roam.org][org-roam]] and org-roam creates a SQLite cache to achieve its performance and dynamicity. I extend this further, adding caching of page keywords, tracking the upstream code but for now a fork of it. I would love to reintegrate this code some time. This elisp code is based on Jethro Kuan's org-roam code and shares copyright with him.
|
|
|
|
#+begin_quote
|
|
Copyright © 2020 Jethro Kuan <jethrokuan95@gmail.com>
|
|
Copyright © 2020 Ryan Rix <ryan@whatthfuck.computer>
|
|
#+end_quote
|
|
|
|
Right now, it's assumed that this code will run on the server which hosts the Elixir service, but that is not a hard requirement -- it could be generated locally and then the =sqlite3= file pushed to the server with [[file:../cce/syncthing.org][Syncthing]] or so.
|
|
|
|
* Rebuilding the Arcology DB within Elixir
|
|
:PROPERTIES:
|
|
:ID: 94fcbe07-5b65-4826-a890-12c46161c2ab
|
|
:END:
|
|
|
|
The org-roam parser runs in Emacs and the Arcology does not. This is squared away by starting the Emacs parser using a shell-command.
|
|
|
|
This code is cool because it has three different types of string escaping in it. Good luck making sense of that!
|
|
|
|
#+begin_src elixir :mkdirp yes :tangle lib/arcology/db_commands.ex
|
|
defmodule Arcology.DbCommands do
|
|
def build_command(), do: _build_command(false)
|
|
def build_command(full_build: fb), do: _build_command(fb)
|
|
|
|
defp _build_command(full_build) do
|
|
arc_dir = Application.get_env(:arcology, :env)[:arcology_directory]
|
|
|
|
org_roam_dir = Application.get_env(:arcology, :env)[:org_roam_source]
|
|
|
|
db_location = Application.get_env(:arcology, :env)[:database]
|
|
|
|
lisp_dir = to_string(:code.priv_dir(:arcology)) <> "/lisp"
|
|
|
|
build_arg =
|
|
case full_build do
|
|
true -> "t"
|
|
false -> "nil"
|
|
end
|
|
|
|
[
|
|
~s<pushd >, lisp_dir, ~s<; emacs -Q --batch >,
|
|
~s<--eval '(setq org-roam-source-directory ">, org_roam_dir, ~s<")' >,
|
|
~s<--eval '(setq org-roam-directory ">, arc_dir, ~s<")' >,
|
|
~s<--eval '(setq org-roam-db-location (expand-file-name ">, db_location, ~s<" ">, File.cwd!, ~s<"))' >,
|
|
~s<-l arcology-batch.el >,
|
|
~s<--eval '(org-roam-db-build-cache >, build_arg ,~s<)' >
|
|
]
|
|
|> Enum.join()
|
|
end
|
|
end
|
|
#+end_src
|
|
|
|
* Mix Tasks
|
|
|
|
These tasks use that build command and send it to =Mix.shell()=, easy.
|
|
|
|
The first rebuilds the DB from scratch:
|
|
|
|
#+begin_src elixir :mkdirp yes :tangle lib/mix/tasks/arcology/build.ex
|
|
defmodule Mix.Tasks.Arcology.Build do
|
|
use Mix.Task
|
|
|
|
@shortdoc "Create the arcology database"
|
|
@impl Mix.Task
|
|
def run(_) do
|
|
Mix.Tasks.Arcology.build_command(full_build: true)
|
|
|> Mix.shell().cmd()
|
|
end
|
|
end
|
|
#+end_src
|
|
|
|
Update an existing DB:
|
|
|
|
#+begin_src elixir :mkdirp yes :tangle lib/mix/tasks/arcology/update.ex
|
|
defmodule Mix.Tasks.Arcology.Update do
|
|
use Mix.Task
|
|
|
|
@shortdoc "Update the arcology database"
|
|
@impl Mix.Task
|
|
def run(_) do
|
|
Mix.Tasks.Arcology.build_command(full_build: false)
|
|
|> Mix.shell().cmd()
|
|
end
|
|
end
|
|
#+end_src
|
|
|
|
These mix tasks start Emacs with no configuration loaded before jamming its own configuration in, =arcology-batch=
|
|
|
|
* =arcology-batch= sets up a batch builder
|
|
|
|
This relies on code from my [[file:../cce/cce.org][Complete Computing Environment]], i'm not sure what to do about that for now, but I don't think it matters so much. This elisp is a "batch" file to be run by the =mix= tasks, loading the required libraries and then configuring org-roam minimally. =org-roam-source-directory=, and optionally =org-roam-directory= is specified in [[file:phoenix.org][Arcology configuration]] and shoved in using the Mix Task above.
|
|
|
|
#+begin_src emacs-lisp :tangle priv/lisp/arcology-batch.el :mkdirp yes
|
|
(unless (boundp 'org-roam-directory)
|
|
(setq org-roam-directory (file-truename "~/org/")))
|
|
|
|
(load-file (expand-file-name "cce/packaging.el" org-roam-directory))
|
|
|
|
(add-to-list 'load-path default-directory)
|
|
(add-to-list 'load-path org-roam-source-directory)
|
|
|
|
(use-package dash)
|
|
(use-package f)
|
|
(use-package s)
|
|
(use-package emacsql)
|
|
(use-package emacsql-sqlite3)
|
|
(require 'subr-x)
|
|
(require 'cl)
|
|
|
|
(require 'org-roam)
|
|
(require 'org-roam-keywords)
|
|
|
|
(setq org-roam-cached-keywords
|
|
'("ARCOLOGY_RSS"
|
|
"ARCOLOGY_KEY"
|
|
"ARCOLOGY_TWITTER"
|
|
"ARCOLOGY_FACEBOOK"
|
|
"ARCOLOGY_OLD_PERMALINK"))
|
|
#+end_src
|