rrix
/
arcology
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
arcology/configuration.org

24 KiB
Raw Permalink Blame History

Arcology Project Configuration

,#+AUTO_TANGLE: vars:org-babel-default-header-args

This file contains user-configuration. Some sections are generated from org-mode tables if you're meant to be editing or extending them. This is a cool feature of org-babel, where you can use tables as data for code which can output more code or even org headings or links. We use this to generate configuration.

The Arcology's Site List

These can be customized by the user:

title key css file link color
The Lion's Rear lionsrear lionsrear.css #cfdcc2
The Arcology Garden garden garden.css #fcf6ed
The Complete Computer cce cce.css #dcdcec
The Arcology Site Engine arcology arcology.css #ccdfff
Local Dev Environment localhost arcology.css #075192
key domain
lionsrear thelionsrear.com
lionsrear rix.si
garden arcology.garden
garden whatthefuck.computer
cce cce.whatthefuck.computer
cce cce.rix.si
arcology engine.arcology.garden
localhost 127.0.0.1
localhost localhost
lionsrear v2.thelionsrear.com
garden v2.arcology.garden
cce cce2.whatthefuck.computer
arcology engine2.arcology.garden

The Code

Oh we doing some code gen in here, this is just gonna generate some JSON for now joining these two tables lul.

(json-encode
 (-map (lambda (siterow)
         (pcase-let* ((`(,title ,key ,css ,link) siterow))
           `((title . ,title)
             (key . ,key)
             (css_file . ,(format "arcology/css/%s" css))
             (link_color . ,link)
             (domains . ,(-map #'cadr
                            (-filter (pcase-lambda (`(,dkey ,domain)) 
                                       (equal key dkey))
                                     domains))))))
       sites))

That generates JSON that goes in to arcology/settings/sites.json for the Arcology Seed Command to use:

<<elisp-join-tables()>>
(thread-last domains
             (-map (pcase-lambda (`(,site ,domain))
                     domain))
             (s-join ","))

NEXT How to re-seed these if you change the configuration?

for now i just blow away the database but gosh!

Per-site CSS

These are generated from a table, too, because I have brainworms. Note that they're basically the same, but you can customize it further to your satisfaction.

Site alert primary secondary success warning white gray2 gray3 gray4 black
default-colors #cc6960 #707231 #ebbe7b #67b4f8 #7e5c41 #fcf6ed #f6e5cb #baad9b #82796c #211f1c
garden #cc6960 #707231 #ebbe7b #67b4f8 #7e5c41 #fcf6ed #f6e5cb #baad9b #82796c #211f1c
lionsrear #cc6960 #707231 #ebbe7b #67b4f8 #7e5c41 #cfdcc2 #87af87 #a0aa96 #82796c #211f1c
cce #cc6960 #707231 #ebbe7b #67b4f8 #7e5c41 #fcfcfc #dcdcec #cacada #808090 #211f1c
arcology #cc6960 #707231 #ebbe7b #67b4f8 #7e5c41 #fcf6ed #f6e5cb #baad9b #82796c #211f1c

The Code 

(pcase-let ((`((,name ,alert ,primary ,secondary ,success ,warning ,white ,gray2 ,gray3 , gray4 ,black))
             (-filter (lambda (row)
                        (equal (first row) site))
                      colors)))
  (s-join "\n"
          (list ":root {"
                (concat "  --alert: " alert ";")
                (concat "  --primary: " primary ";")
                (concat "  --secondary: " secondary ";")
                (concat "  --success: " success ";")
                (concat "  --warning: " warning ";")
                (concat "  --white: " white ";")
                (concat "  --light-gray: " gray2 ";")
                (concat "  --medium-gray: " gray3 ";")
                (concat "  --dark-gray: " gray4 ";")
                (concat "  --black: " black ";")
                "}")))
<<mk-site-colors(site-colors, "default-colors")>>
<<mk-site-colors(site-colors, "cce")>>
<<mk-site-colors(site-colors, "lionsrear")>>
<<mk-site-colors(site-colors, "garden")>>
<<mk-site-colors(site-colors, "arcology")>>

Deploy manifests for Arroyo NixOS Generator or your own manifests

The Arroyo NixOS Generator will integrate this or you can do this yourself. just import the nixos module above, and then configure it as below:

{ pkgs, ... }:

let
  arroyo_rs = pkgs.callPackage /home/rrix/org/arroyo/default.nix {};
  arcology = pkgs.callPackage /home/rrix/org/arcology-django/default.nix { inherit arroyo_rs; };
in {
  imports = [ ./arcology2-module.nix ];

  fileSystems."/media/org" = {
    device = "/home/rrix/org";
    options = ["bind"];
  };

  services.arcology-ng = {
    enable = true;
    packages.arcology = arcology;
    domains = pkgs.lib.splitString "," "<<get_allowed_hosts()>>";
    orgDir = "/media/org";
    folderId = "p1kld-oxnwd";
    dataDir = "/srv/arcology";
    logLevel = "INFO";
  };
}

The NixOS module which defines services.arcology-ng is in Deploying the Arcology.

NEXT the package import needs to be much better than this.

Deploy manifests for a Dynamic Home Manager Configuration

This deploys A Localhost API for the Arcology to any Home Manager user using systemd User Units. See LocalAPI Deployment Options for the list of options that can be changed and how to use it:

{ pkgs, ... }:

let
  arroyo_rs = pkgs.callPackage /home/rrix/org/arroyo/default.nix {};
  arcology = pkgs.callPackage /home/rrix/org/arcology-django/default.nix { inherit arroyo_rs; };
in {
  imports = [ ./arcology-localapi-mod.nix ];

  services.arcology2 = {
    enable = true;
    packages.arcology = arcology;
    folderId = "p1kld-oxnwd";
    environmentFile = "/home/rrix/sync/private-files/.arcology-env";
  };
}

NEXT address the callPackages here…

Service Configuration

these are mostly defaults, a bit of this should be made user-configurable but a lot of this is baked in to how the project works.

Django settings for arcology project.

Generated by 'django-admin startproject' using Django 3.2.22.

For more information on this file, see https://docs.djangoproject.com/en/3.2/topics/settings/

For the full list of settings and their values, see https://docs.djangoproject.com/en/3.2/ref/settings/

Quick-start development settings - unsuitable for production See https://docs.djangoproject.com/en/3.2/howto/deployment/checklist/

from pathlib import Path
import os

from .generators import *

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent.parent
DATABASE_PATH = pathlib.Path(os.getenv("ARCOLOGY_DB_PATH", pathlib.Path(os.getcwd()) / "db.sqlite3")).expanduser()

Environment Variables

ARCOLOGY_DJANGO_SECRET environment variable will be set in the NixOS deployment manifests. this is used for, uhh, security things

SECRET_KEY = os.getenv("ARCOLOGY_DJANGO_SECRET", "devsecret")

Setting ARCOLOGY_ENVIRONMENT=production in the NixOS deployment manifests will modify a bunch of behavior, probably

ARCOLOGY_ENVIRONMENT = os.getenv("ARCOLOGY_ENVIRONMENT", "dev")
DEBUG = (ARCOLOGY_ENVIRONMENT != "production")

The ARCOLOGY_SYNCTHING_KEY environment variable will be set to the Syncthing API key, each deployment will need to specify its own, i'll re-think this when it comes time to deploy this to many places.

SYNCTHING_KEY = os.getenv("ARCOLOGY_SYNCTHING_KEY")

The ARCOLOGY_CACHE_PATH is set to a path that multi-process django can use to cache processed HTML and Atom between processes.

BASE_CACHE_PATH = pathlib.Path(os.environ.get("ARCOLOGY_CACHE_PATH", '/var/tmp/django_cache')).expanduser()
CACHES = {
    'default': {
        'BACKEND': 'django_prometheus.cache.backends.filebased.FileBasedCache',
        'LOCATION': BASE_CACHE_PATH,
    },
}
LOCALAPI_BEARER_TOKEN = os.environ.get("ARCOLOGY_LOCALAPI_BEARER_TOKEN", "changeme!")

NEXT Hostname configuration from arcology.model.Site, eventually

When I have the sites organized in an org-mode table, i'll reapproach the hostname list, and probably before then when i want to test domain-based routing.

ALLOWED_HOSTS = "<<get_allowed_hosts()>>".split(',')

The Arcology is Modular

basically, each org file in this repository, and maybe one or two of your own, are all interoperable and build on top of the base data models of the org docs.

"roam" Arcology Roam Models
"arcology" The Arcology's Data Models and Web Server
"generators" The Arroyo Generators
"syncthonk" Arcology watchsync Command
"sitemap" The Arcology's Site Maps and Discovery Mechanisms 
# Application definition
INSTALLED_APPS = [
    <<gen-config-list(arcology_apps)>>
    "django_htmx",
    "django_prometheus",
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
]
if ARCOLOGY_ENVIRONMENT != "production":
  INSTALLED_APPS = ["localapi"] + INSTALLED_APPS

Internationalization

I'll have to do this at some point. what does an internationalized arcology look like, how does one ship translations of this work?

# Internationalization
# https://docs.djangoproject.com/en/3.2/topics/i18n/

LANGUAGE_CODE = "en-us"
TIME_ZONE = "UTC"
USE_I18N = True
USE_L10N = True
USE_TZ = True

Logging Configuration 

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
        },
    },
    "loggers": {
        "django": {
            "handlers": ["console"],
            "level": os.getenv("DJANGO_LOG_LEVEL", "INFO"),
            "propagate": False,
        },
        "arcology.cache_decorator": { # left as an example to change later.
            "handlers": ["console"],
            "level": "DEBUG",
        }
    },
    "root": {
        "handlers": ["console"],
        "level": os.getenv("ARCOLOGY_LOG_LEVEL", "INFO"),
    },
}

Generator Configuration

NEXT directory configuration 

import pathlib
import os

ARCOLOGY_BASE_DIR = pathlib.Path(os.environ.get("ARCOLOGY_BASE_DIR", "~/org")).expanduser()
ARCOLOGY_EMACS_SNIPPETS_DIR = str(pathlib.Path("~/org/cce/").expanduser())
ARROYO_BASE_DIR = str(pathlib.Path("~/arroyo-nix").expanduser())

Defining extension modules

It should be feasible to add new data types and models and generators to the Arcology system to fit a user's needs. The defaults are specified here.

These models are only persisted if the page is going to be published and are used by the core The Arcology Web Server to set up the page and render backlinks and tag pages and whatnot.

roam.models.Heading Arcology Roam Models's heading entity
roam.models.Reference Arcology Roam Models's org-roam reference entity
roam.models.Tag Arcology Roam Models's heading tag entity
roam.models.HeadingProperty Arcology Roam Models's heading properties map
roam.models.Link Arcology Roam Models page-to-page links
arcology.models.Page The Arcology's Data Models's Page entity
arcology.models.Feed The Arcology's Data Models's Feed entity, probably move to its own module some day…
arcology.models.FeedEntry

The Arroyo Generators are run regardless of whether a file is published. These are a dictionary so that the user can specify a module when invoking the generate Command">Arcology generate Command.

emacs generators.models.EmacsSnippet
epkgs generators.models.EmacsEpkg
nixos generators.models.NixosModule
home-manager generators.models.HomeManagerModule

The Code 

ARCOLOGY_EXTRACTORS = [
  <<gen-config-list(arcology_extractors)>>
]

ARROYO_EXTRACTORS = {
  <<gen-config-dict(arroyo_extractors)>>
}

Enumerating System Roles

I would like to not need these enumerated here, it seems like it should be easier to add a new role than that. But for now here they are so that the DB seed works better.

server the Wobserver Configuration with Arroyo Nixos $
endpoint My NixOS configuration
settop NixOS Set Top Box
droid CCE in Nix On Droid
waterboy waterboy icebox'd data collection platform 
ARROYO_ROLES = [
  <<gen-config-list(arroyo-roles)>>
]

Roam Allowed Keywords

These tables will be used to create an allowlist of #+KEYWORD's to store in the database.

These are used to provide system features or power the various generators:

ARCOLOGY_FEED Add a route that will generate an Atom feed for this page
ARCOLOGY_KEY Add a route that will generate an HTML page for this page
ARCOLOGY_ALLOW_CRAWL Whether the Arcology page will be marked as allowed in robots.txt
ARCOLOGY_TOOT_VISIBILITY This is set to an Mastadan API visibility mode like "public" or "unlisted"
ARROYO_EMACS_MODULE Instruct The Arroyo Generators to include an Emacs snippet in the init.
ARROYO_HOME_MODULE Instruct The Arroyo Generators to import the referenced home-manager module
ARROYO_NIXOS_MODULE Instruct The Arroyo Generators to import the referenced nixos module
ARROYO_HOME_EPKGS Instruct The Arroyo Generators to make a custom Emacs-Lisp package available to Arroyo Emacs
ARROYO_MODULE_WANTS Define a dependency relationship with the referenced org-roam document (this doc depends on that)
ARROYO_MODULE_WANTED Define a dependent relationship from the referenced org-roam document (that doc depends on this)
ARROYO_SYSTEM_ROLE Constrain the Nix expressions on the page to only load in this role (can be specified repeatedly)
ARROYO_SYSTEM_EXCLUDE Prevent the Nix expressions on the page from loading in the listed role (can be specified repeatedly)
ARROYO_DIRENV_DIR Load direnv from a different directory than the loaded buffer's directory. Useful for Org Babel.
AUTHOR Used to populate the author in RSS feeds and page metadata

These are used in some of my custom reports:

BIRTHDAY Used in People pages.
LOCATION Used in People pages.
PRONOUNS Used in People pages.
AGE_OF_MONEY Used in financial review of Monthly Review
JPMORGAN Used in financial review of Monthly Review
SAVINGS Used in financial review of Monthly Review
SPENT_THIS_MONTH Used in financial review of Monthly Review

The Code 

ROAM_ALLOWED_KEYWORDS = [
  <<gen-config-list(roam_allowed_keywords)>>
  <<gen-config-list(my_roam_allowed_keywords)>>
# used for deprecated universal-aggregator feed generation
    "ARROYO_FEEDS",
    "ARROYO_FEED_CATEGORY",
    "ARROYO_FEED_URL",
# unused
    "CCE_HOME_MODULE",
    "ARROYO_EMACS_TANGLED",
    "ARROYO_MODULE_DEP",
]

Ignored Roam Tags

Headings with these tags will not be exported.

NOEXPORT
noexport
Private

The Code 

IGNORED_ROAM_TAGS = [
  <<gen-config-list(ignored_roam_tags)>>
]

Look don't worry about the rest of these 

MIDDLEWARE = [
    "django_prometheus.middleware.PrometheusBeforeMiddleware",
    "django_htmx.middleware.HtmxMiddleware",
    "django.middleware.security.SecurityMiddleware",
    "django.contrib.sessions.middleware.SessionMiddleware",
    "django.middleware.common.CommonMiddleware",
    "django.middleware.csrf.CsrfViewMiddleware",
    "django.contrib.auth.middleware.AuthenticationMiddleware",
    "django.contrib.messages.middleware.MessageMiddleware",
    "django.middleware.clickjacking.XFrameOptionsMiddleware",
    "django_prometheus.middleware.PrometheusAfterMiddleware",
]

ROOT_URLCONF = "arcology.urls"

TEMPLATES = [
    {
        "BACKEND": "django.template.backends.django.DjangoTemplates",
        "DIRS": [],
        "APP_DIRS": True,
        "OPTIONS": {
            "context_processors": [
                "django.template.context_processors.debug",
                "django.template.context_processors.request",
                "django.contrib.auth.context_processors.auth",
                "django.contrib.messages.context_processors.messages",
            ],
        },
    },
]

WSGI_APPLICATION = "arcology.wsgi.application"


# Database
# https://docs.djangoproject.com/en/3.2/ref/settings/#databases

DATABASES = {
    "default": {
        "ENGINE": "django_prometheus.db.backends.sqlite3",
        "NAME": DATABASE_PATH,
    }
}


# Password validation
# https://docs.djangoproject.com/en/3.2/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        "NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator",
    },
    {
        "NAME": "django.contrib.auth.password_validation.MinimumLengthValidator",
    },
    {
        "NAME": "django.contrib.auth.password_validation.CommonPasswordValidator",
    },
    {
        "NAME": "django.contrib.auth.password_validation.NumericPasswordValidator",
    },
]

CSRF_TRUSTED_ORIGINS = list(map(lambda it: f"https://{it}", "<<get_allowed_hosts()>>".split(',')))
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/3.2/howto/static-files/

STATIC_URL = "/static/"

STORAGES = {
    "default": {
        "BACKEND": "django.core.files.storage.FileSystemStorage",
    },
    "staticfiles": {
        "BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage",
    },
}
STATICFILES_DIRS = [
    BASE_DIR / "arcology/static",
    BASE_DIR / "sitemap/static",
]

STATIC_ROOT = os.getenv("ARCOLOGY_STATIC_ROOT", "/var/www/arcology")

# Default primary key field type
# https://docs.djangoproject.com/en/3.2/ref/settings/#default-auto-field

DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"

Helpers 

(mapconcat (lambda (row)
             (format "\"%s\", # %s\n" (first row) (second row)))
           tbl)
(mapconcat (lambda (row)
             (format "\"%s\": \"%s\",\n" (first row) (second row)))
           tbl)