Fork 0

17 KiB

Project Configuration for arcology

This file describes the general Elixir and Phoenix plumbing required to get any Elixir application stood up. In general, when I work in a code module it should get moved in to its own file, but I don't know how to address that for these "infrastructure" elements. For now they'll live here. I also need to design some build systems and build scripts, and set up my .gitignore

Project .gitignore file

This project is intended to be developed as a Literate Programming org-mode application, developed within my org-roam environment, published as part of my Arcology system. The project is its own subdirectory under my org-roam-directory and it can be on yours as well, or provided standalone.

The repo will contain docs and assets and a build script that will call up an Emacs that will "tangle" the files out and build the application. It will not contain the compiled code checked in, the docs must be considered the source of truth. We'll have facilities to "de-tangle" the source code eventually to aide contribution from folks who aren't running Emacs. For now, though, I'm gonna be a bastard 😄

This starts with a .gitignore. When updating this, the order of operation is to delete the file from the index, update this, tangle it, and then commit. it's a bit of a mess, maybe detangling is better. I gotta improve upon this quite a bit still, I would love to automatically generate the gitignore some time.

# elixir and mix outputs

# debug and dumps

# Archives

# The directory NPM downloads your dependencies sources to.

# ansible roles

Files which are provided from tangles will need to be managed here:

# /mix.lock is checked in.

# Arcology module

# ArcologyWeb module

# Channels, Controllers, Views, Templates


# mix tasks


# Assets

# Build crap

# nixos crap

This file should be checked in when it's changed, along with mix.lock

Mix Configuration (/rrix/arcology-elixir/src/branch/main/mix.exs)

Mix is the main tool interface for Elixir, an Elixir app almost always has a mix.exs in its root directory, and this is ours: it uses noweb syntax to tangle the file from multiple source code blocks so that the documentation can be placed around each important segment.

defmodule Arcology.MixProject do
  use Mix.Project


Basic Mix project configuration, appliation version, required elixir version, similar things are returned by project/0, this calls in to some other functions in the module. My Fedora Linux host has Elixir 1.5 installed on it now, I assume I'm going to have to update this sooner or later, though!

def project do
    app: :arcology,
    version: "0.1.1",
    elixir: "~> 1.7",
    elixirc_paths: elixirc_paths(Mix.env()),
    compilers: [:phoenix, :gettext] ++ Mix.compilers(),
    start_permanent: Mix.env() == :prod,
    aliases: aliases(),
    deps: deps(),
    releases: [

These releases are used by mix release and more details can by found in Deploying the Arcology:

arcology: [
  include_executables_for: [:unix],
  steps: [
    (fn rel ->
      File.cp_r("lisp", rel.path <> "/lisp")

Configuration for the OTP application. Type mix help compile.app for more information. This is extended with support for sqlite_ecto2 for the Arcology DB.

  def application do
      mod: {Arcology.Application, []},
      extra_applications: [:logger, :runtime_tools]

The project dependencies are specified here, this is a simple Phoenix web app for now, we'll probably only need a few other things to ship this.

defp deps do
    {:sqlite_ecto2, "~> 2.2"},
    {:symbolic_expression, git: "https://github.com/rob-brown/SymbolicExpression", tag: "1.0.3"},
    {:panpipe, "~> 0.1"},
    {:phoenix, "~> 1.5"},
    {:phoenix_pubsub, "~> 2.0"},
    {:phoenix_pubsub_redis, "~> 3.0"},
    {:phoenix_html, "~> 2.14"},
    {:phoenix_live_reload, "~> 1.3", only: :dev},
    {:gettext, "~> 0.11"},
    {:jason, "~> 1.0"},
    {:plug_cowboy, "~> 2.0"},
    {:memoize, "~>1.3"},
    {:mix_test_watch, "~> 1.0", only: :dev, runtime: false},
    {:file_system, "~> 0.2"}

Aliases are shortcuts or tasks specific to the current project. For example, to create, migrate and run the seeds file at once: mix ecto.setup

  defp aliases do
      "ecto.setup": ["ecto.create", "ecto.migrate", "run priv/repo/seeds.exs"],
      "ecto.reset": ["ecto.drop", "ecto.setup"],
      test: ["arcology.build", "test"]

In :test environment

  defp elixirc_paths(:test), do: ["lib", "test/support"]
  defp elixirc_paths(_), do: ["lib"]

Elixir Formatter Configurations

There are two .formatter.exs files, one in the project root, and one stored with the migrations. I'm not sure if this will be used by the org-babel programming, but I'll provide them for now so that I can git clean

  import_deps: [:ecto, :phoenix],
  inputs: ["*.{ex,exs}", "priv/*/seeds.exs", "{config,lib,test}/**/*.{ex,exs}"],
  subdirectories: ["priv/*/migrations"]

This one is in priv/repo/migrations/

  inputs: ["*.exs"]

Runtime Configurations

This is not noweb-tangled since it's not wrapped in a defmodule, just tangled line-by-line and concatenated.

This file is responsible for configuring your application and its dependencies with the aid of the Mix.Config module:

use Mix.Config

sqlite_ecto2 is the only repository I use, configure the Arcology DB to load from the DB.

config :arcology, Arcology.Repo,
  adapter: Sqlite.Ecto2,
  database: System.get_env("ARCOLOGY_DATABASE") || "./tmp/arcology.db",
  pool: Ecto.Adapters.SQL.Sandbox

config :arcology, ecto_repos: [Arcology.Repo]

The Phoenix endpoint configuration is pretty straightforward, we use Phoenix's PostgreSQL PubSub adapter these days to have a durable message passing channel between frontend and backend. And this damned secret key will have to be provided for production builds somehow, that'll be fun. The only thing that is weird here is that Arcology.PubSub is disabled right now because I'm not using it directly, and don't want to take on a Redis or PG2 dependency just to have one lying about, even if my production systems de-facto have them there's no reason my endpoints and development environments need them running.

config :arcology, ArcologyWeb.Endpoint,
  url: [host: "localhost"],
  secret_key_base: "qjGyFdguDBSm8O7R4TI5M084mtgZ2vPE5p7a/2kTMuDCaPRUz9JiFdLK1RNUBggy",
  render_errors: [view: ArcologyWeb.ErrorView, accepts: ~w(html json)],
  pubsub_server: Arcology.PubSub

Logger configuration is simple, I haven't changed this, though I probably would like to have JSON logs some day.

config :logger, :console,
  format: "$time $metadata[$level] $message\n",
  metadata: [:request_id]

Use Jason for the Phoenix JSON library, this is also a default. I've used Poison in the past for personal projects, but I don't have anything resembling strong feelings.

config :phoenix, :json_library, Jason

My org-mode source files location for the renderer. Even though I have ~/org symlinked on one of my hosts, the elisp code will/would store the actual path, and so I need to be able to switch across that. This will need to be replaced with a system environment variable closer to production deployment. These are used in Arcology DB

config :arcology, :env,
  database: System.get_env("ARCOLOGY_DATABASE") || "./tmp/arcology.db",
  org_roam_source: System.get_env("ORG_ROAM_SOURCE") || "~/Code/org-roam/",
  arcology_directory: System.get_env("ARCOLOGY_DIRECTORY") ||"/home/rrix/org"

Lastly, config.exs loads environment-specific configuration.

import_config "#{Mix.env()}.exs"

Development Configuration

use Mix.Config

The database configuration for local development is pretty simple, a local postgres instance is required, which makes it a bit heavy to do on older hardware, but this is an Elixir decision not mine. There is a sqlite for ecto2, but I only use that in the Arcology project right now.

Disable the cache and enable debugging and hot code reloading, as well as setting up the webpack development server for assets.

config :arcology, ArcologyWeb.Endpoint,
  http: [port: 4000],
  debug_errors: true,
  code_reloader: true,
  check_origin: false,
  watchers: [
    node: [
      cd: Path.expand("../assets", __DIR__)

Configure Phoenix's LiveReload functionality, which watches some file paths and will recompile and reload the browser page if any of the files change:

config :arcology, ArcologyWeb.Endpoint,
  live_reload: [
    patterns: [

In development we don't need timestamps or other metadata in the logs, and provide more stacktrace context when there is an error.

config :logger, :console, format: "[$level] $message\n"
config :phoenix, :stacktrace_depth, 20

"Initialize plugs at runtime for faster development compilation", sure!

config :phoenix, :plug_init_mode, :runtime

Testing Configuration

Testing is pretty simple and similar to development, just less stuff going on. The server doesn't run, the logger is less loud, warnings and errors only. The test suite uses the project itself as the Arcology Directory

use Mix.Config

config :arcology, ArcologyWeb.Endpoint,
  http: [port: 4002],
  server: false

config :logger, level: :warn

config :arcology, :env,
  arcology_directory: File.cwd!,
  database: System.get_env("ARCOLOGY_DATABASE") || "./tmp/arcology-test.db"

config :arcology, Arcology.Repo,
  database: System.get_env("ARCOLOGY_DATABASE") || "./tmp/arcology-test.db"

Production Configuration

use Mix.Config

Note we also include the path to a cache manifest containing the digested version of static files. This manifest is generated by the `mix phx.digest` task, which you should run after static files are built and before starting your production server.

config :arcology, ArcologyWeb.Endpoint,
  http: [:inet6, port: System.get_env("ARCOLOGY_PORT") || 5000],
  url: [host: "dev.arcology.garden", port: System.get_env("ARCOLOGY_PORT")],
  cache_static_manifest: "priv/static/cache_manifest.json",
  server: true,
  root: "."

Don't print debug messages in production.

config :logger, level: :info

Keeping this secrets file in sync is an open thread.

# import_config "prod.secret.exs"

When deployed, we'll probably do SSL termination in nginx, it's not so hard.

Runtime Configurable Elements repeated for Deploying the Arcology with mix release

According to mix release's documentation, the config.exs above is evaluated at compile time, not runtime. I want to be able to specify runtime configuration as process environment injected in to the container at invocation, which I have set up to work for "local" configuration in the System.get_env calls above. To get these to load at runtime, this will configure the Distillery release to inject a smaller runtime-only config.exs which will be evaluated at runtime which uses noweb syntax to transclude code blocks defined above.

import Config


Error Views and other base views

These are default, I'll change them some day.

defmodule ArcologyWeb.ErrorView do
  use ArcologyWeb, :view

  # If you want to customize a particular status code
  # for a certain format, you may uncomment below.
  # def render("500.html", _assigns) do
  #   "Internal Server Error"
  # end

  # By default, Phoenix returns the status message from
  # the template name. For example, "404.html" becomes
  # "Not Found".
  def template_not_found(template, _assigns) do
defmodule ArcologyWeb.ErrorHelpers do
  @moduledoc """
  Conveniences for translating and building error messages.

  use Phoenix.HTML

  @doc """
  Generates tag for inlined form input errors.
  def error_tag(form, field) do
    Enum.map(Keyword.get_values(form.errors, field), fn error ->
      content_tag(:span, translate_error(error), class: "help-block")

  @doc """
  Translates an error message using gettext.
  def translate_error({msg, opts}) do
    # When using gettext, we typically pass the strings we want
    # to translate as a static argument:
    #     # Translate "is invalid" in the "errors" domain
    #     dgettext("errors", "is invalid")
    #     # Translate the number of files with plural rules
    #     dngettext("errors", "1 file", "%{count} files", count)
    # Because the error messages we show in our forms and APIs
    # are defined inside Ecto, we need to translate them dynamically.
    # This requires us to call the Gettext module passing our gettext
    # backend as first argument.
    # Note we use the "errors" domain, which means translations
    # should be written to the errors.po file. The :count option is
    # set by Ecto and indicates we should also apply plural rules.
    if count = opts[:count] do
      Gettext.dngettext(ArcologyWeb.Gettext, "errors", msg, msg, count, opts)
      Gettext.dgettext(ArcologyWeb.Gettext, "errors", msg, opts)
defmodule ArcologyWeb.ErrorViewTest do
  use ArcologyWeb.ConnCase, async: true

  # Bring render/3 and render_to_string/3 for testing custom views
  import Phoenix.View

  test "renders 404.html" do
    assert render_to_string(ArcologyWeb.ErrorView, "404.html", []) == "Not Found"

  test "renders 500.html" do
    assert render_to_string(ArcologyWeb.ErrorView, "500.html", []) == "Internal Server Error"

IDK why I need to hold on to this one.

defmodule ArcologyWeb.LayoutView do
  use ArcologyWeb, :view