17 KiB
Project Configuration for arcology
- Project
.gitignore
file - Mix Configuration (/rrix/arcology-elixir/src/branch/main/mix.exs)
- Elixir Formatter Configurations
- Runtime Configurations
- Error Views and other base views
This file describes the general Elixir and Phoenix plumbing required to get any Elixir application stood up. In general, when I work in a code module it should get moved in to its own file, but I don't know how to address that for these "infrastructure" elements. For now they'll live here. I also need to design some build systems and build scripts, and set up my .gitignore
Project .gitignore
file
This project is intended to be developed as a Literate Programming org-mode application, developed within my org-roam environment, published as part of my Arcology system. The project is its own subdirectory under my org-roam-directory
and it can be on yours as well, or provided standalone.
The repo will contain docs and assets and a build script that will call up an Emacs that will "tangle" the files out and build the application. It will not contain the compiled code checked in, the docs must be considered the source of truth. We'll have facilities to "de-tangle" the source code eventually to aide contribution from folks who aren't running Emacs. For now, though, I'm gonna be a bastard 😄
This starts with a .gitignore. When updating this, the order of operation is to delete the file from the index, update this, tangle it, and then commit. it's a bit of a mess, maybe detangling is better. I gotta improve upon this quite a bit still, I would love to automatically generate the gitignore some time.
# elixir and mix outputs
/_build/
/cover/
/deps/
/doc/
/.fetch
# debug and dumps
erl_crash.dump
npm-debug.log
# Archives
,*.ez
arcology-*.tar
# The directory NPM downloads your dependencies sources to.
/assets/node_modules/
/priv/static/
# ansible roles
/roles/
systemd.service.j2
deploy.yml
Files which are provided from tangles will need to be managed here:
/mix.exs
# /mix.lock is checked in.
.formatter.exs
/config/*.exs
!/config/prod.secret.exs
/tmp/
# Arcology module
/lib/arcology.ex
/lib/arcology/*.ex
/lib/arcology/roam.ex
/lib/arcology/roam/*.ex
/lib/arcology/link_router/*.ex
/lib/arcology/link_router.ex
# ArcologyWeb module
/lib/arcology_web.ex
/lib/arcology_web/*.ex
# Channels, Controllers, Views, Templates
/lib/arcology_web/channels/
/lib/arcology_web/controllers/
/lib/arcology_web/views/
/lib/arcology_web/templates/
/lib/arcology_web/sigma/
/test/arcology_web/controllers/
/test/arcology_web/views/
/test/arcology/
/test/arcology_web/
/test/test_helper.exs
/test/support/
# mix tasks
/lib/mix/
/priv/lisp/arcology.db
/priv/lisp/arcology-batch.el
# Assets
/assets/js/app.js
/assets/js/foundation.js
/assets/js/sw.js
/assets/js/s1gma.js
# Build crap
/rel/artifacts/
/rel/config.exs
/rel/config/config.exs
/rel/vm.args.eex
/bin/build
/Dockerfile
# nixos crap
shell.nix
This file should be checked in when it's changed, along with mix.lock
Mix Configuration (/rrix/arcology-elixir/src/branch/main/mix.exs)
Mix is the main tool interface for Elixir, an Elixir app almost always has a mix.exs in its root directory, and this is ours: it uses noweb syntax to tangle the file from multiple source code blocks so that the documentation can be placed around each important segment.
defmodule Arcology.MixProject do
use Mix.Project
<<project-definition>>
<<application-definition>>
<<dependencies>>
<<mix-aliases>>
<<elixirc-paths>>
end
Basic Mix project configuration, appliation version, required elixir version, similar things are returned by project/0
, this calls in to some other functions in the module. My Fedora Linux host has Elixir 1.5 installed on it now, I assume I'm going to have to update this sooner or later, though!
def project do
[
app: :arcology,
version: "0.1.1",
elixir: "~> 1.7",
elixirc_paths: elixirc_paths(Mix.env()),
compilers: [:phoenix, :gettext] ++ Mix.compilers(),
start_permanent: Mix.env() == :prod,
aliases: aliases(),
deps: deps(),
releases: [
<<releases>>
]
]
end
These releases are used by mix release
and more details can by found in Deploying the Arcology:
arcology: [
include_executables_for: [:unix],
steps: [
:assemble,
(fn rel ->
File.cp_r("lisp", rel.path <> "/lisp")
rel
end),
:tar
]
]
Configuration for the OTP application. Type mix help compile.app
for more information. This is extended with support for sqlite_ecto2
for the Arcology DB.
def application do
[
mod: {Arcology.Application, []},
extra_applications: [:logger, :runtime_tools]
]
end
The project dependencies are specified here, this is a simple Phoenix web app for now, we'll probably only need a few other things to ship this.
defp deps do
[
{:sqlite_ecto2, "~> 2.2"},
{:symbolic_expression, git: "https://github.com/rob-brown/SymbolicExpression", tag: "1.0.3"},
{:panpipe, "~> 0.1"},
{:phoenix, "~> 1.5"},
{:phoenix_pubsub, "~> 2.0"},
{:phoenix_pubsub_redis, "~> 3.0"},
{:phoenix_html, "~> 2.14"},
{:phoenix_live_reload, "~> 1.3", only: :dev},
{:gettext, "~> 0.11"},
{:jason, "~> 1.0"},
{:plug_cowboy, "~> 2.0"},
{:memoize, "~>1.3"},
{:mix_test_watch, "~> 1.0", only: :dev, runtime: false},
{:file_system, "~> 0.2"}
]
end
Aliases are shortcuts or tasks specific to the current project. For example, to create, migrate and run the seeds file at once: mix ecto.setup
defp aliases do
[
"ecto.setup": ["ecto.create", "ecto.migrate", "run priv/repo/seeds.exs"],
"ecto.reset": ["ecto.drop", "ecto.setup"],
test: ["arcology.build", "test"]
]
end
In :test
environment
defp elixirc_paths(:test), do: ["lib", "test/support"]
defp elixirc_paths(_), do: ["lib"]
Elixir Formatter Configurations
There are two .formatter.exs
files, one in the project root, and one stored with the migrations. I'm not sure if this will be used by the org-babel programming, but I'll provide them for now so that I can git clean
[
import_deps: [:ecto, :phoenix],
inputs: ["*.{ex,exs}", "priv/*/seeds.exs", "{config,lib,test}/**/*.{ex,exs}"],
subdirectories: ["priv/*/migrations"]
]
This one is in priv/repo/migrations/
[
inputs: ["*.exs"]
]
Runtime Configurations
This is not noweb-tangled since it's not wrapped in a defmodule
, just tangled line-by-line and concatenated.
This file is responsible for configuring your application and its dependencies with the aid of the Mix.Config module:
use Mix.Config
sqlite_ecto2
is the only repository I use, configure the Arcology DB to load from the DB.
config :arcology, Arcology.Repo,
adapter: Sqlite.Ecto2,
database: System.get_env("ARCOLOGY_DATABASE") || "./tmp/arcology.db",
pool: Ecto.Adapters.SQL.Sandbox
config :arcology, ecto_repos: [Arcology.Repo]
The Phoenix endpoint configuration is pretty straightforward, we use Phoenix's PostgreSQL PubSub adapter these days to have a durable message passing channel between frontend and backend. And this damned secret key will have to be provided for production builds somehow, that'll be fun. The only thing that is weird here is that Arcology.PubSub is disabled right now because I'm not using it directly, and don't want to take on a Redis or PG2 dependency just to have one lying about, even if my production systems de-facto have them there's no reason my endpoints and development environments need them running.
config :arcology, ArcologyWeb.Endpoint,
url: [host: "localhost"],
secret_key_base: "qjGyFdguDBSm8O7R4TI5M084mtgZ2vPE5p7a/2kTMuDCaPRUz9JiFdLK1RNUBggy",
render_errors: [view: ArcologyWeb.ErrorView, accepts: ~w(html json)],
pubsub_server: Arcology.PubSub
Logger configuration is simple, I haven't changed this, though I probably would like to have JSON logs some day.
config :logger, :console,
format: "$time $metadata[$level] $message\n",
metadata: [:request_id]
Use Jason for the Phoenix JSON library, this is also a default. I've used Poison in the past for personal projects, but I don't have anything resembling strong feelings.
config :phoenix, :json_library, Jason
My org-mode source files location for the renderer. Even though I have ~/org
symlinked on one of my hosts, the elisp code will/would store the actual path, and so I need to be able to switch across that. This will need to be replaced with a system environment variable closer to production deployment. These are used in Arcology DB
config :arcology, :env,
database: System.get_env("ARCOLOGY_DATABASE") || "./tmp/arcology.db",
org_roam_source: System.get_env("ORG_ROAM_SOURCE") || "~/Code/org-roam/",
arcology_directory: System.get_env("ARCOLOGY_DIRECTORY") ||"/home/rrix/org"
Lastly, config.exs loads environment-specific configuration.
import_config "#{Mix.env()}.exs"
Development Configuration
use Mix.Config
The database configuration for local development is pretty simple, a local postgres instance is required, which makes it a bit heavy to do on older hardware, but this is an Elixir decision not mine. There is a sqlite for ecto2, but I only use that in the Arcology project right now.
Disable the cache and enable debugging and hot code reloading, as well as setting up the webpack development server for assets.
config :arcology, ArcologyWeb.Endpoint,
http: [port: 4000],
debug_errors: true,
code_reloader: true,
check_origin: false,
watchers: [
node: [
"node_modules/webpack/bin/webpack.js",
"--mode",
"development",
"--watch-stdin",
cd: Path.expand("../assets", __DIR__)
]
]
Configure Phoenix's LiveReload functionality, which watches some file paths and will recompile and reload the browser page if any of the files change:
config :arcology, ArcologyWeb.Endpoint,
live_reload: [
patterns: [
~r"priv/static/.*(js|css|png|jpeg|jpg|gif|svg)$",
~r"priv/gettext/.*(po)$",
~r"lib/arcology_web/{live,views}/.*(ex)$",
~r"lib/arcology_web/templates/.*(eex)$"
]
]
In development we don't need timestamps or other metadata in the logs, and provide more stacktrace context when there is an error.
config :logger, :console, format: "[$level] $message\n"
config :phoenix, :stacktrace_depth, 20
"Initialize plugs at runtime for faster development compilation", sure!
config :phoenix, :plug_init_mode, :runtime
Testing Configuration
Testing is pretty simple and similar to development, just less stuff going on. The server doesn't run, the logger is less loud, warnings and errors only. The test suite uses the project itself as the Arcology Directory
use Mix.Config
config :arcology, ArcologyWeb.Endpoint,
http: [port: 4002],
server: false
config :logger, level: :warn
config :arcology, :env,
arcology_directory: File.cwd!,
database: System.get_env("ARCOLOGY_DATABASE") || "./tmp/arcology-test.db"
config :arcology, Arcology.Repo,
database: System.get_env("ARCOLOGY_DATABASE") || "./tmp/arcology-test.db"
Production Configuration
use Mix.Config
Note we also include the path to a cache manifest containing the digested version of static files. This manifest is generated by the `mix phx.digest` task, which you should run after static files are built and before starting your production server.
config :arcology, ArcologyWeb.Endpoint,
http: [:inet6, port: System.get_env("ARCOLOGY_PORT") || 5000],
url: [host: "dev.arcology.garden", port: System.get_env("ARCOLOGY_PORT")],
cache_static_manifest: "priv/static/cache_manifest.json",
server: true,
root: "."
Don't print debug messages in production.
config :logger, level: :info
Keeping this secrets file in sync is an open thread.
# import_config "prod.secret.exs"
When deployed, we'll probably do SSL termination in nginx
, it's not so hard.
Runtime Configurable Elements repeated for Deploying the Arcology with mix release
According to mix release's documentation, the config.exs
above is evaluated at compile time, not runtime. I want to be able to specify runtime configuration as process environment injected in to the container at invocation, which I have set up to work for "local" configuration in the System.get_env
calls above. To get these to load at runtime, this will configure the Distillery release to inject a smaller runtime-only config.exs
which will be evaluated at runtime which uses noweb syntax to transclude code blocks defined above.
import Config
<<arcology-repo-config>>
<<arcology-env>>
<<arcology-endpoint>>
Error Views and other base views
These are default, I'll change them some day.
defmodule ArcologyWeb.ErrorView do
use ArcologyWeb, :view
# If you want to customize a particular status code
# for a certain format, you may uncomment below.
# def render("500.html", _assigns) do
# "Internal Server Error"
# end
# By default, Phoenix returns the status message from
# the template name. For example, "404.html" becomes
# "Not Found".
def template_not_found(template, _assigns) do
Phoenix.Controller.status_message_from_template(template)
end
end
defmodule ArcologyWeb.ErrorHelpers do
@moduledoc """
Conveniences for translating and building error messages.
"""
use Phoenix.HTML
@doc """
Generates tag for inlined form input errors.
"""
def error_tag(form, field) do
Enum.map(Keyword.get_values(form.errors, field), fn error ->
content_tag(:span, translate_error(error), class: "help-block")
end)
end
@doc """
Translates an error message using gettext.
"""
def translate_error({msg, opts}) do
# When using gettext, we typically pass the strings we want
# to translate as a static argument:
#
# # Translate "is invalid" in the "errors" domain
# dgettext("errors", "is invalid")
#
# # Translate the number of files with plural rules
# dngettext("errors", "1 file", "%{count} files", count)
#
# Because the error messages we show in our forms and APIs
# are defined inside Ecto, we need to translate them dynamically.
# This requires us to call the Gettext module passing our gettext
# backend as first argument.
#
# Note we use the "errors" domain, which means translations
# should be written to the errors.po file. The :count option is
# set by Ecto and indicates we should also apply plural rules.
if count = opts[:count] do
Gettext.dngettext(ArcologyWeb.Gettext, "errors", msg, msg, count, opts)
else
Gettext.dgettext(ArcologyWeb.Gettext, "errors", msg, opts)
end
end
end
defmodule ArcologyWeb.ErrorViewTest do
use ArcologyWeb.ConnCase, async: true
# Bring render/3 and render_to_string/3 for testing custom views
import Phoenix.View
test "renders 404.html" do
assert render_to_string(ArcologyWeb.ErrorView, "404.html", []) == "Not Found"
end
test "renders 500.html" do
assert render_to_string(ArcologyWeb.ErrorView, "500.html", []) == "Internal Server Error"
end
end
IDK why I need to hold on to this one.
defmodule ArcologyWeb.LayoutView do
use ArcologyWeb, :view
end