19 KiB

Raw Permalink Blame History

Arcology Phoenix Setup

nix-shell for base dependencies
nix build for deploying the project
mix.exs project configuration
Runtime Project Configuration
- Dev
- Test
- NEXT Prod
- Runtime

nixpkgs has pretty decent support for Elixir projects with some good helpers. In 2021 I tried an earlier incarnation of The Arcology Project in Elixir+Phoenix, got it working on my laptop but when time came to use Nix to put together a Docker container I couldn't get ecto_sqlite3 to build. Since then, I have shipped a fully function Arcology FastAPI prototype, but I am still quite interested in shipping this on the BEAM. Elixir is cool, and the Arcology Public Router could "just" be a Phoenix.Plug. Now that I have a proper database layer in the Arroyo Arcology Generator many of the design shortcomings of the original Arcology Phoenix prototype can be worked around. I'll be writing a Content-Addressed Store, too, I think, for caching the pages.

`nix-shell` for base dependencies

Install these packages from nixpkgs:

elixir_1_14
elixir_ls
inotify-tools
mix2nix

# probably will need this.... 🥴
nodePackages.node2nix

Set these basic environment variables in the shell:

LANG="C.UTF-8";
ERL_AFLAGS="-kernel shell_history enabled";

Put dependency files in $PWD/.nix-shell: (This is taken from elixir forum)

NIX_SHELL_DIR="$PWD/.nix-shell";
MIX_HOME="$NIX_SHELL_DIR/.mix";
MIX_ARCHIVES="$MIX_HOME/archives";
HEX_HOME="$NIX_SHELL_DIR/.hex";
PATH="$HEX_HOME/bin:$MIX_HOME/escripts:$MIX_HOME/bin:$PATH";
LIVEBOOK_HOME="$PWD";

Install a small wrapper script which will install hex package manager and the Phoenix mix tasks; run setup-mix-phx the first time this project is set up.

(writeScriptBin "setup-mix-phx" ''
 ${elixir}/bin/mix local.hex
 ${elixir}/bin/mix local.rebar
 ${elixir}/bin/mix archive.install hex phx_new
 '')

Assemble all that in to a pkgs.mkShell for nix-shell

{ ... }:

let
  pkgs = import <nixpkgs> {};
  # import <arroyo> {};
in with pkgs; mkShell {
  packages = [
    <<packages>>
  ];

  shellHook = ''
    export <<environment>>

    <<shellHook>>

    ${elixir}/bin/mix --version
    ${elixir}/bin/iex --version
  '';
}

And set up direnv">direnv (don't forget to direnv allow">direnv allow):

use nix

This doesn't set up PostgreSQL or anything like that – we're in roam:Sqlite country.

`nix build` for deploying the project

I had a fully functional Arcology Phoenix a number of years back but wasn't able to get ecto_sqlite3 to build in NixOS. Better start on with that now!

Start with a stub default.nix that sets up a callPackage:

{ ... }:

let pkgs = import <nixpkgs> {};
in
  pkgs.callPackage ./arcology.nix {}

Then this thing can have its nix dependencies declared:

{ lib, beamPackages, callPackage, ... }:

beamPackages.mixRelease rec {
  pname = "arcology";
  version = "0.0.1";

  src = ./.;

  dontStrip = true;

  <<override-mix-deps>>

  meta = with lib; {
    description = "Arcology Org-mode Web Engine";
    homepage = "https://engine.arcology.garden";
    license = licenses.unfree;
    maintainers = with maintainers; [ rrix ];
  };
}

Nix's mix deps are provided by mix2nix, run shell:mix2nix > mix.nix when the mix.exs deps are updated;

this basically works, except that exqlite which ecto_sqlite3 uses as a database driver. It will try to write "something" to XDG_CACHE_HOME … I'm not sure why I need to define this twice – defining it in only one of the root Arcology mixRelease or exqlite's mixRelease will cause the build to fail…

XDG_CACHE_HOME = "/tmp/elixir-cache";
mixNixDeps = import ./mix.nix {
  inherit beamPackages lib;
  overrides = (final: prev: {
    exqlite = prev.exqlite.overrideAttrs (pprev: pprev // {
      XDG_CACHE_HOME = "/tmp/elixir-cache";
    });
  });
};

`mix.exs` project configuration

def project do
  [
    app: :arcology,
    version: "0.1.0",
    elixir: "~> 1.14",
    elixirc_paths: elixirc_paths(Mix.env()),
    start_permanent: Mix.env() == :prod,
    aliases: aliases(),
    deps: deps()
  ]
end

This instructs Mix to load roam:Arcology.Application as the entrypoint:

def application do
  [
    mod: {Arcology.Application, []},
    extra_applications: [:logger, :runtime_tools]
  ]
end

Specifies which paths to compile per environment.


defp elixirc_paths(:test), do: ["lib", "test/support"]
defp elixirc_paths(_), do: ["lib"]

Here's what we depend on:

defp deps do
  [
    {:phoenix, "~> 1.7.1"},
    {:phoenix_ecto, "~> 4.4"},
    {:ecto_sql, "~> 3.6"},
    {:ecto_sqlite3, ">= 0.0.0"},
    {:phoenix_html, "~> 3.3"},
    {:phoenix_live_reload, "~> 1.2", only: :dev},
    {:phoenix_live_view, "~> 0.18.16"},
    {:floki, ">= 0.30.0", only: :test},
    {:phoenix_live_dashboard, "~> 0.7.2"},
    {:esbuild, "~> 0.5", runtime: Mix.env() == :dev},
    {:tailwind, "~> 0.1.8", runtime: Mix.env() == :dev},
    {:swoosh, "~> 1.3"},
    {:finch, "~> 0.13"},
    {:telemetry_metrics, "~> 0.6"},
    {:telemetry_poller, "~> 1.0"},
    {:gettext, "~> 0.20"},
    {:jason, "~> 1.2"},
    {:plug_cowboy, "~> 2.5"}
  ]
end

Aliases are shortcuts or tasks specific to the current project. For example, to install project dependencies and perform other setup tasks, run shell:mix setup. See the documentation for `Mix` for more info on aliases.

defp aliases do
  [
    setup: ["deps.get", "ecto.setup", "assets.setup", "assets.build"],
    "ecto.setup": ["ecto.create", "ecto.migrate", "run priv/repo/seeds.exs"],
    "ecto.reset": ["ecto.drop", "ecto.setup"],
    test: ["ecto.create --quiet", "ecto.migrate --quiet", "test"],
    "assets.setup": ["tailwind.install --if-missing", "esbuild.install --if-missing"],
    "assets.build": ["tailwind default", "esbuild default"],
    "assets.deploy": ["tailwind default --minify", "esbuild default --minify", "phx.digest"]
  ]
end

defmodule Arcology.MixProject do
  use Mix.Project

  <<project>>

  <<application>>

  <<paths>>

  <<deps>>

  <<aliases>>
end

Runtime Project Configuration

# This file is responsible for configuring your application
# and its dependencies with the aid of the Config module.
#
# This configuration file is loaded before any dependency and
# is restricted to this project.

# General application configuration
import Config

Tell Phoenix to load roam:Arcology.Repo

config :arcology,
  ecto_repos: [Arcology.Repo]

Configure ArcologyWeb.Endpoint">ArcologyWeb.Endpoint, instruct it to use Arcology.PubSub as its message-passing layer.

config :arcology, ArcologyWeb.Endpoint,
  url: [host: "localhost"],
  render_errors: [
    formats: [html: ArcologyWeb.ErrorHTML, json: ArcologyWeb.ErrorJSON],
    layout: false
  ],
  pubsub_server: Arcology.PubSub,
  live_view: [signing_salt: "OgcteMC0"]

Configures the mailer

By default it uses the "Local" adapter which stores the emails locally. You can see the emails in your browser, at "/dev/mailbox".

For production it's recommended to configure a different adapter at the `config/runtime.exs`.

config :arcology, Arcology.Mailer, adapter: Swoosh.Adapters.Local

Configure esbuild for Javascript packaging. Sure beats a webpack setup!! I hope!!!!

config :esbuild,
  version: "0.14.41",
  default: [
    args:
      ~w(js/app.js --bundle --target=es2017 --outdir=../priv/static/assets --external:/fonts/* --external:/images/*),
    cd: Path.expand("../assets", __DIR__),
    env: %{"NODE_PATH" => Path.expand("../deps", __DIR__)}
  ]

Configure Tailwind for components CSS; I might just ship the same 40 CSS rules I've been using for a while, though, tbh. This stuff is auto-generated:

# Configure tailwind (the version is required)
config :tailwind,
  version: "3.2.4",
  default: [
    args: ~w(
      --config=tailwind.config.js
      --input=css/app.css
      --output=../priv/static/assets/app.css
    ),
    cd: Path.expand("../assets", __DIR__)
  ]

Configures Elixir's Logger:

config :logger, :console,
  format: "$time $metadata[$level] $message\n",
  metadata: [:request_id]

Use Jason for JSON parsing in Phoenix; in the past I used Poison which require a NIF, but this seems like a good middle-ground between fast and cheap.

config :phoenix, :json_library, Jason

Import environment specific config. This must remain at the bottom of this file so it overrides the configuration defined above.

import_config "#{config_env()}.exs"

Dev

This will override the code in the parent config.exs.

import Config

In production this will read an environment variable, but in dev we can just use the local directory to store the Arroyo Arcology Generator's DB.

config :arcology, Arcology.Repo,
  database: Path.expand("../arcology.db", Path.dirname(__ENV__.file)),
  pool_size: 5,
  stacktrace: true,
  show_sensitive_data_on_connection_error: true

Dev build will bind to http://localhost:4000 and have debug helpers installed code reloading enabled for Elixir, CSS, and Javascript.

config :arcology, ArcologyWeb.Endpoint,
  # Binding to loopback ipv4 address prevents access from other machines.
  # Change to `ip: {0, 0, 0, 0}` to allow access from other machines.
  http: [ip: {127, 0, 0, 1}, port: 4000],
  check_origin: false,
  code_reloader: true,
  debug_errors: true,
  secret_key_base: "PkJnuIpBpXQpUqGrc7ynUHYyXvAB40b03rjay/9WuKmzEN9H9GaELBdGOok2GQih",
  watchers: [
    esbuild: {Esbuild, :install_and_run, [:default, ~w(--sourcemap=inline --watch)]},
    tailwind: {Tailwind, :install_and_run, [:default, ~w(--watch)]}
  ]

Phoenix's LiveReload is pretty slick – it will automatically reload your browser page if CSS, JS, or Elixir Web modules, etc are changed:

config :arcology, ArcologyWeb.Endpoint,
  live_reload: [
    patterns: [
      ~r"priv/static/.*(js|css|png|jpeg|jpg|gif|svg)$",
      ~r"priv/gettext/.*(po)$",
      ~r"lib/arcology_web/(controllers|live|components)/.*(ex|heex)$"
    ]
  ]

Enable dev routes for dashboard and mailbox

config :arcology, dev_routes: true

Do not include metadata nor timestamps in development logs

config :logger, :console, format: "[$level] $message\n"

Set a higher stacktrace during development. The auto-generated comment says "Avoid configuring such in production as building large stacktraces may be expensive."

config :phoenix, :stacktrace_depth, 20

Initialize plugs at runtime for faster development compilation:

config :phoenix, :plug_init_mode, :runtime

Disable swoosh api client as it is only required for production adapters, this is for the Mailer, and I probably don't care to set this up in prod, but yanno…

config :swoosh, :api_client, false

Test

I promise myself I'll figure out how to write tests for Arcology this time around………….

import Config

# Configure your database
#
# The MIX_TEST_PARTITION environment variable can be used
# to provide built-in test partitioning in CI environment.
# Run `mix help test` for more information.
config :arcology, Arcology.Repo,
  database: Path.expand("../arcology_test.db", Path.dirname(__ENV__.file)),
  pool_size: 5,
  pool: Ecto.Adapters.SQL.Sandbox

# We don't run a server during test. If one is required,
# you can enable the server option below.
config :arcology, ArcologyWeb.Endpoint,
  http: [ip: {127, 0, 0, 1}, port: 4002],
  secret_key_base: "4EDyvZHi6tCmgax6tx/v9e6L9kQmXt8qnhsqYE78k7P82uy71Ge7Zl6jPMolA+RR",
  server: false

# In test we don't send emails.
config :arcology, Arcology.Mailer, adapter: Swoosh.Adapters.Test

# Disable swoosh api client as it is only required for production adapters.
config :swoosh, :api_client, false

# Print only warnings and errors during test
config :logger, level: :warning

# Initialize plugs at runtime for faster test compilation
config :phoenix, :plug_init_mode, :runtime

NEXT Prod

This is mostly overridden by the runtime configuration below.

import Config

# For production, don't forget to configure the url host
# to something meaningful, Phoenix uses this information
# when generating URLs.

# Note we also include the path to a cache manifest
# containing the digested version of static files. This
# manifest is generated by the `mix phx.digest` task,
# which you should run after static files are built and
# before starting your production server.
config :arcology, ArcologyWeb.Endpoint, cache_static_manifest: "priv/static/cache_manifest.json"

# Configures Swoosh API Client
config :swoosh, api_client: Swoosh.ApiClient.Finch, finch_name: Arcology.Finch

# Do not print debug messages in production
config :logger, level: :info

# Runtime production configuration, including reading
# of environment variables, is done on config/runtime.exs.

Runtime

These helpers are used to set up the process with environment variables at runtime. These will be important when it comes to deploying Arcology on the Wobserver.

Only start the web server if PHX_SERVER is set; this is done automatically by the bin/server wrapper script installed by mix phx.gen.release.

if System.get_env("PHX_SERVER") do
  config :arcology, ArcologyWeb.Endpoint, server: true
end

Only prod is runtime configured:

if config_env() == :prod do

The database is pointed to with DATABASE_PATH:

  database_path =
    System.get_env("DATABASE_PATH") ||
      raise """
      environment variable DATABASE_PATH is missing.
      For example: /etc/arcology/arcology.db
      """
  
  config :arcology, Arcology.Repo,
    database: database_path,
    pool_size: String.to_integer(System.get_env("POOL_SIZE") || "5")

The secret key base is used to sign/encrypt cookies and other secrets. A default value is used in config/dev.exs and config/test.exs but you want to use a different value for prod and you most likely don't want to check this value into version control, so we use an environment variable instead.

  secret_key_base =
    System.get_env("SECRET_KEY_BASE") ||
      raise """
      environment variable SECRET_KEY_BASE is missing.
      You can generate one by calling: mix phx.gen.secret
      """

PHX_HOST and PORT are used to configure the route/URL generation. The app is configured to listen only to the loopback IPv4 interface since Nginx will be handling the actual edge and SSL and whatnot. Otherwise it would be configured according to Plug.Cowboy documentation.

  host = System.get_env("PHX_HOST") || "example.com"
  port = String.to_integer(System.get_env("PORT") || "4000")

  config :arcology, ArcologyWeb.Endpoint,
    url: [host: host, port: 443, scheme: "https"],
    http: [
      ip: {127, 0, 0, 1},
      port: port
    ],
    secret_key_base: secret_key_base

These aren't used right now:

  # ## SSL Support
  #
  # To get SSL working, you will need to add the `https` key
  # to your endpoint configuration:
  #
  #     config :arcology, ArcologyWeb.Endpoint,
  #       https: [
  #         ...,
  #         port: 443,
  #         cipher_suite: :strong,
  #         keyfile: System.get_env("SOME_APP_SSL_KEY_PATH"),
  #         certfile: System.get_env("SOME_APP_SSL_CERT_PATH")
  #       ]
  #
  # The `cipher_suite` is set to `:strong` to support only the
  # latest and more secure SSL ciphers. This means old browsers
  # and clients may not be supported. You can set it to
  # `:compatible` for wider support.
  #
  # `:keyfile` and `:certfile` expect an absolute path to the key
  # and cert in disk or a relative path inside priv, for example
  # "priv/ssl/server.key". For all supported SSL configuration
  # options, see https://hexdocs.pm/plug/Plug.SSL.html#configure/1
  #
  # We also recommend setting `force_ssl` in your endpoint, ensuring
  # no data is ever sent via http, always redirecting to https:
  #
  #     config :arcology, ArcologyWeb.Endpoint,
  #       force_ssl: [hsts: true]
  #
  # Check `Plug.SSL` for all available options in `force_ssl`.

  # ## Configuring the mailer
  #
  # In production you need to configure the mailer to use a different adapter.
  # Also, you may need to configure the Swoosh API client of your choice if you
  # are not using SMTP. Here is an example of the configuration:
  #
  #     config :arcology, Arcology.Mailer,
  #       adapter: Swoosh.Adapters.Mailgun,
  #       api_key: System.get_env("MAILGUN_API_KEY"),
  #       domain: System.get_env("MAILGUN_DOMAIN")
  #
  # For this example you need include a HTTP client required by Swoosh API client.
  # Swoosh supports Hackney and Finch out of the box:
  #
  #     config :swoosh, :api_client, Swoosh.ApiClient.Hackney
  #
  # See https://hexdocs.pm/swoosh/Swoosh.html#module-installation for details.

end

19 KiB Raw Permalink Blame History Unescape Escape