Ryan Rix
commit
c27fe661f8
|
|
@ -0,0 +1,321 @@
|
|||
:PROPERTIES:
|
||||
:ID: arcology/fastapi
|
||||
:ROAM_ALIASES: "Arcology FastAPI"
|
||||
:END:
|
||||
#+TITLE: Arcology Python Prototype
|
||||
#+AUTO_TANGLE: t
|
||||
#+filetags: :Project:
|
||||
|
||||
I learned a lot in building [[id:1d917282-ecf4-4d4c-ba49-628cbb4bb8cc][The Arcology Project]] the first time around, and now that I have [[id:26762cec-7934-4275-8f0b-731ee0e22e07][Migrated to org-roam v2]] I need to evaluate the project, fix it, and get it running again.
|
||||
|
||||
Over the last few months, I have been playing with a [[id:cce/python][Python]] package called [[https://fastapi.tiangolo.com/][FastAPI]] and loving the "batteries included" approach with modern Python 3 and an Flask- or Express-like router model rather than a full MVC framework which I was working with on [[id:cce/elixir][Elixir]]...
|
||||
|
||||
* Frontmatter
|
||||
|
||||
** =requirements.txt= and mach-nix setup
|
||||
|
||||
#+begin_src txt :tangle requirements.txt :comments none
|
||||
fastapi
|
||||
uvicorn[standard]
|
||||
sqlmodel
|
||||
sexpdata
|
||||
#+end_src
|
||||
|
||||
#+begin_src nix -r :tangle mach.nix
|
||||
{ ... }:
|
||||
let
|
||||
mach-nix = import (builtins.fetchGit {
|
||||
url = "https://github.com/DavHau/mach-nix";
|
||||
ref = "refs/tags/3.3.0";
|
||||
}) {
|
||||
pkgs = import <nixpkgs> {};
|
||||
};
|
||||
in
|
||||
mach-nix.mkPythonShell {
|
||||
requirements = builtins.readFile ./requirements.txt;
|
||||
_.websockets.patchPhase = ""; # (ref:websockets)
|
||||
}
|
||||
#+end_src
|
||||
|
||||
in [[(websockets)]] reference, [[file:~/.nix-defexpr/channels/nixpkgs/pkgs/development/python-modules/websockets/default.nix::patchPhase = ''][patchPhase in websockets is broken...]]
|
||||
|
||||
[[https://github.com/DavHau/mach-nix/issues/324][sqlmodel]] is uninstallable in =mach-nix= right now, just use venv...
|
||||
|
||||
#+begin_src nix :tangle shell.nix
|
||||
let
|
||||
pkgs = import <nixpkgs> {};
|
||||
in pkgs.mkShell {
|
||||
packages = [
|
||||
pkgs.python3
|
||||
];
|
||||
}
|
||||
#+end_src
|
||||
|
||||
- [[id:45fc2a02-fcd0-40c6-a29e-897c0ee7b1c7][direnv]] will get a lot of the way, but without mach we still need:
|
||||
- [[elisp:(pyvenv-activate "~/org/arcology-fastapi/env")]] and turn on =pyvenv-mode=...
|
||||
- then we can click for deps [[shell:pip install -r requirements.txt &]]
|
||||
|
||||
* Model the [[id:cce/org-roam][org-roam]] schema in SQLModel
|
||||
|
||||
Getting these to all fit together is gonna hopefully not be too fiddly. I am happy that I don't have to coerce all these primary keys and shit, I just need to be able to read from the database and walk among ORM objects to extract an Arcology Page out of it
|
||||
|
||||
** Arcology "just" reads the [[id:cce/org-roam][org-roam]] database
|
||||
|
||||
#+begin_src python :tangle arcology/models.py
|
||||
from sqlmodel import create_engine
|
||||
|
||||
sqlite_file_name = "/home/rrix/.emacs.d/org-roam.db"
|
||||
sqlite_url = f"sqlite:///{sqlite_file_name}"
|
||||
|
||||
def make_engine():
|
||||
return create_engine(sqlite_url, echo=True)
|
||||
|
||||
engine = make_engine()
|
||||
|
||||
from typing import Optional, List
|
||||
from sqlmodel import Field, Relationship, SQLModel
|
||||
|
||||
from arcology.parse import parse_sexp
|
||||
#+end_src
|
||||
|
||||
*** NEXT make path configurable
|
||||
|
||||
bring in a configuration framework in this?
|
||||
|
||||
** Nodes have-many Links and Nodes have-many-to-many Nodes through Links
|
||||
:LOGBOOK:
|
||||
- State "INPROGRESS" from "NEXT" [2021-09-21 Tue 19:35]
|
||||
:END:
|
||||
|
||||
Setting up the links here is a bit fussy; the class key =source_ref= is the database key =source= and =dest_ref= similarly maps to =dest=; this requires some extra bits to be defined to properly de-reference these, especially when using it as the LinkModel: the link models need to know to query against =source_ref= and =dest_ref!
|
||||
|
||||
#+begin_src sql
|
||||
CREATE TABLE links (pos NOT NULL, source NOT NULL, dest NOT NULL, type NOT NULL, properties NOT NULL,
|
||||
FOREIGN KEY (source) REFERENCES nodes (id) ON DELETE CASCADE);
|
||||
#+end_src
|
||||
|
||||
#+begin_src python :tangle arcology/models.py
|
||||
class Link(SQLModel, table=True):
|
||||
__tablename__ = 'links'
|
||||
pos: str
|
||||
source_ref: Optional[str] = Field(primary_key=True, foreign_key="nodes.id",
|
||||
sa_column_kwargs=dict(name='source'))
|
||||
dest_ref: Optional[str] = Field(primary_key=True, foreign_key="nodes.id",
|
||||
sa_column_kwargs=dict(name='dest'))
|
||||
type: str
|
||||
properties: str
|
||||
|
||||
source: Optional["Node"] = Relationship(# back_populates="outlinks",
|
||||
sa_relationship_kwargs=dict(
|
||||
foreign_keys="[Link.source_ref]",
|
||||
))
|
||||
dest: Optional["Node"] = Relationship(back_populates="backlinks",
|
||||
sa_relationship_kwargs=dict(
|
||||
foreign_keys="[Link.dest_ref]",
|
||||
))
|
||||
#+end_src
|
||||
|
||||
** Node is the "root" object
|
||||
:LOGBOOK:
|
||||
- State "INPROGRESS" from [2021-09-21 Tue 18:31]
|
||||
:END:
|
||||
|
||||
[[id:cce/org-roam][org-roam]] v2 is node-oriented, but Arcology is file-oriented.
|
||||
|
||||
#+begin_src sql
|
||||
CREATE TABLE nodes (id NOT NULL PRIMARY KEY, file NOT NULL, level NOT NULL, pos NOT NULL,
|
||||
todo , priority , scheduled text, deadline text, title , properties , olp ,
|
||||
FOREIGN KEY (file) REFERENCES files (file) ON DELETE CASCADE);
|
||||
#+end_src
|
||||
|
||||
=file_ref= is defined strangely so that the relationship can be the reasonable one to access. Similarly, there are =source= and =dest= hackery-pokery here, as well.
|
||||
|
||||
#+begin_src python :tangle arcology/models.py :mkdirp yes
|
||||
class Node(SQLModel, table=True):
|
||||
__tablename__ = 'nodes'
|
||||
id: Optional[int] = Field(default=None, primary_key=True)
|
||||
file_ref: Optional[str] = Field(foreign_key="files.file",
|
||||
sa_column_kwargs={'name': 'file'})
|
||||
level: int
|
||||
pos: int
|
||||
title: str
|
||||
|
||||
todo: str
|
||||
priority: str
|
||||
|
||||
scheduled: str
|
||||
deadline: str
|
||||
|
||||
properties: str
|
||||
olp: str
|
||||
|
||||
file: List["File"] = Relationship(back_populates="nodes")
|
||||
aliases: List["Alias"] = Relationship(back_populates="node")
|
||||
tags: List["Tag"] = Relationship(back_populates="node")
|
||||
references: List["Reference"] = Relationship(back_populates="node")
|
||||
|
||||
backlinks: List["Link"] = Relationship(back_populates="source",
|
||||
sa_relationship_kwargs=dict(
|
||||
foreign_keys="[Link.source_ref]",
|
||||
primaryjoin="Node.id==Link.source_ref",
|
||||
))
|
||||
outlinks: List["Link"] = Relationship(back_populates="dest",
|
||||
sa_relationship_kwargs=dict(
|
||||
foreign_keys="[Link.dest_ref]",
|
||||
primaryjoin="Node.id==Link.dest_ref",
|
||||
))
|
||||
|
||||
# https://github.com/tiangolo/sqlmodel/issues/89
|
||||
backlink_files: List["Node"] = Relationship(
|
||||
back_populates="outlink_files",
|
||||
link_model=Link,
|
||||
sa_relationship_kwargs=dict(
|
||||
primaryjoin="Node.id==Link.source_ref",
|
||||
secondaryjoin="Node.id==Link.dest_ref",
|
||||
)
|
||||
)
|
||||
outlink_files: List["Node"] = Relationship(
|
||||
back_populates="backlink_files",
|
||||
link_model=Link,
|
||||
sa_relationship_kwargs=dict(
|
||||
primaryjoin="Node.id==Link.dest_ref",
|
||||
secondaryjoin="Node.id==Link.source_ref",
|
||||
)
|
||||
)
|
||||
#+end_src
|
||||
|
||||
** Nodes have-one file
|
||||
:LOGBOOK:
|
||||
- State "INPROGRESS" from [2021-09-21 Tue 18:31]
|
||||
:END:
|
||||
|
||||
The node at level-0 is the File Properties node.
|
||||
|
||||
#+begin_src sql
|
||||
CREATE TABLE files (file UNIQUE PRIMARY KEY, hash NOT NULL,
|
||||
atime NOT NULL, mtime NOT NULL);
|
||||
#+end_src
|
||||
|
||||
#+begin_src python :tangle arcology/models.py :mkdirp yes
|
||||
class File(SQLModel, table=True):
|
||||
__tablename__ = 'files'
|
||||
file: Optional[str] = Field(default=None, primary_key=True)
|
||||
hash: str
|
||||
atime: str
|
||||
mtime: str
|
||||
|
||||
nodes: List["Node"] = Relationship(back_populates="file")
|
||||
#+end_src
|
||||
|
||||
** Nodes have-many Aliases
|
||||
:LOGBOOK:
|
||||
- State "INPROGRESS" from "NEXT" [2021-09-21 Tue 19:30]
|
||||
:END:
|
||||
|
||||
#+begin_src sql
|
||||
CREATE TABLE aliases (node_id NOT NULL, alias ,
|
||||
FOREIGN KEY (node_id) REFERENCES nodes (id) ON DELETE CASCADE);
|
||||
#+end_src
|
||||
|
||||
#+begin_src python :tangle arcology/models.py :mkdirp yes
|
||||
class Alias(SQLModel, table=True):
|
||||
__tablename__ = 'aliases'
|
||||
node_id: str = Field(primary_key=True, foreign_key='nodes.id')
|
||||
alias: str
|
||||
|
||||
node: Optional["Node"] = Relationship(back_populates="aliases")
|
||||
|
||||
def __str__(self):
|
||||
return parse_sexp(self.alias)
|
||||
#+end_src
|
||||
|
||||
*** DONE un-wrap Alias.__str__
|
||||
:LOGBOOK:
|
||||
- State "DONE" from "NEXT" [2021-09-21 Tue 21:22]
|
||||
:END:
|
||||
|
||||
** Nodes have-many Tags
|
||||
:LOGBOOK:
|
||||
- State "INPROGRESS" from "NEXT" [2021-09-21 Tue 19:31]
|
||||
:END:
|
||||
|
||||
#+begin_src sql
|
||||
CREATE TABLE tags (node_id NOT NULL, tag ,
|
||||
FOREIGN KEY (node_id) REFERENCES nodes (id) ON DELETE CASCADE);
|
||||
#+end_src
|
||||
|
||||
#+begin_src python :tangle arcology/models.py :mkdirp yes
|
||||
class Tag(SQLModel, table=True):
|
||||
__tablename__ = 'tags'
|
||||
node_id: str = Field(primary_key=True, foreign_key='nodes.id')
|
||||
tag: str
|
||||
|
||||
node: Optional["Node"] = Relationship(back_populates="tags")
|
||||
|
||||
def __str__(self):
|
||||
return parse_sexp(self.tag)
|
||||
#+end_src
|
||||
|
||||
*** DONE un-wrap Tag.__str__
|
||||
:LOGBOOK:
|
||||
- State "DONE" from "NEXT" [2021-09-21 Tue 21:22]
|
||||
:END:
|
||||
|
||||
** Nodes have-many References
|
||||
:LOGBOOK:
|
||||
- State "INPROGRESS" from "NEXT" [2021-09-21 Tue 19:32]
|
||||
:END:
|
||||
|
||||
#+begin_src sql
|
||||
CREATE TABLE refs (node_id NOT NULL, ref NOT NULL, type NOT NULL,
|
||||
FOREIGN KEY (node_id) REFERENCES nodes (id) ON DELETE CASCADE);
|
||||
#+end_src
|
||||
|
||||
#+begin_src python :tangle arcology/models.py :mkdirp yes
|
||||
class Reference(SQLModel, table=True):
|
||||
__tablename__ = 'refs'
|
||||
node_id: str = Field(primary_key=True, foreign_key='nodes.id')
|
||||
ref: str
|
||||
|
||||
node: Optional["Node"] = Relationship(back_populates="references")
|
||||
|
||||
def __str__(self):
|
||||
return parse_sexp(self.ref)
|
||||
#+end_src
|
||||
|
||||
*** DONE Unwrap Reference.__str__
|
||||
:LOGBOOK:
|
||||
- State "DONE" from "NEXT" [2021-09-21 Tue 21:17]
|
||||
:END:
|
||||
|
||||
** Translate in/out of s-expression forms
|
||||
|
||||
Use [[https://sexpdata.readthedocs.io/en/latest/][sexpdata]] to decode some of the keys. At some point I could do some hackery-pokery to monkeypatch this in to some points. For now it'll be great in =__str__= and some property access methods.
|
||||
|
||||
#+begin_src python :tangle arcology/parse.py
|
||||
import sexpdata as sexp
|
||||
|
||||
def parse_sexp(in_sexp: str):
|
||||
return sexp.loads(in_sexp)
|
||||
#+end_src
|
||||
|
||||
** Attach [[id:arroyo/system-cache][arroyo-db]] for Keywords
|
||||
** higher-level =arcology.Page= class
|
||||
|
||||
The =arcology.page.Page= (aliased as =arcology.Page= elsewhere=
|
||||
|
||||
#+begin_src python :tangle arcology/page.py
|
||||
|
||||
#+end_src
|
||||
|
||||
* inotify-watcher
|
||||
|
||||
** [[id:arroyo/system-cache][arroyo-db]] batch update functions and script
|
||||
** pandoc or whatever
|
||||
|
||||
* fastapi
|
||||
|
||||
** "local" routing
|
||||
** URL re-writing
|
||||
** pandoc memoization
|
||||
**
|
||||
|
|
@ -0,0 +1,124 @@
|
|||
from sqlmodel import create_engine
|
||||
|
||||
sqlite_file_name = "/home/rrix/.emacs.d/org-roam.db"
|
||||
sqlite_url = f"sqlite:///{sqlite_file_name}"
|
||||
|
||||
def make_engine():
|
||||
return create_engine(sqlite_url, echo=True)
|
||||
|
||||
engine = make_engine()
|
||||
|
||||
from typing import Optional, List
|
||||
from sqlmodel import Field, Relationship, SQLModel
|
||||
|
||||
from arcology.parse import parse_sexp
|
||||
|
||||
class Link(SQLModel, table=True):
|
||||
__tablename__ = 'links'
|
||||
pos: str
|
||||
source_ref: Optional[str] = Field(primary_key=True, foreign_key="nodes.id",
|
||||
sa_column_kwargs=dict(name='source'))
|
||||
dest_ref: Optional[str] = Field(primary_key=True, foreign_key="nodes.id",
|
||||
sa_column_kwargs=dict(name='dest'))
|
||||
type: str
|
||||
properties: str
|
||||
|
||||
source: Optional["Node"] = Relationship(# back_populates="outlinks",
|
||||
sa_relationship_kwargs=dict(
|
||||
foreign_keys="[Link.source_ref]",
|
||||
))
|
||||
dest: Optional["Node"] = Relationship(back_populates="backlinks",
|
||||
sa_relationship_kwargs=dict(
|
||||
foreign_keys="[Link.dest_ref]",
|
||||
))
|
||||
|
||||
class Node(SQLModel, table=True):
|
||||
__tablename__ = 'nodes'
|
||||
id: Optional[int] = Field(default=None, primary_key=True)
|
||||
file_ref: Optional[str] = Field(foreign_key="files.file",
|
||||
sa_column_kwargs={'name': 'file'})
|
||||
level: int
|
||||
pos: int
|
||||
title: str
|
||||
|
||||
todo: str
|
||||
priority: str
|
||||
|
||||
scheduled: str
|
||||
deadline: str
|
||||
|
||||
properties: str
|
||||
olp: str
|
||||
|
||||
file: List["File"] = Relationship(back_populates="nodes")
|
||||
aliases: List["Alias"] = Relationship(back_populates="node")
|
||||
tags: List["Tag"] = Relationship(back_populates="node")
|
||||
references: List["Reference"] = Relationship(back_populates="node")
|
||||
|
||||
backlinks: List["Link"] = Relationship(back_populates="source",
|
||||
sa_relationship_kwargs=dict(
|
||||
foreign_keys="[Link.source_ref]",
|
||||
primaryjoin="Node.id==Link.source_ref",
|
||||
))
|
||||
outlinks: List["Link"] = Relationship(back_populates="dest",
|
||||
sa_relationship_kwargs=dict(
|
||||
foreign_keys="[Link.dest_ref]",
|
||||
primaryjoin="Node.id==Link.dest_ref",
|
||||
))
|
||||
|
||||
# https://github.com/tiangolo/sqlmodel/issues/89
|
||||
backlink_files: List["Node"] = Relationship(
|
||||
back_populates="outlink_files",
|
||||
link_model=Link,
|
||||
sa_relationship_kwargs=dict(
|
||||
primaryjoin="Node.id==Link.source_ref",
|
||||
secondaryjoin="Node.id==Link.dest_ref",
|
||||
)
|
||||
)
|
||||
outlink_files: List["Node"] = Relationship(
|
||||
back_populates="backlink_files",
|
||||
link_model=Link,
|
||||
sa_relationship_kwargs=dict(
|
||||
primaryjoin="Node.id==Link.dest_ref",
|
||||
secondaryjoin="Node.id==Link.source_ref",
|
||||
)
|
||||
)
|
||||
|
||||
class File(SQLModel, table=True):
|
||||
__tablename__ = 'files'
|
||||
file: Optional[str] = Field(default=None, primary_key=True)
|
||||
hash: str
|
||||
atime: str
|
||||
mtime: str
|
||||
|
||||
nodes: List["Node"] = Relationship(back_populates="file")
|
||||
|
||||
class Alias(SQLModel, table=True):
|
||||
__tablename__ = 'aliases'
|
||||
node_id: str = Field(primary_key=True, foreign_key='nodes.id')
|
||||
alias: str
|
||||
|
||||
node: Optional["Node"] = Relationship(back_populates="aliases")
|
||||
|
||||
def __str__(self):
|
||||
return parse_sexp(self.alias)
|
||||
|
||||
class Tag(SQLModel, table=True):
|
||||
__tablename__ = 'tags'
|
||||
node_id: str = Field(primary_key=True, foreign_key='nodes.id')
|
||||
tag: str
|
||||
|
||||
node: Optional["Node"] = Relationship(back_populates="tags")
|
||||
|
||||
def __str__(self):
|
||||
return parse_sexp(self.tag)
|
||||
|
||||
class Reference(SQLModel, table=True):
|
||||
__tablename__ = 'refs'
|
||||
node_id: str = Field(primary_key=True, foreign_key='nodes.id')
|
||||
ref: str
|
||||
|
||||
node: Optional["Node"] = Relationship(back_populates="references")
|
||||
|
||||
def __str__(self):
|
||||
return parse_sexp(self.ref)
|
||||
|
|
@ -0,0 +1 @@
|
|||
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
import sexpdata as sexp
|
||||
|
||||
def parse_sexp(in_sexp: str):
|
||||
return sexp.loads(in_sexp)
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
{ ... }:
|
||||
let
|
||||
mach-nix = import (builtins.fetchGit {
|
||||
url = "https://github.com/DavHau/mach-nix";
|
||||
ref = "refs/tags/3.3.0";
|
||||
}) {
|
||||
pkgs = import <nixpkgs> {};
|
||||
};
|
||||
in
|
||||
mach-nix.mkPythonShell {
|
||||
requirements = builtins.readFile ./requirements.txt;
|
||||
_.websockets.patchPhase = ""; #
|
||||
}
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
fastapi
|
||||
uvicorn[standard]
|
||||
sqlmodel
|
||||
sexpdata
|
||||
Loading…
Reference in New Issue