rrix
/
arcology
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

add feed discovery to <head>

main
Ryan Rix 2022-07-27 18:31:08 -07:00
parent 862b982e14
commit bc20ab278e
6 changed files with 138 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
arcology-fastapi.org
View File

@ -203,6 +203,9 @@ app.mount("/static", StaticFiles(directory=static_directory), name="static")
{% block head %}
<title>{{ page.get_title() }} - {{ site.title }}</title>
{% for feed in feeds %}
<link rel="alternate" type="application/atom+xml" href="{{ feed[0] }}" title="{{ feed[1] }}" />
{% endfor %}
{% endblock %}
{% block body %}

85
arcology-feedgen.org
View File

@ -3,6 +3,7 @@
:ROAM_ALIASES: "Arcology Feed Generator" "Arcology Atom Generator"
:END:
#+TITLE: Arcology's Atom Pandoc Filter + Template
#+filetags: :Project:
#+AUTO_TANGLE: t
#+ARCOLOGY_KEY: arcology/feed-gen
@ -18,7 +19,7 @@ I'm shelling out to =pandoc= directly. Probably shouldn't have reached for that
import re
from fastapi import Response, HTTPException
import asyncio
import asyncio
from sqlmodel import Session
from async_lru import alru_cache
from typing import Optional
@ -178,7 +179,87 @@ $endfor$
And this can be confirmed to work with e.g. [[id:20220523T154158.992219][The Lion's Rear Site Feed]]:
#+begin_src shell :results verbatim :exports both
pandoc ../thelionsrear_updates.org --lua-filter=arcology/pandoc/make-atom.lua --template=arcology/pandoc/atom.xml -s
pandoc ../thelionsrear_updates.org --lua-filter=arcology/pandoc/make-atom.lua --template=arcology/pandoc/atom.xml -s
#+end_src
#+results:
* Listing the Arcology Feeds
Since the feeds exist in the [[id:arroyo/system-cache][Arroyo Cache]] K/V/F store, they can be extracted to shove in to the <head> for example.
This is a poor data modeling, however, and it's like that we will benefit from an [[id:arcology/arroyo-page][Arroyo Arcology Generator]] which extracts =ARCOLOGY_FEED= entities to associate them to the page/file they're embedded in.
#+begin_src python :tangle arcology/feeds.py
from typing import List
from sqlmodel import select, Session
from arcology.arroyo import Keyword
from arcology.parse import parse_sexp
from arcology.key import ArcologyKey
#+END_SRC
These helpers prepare the data for =make_feed_entries=. =get_feed_keys= will return the list of all =ARCOLOGY_FEED= routing keys, and =get_feed_files= returns the files associated with those keys.
#+begin_src python :tangle arcology/feeds.py :noweb yes
def arcology_feed_q():
return select(Keyword).where(Keyword.keyword=='"ARCOLOGY_FEED"')
def get_feed_keys(session) -> List[str]:
return [parse_sexp(row.value) for row in session.exec(arcology_feed_q())]
def get_feed_files(session) -> List[str]:
return [parse_sexp(row.file) for row in session.exec(arcology_feed_q())]
#+END_SRC
=make_feed_entries= exposes just why the data model is a bit weak.
We have to build the mapping using the return of =get_feed_files= so that the feeds' pages' titles can be applied in the final return value.
We use the =site_key= to make sure it's filtered to only show feeds related to the current [[id:20211219T144255.001827][Arcology Site]]. It's certainly simpler to show *all* feeds for *all* sites, but in the future I may want to have sites which are at least somewhat hidden, and so showing them in the global feed discovery mechanism is quite a silly thing to build in. If the site keys don't match, the title isn't added to the dict...
#+begin_src python :noweb-ref populateDict
feed_page_titles = dict() # file -> title
for feed_file in get_feed_files(session):
p = Page.from_file(feed_file, session)
if p.get_site().key == site_key:
feed_page_titles[feed_file] = p.get_title()
#+end_src
If the file isn't set in the =feed_page_titles= dict, we know that it's been skipped. The feed URL is generated using [[id:arcology/arroyo/key][arcology.key.ArcologyKey]], and the title and URL are added to the return list in a tuple.
#+begin_src python :noweb-ref populateRetVal
ret = list()
for feed_key in get_feed_keys(session):
feed_url = ArcologyKey(feed_key).to_url()
feed_file = Keyword.get("ARCOLOGY_FEED", feed_key, session=session).filename()
if feed_page_titles.get(feed_file, None):
ret.append((feed_url, feed_page_titles[feed_file]))
#+end_src
Splat!
#+begin_src python :tangle arcology/feeds.py :noweb yes
def make_feed_entries(site_key: str, session):
<<populateDict>>
<<populateRetVal>>
return ret
#+end_src
** NEXT [[id:arcology/arroyo-page][Arroyo Arcology Generator]] for =ARCOLOGY_FEED= keys
All of this becomes much simpler with a [[id:arcology/arroyo-page][Arroyo Arcology Generator]] schema like, maybe, this:
#+begin_src elisp
(arcology-feeds
[(file :not-null)
(key :not-null)
(title :not-null)
(site :not-null)
(hash :not-null)])
#+end_src
then things like =select(Feed.key, Feed.title).where(Feed.site == "lionsrear")= is trivial.

8
arcology-router.org
View File

@ -38,9 +38,12 @@ templates = Jinja2Templates(directory=templ_dir)
This thing is responsible for loading the [[id:arcology/arroyo/page][Arcology Page]], and generating an HTML response and packaging it in to a FastAPI response format. It does a lot and the fact that it's pulling modules from all over the code base gives me great anxiety! this is probably something to really consider refactoring or putting better abstractions in to the Page module... or maybe not.
#+begin_src python :tangle arcology/routing/util.py :mkdirp yes
from fastapi import HTTPException
import asyncio
from fastapi import HTTPException
from arcology.feeds import make_feed_entries
async def render_page_from_key(request: Request, key: str, engine, site) -> Optional[templates.TemplateResponse]:
with Session(engine) as session:
p = Page.from_key(key, session)
@ -50,12 +53,15 @@ async def render_page_from_key(request: Request, key: str, engine, site) -> Opti
bhtml = await asyncio.create_task(p.backlink_html())
document = html.rewrite_html(dhtml, session)
backlink = html.rewrite_html(bhtml, session)
feeds = make_feed_entries(p.get_site().key, session)
return templates.TemplateResponse("page.html.j2", dict(
site=site,
page=p,
document=document,
backlink=backlink,
request=request,
feeds=feeds,
))
#+end_src

36
arcology/feeds.py
View File

@ -1,7 +1,7 @@
import re
from fastapi import Response, HTTPException
import asyncio
import asyncio
from sqlmodel import Session
from async_lru import alru_cache
from typing import Optional
@ -54,3 +54,37 @@ def hydrate_feed(filename: str, xml: str, session) -> str:
out_xml = re.sub(r'{ARCOLOGY_FEED_PAGE}', akey.to_url(), out_xml)
out_xml = rewrite_html(out_xml, session) # lol dangerous
return out_xml
from typing import List
from sqlmodel import select, Session
from arcology.arroyo import Keyword
from arcology.parse import parse_sexp
from arcology.key import ArcologyKey
def arcology_feed_q():
return select(Keyword).where(Keyword.keyword=='"ARCOLOGY_FEED"')
def get_feed_keys(session) -> List[str]:
return [parse_sexp(row.value) for row in session.exec(arcology_feed_q())]
def get_feed_files(session) -> List[str]:
return [parse_sexp(row.file) for row in session.exec(arcology_feed_q())]
def make_feed_entries(site_key: str, session):
feed_page_titles = dict() # file -> title
for feed_file in get_feed_files(session):
p = Page.from_file(feed_file, session)
if p.get_site().key == site_key:
feed_page_titles[feed_file] = p.get_title()
ret = list()
for feed_key in get_feed_keys(session):
feed_url = ArcologyKey(feed_key).to_url()
feed_file = Keyword.get("ARCOLOGY_FEED", feed_key, session=session).filename()
if feed_page_titles.get(feed_file, None):
ret.append((feed_url, feed_page_titles[feed_file]))
return ret

8
arcology/routing/util.py
View File

@ -11,9 +11,12 @@ from arcology.config import get_settings, Environment
templ_dir = "arcology/templates"
templates = Jinja2Templates(directory=templ_dir)
from fastapi import HTTPException
import asyncio
from fastapi import HTTPException
from arcology.feeds import make_feed_entries
async def render_page_from_key(request: Request, key: str, engine, site) -> Optional[templates.TemplateResponse]:
with Session(engine) as session:
p = Page.from_key(key, session)
@ -23,10 +26,13 @@ async def render_page_from_key(request: Request, key: str, engine, site) -> Opti
bhtml = await asyncio.create_task(p.backlink_html())
document = html.rewrite_html(dhtml, session)
backlink = html.rewrite_html(bhtml, session)
feeds = make_feed_entries(p.get_site().key, session)
return templates.TemplateResponse("page.html.j2", dict(
site=site,
page=p,
document=document,
backlink=backlink,
request=request,
feeds=feeds,
))

3
arcology/templates/page.html.j2
View File

@ -7,6 +7,9 @@
{% block head %}
<title>{{ page.get_title() }} - {{ site.title }}</title>
{% for feed in feeds %}
<link rel="alternate" type="application/atom+xml" href="{{ feed[0] }}" title="{{ feed[1] }}" />
{% endfor %}
{% endblock %}
{% block body %}