9.0 KiB
Gnus Adaptive Scoring
I believe that Gnus has a single feature which keeps me using it, stuck around me like the world's lightest anchor. Gnus has a very powerful "scoring" system, a message sorting algorithm where each message and each mailing list and each IMAP folder will develop over time a score which can be used as a sorting key to bubble the most relevant things to the top of my attention.
This scoring system can be programmed with explicit commands: "When you see a message from my mom, make sure that sorts to the top (+10000 points)". "Never show me emails from this person who sends me angry emails any time I publish a document. (-5000 points)" "When my CPA or healthcare provider email me, I need to see those mails. (+1000 points)"
This scoring system, critically, can also be scored with an automatically evolving "Adaptive Score" file. Gnus adds a single bit of information that must be provided while I am reading my mail: whether this email is relevant or irrelevant. When Gnus closes a folder, it will look at each mail, and will add or subtract points to the people, words, topics, sentences and groups I interact with most often.
These score files can be combined in to a system which learns from use over time but can also be overridden by the user when the system fails to intuit properly, and this is not a kludge like when GMail fails to mark a message as important, going in and cleaning up behind it and hoping it doesn't happen next time. If I have a need to see a class of messages, I have a firm guarantee that they will be near the top of my inbox.
Anything that scores below a certain threshold is semi-visible: The messages appear at in the list of messages, most likely sorted to the bottom, and they can be selected if the sender or the subject interest me, but by default they're marked as read, swept out of the inbox.
With this simple workflow, with this one bit of information and a terribly inefficient "Artificial Stupidity", I am confident in my ability to triage both mail and news sources. This provides a highly personalized news reading expierence with minimal overhead and the more I use it the smarter it becomes.
As an example: When I would take time off at Uber, I would often do so for two weeks or more at a time, and when I would do so I would fully disconnect. You could page me if you needed me, you could call me and I'd respond, and my team knew this, but I would come back to work with six or seven hundred unread mails, some critical, some that could be answered by maybe me and three other folks, a lot of update emails and newsletters, and a fair bit of crap. And I would often be free of the pile by lunch time, having brought my inbox down to the 30 or so emails that I would actually need to address, and I would still have a vague idea of what had happened in the two weeks by reading the tl;dr section often included in the top of update and newsletter emails. I heavily attribute my success as a technical lead in that highly-social distributed development environment to my ability to process and filter information effectively, and retain the important bits to disseminate to my colleagues and include in development practice.
But at what cost? Each directory has to be processed locally for scores – there's not really a great efficient way to do this on the backend. I had some idea for IMAP-based Smart Scoring but never bothered figuring out how to implement it. But running this sort of algorithm on the client is fine for some folks, especially for me because Personal Software Can Be Shitty, it doesn't need to scale to web-scale or even community-scale if it just runs on my device. Shipping the score files around is a bit annoying but definitely viable with Syncthing. I would love to have this sort of behavior for more clients like in the Fediverse. I had, for some time, a Universal Aggregator script which would ingest my Twitter posts in to Maildir so that I could read them in Gnus and run the adaptive scoring on them but that is not a good way to read twitter. It works great on RSS feeds though where people are less likely to talk "around" a topic and more directly write about a topic.
It's difficult for me to express how powerful this system becomes as it ages, though. I have yet to find a news reader or email client which can emulate these features, which has lead to this strange situation where I don't want to read my email on my phone, where I must be on a desktop or a laptop to be able to keep up with the world around me. There are many systems which will claim to have a "smart" recommendation engine. Because Gnus can rely on the horsepower of my own machine, rather than some CPU time on a datacenter cluster that is being tracked for cost efficiency, my system can get away with brute-forcing this behavior in a way that may make many uncomfortable because it is not an optimal solution, and it is not a good model for how you want to solve this at scale.
roam:#MePersonally, I believe that the Technology industry's reliance on opaque machine learning models is creating inequitable technology, and has been part of a decades long trend of stripping users of control and telling us that we're better off for it. It's only now that we are starting to grapple with the negative effects and costs of automated decision making, and we are finding much to be mad about. Simple, transparent systems which are not immediately effective but which grow over time in a healthy auditable fashion, are an improvement over the network weight models which are easy to scale but effectively magic to the public and to even technical pundits and thinkers.
Configuring Adaptive Scoring
(provide 'cce/gnus-adaptive)
When Gnus opens a folder it will run each message through a guantlet of tests, in the order specified below. Critically, the last one is the total-score function, which will sort the folder primarily on the total-score. When they're equal, it's sorted by date, or the order it was received. Simple stuff.
(setq gnus-thread-sort-functions
'(gnus-thread-sort-by-number
gnus-thread-sort-by-date
gnus-thread-sort-by-total-score))You can make Gnus adaptive simply by asking it to adapt. A powerful idea. Setting the weights for each type of mark applied to a message is easy as well, through gnus-default-adaptive-score-alist, and can be done based on both the sender and subject. I modify this alist to score-up messages which I mark as "ticked" or important or flagged, or whatever your system thinks about it. If I can't be fucked to read the things, they'll slowly be pushed down.
(setq gnus-use-adaptive-scoring '(word line))
(setq gnus-adaptive-word-length-limit 5)
(setq gnus-default-adaptive-score-alist
'((gnus-unread-mark)
(gnus-ticked-mark (from 4))
(gnus-dormant-mark (from 5))
(gnus-del-mark (from -4) (subject -1))
(gnus-read-mark (from 4) (subject 2))
(gnus-expirable-mark (from -1) (subject -1))
(gnus-killed-mark (from -1) (subject -3))
(gnus-kill-file-mark)
(gnus-ancient-mark)
(gnus-low-score-mark)
(gnus-catchup-mark (from -1) (subject -1))))
The weight files that are saved are large S-Expressions suitable to be read and evaluated, written to ~/News, which I have shared with my machines using Syncthing – all of my score files are kept in sync and when there are conflicts I just have to pick one and run with it. I want Gnus to not write the caches to file names which are not necessarily safe, especially since they were being run over WebDav in the past, and I maintain this configuration largely as a backwards compatibility need:
(setq nnheader-file-name-translation-alist '((?: . ?_) (?\[ . ?_) (?\] . ?_)))
Finally, I mentioned in the introductory text above that this also worked on groups, where each mailing list, IMAP folder, or news collection could also have its own score. This is done by adding a hook which is run when the Gnus summary buffer is closed. When I am done reading a folder, this buffer will close, and gnus-summary-bubble-group will be called. Every so often in the group buffer I will type \GSv to sort all of the groups by how often I read them.
(add-hook 'gnus-summary-exit-hook 'gnus-summary-bubble-group)