complete-computing-environment/datasette-dogsheep.org

Datasette: An open source multi-tool for exploring and publishing data

• NEXT pull ~/DataExports from Virtuous Cassette and move to Syncthing
• Inaturalist to Sqlite
• Twitter to Sqlite
• home-manager availability
• move these to rixpkgs

Archive Tools

Datasette is a tool for exploring and publishing data. It helps people take data of any shape, analyze and explore it, and publish it as an interactive website and accompanying API.

Datasette is aimed at data journalists, museum curators, archivists, local governments, scientists, researchers and anyone else who has data that they wish to share with the world. It is part of a wider ecosystem of 40 tools and 93 plugins dedicated to making working with structured data as productive as possible.

These could be used to generate org-mode files for The Arcology Project and my Knowledge Base….

NEXT pull ~/DataExports from Virtuous Cassette and move to Syncthing

Inaturalist to Sqlite

Slurp: shell:inaturalist-to-sqlite ~/DataExports/inaturalist.db rrix &

{ pkgs ? import <nixpkgs>,
  python ? pkgs.python3,
  ... }:

python.pkgs.buildPythonPackage {
  pname = "inaturalist-to-sqlite";
  version = pkgs.lib.pkgVersions.inaturalist-to-sqlite.version;
  src = pkgs.callPackage pkgs.lib.pkgVersions.inaturalist-to-sqlite.src {};

  propagatedBuildInputs = with python.pkgs; [
    sqlite-utils
    click
    requests
  ];

  checkInputs = with python.pkgs; [
    pytest
  ];

  meta = with pkgs.lib; {
    description = "Create a SQLite database containing your observation history from iNaturalist";
    homepage = "https://github.com/dogsheep/inaturalist-to-sqlite";
    platforms = platforms.unix;
    maintainers = with maintainers; [rrix];
    license = with licenses; [ asl20 ];
  };
}

Twitter to Sqlite

• Slurp favorites: shell:twitter-to-sqlite favorites --auth ~/sync/auth.json ~/"DataExports/twitter-faves.db&%2334;%20&
• Slurp followers: shell:twitter-to-sqlite followers --auth ~/sync/auth.json ~/"DataExports/twitter-follows.db&%2334;%20&
• Track Twitter Following Deltas

sqlite> SELECT id, full_text FROM tweets WHERE ROWID IN (SELECT ROWID FROM tweets_fts WHERE tweets_fts MATCH 'quick take' ORDER BY rank);
1545048960699551744|@deejaygeejaygee quick take https://twitter.com/MartinLundfall/status/1545048960699551744/video/1

{ pkgs ? import <nixpkgs>,
  python ? pkgs.python3,
  ... }:

python.pkgs.buildPythonPackage {
  pname = "twitter-to-sqlite";

  version = pkgs.lib.pkgVersions.twitter-to-sqlite.version;
  src = pkgs.callPackage pkgs.lib.pkgVersions.twitter-to-sqlite.src {};

  propagatedBuildInputs = with python.pkgs; [
    sqlite-utils
    requests-oauthlib
    python-dateutil
  ];

  patchPhase = ''
    substituteInPlace setup.py --replace '"requests-oauthlib~=1.2.0",' '"requests-oauthlib~=1.3.0",'
  '';

  checkInputs = with python.pkgs; [
    pytest
  ];

  meta = with pkgs.lib; {
    description = "Save data from Twitter to a SQLite database.";
    homepage = "https://github.com/dogsheep/twitter-to-sqlite";
    platforms = platforms.unix;
    maintainers = with maintainers; [rrix];
    license = with licenses; [ asl20 ];
  };
}

home-manager availability

Use these in home-manager like so:

{ pkgs, ... }:

{
  home.packages = [ pkgs.inaturalist-to-sqlite pkgs.twitter-to-sqlite ];
}

DONE move these to rixpkgs

• State "DONE" from "NEXT" [2022-12-23 Fri 15:21]