complete-computing-environment/vpype-vsketch.org

:PROPERTIES:
:ID:       20220506T155821.572980
:END:
#+title: Back on to the Plotter Train with vsketch
#+AUTO_TANGLE: t

* What are =vsketch= and =vpype=
:PROPERTIES:
:ID:       20220506T155905.773161
:ROAM_REFS: https://github.com/abey79/vpype/ https://vsketch.readthedocs.io/en/latest/index.html
:ROAM_ALIASES: "vsketch and vpype" vpype vsketch
:END:
=vsketch= is a plotter-centric [[id:cce/python][Python]] [[id:80864655-e79c-48d3-825c-d311125a3a92][Generative Art]] library with an API based loosely around Processing.
=vpype= is a post-processing library which helps you make designs which plot better.

I do all my interactive art in one place so that i can have =nix-shell= etc in there.. Start a shell:

#+begin_src emacs-lisp
(setq cce/vsketch-dir "~/Code/plotting/")
(let ((default-directory cce/vsketch-dir))
  (shell "*vsketch*"))
#+end_src

* Packaging =vpype= and =vsketch= in [[id:20221021T115008.329657][Arroyo Nix Pkgs]]

I previously maintained these in a [[id:c75d20e6-8888-4c5a-ac97-5997e2f1c711][nixpkgs]] fork at  [[https://github.com/rrix/nixpkgs/tree/rrix/vpype-vsketch][github.com/rrix/nixpkgs@rrix/vpype-vsketch]] but maintaining that and dealing with the occasional merge conflicts is not worth just making [[id:20221021T115008.329657][Arroyo Nix]] an overlay that adds these packages to the =nixpkgs= namespace.

** vpype + extensions
=vpype= has a number of extensions available, we'll set them all up in here; i wish I could figure out why I can't =callPackage= in these dependencies which are defined in the overlay...

#+begin_src nix :tangle ~/arroyo-nix/pkgs/vpype/default.nix :mkdirp yes
{ lib
, callPackage
, fetchFromGitHub
, pnoise
, svgelements
, watchfiles
, qasync
, hnswlib
}:

lib.makeExtensible (self: {
  vpype-minimal = callPackage ./common.nix rec {
    inherit pnoise; inherit svgelements;
  };

  vpype = callPackage ./common.nix rec {
    inherit pnoise; inherit svgelements;

    plugins = [ self.hatched self.flow-imager self.vectrace ];
  };

  # vsketch uses vpype, not a vpype plugin
  vsketch = callPackage ./vsketch.nix {
    inherit pnoise; inherit qasync; inherit watchfiles;

    vpype = self.vpype;
  };

  # plugins
  hatched = callPackage ./hatched.nix { vpype = self.vpype-minimal; };
  flow-imager = callPackage ./flow-imager.nix {
    inherit hnswlib;
    
    vpype = self.vpype-minimal;
  };
  vectrace = callPackage ./vectrace.nix {
    inherit svgelements;
    vpype = self.vpype-minimal;
  };
})
#+end_src

*** vpype package

#+begin_src nix :tangle ~/arroyo-nix/pkgs/vpype/common.nix
{ lib
, buildPythonPackage
, fetchFromGitHub

, asteval
, cachetools
, click
, multiprocess
, numpy
, pnoise
, scipy
, setuptools
, shapely
, svgelements
, svgwrite
, tomli
, pkgs

# gui dependencies
, glcontext
, matplotlib
, moderngl
, pillow
, pyside2

, poetry-core

, plugins ? []
}:

buildPythonPackage rec {
  pname = "vpype";
  version = "1.11.0-alpha.1";

  src = fetchFromGitHub {
    # https://github.com/abey79/vpype
    owner = "abey79";
    repo = "vpype";
    rev = "6a099b7b4f3d756f7fa8052049be365d0314c5ee";
    sha256 = "sha256-C15bO+Z5u29mfGULgK173AYWnYlPesdzrxPqb0j/goc=";
  };

  format = "pyproject";

  propagatedBuildInputs = [
    cachetools click multiprocess numpy pnoise shapely scipy setuptools
    svgelements svgwrite tomli asteval

    matplotlib glcontext moderngl pillow pyside2

    poetry-core
  ] ++ plugins;

  postPatch = ''
    substituteInPlace pyproject.toml --replace 'matplotlib = { version = ">=3.3.2,<3.6.0", optional = true }' 'matplotlib = { version = ">=3.3.2,<3.8.0", optional = true }'
    substituteInPlace pyproject.toml --replace 'pnoise = "^0.1.0"' 'pnoise = "^0.2.0"'
    substituteInPlace pyproject.toml --replace 'setuptools = "^51.0.0"' 'setuptools = "^61.0.0"'
  '';

  postInstall = ''
    rm $out/lib/python*/site-packages/LICENSE
    rm $out/lib/python*/site-packages/README.md
  '';

  meta = with lib; {
    description = "The Swiss Army knife of vector graphics for pen plotters";
    homepage = "https://github.com/abey79/vpype/";
    platforms = platforms.unix;
    maintainers = with maintainers; [rrix];
    license = with licenses; [ mit ];
  };
}
#+end_src

*** hatched
:PROPERTIES:
:ID:       20221022T234938.254085
:END:
:LOGBOOK:
- State "DONE"       from "INPROGRESS" [2022-10-23 Sun 15:34]
- State "INPROGRESS" from              [2022-10-22 Sat 23:49]
:END:

#+begin_src nix :tangle ~/arroyo-nix/pkgs/vpype/hatched.nix
{ lib
, buildPythonPackage
, fetchFromGitHub

, poetry-core

, click
, scikitimage
, svgwrite
, opencv4
, shapely
, matplotlib
, vpype

, qt5
}:

buildPythonPackage rec {
  pname = "hatched";
  version = "0.3.0a0";

  src = fetchFromGitHub {
    owner = "plottertools";
    repo = "hatched";
    rev = "6fbaf9baf5f71ee954384e8e61c8bdd469048d07";
    sha256 = "0wq3dk6x732mndq3n85f6cmdqpnx5s102p15d155j9pbgz4nm1xa";
    # date = "2022-10-17T15:28:37+02:00";
  };

  doCheck = false;
  nativeBuildInputs = [ qt5.wrapQtAppsHook ];
  propagatedBuildInputs = [
    click
    matplotlib
    opencv4
    scikitimage
    shapely
    svgwrite
  ];

  buildInputs = [
    vpype
  ];

  postPatch = ''
    substituteInPlace setup.py --replace '"opencv-python-headless"' '"opencv"'
  '';

  meta = with lib; {
    description = "Library and vpype plug-in to convert images to plotter-friendly, hatched patterns.";
    homepage = "https://github.com/plottertools/hatched";
    platforms = platforms.unix;
    maintainers = with maintainers; [rrix];
    license = with licenses; [ mit ];
  };
}
#+end_src

*** INPROGRESS vpype-flow-imager
:PROPERTIES:
:ID:       20221022T234940.053367
:END:
:LOGBOOK:
- State "INPROGRESS" from "NEXT"       [2022-10-22 Sat 23:49]
:END:

#+begin_src nix :tangle ~/arroyo-nix/pkgs/vpype/flow-imager.nix
{ lib
, buildPythonPackage
, fetchFromGitHub

, click
, hnswlib
, opencv4
, opensimplex
, pillow
, scikitimage
, tqdm
, vpype
}:

buildPythonPackage rec {
  pname = "vpype-flow-imager";
  version = "1.0.7";

  src = fetchFromGitHub {
    owner = "serycjon";
    repo = "vpype-flow-imager";
    rev = "0a419e46be6440ad779516eb647c1396726048d2";
    sha256 = "13z1xbdfxpsrmwdbh1y901rnz733khblkvlqxlcr8ph5b5nhlq6n";
    # date = "2022-06-25T23:32:35+02:00";
  };

  doCheck = false;
  propagatedBuildInputs = [
    click
    hnswlib
    opencv4
    opensimplex
    pillow
    scikitimage
    tqdm
  ];

  buildInputs = [
    vpype
  ];

  postPatch = ''
    substituteInPlace setup.py --replace "'opencv-python-headless'" "'opencv'" \
                               --replace "'opensimplex==0.4'" "'opensimplex>=0.4,<0.5'"
  '';

  meta = with lib; {
    description = "vpype plug-in to convert images to flow field line art inspired by Sean M. Puckett's work and the \"Creating evenly-spaced streamlines of arbitrary density\" paper by Jobard and Lefer.";
    homepage = "https://github.com/serycjon/vpype-flow-imager";
    platforms = platforms.unix;
    maintainers = with maintainers; [rrix];
    license = with licenses; [ mit ];
  };
}
#+end_src

*** INPROGRESS vpype-vectrace
:LOGBOOK:
- State "INPROGRESS" from "NEXT"       [2022-10-23 Sun 15:34]
:END:

#+begin_src nix :tangle ~/arroyo-nix/pkgs/vpype/vectrace.nix
{ lib
, buildPythonPackage
, fetchFromGitHub

, click
, pillow
, svgelements
, vpype
}:

buildPythonPackage rec {
  pname = "vpype-vectrace";
  version = "0.1.2";

  src = fetchFromGitHub {
    owner = "tatarize";
    repo = "vpype-vectrace";
    rev = "de7341865b304c5842ab9f33afadb476e5a8b588";
    sha256 = "1sf3kky7qhmpma0ga0l0w2fxfqgi1688v2s107ij5c9mdj1p05z6";
    # date = "2022-04-14T10:35:27-07:00";
  };

  doCheck = false;
  propagatedBuildInputs = [
    click
    pillow
    svgelements
  ];

  buildInputs = [
    vpype
  ];

  meta = with lib; {
    description = "Vpype plugin for vector tracing";
    homepage = "https://github.com/tatarize/vpype-vectrace";
    platforms = platforms.unix;
    maintainers = with maintainers; [rrix];
    license = with licenses; [ mit ];
  };
}
#+end_src

*** NEXT vpype-perspective
*** NEXT vpype-pixelart

** vsketch
:LOGBOOK:
- State "INPROGRESS" from              [2022-10-21 Fri 18:48]
:END:

#+begin_src nix :tangle ~/arroyo-nix/pkgs/vpype/vsketch.nix
{ lib
, buildPythonPackage
, fetchFromGitHub

, poetry-core

, colorama
, cookiecutter
, matplotlib
, multiprocess
, numpy
, pnoise
, pyside2
, qasync
, shapely
, vpype
, watchfiles

, qt5
}:

let
  vpype-rev = "6a099b7b4f3d756f7fa8052049be365d0314c5ee";
in
buildPythonPackage rec {
  pname = "vsketch";
  version = "1.0.0-alpha.0";

  src = fetchFromGitHub {
    owner = "abey79";
    repo = "vsketch";
    rev = "1e9ebc540a3ef873d39e04e728a8e96a3faedb80";
    sha256 = "1s3pjrfp1hcsyxfg0dr4xms8qhp4k4gyhzfvy67mwwk0bbhps41s";
    # date = "2022-06-27T11:14:11+02:00";
  };

  format = "pyproject";

  nativeBuildInputs = [ qt5.wrapQtAppsHook ];
  propagatedBuildInputs = [
    pnoise qasync watchfiles
    shapely numpy pyside2
    colorama cookiecutter matplotlib multiprocess

    poetry-core

    vpype
  ];

  postPatch = ''
    substituteInPlace pyproject.toml --replace 'vpype = {extras = ["all"], git = "https://github.com/abey79/vpype", rev = "${vpype-rev}"}' 'vpype = "^1.11.0a0"' \
                                     --replace 'Shapely = {extras = ["vectorized"], version = "1.8.2"}' 'Shapely = {extras = ["vectorized"], version = "^2.0"}'

    sed -i '/PySide2/d' pyproject.toml # no idea why there isnt a dist-info written...
  '';

  dontWrapQtApps = true;
  preFixup = ''
    makeWrapperArgs+=("''${qtWrapperArgs[@]}")
  '';

  postInstall = ''
    rm $out/lib/python*/site-packages/LICENSE
    rm $out/lib/python*/site-packages/README.md
  '';

  meta = with lib; {
    description = "Plotter generative art environment";
    homepage = "https://github.com/abey79/vsketch/";
    platforms = platforms.unix;
    maintainers = with maintainers; [rrix];
    license = with licenses; [ mit ];
  };
}
#+end_src

** Dependency packages
*** NEXT pyside6 and upgrade vsketch version
*** pnoise
:LOGBOOK:
- State "DONE"       from "INPROGRESS" [2022-10-21 Fri 15:07]
- State "INPROGRESS" from "NEXT"       [2022-10-21 Fri 15:05]
:END:

#+begin_src nix :tangle ~/arroyo-nix/pkgs/pnoise.nix
{ lib
, buildPythonPackage
, fetchFromGitHub
, numpy
}:

buildPythonPackage rec {
  version =  "0.2.0";
  src = fetchFromGitHub {
    owner = "plottertools";
    repo = "pnoise";
    rev = version;
    sha256 = "sha256-JwWzLvgCNSLRs/ToZNFH6fN6VLEsQTmsgxxkugwjA9k=";
  };

  pname = "pnoise";

  propagatedBuildInputs = [ numpy ];

  meta = with lib; {
    description = "pnoise is a pure-Python, Numpy-based, vectorized port of Processing's noise() function.";
    homepage = "https://github.com/plottertools/pnoise/";
    platforms = platforms.unix;
    maintainers = with maintainers; [rrix];
    license = with licenses; [ lgpl21 ];
  };
}
#+end_src

*** hnswlib

#+begin_src nix :tangle ~/arroyo-nix/pkgs/hnswlib.nix
{ lib
, buildPythonPackage
, fetchFromGitHub
, numpy
, pybind11
}:

buildPythonPackage rec {
  version =  "0.6.1";
  src = fetchFromGitHub {
    owner = "nmslib";
    repo = "hnswlib";
    rev = "443d667478fddf1e13f2e06b1da4e1ec3a9fe716";
    sha256 = "0vxi1xlk2slw1hrvbvacy6g7rypggm6a1hzayxpbi001982sh1dh";
    # date = "2022-04-15T19:58:29-07:00";
  };

  pname = "hnswlib";

  propagatedBuildInputs = [ numpy ];
  buildInputs = [ pybind11 ];
  postPatch = ''
    rm python_bindings/setup.py
  '';

  meta = with lib; {
    description = "Header-only C++ HNSW implementation with python bindings.";
    homepage = "https://github.com/nmslib/hnswlib/";
    platforms = platforms.unix;
    maintainers = with maintainers; [rrix];
    license = with licenses; [ asl20 ];
  };
}
#+end_src

*** svgelements
:LOGBOOK:
- State "DONE"       from "INPROGRESS" [2022-10-21 Fri 15:08]
- State "INPROGRESS" from "NEXT"       [2022-10-21 Fri 15:08]
:END:

#+begin_src nix :tangle ~/arroyo-nix/pkgs/svgelements.nix
# https://github.com/meerk40t/svgelements
{ lib
, buildPythonPackage
, fetchFromGitHub
, pytest
, numpy
, scipy
}:

buildPythonPackage rec {
  version = "1.6.12"; # not quite!
  src = fetchFromGitHub {
    owner = "meerk40t";
    repo = "svgelements";
    rev = "761bb315a6c12a8fcea990276570780a07fc492f";
    sha256 = "sha256-DQU+88Twt6J2TLa745kgS9UKNcYxLpH+ti8uxymS4Rw=";
  };

  pname = "svgelements";

  propagatedBuildInputs = [ numpy scipy ];

  nativeCheckInputs = [ pytest ];
  checkPhase = ''
    pytest test
  '';

  meta = with lib; {
    description = "svgelements does high fidelity SVG parsing and geometric rendering.";
    homepage = "https://github.com/meerk40t/svgelements";
    platforms = platforms.unix;
    maintainers = with maintainers; [rrix];
    license = with licenses; [ mit ];
  };
}

#+end_src

*** qasync
:LOGBOOK:
- State "DONE"       from "INPROGRESS" [2022-10-21 Fri 15:11]
- State "INPROGRESS" from "NEXT"       [2022-10-21 Fri 15:09]
:END:

#+begin_src nix :tangle ~/arroyo-nix/pkgs/qasync.nix
{ lib
, buildPythonPackage
, fetchFromGitHub
, pyside2
, pytest
}:

buildPythonPackage rec {
  version =  "0.23.0";
  src = fetchFromGitHub {
    owner = "CabbageDevelopment";
    repo = "qasync";
    rev = "58882735229b0d17836621d7d09ce02a6f80789d";
    sha256 = "sha256-2wustBtShydCXM5L5IQbOaZ2BfGxbIPwLZ8sRfxFnM4=";
  };

  pname = "qasync";

  propagatedBuildInputs = [ pyside2 ];

  # sigabrt
  # checkInputs = [pytest];
  # checkPhase = ''
  #   pytest
  # '';

  doCheck = false;

  meta = with lib; {
    description = "qasync allows coroutines to be used in PyQt/PySide applications by providing an implementation of the PEP 3156 event-loop.";
    homepage = "https://github.com/CabbageDevelopment/qasync";
    platforms = platforms.unix;
    maintainers = with maintainers; [rrix];
    license = with licenses; [ lgpl21 ];
  };
}
#+end_src

*** watchfiles
:LOGBOOK:
- State "DONE"       from "INPROGRESS" [2022-10-21 Fri 15:11]
- State "INPROGRESS" from "NEXT"       [2022-10-21 Fri 15:10]
:END:

#+begin_src nix :tangle ~/arroyo-nix/pkgs/watchfiles/default.nix :mkdirp yes
{ stdenv
, lib
, anyio
, buildPythonPackage
, fetchFromGitHub
, rustc
, cargo
, rustPlatform
, setuptools-rust
, pythonOlder
, dirty-equals
, pytest-mock
, pytest-timeout
, pytestCheckHook
, python
}:

buildPythonPackage rec {
  pname = "watchfiles";
  version = "0.18.0";
  format = "pyproject";

  disabled = pythonOlder "3.7";

  src = fetchFromGitHub {
    owner = "samuelcolvin";
    repo = pname;
    rev = "refs/tags/v${version}";
    hash = "sha256-biGGn0YAUbSO1hCJ4kU0ZWlqlXl/HRrBS3iIA3myRI8=";
  };

  cargoDeps = rustPlatform.fetchCargoTarball {
    inherit src;
    name = "${pname}-${version}";
    hash = "sha256-nmkIKA4EDMOeppOxKwLSh3oREInlDIcFzE7/EYZRGKY=";
  };

  patches = [
    ./pyproject-define-version.patch
  ];

  postPatch = ''
     substituteInPlace pyproject.toml --subst-var-by "version" "${version}"
  '';

  nativeBuildInputs = [
    cargo
    rustc
  ] ++ (with rustPlatform; [
    cargoSetupHook
    maturinBuildHook
  ]);

  propagatedBuildInputs = [
    anyio
  ];

  preCheck = ''
    rm -rf watchfiles
  '';

  nativeCheckInputs = [
    dirty-equals
    pytest-mock
    pytest-timeout
    pytestCheckHook
  ];

  pythonImportsCheck = [
    "watchfiles"
  ];

  meta = with lib; {
    broken = stdenv.isDarwin;
    description = "Simple, modern file watching and code reload";
    homepage = "https://watchfiles.helpmanual.io/";
    license = licenses.mit;
    maintainers = with maintainers; [ fab ];
  };
}
#+end_src

#+begin_src diff :tangle ~/arroyo-nix/pkgs/watchfiles/pyproject-define-version.patch :comments none
--- a/pyproject.toml	2022-10-21 16:28:04.669544358 -0700
+++ b/pyproject.toml	2022-10-21 16:25:26.582827217 -0700
@@ -35,10 +35,10 @@
     'Topic :: System :: Filesystems',
     'Framework :: AnyIO',
 ]
+version = "@version@"
 dynamic = [
     'license',
     'readme',
-    'version'
 ]
 
 [project.scripts]
#+end_src

* Installing =vsketch= in [[id:cce/home-manager][home-manager]]

With the packages crammed in to Arroyo using an overlay, it's easy enough to add these in:

#+ARROYO_HOME_MODULE: hm/vsketch.nix
#+ARROYO_SYSTEM_ROLE: endpoint
#+begin_src nix :tangle ~/arroyo-nix/hm/vsketch.nix
{ pkgs, ... }:

{
  home.packages = [ pkgs.vpypePackages.vsketch pkgs.vpypePackages.vpype ];
}
#+end_src

* Installing =vsketch= in a VM

I tried a few times a few ways to get this =vsketch= thing working on NixOS before giving up and setting up a [[id:fedora_linux][Fedora Linux]] VM... Running [[id:cce/python][Python]] stuff on NixOS is still so painful...

,#+ARROYO_NIXOS_MODULE: nixos/qemu.nix
#+begin_src nix :tangle ~/arroyo-nix/nixos/qemu.nix
{ pkgs, ... }:
{:tangle ~/arroyo-nix/ages = [ pkgs.virt-manager ];
  virtualisation.libvirtd = {
    enable = true;
    onBoot = "ignore";
  };
}
#+end_src

- install fedora 35
- set up a filesystem share "add hardware" in virt-manager ui, while it installs
- =sudo dnf install pipx=
- =sudo mount -t 9p -o trans=virtio plotting /mnt/plotting -o version=9p2000.L=
- =systemctl enable sshd --now=
- =pipx install "vpype[all]"=
- =pipx inject vpype git+https://github.com/abey79/vsketch --include-apps=

* NEXT extract versions to [[id:cce/version_pins][Nix Version Pins]]