CVAT is a really neat labelling platform, online + free on-premise w/ Docker.
(Github: cvat-ai/cvat: Annotate better with CVAT, the industry-leading data engine for machine learning. Used and trusted by teams at any scale, for data of any scale.)

I like it more than label studio for images, has more functions, but is also “heavier” / bulkier.

Love how it supports even 600mb 4-channel TIFF satellite images and is quite fast at that.

Bits:

• Enable auto-save every N minutes
• <C-a> for snipping polygons to existing polygon ponits
• “Backup project” to re-import later, “Export project” to get e.g. YOLO annotations

Pydantic’s FilePath is like Path except that the file has to exist and be a file.

BUT FilePath when validating expects a string as input, not a Path! (in other words: FilePath(Path) doesn’t seem to work)

So when I create a Validator that converts str into Path ¹:

@field_validator("filename", mode="before") 
@classmethod 
def parse_filename(cls, value: str | Path) -> Path: 
	return Path(value)

I get a wonderful

>       doc = UCFDocument.model_validate_json(json_string)
E       pydantic_core._pydantic_core.ValidationError: 1 validation error for UCFDocument
E       filename
E         Input is not a valid path for <class 'pathlib.Path'> [type=path_type, input_value=PosixPath('/home/sh/w/cor...n/doc.pdf_data/doc.pdf'), input_type=PosixPath]

tests/ucf/test_data_structures.py:179: ValidationError

Again, the error is a PosixPath not being a Path, though it is one:

E         Input is not a valid path for <class 'pathlib.Path'> [type=path_type, input_value=PosixPath('/home/sh/w/cor...n/doc.pdf_data/doc.pdf'), input_type=PosixPath]

# explicitly expecting a PosixPath creates an even better
E         Input is not a valid path for <class 'pathlib.PosixPath'> [type=path_type, input_value=PosixPath('/home/sh/w/cor...n/doc.pdf_data/doc.pdf'), input_type=PosixPath]

Not intuitive at all.

Solution is to give FilePath strings and only strings, or drop FilePath to begin with.

├── pydantic v2.10.6
│   ├── annotated-types v0.7.0
│   ├── pydantic-core v2.27.2
│   │   └── typing-extensions v4.12.2

The Open LLM Leaderboard is dead¹, as good time as any to look for new eval stuff!

• HF universe

  • The Open LLM Leaderboard people are actually OpenEvals (OpenEvals), and they created other cool stuffs
    • huggingface/lighteval: Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends eval suite
      • I like their documentation for new task / new model
    • huggingface/evaluation-guidebook: Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!
      • the contents are awesome, w/ examples, model-as-a-judge, etc.
  • evaluate-metric (Evaluate Metric) recommended by them as a guide to existing metrics

• Harnesses

  • Evalverse: Unified and Accessible Library for Large Language Model Evaluation meta-thing that can run different harnesses based on the target. Github repo archived? UpstageAI/evalverse: The Universe of Evaluation. All about the evaluation for LLMs.
  • open-compass/opencompass: OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.
    • don’t really like their documentation on adding datasets/models
  • The venerable stanford-crfm/helm: Holistic Evaluation of Language Models (HELM)
  • openai/evals: Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

• Resources / articles:

  • How to evaluate by Meta: Validation | How-to guides
  • HF eval guidebook: huggingface/evaluation-guidebook

Backup and Restore | Gitea Documentation has the full detailed story.

The easy stupid way for backing up gitea running in docker, untested and allegedly will fail if DB was being used during dumping.

docker exec -it --user git gitea-container bash

gitea dump

# then outside the container, copy from the gitea container to host OSooj

docker cp gitea-container:/whatveer/gitea-dump.zip /tmp

Importing: the docs don’t have the correct paths, not easy to follow.

EDIT: if your docker has /data mounted somewhere local, just copying that directory somewhere might work.

• M.2 slots have keys: How do I install an M.2 SSD on my computer? - Transcend Information, Inc.
• B+M means both B and M slots are acceptable for the B+M module

CLI: sudo lshw -C disk tells you all disks h

In fsspec fs.copy() doesn’t really work from local to remote, also existing or not-existing directories etc.

Their documentation has a whole page on this: Copying files and directories — fsspec 2024.10.0.post13+gdbed2ec.d20241115 documentation

GitLab CLI - glab | GitLab Docs: CLI thingy to interact with gitlab.

TL;DR: glab ci status or whatever.

It’s really neat and has a cool CLI interface, either you set things through flags or you get a neat menu to choose from!

Smart enough to parse current directory!

Git Tag: A Tutorial for Tagging Releases in Git - DEV Community

#ligthweight tag
git tag v1.0.0

# full
git tag -a v1.0.0 -m "Releasing version v1.0.0"

Tags don’t get pushed automatically. For this, git push origin v1.0.0

• Writing your pyproject.toml - Python Packaging User Guide
• uv uses hatch which has this to say about dynamic versioning: Configuring project metadata - Hatch

[project] 
dynamic = ["version"] 

[tool.hatch.version] 
path = "..."

Path is a python file w/ version info. If using src layout, src has to be included in the path.j

For uv, this works. Described for example here: Versioning Python Projects with Hatch (I like __init__.py though, not about as that guide does)

uv-dynamic-versioning · PyPI exists but I don’t really see why.

Because every single goddamn time

Command line: How do you rotate a PDF file 90 degrees? - Unix & Linux Stack Exchange

pdftk input.pdf cat 1-endwest output output.pdf

1- is needed because page range, here for all pages.

endwest etc from man page:

 [<begin page number>[-<end page number>[<qualifier>]]][<page rotation>]

The qualifier can be even or odd, and the page rotation can be north, south, east, west, left, right, or down.
Each option sets the page rotation as follows (in degrees): north: 0, east: 90, south: 180, west: 270, left: -90, right: +90, down: +180. left, right, and down make relative adjustments to a page’s rotation.

• Just got sold on this as a way to automate creation of config maps, secrets and friends!
  • Though allegedly hard-ish to learn
• Need to look deeper.

This is brilliant: Git, rewrite previous commit usernames and emails - Stack Overflow

TL;DR

git config --global alias.change-commits '!'"f() { VAR=\$1; OLD=\$2; NEW=\$3; shift 3; git filter-branch --env-filter \"if  \\\"\$\`echo \$VAR\`\\\" = '\$OLD' ; then export \$VAR='\$NEW'; fi\" \$@; }; f"

Then

git change-commits GIT_AUTHOR_NAME "old name" "new name"

# last 10 commits
git change-commits GIT_AUTHOR_EMAIL "old@email.com" "new@email.com" HEAD~10..HEAD

Depending on why I need this, I may need also GIT_COMMITTER_[NAME/EMAIL]

For multiple times, I created an change-commit-f that forces overwiriting the backup:

git config --global alias.change-commits-f '!'"f() { VAR=\$1; OLD=\$2; NEW=\$3; shift 3; git filter-branch -f --env-filter \"if  \\\"\$\`echo \$VAR\`\\\" = '\$OLD' ; then export \$VAR='\$NEW'; fi\" \$@; }; f"

A quick tldr from other answer, may be better but untested:

git config alias.change-commits '!'"f() { VAR=\$1; OLD=\$2; NEW=\$3; shift 3; git filter-branch --env-filter \"if  \\\"\$\`echo \$VAR\`\\\" = '\$OLD' ; then export \$VAR='\$NEW'; fi\" \$@; }; f "
git change-commits GIT_AUTHOR_NAME "<Old Name>" "<New Name>" -f
git change-commits GIT_AUTHOR_EMAIL <old@email.com> <new@email.com> -f
git change-commits GIT_COMMITTER_NAME "<Old Name>" "<New Name>" -f
git change-commits GIT_COMMITTER_EMAIL <old@email.com> <new@email.com> -f

(Previously: 220408-1822 Gitlab ‘you cannot push commits for ..’ error)

Lines of code that beat A/B testing (2012) | Hacker News / 20 lines of code that will beat A/B testing every time

TL;DR: A/B testing that automatically increases how often “good” versions are shown

First impressions

• INCEpTION User Guide

  • Docker works:
    • docker run -it --name inception -p8080:8080 ghcr.io/inception-project/inception:35.1
    • $ docker run -it --name inception -v /srv/inception:/export -p8080:8080 ghcr.io/inception-project/inception:35.1
      • data will be in /srv/inception

• Creating a project automatically fills it with sample data:

• Tagsets

  • Export format is:

  {
    "name" : "BBK",
    "description" : null,
    "language" : null,
    "tags" : [ {
      "tag_name" : "aaa_human_processed",
      "tag_description" : null
    }, {
      "tag_name" : "block",
      "tag_description" : null
    } ],
    "create_tag" : false
  }
  

  • Import format: I can get it do do only txt, one tag per line, first line is name of tagset

• A layer has to be linked to a feature (string) which then can be linked to a tagset: (INCEpTION User Guide)

  • then you can add keybindings manually
  • and the “editor type” for the tag list is neat, “Radio group” works nicely for tagsets it doesn’t consider small

• annotations get saved automatically

• in the viewer, you can set dynamic for annotations differing based on color

Import/Export

• Admin->Export can
  • export the entire projects
  • project + separately a copy of the anntations
• To import a project: admin->projects, where you can create a new project, you can also import. You import the .zip without extraction

Features

• Supports overlapping annos
• Supports annos across page boundaries
• Doesn’t support gaps in annos!

Resources

• Not-official but really nice tutorial: 12. INCEpTION — New Languages for NLP

Using uv as your shebang line – Rob Allen (HN comments) and more detailed article on this: Lazy self-installing Python scripts with uv

But especially Defining Python dependencies at the top of the file – Rob Allen and the PEP 723 – Inline script metadata | peps.python.org

You can add uv to the shebang line as

#!/usr/bin/env -S uv run --script

And you can set requirements by adding this under the shebang line:

# /// script
# requires-python = ">=3.11"
# dependencies = [
#   "flickrapi",
# ]
# ///

Then you can uv run sync-flickr-dates.py

Full package:

#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
#   "flickrapi",
# ]
# ///
import flickrapi
print("\nI am running")

❯ chmod +x test.py
❯ ./test.py
Installed 11 packages in 134ms

I am running!

Neat!

git - How to cherry-pick multiple commits - Stack Overflow:

For one commit you just pase its hash.
For multiple you list them, in any order.
For a range, you do oldest-latest but add ~, ^ or ~1 to the oldest to include it. Quoting directly from the SO answer:

# A. INCLUDING the beginning_commit
git cherry-pick beginning_commit~..ending_commit
# OR (same as above)
git cherry-pick beginning_commit~1..ending_commit
# OR (same as above)
git cherry-pick beginning_commit^..ending_commit 

# B. NOT including the beginning_commit
git cherry-pick beginning_commit..ending_commit

So, given that kubectl cp was never reliable ever for me, leading to many notes here, incl. 250115-1052 Rancher much better way to copy data to PVCs with various hacks and issues like 250117-1127 Splitting files, 250117-1104 Unzip in alpine is broken issues etc. etc. etc.

For many/large files, I’d have used rsync, for which ssh access is theoretically needed. Not quite!

rsync files to a kubernetes pod - Server Fault

ksync.sh (EDIT Updated by ChatGPT to support files with spaces):

if [ -z "$KRSYNC_STARTED" ]; then
    export KRSYNC_STARTED=true
    exec rsync --blocking-io --rsh "$0" "$@"
fi

# Running as --rsh
namespace=''
pod=$1
shift

# If user uses pod@namespace, rsync passes args as: {us} -l pod namespace ...
if [ "X$pod" = "X-l" ]; then
    pod=$1
    shift
    namespace="-n $1"
    shift
fi

# Execute kubectl with proper quoting
exec kubectl $namespace exec -i "$pod" -- "$@"

Usage is same as rsync basically :

./ksync.sh -av --info=progress2 --stats /local/dir/to/copy/  PODNAME@NAMESPACE:/target/dir/

(Or just --progress for per-file instead of total progress).

Rsync needs to be installed on server for this to work.

For flaky connections (TODO document better): -hvvrPt --timeout1 and while ! rsync ..; do sleep 5; done ¹

TL;DR pipx inject target_app package_to_inject

• pipx is awesome, and can install apps in their own virtualenv
• If you try e.g. pipx psutil it refuses, it’s a library, not an app
  • and tells you to use pip
• If I want psutil for the MemoryGraph widget in (pipx install-ed) qtile, that doesn’t help
• What does: pipx inject qtile psutil

❯ pipx inject qtile psutil
  injected package psutil into venv qtile
done! ✨ 🌟 ✨

If no real config thingy is required/wanted, then this works (stolen from Parsing Dictionary-Like Key-Value Pairs Using Argparse in Python | Sumit’s Space)¹:

def parse_args():
    class ParseKwargs(argparse.Action):
        def __call__(self, parser, namespace, values, option_string=None):
            setattr(namespace, self.dest, dict())
            for value in values:
                key, value = value.split("=")
                getattr(namespace, self.dest)[key] = value
    parser.add_argument("--no-pics", action="store_true", help="Predict only on videos")
	# ...

    parser.add_argument(
        "-k",
        "--kwargs",
        nargs="*",
        action=ParseKwargs,
        help="Additional inference params, e.g.: batch=128, conf=0.2.",
    )

#!/bin/bash

BATTINFO=$(acpi -b)
LIM="00:15:00"
if  grep Discharging) && $(echo $BATTINFO | cut -f 5 -d " ") < $LIM ; then
  # DISPLAY=:0.0 /usr/bin/notify-send "low battery" "$BATTINFO"
  dunstify "low battery" "$BATTINFO"
fi

For this, install and run on startup dunst, then cron job for the above.

• Assigning Pods to Nodes | Kubernetes
  • Operators

spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: kubernetes.io/hostname
                operator: NotIn
                values:
                  - node_to_avoid

(operator: In for the list of the allowed nodes)

E.g. to upload it somewhere where it’s hard to upload large files

See also: 250117-1104 Unzip in alpine is broken

# split
split -b 2G myfile.zip part_
# back
cat part_* > myfile.zip

docker - Why do I get “unzip: short read” when I try to build an image from Dockerfile? - Stack Overflow:

TL;DR alpine’s unzip is busyboxes, and fails for me with

/data/inference_data # unzip rd1.zip
Archive:  rd1.zip
unzip: short read

apk add unzip installs the same real one I have on all other computers, and then it works.

Related: 250115-1238 Adding wandb to a CLI yolo run

• Reference are surprisingly hard to find on the website: results - Ultralytics YOLO Docs

• yolo detect train model=yolo11s.pt data=/data/data/data.yaml project=/data/project/ epochs=500 imgsz=640 device=0,1 name=yolo11s-aug-500epochs-full

• YOLOv11 sets default batch_size 16, one can set -1 for it to automatically pick one that’s 60% of GPU, or 0.8 to automatically pick one that’s 80% of GPU

• To decrease verbosity in predictions, verbose=False to model.predict() (and `.track()) works¹.

• Changing imgsz= to something lower may not necessarily make it faster, if a model was trained with a certain size it may predict faster at that size (e.g. OSCF/TrapperAI-v02.2024 predicts at 40+ iterations per second when resized to 640 and ~31 when left to its default 1024pd)

  • Resizing (if provided a single int, not a tuple) works by making the larger side of the image equal to the given one, if padding is needed grey is used(?)

• Half-life precision (if supported by GPU) is really cool! half=True makes stuff faster (no idea about prediction quality yet)

  • And batch size obviously

• vid_stride predicts every Nth video frame, was almost going to write that myself

All-in-all I like ultralytics/YOLO

Assuming you’re doing a YOLO run w/

yolo detect train model=yolo11s.pt data=/data/data/data.yaml project=/data/project/ epochs=500 imgsz=640 device=0,1 name=yolo11s-aug-500epochs-full

• name there becomes training run name in wandb + directory name in /data/projct
  • (project on wandb will be `-data-project-)
• To enable wandb:

pip install wandb
yolo settings wandb=True
wandb login 

Or if you’re inside an ultralytics:latest Docker container,

apt install -y bash screen
bash
pip install wandb
yolo settings wandb=True
wandb login 
screen 
yolo detect train model=yolo11s.pt data=/data/data/data.yaml project=/data/project/ epochs=500 imgsz=640 device=0,1 name=yolo11s-aug-500epochs-full

Also useful:

# get a model file
wandb artifact get /proje:ject/run_alxxxpy7_model:v0 --root target_director/

TL;DR:
git clean -d -f .

If a broken clone / switch leaves stray broken files: error: The following untracked working tree files would be overwritten by checkout:, that fixes it.¹

apiVersion: v1
kind: Pod
metadata:
  name: temp-pod
  namespace: CHANGEME-ns
spec:
  restartPolicy: Never
  containers:
    - name: temp-pod
      image: alpine:latest # Use a lightweight image
      command: ["sleep", "3600"] # Keep the pod running for 1 hour
      volumeMounts: # Mount the persistent volume
        - name: data
          mountPath: /data
  volumes: # Specify the persistent volume claim
    - name: data
      persistentVolumeClaim:
        claimName: CHANGEME-pvc

So basically trivial temporary pod done specifically to copy stuff to a PVC that auto-kills itself in 1h w/ no need to do anything for this — ty KM for this!¹

Then one can happily copy stuff from there using the usual:

❯ kubectl cp CHANGEME-ns/temp-pod:/data/project/train7/ .
# as well as 
kubectl exec -it temp-pod -n CHANGEME-ns -- sh

ADDITIONALLY, I had issues cp ing things larger than 6gb. Reminder to myself that one can tar things or just transfer in separate parts.

I had exotic not enough shared memory crashes, ty GC for giving me these lines I do not yet understand but that seem to work, later I’ll dig into why (TODO)

apiVersion: v1
kind: Pod
metadata:
  name: CHANGEME
  namespace: CHANGEME-ns
spec:
  restartPolicy: Never
  containers:
    - name: sh-temp-yolo-container-3
      image: ultralytics/ultralytics:latest
      command: ["/bin/sh", "-c"]
      args: 
        - "yolo detect train model=yolo11s.pt data=/data/data/data.yaml project=/data/project/ epochs=30 imgsz=640 device=0,1"
      resources:
        requests:
          nvidia.com/gpu: "2" # GPUs for each training run
          ephemeral-storage: "12Gi"
        limits:
          nvidia.com/gpu: "2" # same as requests nvidia.com/gpu
          ephemeral-storage: "14Gi"
      volumeMounts: # Mount the persistent volume
        - name: data
          mountPath: /data
        - name: shared-memory
          mountPath: /dev/shm
  volumes: 
    - name: shared-memory
      emptyDir:
        medium: Memory
    - name: data
      persistentVolumeClaim:
        claimName: sh-temp-yolo-pvc

Both requests AND limits, as well as mount shared memory in volumeMounts + volumes.

For one-off HTML exports, found the plugin KosmosisDire/obsidian-webpage-export: Export html from single files, canvas pages, or whole vaults. Direct access to the exported HTML files allows you to publish your digital garden anywhere. Focuses on flexibility, features, and style parity.

It exports both the vault and individual pages, and adds things like toc on the left and toggles and optionally file browsing. Much better than the other pandoc-based export plugin that I could not get to work reliably for exporting good-looking HTML

• Qutebrowser:
  :set colors.webpage.darkmode.enabled true Really neat actually! ALSO: colors.webpage.preferred_color_scheme: dark tells websites my preference
• Nvim: :colorscheme zaibatsu
• Redshift: redshift -r -P -O 4000 -b 0.3
• If I’m doing all that, I probably want to mute my speakers as well

Nested list comprehensions are a horrible idea because they are hard to parse, and I never understood them, BUT.¹

python - How do I make a flat list out of a list of lists? - Stack Overflow has a discussion in the accepted answer about the suggested syntax to flatten lists, and I get it now.

flat_list = [
    x
    for xs in xss
    for x in xs
]

# equivalent to 
flat_list = []

for xs in xss:
    for x in xs:
        flat_list.append(x)

So,

[x for xs in xss for x in xs]

Comments:

I found the syntax hard to understand until I realized you can think of it exactly like nested for loops. for sublist in l: for item in sublist: yield item

[leaf for tree in forest for leaf in tree]

I kept looking here every time I wanted to flatten a list, but this gif is what drove it home: i.sstatic.net/0GoV5.gif 

GIF IN QUESTION, after which it clicked for me:

The first element is the one that gets returned!

for tree in forest: for leaf in tree: return leaf
[leaf (for tree in forest, for leaf in tree)]
[leaf (for tree in forest for leaf in tree)]
[leaf for tree in forest for leaf in tree]

Found Understanding nested list comprehension syntax in Python — /var/ which expands on this, quoting PEP

It is proposed to allow conditional construction of list literals using for and if clauses. They would nest in the same way for loops and if statements nest now.

It then shows:

for x in non_flat:
    if len(x) > 2
        for y in x:
            y
# equivaent to
>>> [ y for x in non_flat if len(x) > 2 for y in x ]

MIND. BLOWN.

I’m not sure “this requires you to understand Python syntax” is an argument against using a given technique in Python This is about itertools.chain(*list, which is the way to go imo. But still, * is python syntax, otherwise there are more or less readable ways to do thigs and nested comprehensions are rarely worth it

TL;DR clone to unencrypted directory

error: unable to create file datasets/processed/GitHub-Mensch-Animal_Finetuned/data/val/labels/1713256557366,hintergrund-meister-lampe-geht-das-licht-aus-vom-rueckgang-der-arten-tierische-und-pflanzliche-neubuerger-108~v-16x9@2dM-ad6791ade5eb8b5c935dd377130b903c4b5781d8.txt: File name too long

error: cannot stat ‘datasets/processed/GitHub-Mensch-Animal_Finetuned/data/val/images/1713256557366,hintergrund-meister-lampe-geht-das-licht-aus-vom-rueckgang-der-arten-tierische-und-pflanzliche-neubuerger-108~v-16x9@2dM-ad6791ade5eb8b5c935dd377130b903c4b5781d8.jpg’: File name too long

The usual solution¹ is to set longpaths = true in git config or during clone (git clone -c core.longpaths=true <repo-url>)

Didn’t solve this for me.

BUT apparently my encrypted $HOME has something to do with this, because filenames can get longer (?) in this case and go over the limit?.. git checkout-index: unable to create file (File name too long) - Stack Overflow

And one solution is to clone to /tmp or whatever is not encrypted by encryptfs.

(And in my case I could rename these files in a commit in /tmp and after that it worked, as long as I don’t check out the revisions with the long filenames)

• Goal: centerline trace pngs
• Inscape
  • Script, extension, now part of Inkscape: inkscape-centerline-trace/centerline-trace.py at master · fablabnbg/inkscape-centerline-trace
  • Inkscape as a purely command line tool : r/Inkscape reddit on using the CLI and running the extension standalone and how it doesn’t really work
  • Can’t get it to work using CLI either, there’s just bitmap-trace that has no centerline option
• Not inkscape
  • autotrace/autotrace: bitmap to vector graphics converter
  • Potrace
    • no centerline

Autotrace is awesome!

This alone works really nicely:

autotrace -centerline AMPERSAND.png  -output-file AMPERSAND.svg

Fish script for batch processing, courtesy of ChatGPT:

#!/usr/bin/fish

# Check if autotrace is installed
if not type -q autotrace
    echo "autotrace is not installed. Please install it first."
    exit 1
end

# Loop through each .png file provided as an argument
for file in $argv
    # Check if the file extension is .png
    if string match -r '\.png$' $file
        # Set the output filename by replacing .png with .svg
        set output_file (string replace -r '\.png$' '.svg' $file)

        # Execute autotrace with centerline option
        autotrace -centerline $file -output-file $output_file

        # Confirmation message
        echo "Processed $file to $output_file"
    else
        echo "Skipping $file: not a .png file"
    end
end

And a more simple one:

#!/usr/bin/fish
for file in $argv
    autotrace -centerline $file -output-file "$file.svg"
end

Options

ChatGPT says this:

autotrace -centerline -input-format png -output-format svg -output-file traced_dejavu.svg -dpi 300 -error-threshold 0.5 -corner-threshold 85 -filter-iterations 2 -noise-removal 0.99 -line-threshold 0.5 -corner-surround 3

(et 1 is best)

Using the Command Line - Inkscape Wiki

• inkscape action-list shows all available actions

• man inkscape is the latest and best

• inkscape AMPERSAND.png --export-type="svg" --export-area-page --batch-process works but asks me about import options

inkscape --shell, man page gives examples:

file-open:file1.svg; export-type:pdf; export-do; export-type:png; export-do
                   file-open:file2.svg; export-id:rect2; export-id-only; export-filename:rect_only.svg; export-do

OK this works for no questions about how to import it:

> file-open:AMPERSAND.png
> export-filename:AM.svg
> export-do

• Quarto markdown is a superset of pandoc markdown, unrelated article about the latter: Everything pandoc Markdown can do
• Pandoc has a python library: Pandoc (Python Library)
• panflute is a more python way to parse docs and write pandoc filters, and I love it: User guide — panflute 2.3.0 documentation

I missed an ability to recursively look for elements matching a condition in panflute, so:

def _recursively_find_elements(
    element: Element | list[Element], condition: Callable
) -> list[Element]:
    """Return panflute element(s) and their descendants that match conditition.
    """
    results = list()

    def action(el, doc):
        if condition(el):
            results.append(el)

    if not isinstance(element, list):
        element = [element]

    for e in element:
        e.walk(action)


    return results

# sample condition
def is_header(e) -> bool:
	cond = e.tag == "Header" and e.level == 2  # and "data-pos" in e.attributes
	return cond

Ah, to read:

ddoc = pf.convert_text(
	markdown,
	input_format="commonmark_x+raw_html+bracketed_spans+fenced_divs+sourcepos",
	output_format="panflute",
)

To output readably:

pf.stringify(el).strip() 

Pandoc/panflute get line numbers of elements

• input_format has to be commonmark[_x]+sourcepos
  • sourcepos isn’t too well documented, only w/ commonmark
  • it basically sets el.attributes['data-pos'] a la 126:1-127:1
  • line_no always matching what I expect

def _parse_data_pos(p: str) -> tuple[tuple[int, int], tuple[int, int]]:
	"""Parse data-pos string to (line, char) for start and end.
	
	Example: '126:1-127:1' -> ((126, 1), (127, 1))
	
	Arguments:
		p: data-pos string as generated by commonmark+sourcepos extension.
	"""
	start, end = p.split("-")
	start_l, start_c = start.split(":")
	end_l, end_ch = end.split(":")
	return (int(start_l), int(start_c)), (int(end_l), int(end_ch))

One control, play/pause.

const buttons = document.querySelectorAll('.play-pause-btn');

buttons.forEach(button => {
    const audio = document.getElementById(button.dataset.audio);

    button.addEventListener('click', () => {
        if (audio.paused) {
            // Pause all other audio files
            document.querySelectorAll('audio').forEach(a => {
                a.pause();
                a.currentTime = 0; // Reset other audio files
            });
            document.querySelectorAll('.play-pause-btn').forEach(btn => {
                btn.textContent = '▶';

            });

            // Play the clicked audio
            audio.play();
            button.textContent = '⏸︎';

        } else {
            audio.pause();
            button.textContent = '▶';
        }
    });

    // Reset button icon when audio ends
    audio.addEventListener('ended', () => {
        button.textContent = '▶';
    });
});

Multiple players:

<div class="player-container">
	<button class="play-pause-btn" data-audio="audio1">▶️</button>
	<audio id="audio1" src="audio1.mp3"></audio>
</div>

<div class="player-container">
	<button class="play-pause-btn" data-audio="audio2">▶️</button>
	<audio id="audio2" src="audio2.mp3"></audio>
</div>

.player-container {
    display: inline;
    vertical-align: text-bottom;
    align-items: center;
    margin-bottom: 20px;
}
.play-pause-btn {
    font-size: 32px;
    background: none;
    border: none;
    cursor: pointer;
    margin-right: 10px;
}

Fish Shell function for sourcing standard .env files :

 . (sed 's/^/export /' .env | psub)

(And yet another mention of Taskfile that I’ll definitely look into nowG)

I want to automatically get the PDF version of quarto/reveal presentations. The usual way would be to open the presentation in export mode e, then print with no margins through the usual print window.

I want to do this automatically as part of a CI/CD pipeliene.

Options

• Generate PDF from revealjs presentation (consistent with PDF Export in revealjs) · quarto-dev/quarto-cli · Discussion #7018
• selenium-print · PyPI / bubblegumsoldier/selenium-print uses selenium+chromium to do this.
• PDF Export | reveal.js
  • Support other PDF export tools for revealjs (e.g decktape) · Issue #4677 · quarto-dev/quarto-cli

Selenium-print

selenium-print · PyPI / bubblegumsoldier/selenium-print uses selenium+chromium to do this.

As for the printing options available in Chrome, this looks relevant:

selenium-print/seleniumprint/drivers/chrome_pdf_driver.py at main · bubblegumsoldier/selenium-print

        pdf = self.driver.execute_cdp_cmd("Page.printToPDF", {"printBackground": True})

OK, so it’s all a static option.

Chrome DevTools Protocol - Page domain has the other available options — which is what I need.

The rest of the code feels like a wrapper to this — maybe I can drop the entire library and just use these single bits?

Decttape (?)

• Generate PDF from revealjs presentation (consistent with PDF Export in revealjs) · quarto-dev/quarto-cli · Discussion #7018

• Command substitutions and variables: Fish for bash users — fish-shell 3.7.1 documentation
• string collect: string-collect - join strings into one — fish-shell 3.7.1 documentation
• SO: Read file into environment variable? · Issue #7323 · fish-shell/fish-shell:

set VARNAME (cat ~/myfile | string collect)

Here string collect makes sure it’s a multiline variable instead of an array composed of one element per line.

TL;DR use your own personal settings, then “dev settings” (!), then create one but set the resource owner to the organization.

(As of 2024-10-14. Hard to find clear AND CORRECT documentation on this.)

Create access token for organization · community · Discussion #74701.

Gitlab mirroring didn’t work for me after trying for hours, I give up. CI/CD it is.

• Refs:
  • Predefined CI/CD variables reference | GitLab
  • How do I push to a repo from within a gitlab CI pipeline? - Stack Overflow

• Font Finder By Image — Fontspring Matcherator allows manually specifying glyphs and shows all paid fontns
• WhatTheFont Font Finder - Identify Fonts by Image is the very best, instantly parsed the noisy image and the fonts looked much closer (and found some very famous ones missed by the first)

On a slow connection or other constraints, you can check out only the last N commits or only a specific branch:

git clone --depth 3 --branch some-branch https://some-repo.org

Git - git-clone Documentation

TIL¹:

• ./git/info/exclude is your local .gitignore outside the repository tree!
• git update-index --assume-unchanged .gitignore makes git stop checking the changes for that file in the working tree. --no-assume-unchanged to take it back.²³

Finally a place for my local ignores that won’t appear in autocomplete suggestions for git add and friends. In Pycharm I have changelists, and now I finally have a solution for my just-as-usual vim/git/CLI workflow as well.

BUT:

• exclude won’t work if the file is already tracked (says SO but for me it works?..)

As of 2024-10-02 20:16 at least.

give me the ag command to look inside markdown and yaml files only

GPT4o¹:

`ag --include='*.md' --include='*.yaml' --include='*.yml' 'search_pattern'`

GPT4²:

`ag "search_pattern" --markdown --yaml`

github/git-sizer: Compute various size metrics for a Git repository, flagging those that might cause problems (linked by Repository limitations and recommendations)

TL;DR too many files? too large one? too many tags? Find out!

yay took forever to compress pycharm. Like, 5-10 mins at least.

TODO documentation, but if you don’t care about package size but care about speed, this will speed up everything considerably:

PKGEXT='.pkg.tar' yay -S pycharm-professional

Wanted to do sth like this all the time, but the help basically told me to copypaste multiple arguments etc.

Will come back to bite me but nice to have the option I guess

Question: How can I install packages without having to confirm? · Issue #1033 · Jguer/yay:

echo y | LANG=C yay --noprovides --answerdiff None --answerclean None --mflags "--noconfirm" $PKGNAME

(--noconfirm is not documented in help and man, and of course can create problems¹)

BUT ALSO

yes is a command that exists.

Then: yes | yay ... or yes | LANG=C yay

And generally really neat to have a command for answering “yes” to prompts.

And then quick unsafe bad dangerous command to update everything that includes 241001-1512 Yay better compression algos:

echo y | LANG=C PKGEXT='.pkg.tar'  yay  --answerdiff None --answerclean None --mflags "--noconfirm"

• You can share lists in Google Maps, making them editable by everyone with a google account
• Google My Maps is much cooler, with layers etc

Google My Maps

• Really nice tutorial on everything: Visualize your data on a custom map using Google My Maps – Google Earth Outreach
• To share it editable, you share -> Share on Drive
• drag and drop doesn’t always work
• style and colors
  • layer can be styled uniformly (easy)
  • style by data column: per-field, e.g. description, like a groupby
  • individual styles per element
• you can drag and drop items between layers only if they’re styled individually??
• If you style by e.g. description, assign icons/colors, then style individually to move them, then back to style by description, it seems to save the previous settings

In pycharm, updated black to use these args: --preview --enable-unstable-feature string_processing $FilePath$

The (future of the) Black code style - Black 24.8.0 documentation has both the preview style and the unstable features one can enable (by passing the flag multilple times).

string_processing breaks long strings into multiple shorter ones, one on each line.

• Useful:

  • Running mypy and managing imports - mypy 1.12.0+dev.ecfab6a02415c46eda5717ec6ee9bfac8115c1e9.dirty documentation evergreen
  • Error codes - mypy 1.12.0+dev.ecfab6a02415c46eda5717ec6ee9bfac8115c1e9.dirty documentation
  • The mypy configuration file - mypy 1.12.0+dev.ecfab6a02415c46eda5717ec6ee9bfac8115c1e9.dirty documentation

• mypy cares about packages, especially __init__.py files — the uppermost dir with such a file will be the root package. See Mapping files to modules - Running mypy and managing imports - mypy 1.12.0+dev.ecfab6a02415c46eda5717ec6ee9bfac8115c1e9.dirty documentation

• This is needed to do per-package configs

[mypy-mycode.package.etc]
disable_error_code = attr-defined, union-attr

• Problem: the pycharm extension I have crashes in the last version. :(

• Main and best: python - How to run Pylint with PyCharm - Stack Overflow

  • idea of multiple ’external tools’ for diff pylint options: HOWTO - Setup pylint with PyCharm and the PyLint Plugin

My changes:

• arguments: --msg-template="{abspath}:{line:5d},{column:2d}: {C}/{msg} ({symbol})" --output-format=colorized "$FilePath$"
  • note ", otherwise it failed for me, and {C} — for the message class see man page (or below) for list of format string options.
• output filters for on-click: $FILE_PATH$:\s*$LINE$\,\s*$COLUMN$

I had two options as separate tools:

• pylint for a single file
• pylint for the module — same as above, but
  • with $FileParentDir$ at the end.
  • --recursive=y — fails on no __init__.py otherwise

Worked neatly with a .pylintrc file in repo root with e.g.

[tool.pylint.main]

[tool.pylint.basic]
# "too-few-public-methods", disable
min-public-methods=0

function-naming-style="camelCase"
argument-naming-style="camelCase"
method-naming-style="camelCase"
variable-naming-style="camelCase"
attr-naming-style="camelCase"

[tool.pylint."messages control"]
disable = [
    "fixme",  # TODOs
    "import-error",  # runner has them in its environment
    "import-outside-toplevel",  # explicit requirement of XXX to import where used
    "duplicate-code"  # entangling different extensions/modules is not the solution
]

Pylint format string options from man pylint:

path   relative path to the file

abspath
	  absolute path to the file

line   line number

column column number

end_line
	  line number of the end of the node

end_column
	  column number of the end of the node

module module name

obj    object within the module (if any)

msg    text of the message

msg_id the message code (eg. I0011)

symbol symbolic name of the message (eg. locally-disabled)

C      one letter indication of the message category

category
	  fullname of the message category

For example, the former (pre 1.0) default format can be obtained with:

  pylint --msg-template='{msg_id}:{line:3d},{column}: {obj}: {msg}'

Bonus

If I have this post open, I’ll need this one anyway: Messages control - Pylint 4.0.0-dev0 documentation

So Slicer seems to use both and I need to as well, so I’ll have to learn that sooner or later.

• VTK
  • VTK advertises their Book, available online, which looks like an excellent intro about everything, terminology etc.
    • Chapter 5 - Data Representation — VTK Book
  • wiki: VTK/Tutorials - KitwarePublic but superficially less useful to me?..
• ITK

  • open-source, cross-platform library that provides developers with an extensive suite of software tools for image analysis (About | ITK)

  • Python Quick Start - ITK’s documentation

  • Examples — v5.4.0

    • Common especially has interesting things like add noise etc.: Examples — v5.4.0

  • there’s a python package docu:

    • Quick start guide — ITKPythonPackage documentation
• Slicer
  • Slicer examples as usual as always: Script repository — 3D Slicer documentation
    • Segment effects! Script repository — 3D Slicer documentation
      • Fills all internal holes inside segments. For example, fills in empty regions inside vertebral bodies segmented with Grow from seeds with intensity-based masking.
        • shows many other cool useful things as well, incl. adding segments

Is ‘file’ a keyword in python? - Stack Overflow

TL;DR python3 doesn’t care about file, regardless of what syntax highlighters think about this.

TL;DR:

• pydocstyle != flake8-docstrings · PyPI!
• read the pypy documentation
• docstring-convention=google in flake8 config file, ignores there as well together with the rest.

pydocstyle / python-flake8-docstrings is a thing. Forgot I had it installed and spent a lot of time trying to understand pycharm’s output

Usage — pydocstyle 0.0.0.dev0 documentation flake8-docstrings · PyPI

To ignore things, you don’t do:

[pydocstyle]
convention = google
ignore = D401

It’s either ignore or convention. Which quietly happened in the background, and I thought it doesn’t read my config file since D401 was still shown.

Correct would be this:

[pydocstyle]
convention = google
add-ignore = D401

EDIT GodDAMN it, pydocsyle parsing a config file doesn’t mean that flake8(-..docstring) will.

Reading the flake8 plugin docs, I should add THIS and to flake8 config file. Ignores also are set up there using the usual way.

docstring-convention=google

And the pydoctest config file search and add-ignore is irrelevant. God lost so much time to this

EDIT: this may be all wrong, in the debugger I can just edit the code and it gets automatically used by slicer — logical, since it’s just a CLI script.

It’s the .xml with the interface that’s problematic and it can’t be reloaded using the method below, or at least I couldn’t.

A scriptedcli module imported to Slicer doesn’t show for me the usual “reload” buttons as seen in scripted modules in dev. mode. To develop, I need my changes I need to reload it w/o restarting 3dslicer.

Based on this more complete example¹ linked in the script repository²

>>> mpath = "/full/path/to/the/my_module.py"
>>> factoryManager = slicer.app.moduleManager().factoryManager()
>>> factoryManager.registerModule(qt.QFileInfo(mpath))
>>> factoryManager.loadModules(["my_module"])
True

BONUS:

• To debug such a module using pycharm, adding the pydev lines inside it works (and official slicer debug extension doesn’t):

import pydevd_pycharm
pydevd_pycharm.settrace('localhost', port=5678, stdoutToServer=True, stderrToServer=True)

Links

• GUI
  • XML reference Documentation/Nightly/Developers/SlicerExecutionModel - Slicer Wiki

Debugging

TL;DR Pycharm Professional instructions work

• These instructions: SlicerRt/SlicerDebuggingTools: Extension for 3D Slicer containing various tools for module development and debugging¹
  • No need to change anything in module code
• During debugging everything freezes
• You can use pycharm python CLI to look around etc. — neat!

not TL;DR

• SlicerRt/SlicerDebuggingTools: Extension for 3D Slicer containing various tools for module development and debugging¹
  • needs pycharm Pro
  • SlicerDebuggingTools/PyDevRemoteDebug/PyDevRemoteDebug.py at master · SlicerRt/SlicerDebuggingTools
  • source of the 3dslicer bits
• pdb-attach · PyPI can attach to random programs,
  • os.getpid() is my friend
• Documentation/4.5/Developers/FAQ/Python Scripting - Slicer Wiki at some point it was possible to use python-remote-pdb
• ionelmc/python-remote-pdb: Remote vanilla PDB (over TCP sockets). 2y ago last commit
• I can use these same instructions mostly for pdb-attach, I see no reason it shouldn’t work, python is python
• it Kiiiinda works but breaks after the first button
• Giving up for now
• Update: installed Pycharm Pro, works!
  • Pycharm Pro egg was in /opt/pycharm-professional/debug-eggs/pydevd-pycharm.egg

Basics

Starting Slicer

Loads the module automatically (for some reason doing it manually doesn’t preserve it across restarts?..)

Slicer  --additional-module-path /path/to/my/extension/one/lower/than/what/i/would/pick/in/Slicer/GUI

Logging

• Not directly mentioned in the help because they state it’s always better to use the debugger²
• Slicer/Base/Python/slicer/slicerqt.py at main · Slicer/Slicer has the slicer setup
• I can do logging.warning("test") in the python console and it works
• Slicer settings allow changing the loglevel of its python console!

Module types: CLI modules

Different types of modules exist: Module Overview — 3D Slicer documentation

In the first tests there was a lot of latency.

Solution: CLI modules (not extensions) that can run in the background and whose status can be queried, which can be added to new extension from extension editor.

Now I understad this better:

• CLI modules are simple I/O, non-blocking, that’s where the networkig logic should go to
• scripted are python scripts with GUIs

• This bit about running CLI in the backgroundPython FAQ — 3D Slicer documentation

Looking into the sample code

Module of type scripted

• Created an extension of type cmake, to which I added a module of type scripted. It’s the sample thresholding extension
  • Base class: Slicer/Base/Python/slicer/ScriptedLoadableModule.py at main · Slicer/Slicer
• Sample data
  • It shows how to add sample data to the extension for testing
  • It can then be found in the Sample Data module, named after the extension itself!

Module of type scriptedCLI

• I hope it’s the same as type CLI, but only with python

• I think it’s lassoan/SlicerPythonCLIExample: Example extension for 3D Slicer that demonstrates how to make a Python script available as a CLI module

• Is a CLI script w/ an XML that has module metadata + examples

  • CLI module goes in the same examles menu
  • and a basic UI has been generated for it? Wow
  • Running it from the UI actually shows a progress bar and it doesn’t block the interface! Woohoo!

• Adding it added it to CMakeLists.xml: add_subdirectory(my_scripted_cli)

• python dev tools actually show the CLI used to run a CLI module!

  • the input arguments are filepaths, and slicer automatically generates and then deletes temporary files for this when running

• can’t debug it the usual way, I guess I’ll have to add the pydevd-pycharm things to the code

[VTK] /opt/3dslicer/bin/python-real /home/.../cli_model_service.py /tmp/Slicer-sh/IACHCI_vtkMRMLScalarVolumeNodeD.nrrd 1 /tmp/Slicer-sh/IACHCI_vtkMRMLScalarVolumeNodeF.nrrd 

(Auto-creating) GUI

• Slicer/Docs/developer_guide/parameter_nodes/gui_creation.md at main · Slicer/Slicer seems interesting

• CLI modules have an interface specified by the .xml file, and the interface is generated based on it

  • couldn’t find any documentation on how to write the XML: Documentation/Nightly/Developers/SlicerExecutionModel - Slicer Wiki

• Bits about the XML

  • channel is either input or output, and only for pics.
  • No parameter names, just their index

• If I read the volume as volume, it uses the .nrrd file extension by default — and I want .nii

  • Script repository deals with these settings, but I can’t use them from within a CLI module Script repository — 3D Slicer documentation

Auto-loading the module at startup

Module from console with arguments - Support - 3D Slicer Community

Slicer.exe --python-code "selectModule('SampleData'); getModuleGui('SampleData').setCategoryVisible('BuiltIn', False)"

File operations

Script repository — 3D Slicer documentation:

# Create a new directory where the scene will be saved into
import time
sceneSaveDirectory = slicer.app.temporaryPath + "/saved-scene-" + time.strftime("%Y%m%d-%H%M%S")
if not os.access(sceneSaveDirectory, os.F_OK):
  os.makedirs(sceneSaveDirectory)

Inspiration

• User_Manual.md · develop · OpenDose / SlicerOpenDose3D · GitLab
  • openDose3D is a cool example extension that shows many things
• SlicerTotalSegmentator/TotalSegmentator/TotalSegmentator.py at main · lassoan/SlicerTotalSegmentator

  		outputSegmentation.AddDefaultStorageNode()
  		outputSegmentation.GetStorageNode().SetFileName(output_segmentation_path)
  		outputSegmentation.GetStorageNode().ReadData(outputSegmentation)
  

Useful bits from documentation

• There’s a node type for tables
  • statistics become a table
  • also: many modules exist
    • Documentation/Nightly/Extensions/MeshStatistics - Slicer Wiki / DCBIA-OrthoLab/MeshStatisticsExtension
      • does a lot including creating a table GUI programmatically and CSV export etc.
• Colors: are not just colors, but class/label names as well

• Main links

  • Extensions — 3D Slicer documentation¹
  • Extensions Manager — 3D Slicer documentation²

• slicerrc.py is a thing! sample: 3D-Slicer-Scripts/.slicerrc.py at master · jzeyl/3D-Slicer-Scripts · GitHub

• Slicer custom application deployment to many computers - Development - 3D Slicer Community

  • SlicerQReads.exe --python-code "folder='S:\SlicerQREADS\TestImages'; import os; slicer.util.loadVolume(folder + '/' + os.listdir(folder)[0], {'singleFile': False})"
  • alternatively: Slicer.exe --python-script "/full/path/to/myscript.py" --no-splash --no-main-window³

• Slicer can run in Docker⁴

• Slicer can run in webbrowser⁵

• Slicer can run in jupyter⁵, including partially (e.g. only a single view etc!)

Slicer --help:

  --testing                                     Activate testing mode. It implies --disable-settings and --ignore-slicerrc. (default: false)
  --disable-python                              Disable python support. This is equivalent to build the application with Slicer_USE_PYTHONQT=OFF.
  --python-script                               Python script to execute after slicer loads.
  --python-code                                 Python code to execute after slicer loads.
  -c                                            Python code to execute after slicer loads. By default, no modules are loaded and Slicer exits afterward.
  --ignore-slicerrc                             Do not load the Slicer resource file (~/.slicerrc.py).
  --additional-module-path                      Additional module path to consider when searching for modules to load.
  --additional-module-paths                     List of additional module path to consider when searching for modules to load.

• Slicer modules — todo, CLI things with easy arguments

  • Documentation/Nightly/Developers/Modules - Slicer Wiki

• Slicer extensions/modules creation

  • Developing extension in 3D Slicer - Google Slides presentation
    • 
    • Sharing extensions there: TL;DR use slicer publishing thing to publish to own repo, then PR to slicer main extensions index repo (GitHub - Slicer/ExtensionsIndex: Slicer extensions index)
      • Sample JSON: ExtensionsIndex/SurfaceWrapSolidify.json at main · Slicer/ExtensionsIndex · GitHub
      • Pull requests · Slicer/ExtensionsIndex has code for existing recent extensions that may be useful

• Extension manager

  • Extensions Manager — 3D Slicer documentation ²
  • Can have different extension server URI!
  • Updating:
    • Extensions Manager — 3D Slicer documentation
    • TL;D/R extension manager can do that incl. automatically

• CI/CD exists for extensions published officially

• Misc:

  • Script repository — 3D Slicer documentation has sample uses of slicer CLI!
  • 3D-Slicer-Scripts/.slicerrc.py at master · jzeyl/3D-Slicer-Scripts · GitHub

• Script repository — 3D Slicer documentation has this:

  • This code snippet can be useful for sharing code in development without requiring a restart of Slicer.

  • gist.githubusercontent.com/pieper/a9c0ba57de3833c9f5aea68247bda597/raw/82d73d85340ffd205b01fc339b096a8607cfe02b/InstallSlicerModuleFromGit.py
  • TODO, does this mean updating extensions without restart?
  • EDIT: I can reload the code of the extension as-is

• Fazit:

  • extensions can be built (C++) or not (python)
  • enabling:
    • add to CLI path to “additional module paths” path to place where .py extension is locally
      • git pull etc.?
      • get a .zip from the repo and use the python code to get it from there and update
    • make our own extension manager repo thing and use automatic updating functionality

• Other bits

  • Making Slicer simpler by hiding many things: Simple 3D Slicer based image viewer
  • And hiding a module UI: Slicer/Modules/Scripted/DICOMPlugins/DICOMScalarVolumePlugin.py at 022a4f1a347ebb24509a50f6c8928ae26d3c5955 · Slicer/Slicer

inotifywait(1) - Linux man page

#!/bin/sh
while inotifywait --format '%:e %f' p.* *.bib; do
  sleep 1  # files get moved often
  echo "rsync!"
  rsync -avzirh --progress  p.* *.bib me@server.net:dir
done

boltctl monitor shows changes as they happens — nice when something breaks. I used to do a lot of watch boltctl list before in this exact scenario

(having a monitor subcommand in your thing says a lot about stuff, actually, — the best interpretation is that it’s written with developers in mind)

Very proud of this idea, haha :)

For absolute positioned elements one needs to “guess” the correct sizes etc., may not be trivial. Using a pixel ruler won’t help because the .absolute coords don’t map to the screen ones.

Solution:

.grid {
  // https://stackoverflow.com/questions/4191260/drawing-a-grid-using-css
  background-image:
    repeating-linear-gradient(lightblue 0 2px, transparent 2px 100%),
    repeating-linear-gradient(90deg, lightblue 0 2px, transparent 2px 100%);
  background-size: 50px 50px;
  //border: 2px solid red;
}

.smallgrid {
  background-image:
    repeating-linear-gradient(#ccc 0 1px, transparent 1px 100%),
    repeating-linear-gradient(90deg, #ccc 0 1px, transparent 1px 100%);
  background-size: 10px 10px;
  width: 100%;
  height: 100%;
}

::::: {.absolute left=0 right=0 top=0 bottom=0 .grid}
::: {.smallgrid}
:::
:::::

It’s not exactly aligned but close enough to judge the widths and heights involved!

Rectangle in the pic is:

:::: {.redrect .absolute right=200 top=130 width=200 height=100}
::::

(And the small grid can be omitted if not needed by removing the div in the middle. )

For bonus points, can be saved as a file and included if needed:

{\{< include ../_shared/smallgrid.qmd >}\}

ALSO:

// show the border of the slide
.reveal .slides {
  border: 1px dashed red;
}

DAMN.

Figures – Quarto:

:::: {layout="1,1,1],[1],[1,1,1" layout-valign="center"}

if it’s a layout I do get a valign thing. I just have to use layouts, not columns. Damn.

This doesn’t work on SVGs for me:

![](images/02/svgs/02_seqsiii.svg){.absolute top=0 left=0}

This does:

::: {.absolute top=0 left=0}
![](images/02/svgs/02_seqsiii.svg)
:::

SO.

One can literally generate a plot w/ graphviz, export as svg, open and edit in inkscape, save as optimized svg, paste into quarto, and manually add the correct class="fragment" data-fragment-index=4 bits to the relevant groups.

Insert with the usual {=html} thing.

Damn.

LeaderLine

almost works with quarto, except that line.position needs to be called on slide change — ergo likely won’t work in pdf export mode.

maurosilber/pyfragments: Animated figures for Quarto installs as-is

IT WORKS FOR PDF MODE TOO! And uses fragments

# | fig-width: 15
# | fig-height: 10
# | output: asis
import matplotlib.pyplot as plt
from pyfragments import AnimatedFigure

with AnimatedFigure() as ani:
# Data for plotting
  train_loss = [1.0, 0.6, 0.3, 0.2, 0.1]
  val_loss = [1.2, 0.8, 0.5, 0.4, 0.6]
  iterations = [1, 2, 3, 4, 5]

  plt.xlim(0, 6)
  plt.ylim(0, 1.5)

# Labels and legend
  plt.xlabel('Trainingsiterationen', fontsize=14)
  plt.ylabel('Loss', fontsize=14)
  plt.legend(loc='upper right', fontsize=12)

# Plot
  for x in range(len(train_loss)+1):
    with ani.fragment():
      plt.plot(iterations[:x], train_loss[:x], 'bo-', label='Training', linewidth=2)
      plt.plot(iterations[:x], val_loss[:x], 'go-', label='Validation', linewidth=2)

# Add a dashed vertical line in the middle
  mid_iteration = (iterations[0] + iterations[-1]) / 2

  with ani.fragment():
    plt.axvline(x=mid_iteration, color='r', linestyle='--', linewidth=1.5)

I couldn’t make a graph large enough to fill the screen, increasing fig-width didn’t help.

Solution: increase both fig-width and fig-height!

{dot}
//| fig-width: 12
//| fig-height: 9
digraph G {
    rankdir=LR;
	..

Execution Options – Quarto has figure options that lists the default sizes of figures based on output formats. Couldn’t find them because was looking in figures/graphviz etc. pages, not execution. …

For reveal slides, it’s 9 x 5.

Apparently it didn’t want to increase the 5 till I explicitly told it to, then width started increasing as well.

Both superscripts and subscripts

… are hard and you have to use a table.

digraph Neural_Network {
    rankdir=LR;
    ranksep=1.3;
    
    node [shape=circle, style=filled, fontcolor=white, fontsize=25, fillcolor="blue", color="black"];

    subgraph cluster_0 {
        node [fillcolor="#2c3e50", style="filled"];
        x1 [label=<
        <TABLE border="0" cellborder="0" cellspacing="0">
        <TR><TD rowspan="2" style="...">X</TD><TD style="...">1</TD></TR>
        <TR>                                  <TD style="...">1</TD></TR>
        </TABLE>>];
        x2 [label=<
        <TABLE border="0" cellborder="0" cellspacing="0">
        <TR><TD rowspan="2" style="...">X</TD><TD style="...">1</TD></TR>
        <TR>                                  <TD style="...">2</TD></TR>
        </TABLE>>];
    }

    sum [label=<∑<FONT color="yellow" point-size="10">(⎰)</FONT>>,  fillcolor="#27ae60", width=0.8, height=0.8, fixedsize=true];
    y [label=<y<sup>1</sup>>];
    
    edge [style=solid, color="#2c3e50"];
    x2 -> sum;
    x1 -> sum;

    edge [style=solid, color="#27ae60"];
    sum -> y;

    {rank=same; x1; x2;}
}

Commit messages starting with #14 whatever are awkward as # is the default comment in git rebase and friends.

git config core.commentchar ";"

fixes that for me.

For a one-time thing this works as well:

git -c core.commentChar="|" commit --amend

(escaping - Escape comment character (#) in git commit message - Stack Overflow)

Misc

• Papers
  • Comparison of Three 3D Segmentation Software Tools for Hip Surgical Planning - PMC: compares 3d slicer, Syngo, and Materialize¹
    • 
  • Comprehensive Review of 3D Segmentation Software Tools for MRI Usable for Pelvic Surgery Planning - PMC²: 12 software tools
    • really nice criteria, incl. extensibility etc.
    • 
    • 
    • TL;DR: ITK-SNAP most usable, Myrian Studio second but windows-only, 3d Slicer third but neat because easily extendable. Seg3d as also mentioned as aving very good usability
• Misc
  • https://github.com/neurolabusc/niivue-images sample nii images for evaluating nii viewers/editors

+ 3d Slicer

• Main file: 240715-1653 Notes on 3d-slicer, 240718-1530 First python 3d slicer interactions
• License:
  • About 3D Slicer — 3D Slicer documentation
  • custom license: BSD-style, broadly open source
    • no GPL/copyleft code allowed because it’d make it harder to mix slicer w/ private intellectual property
    • commercial use OK
    • 3d slicer based products — only some are listed, because one can use it and not disclose it and it’s fine
• Language: python
• extensible in python as well
  • has many cool extensions: @KitwareMedical/slicer-extensions-webapp
  • includes MONAILabel active learning extension plugin
  • a whole category for developer tools! @KitwareMedical/slicer-extensions-webapp
  • Extension Wizard — 3D Slicer documentation
    • https://extensions.slicer.org/catalog/Shape Analysis/32952/linux has extensions about stats
  • ai-assisted-annotation-client/slicer-plugin/README.md at master · NVIDIA/ai-assisted-annotation-client
• Very active community and frequent updates
• Downsides
  • J:
    • too specific for medical images
    • presets hard to customize (I agree!)
    • bad documentation (I think fixed now)
  • but: “Best of the free ones”
• Ha: https://github.com/NVIDIA/ai-assisted-annotation-client/blob/master/slicer-plugin/README.md

+ Syngo

• syngo.via OpenApps – Siemens Healthineers Deutschland
• Paid.
• Best according to ¹
• No easy way to test.

MeVisLab

• https://www.mevislab.de/
• Commercial and non-commercial version available
• Documentation:
  • Getting Started
• SDK for scripting exists
  • python QT bindings available
  • and generally feels expandable w/ python, shows python errors in the main screen for example, which is a good sign
• Official video tutorials exist, incl. for application development: MeVisLab - YouTube
• Download: a single .bin file that has to be run
• I don’t really like the website but the program itself feels nice!
• Nice pipeline visualization: and adding blocks — much less chaotic than 3d slicer (at first glance)
• overall good impression
• python scripting has a nice reference etc.

ITK-SNAP

• TODO
• Winner of ² due to its usability, does segmentation only but does it well
• available on AUR
• tutorial: TutorialSectionVolumes
• I really like it as first impression
• Has a distributed segmentation service already implemented!
  • ITK-SNAP Distributed Segmentation Services
  • 
  • ASHS Documentation - Overview sample of such a service
  • (can’t test it because pyqt6 present but not found)
• has volumes and statistics, maybe expandable
• BUT can’t find easy ways to add plugins

+ 3D Seg

• Open Source, segmentation-first
• for linux has to be compiled from source
• last meaningful commits from ~3 years ago: Commits · SCIInstitute/Seg3D, last release 2021 as well
• tried to build it, as ./build, failed because can’t find svn for
• Has detailed-ish developer documentation as videos! https://www.sci.utah.edu/cibc-software/seg3d.html
• Couldn’t build locally sadly, error in the middle of it
• Giving up

Myrian

• commercial, but non-commercial exists, and can be extended with plugins

NeuroMorph

• NeuroMorph ‐ EPFL
  • NeuroMorph | BIII
• Blender-based!

• NeuroMorph is a set of tools designed to import, analyze, and visualize mesh models in Blender. It has been developed specifically for the morphological analysis of 3D objects derived from serial electron microscopy images of brain tissue, but much of its functionality can be applied to any 3D mesh.

ilastic

• ilastik - ilastik
• integrat-able with fiji: ilastik/ilastik4ij: ImageJ plugins to run ilastik workflows and with everything else
• BioImage.IO integration as well, btw another model zoo

Fiji

• Fiji: ImageJ, with “Batteries Included”
• Legendary
• infinite plugins, very well known, downloadable as executable
• not too easy to use in my subjective opinion, menus inside menus, but…

Kitware glance

https://kitware.github.io/glance/doc/index.html

• web-based, extendable
• Kitware/glance: Kitware Glance is an open-source web application developed at Kitware for visualizing volumetric images, molecular structures, geometric objects, and point clouds. It is part of the ParaView platform and can serve as a foundation for building custom web-based visualization applications involving ITK.js and VTK.js.

Biaflows

• https://biaflows-sandbox.neubias.org/#/
• fully-online and can run models
• I don’t like the 3d viewer and I think it’s segmentation only, no easy way to generate/show stats

Why is the number of voxels different from the number of matrices? - Support - 3D Slicer Community

>>> raw = getNode('probe1_0000.nii.gz')
>>> slicer.util.arrayFromVolume(raw).shape
(180, 180, 500)
>>> 

doesn’t work for segmentations because not a volume.

Image dimensions can be seen from volume information And are identical in both segmentations and original

I found smoothing in view controllerS!

In Slice controllers there’s this arrow button that allows to set basically layers AND SMOOTHING per slice!

So everything is as expected wrt smoothing — there was none.

Zooming in slices w/ CTRL!

// ref: 240701-1752 NII MRI annotation tools

• 3D Slicer image computing platform | 3D Slicer
• DICOM seems primary format¹ but nifti is supported
• Installing from AUR

• Importing 2x .nii as directory worked as-is

• Volumes can do settings per file, incl.:

  • annotations threshold starts at 1

• Volume renderings gives 3d views?

• Annotations can be imported as annotations/segmentations through add data! Then they are parsed semantically

• One can segment N slices and it magically creates a 3d shape out of it!

• Exporting annos as nifti possible through “export” in Segmentation module ²(not Segment Editor!)

• https://www.youtube.com/watch?v=CxwxYtizdhE resampling volumes

• Diagrams

  • A Quarto tip a day - Diagrams

• This shows how to use fragments inside code/diagram segments: Fragment Layout Elements in Reveal.JS · quarto-dev/quarto-cli · Discussion #3685

• Non-quarto solutions:

  • Using code to generate graphs: GraphViz as figure when generated by R code · quarto-dev/quarto-cli · Discussion #5949
  • mapio/GraphvizAnim: A tool to create animated graph visualizations, based on graphviz.

Simplest way possible

• Multiple fragments one after the other with less and less invisible graph parts
  • style=invis: sum [label="∑", fillcolor="#27ae60", shape=circle, style=invis];¹

This makes everything invisible by default, uncommenting the lines makes the parts visible

digraph G {
    rankdir=LR;
    node [shape=circle, style=invis, fontcolor=white, fontsize=18, fillcolor="white", color="white"];
    edge [color="white", fontcolor="white"];
    graph [color="white"];
  
    
  subgraph cluster_1 {
    #label = "Inputs";
    // uncomment for visibility:
    node [fillcolor="#2c3e50", style="filled"]
    
    n0 [label=<1>];
    n1 [label=<x<SUB>1</SUB>>];
    n2 [label=<x<SUB>2</SUB>>];
    n3 [label=<x<SUB>n</SUB>>];

    { rank=same; n0; n1; n2; n3; }
  }
  
  subgraph arrows {
    // uncomment for visibility of arrows:
    edge [color="black", fontcolor="black"];
    node [style="filled", fillcolor="#27ae60"];
    
    
    n0 -> sum [label=<w<SUB>0</SUB>>];
    n1 -> sum [label=<w<SUB>1</SUB>>];
    n2 -> sum [label=<w<SUB>2</SUB>>];
    n3 -> sum [label=<w<SUB>n</SUB>>];
 
    sum [label="∑"];
  }
    
  subgraph end {
    // uncomment for visibility
    edge [color="black", fontcolor="black"];
    node [style="filled", fillcolor="#27ae60"];

    activation [label="∫", fillcolor="#f39c12", shape=circle];

    // Output node
    output [label="y", fillcolor="#8e44ad"];

    // Other edges
    sum -> activation;
    activation -> output;

   }

}

• The very best QR code generator is https://hovercode.com/
• DuckDuckGo can generate simple ones as well: qr https://www.eff.org/ at DuckDuckGo

Basics

Graphviz Online is a really nice viewer

Grouping

• You can set attributes globally or per subgraph
• edge [color="white"] makes all child edges white, unless overwritten

Labeling things

I figured out this way to label different ranks by using an invisible pseudo-graph: viewer

This is the chunk:

  subgraph labels {
    edge [fontcolor="black"];
    node [style="filled", fontcolor="black", fontsize=10];
      
    node1 [label="Inputs"];
    node2 [label="Weights"];
    node3 [label="Sum"];
    node4 [label="Activation\nfunction"];
    node5 [label="Output"];
    
    node1 -> node2 -> node3 -> node4 -> node5;
    
    {rank=same; node1;n0;n1; n2; n3;}

    
    {rank=same; node3;sum;}
  }

Note using ranks to align inputs/sum to the correct level; also the use of \n in the label.

Adding labels under nodes

• xlabel, and use HTML to make it a diff color than the main label. dot - Graphviz: Putting a Caption on a Node In Addition to a Label - Stack Overflow Graphviz: Distinct colors for xlabel and nodelabel - Stack Overflow

• To get images, rename the pptx to .zip¹ and just copy them from the archive
  • zipinfo lists everything

• ssine/pptx2md: a pptx to markdown converter does this automatically!

  • --qmd
  • Had to install deps manually
  • In LibreOffice could delete the footer and header in the master slide, giving me clean slides.
    • LibreOffice could delete header only, but not the footer — had to use MS Office for this

• // Ha: [Falk358/quarto_translator: A python project for translating quarto markdown files using the openai API](https://github.com/Falk358/quarto_translator+

  • Tested it on a small file, it actually works nicely

LaTeX

Colorful equations

How do I write LaTeX with colors in quarto presentations? · quarto-dev/quarto-cli · Discussion #4429

$$\begin{aligned}
\color{green}{\bf{Net Benefit}} = \frac{\color{green}{\bf{TP}}}{\text{N}} - \frac{\color{red}{\bf{FP}}}{\text{N}} * {\frac{{p_{t}}}{{1 - p_{t}}}}
\end{aligned}$$

$$\color{red}{w_0} + x_1*w_1 + x_2*w_2 + x_1*w_1$$  

It uses the usual names: LaTeX Color Definitions

Actually² these: MathJax-src/ts/input/tex/color/ColorConstants.ts at master · mathjax/MathJax-src

HA, it can even do usual RGB hex colors!

The useful to me are Cardinal/Carmine and Cerulean/Blue?..

Size of formulas

• Modifying the MathJax size · quarto-dev/quarto-cli · Discussion #2392 only global

• Mathjax supports things like \tiny{w_..}! (Changing mathjax’s font size - Stack Overflow)

Quarto align stuff right

Quarto: How to right justify text in PDF and HTML - Stack Overflow:

\`\`\`{=html}
<style>
.flushright {
   text-align: right;
}
</style>

::: {.flushright}

But actually,

::: {style="text-align: right;"}
$\small{E(W) = \frac{1}{n} \sum_{i=1}^{n} \mathcal{L} \left( \color{blue}{f(x^{(i)}; W)}, \color{red}{y_{true}^{(i)}} \right)}$
:::

Height and aspect ratios

Preview in a long/high browser window to see what the actual width of the presentation is. PDF export also uses the actual width

Misc

• ✕ aka MULTIPLICATION X is a really nice cross mark to use in random pictures.³
  • \times in latex
• ⮕ is a good arrow.
  • → is better (→)
  • more: HTML Unicode UTF-8

Context: in a paper, using PDF graphics is much better than e.g. PNG, because antialiasing (230628-1313 Saving plots matplotlib seaborn plotly PDF). Assume you need to crop one, and taking a PNG screenshot won’t cut it, because it would defeat the entire purpose.

LaTeX

TL;DR: in \includegraphics, trim=left bottom right top (in cm)

positioning - How can I crop included PDF documents? - TeX - LaTeX Stack Exchange:

includegraphics[clip, trim=0.5cm 11cm 0.5cm 11cm, width=1.00\textwidth]{PDFFILE.pdf}

BUT! If I’m ever in this situation, inkscape can really nicely open a PDF and export only the selected things, also as PDF.

Inkscape (much better)

1. When importing, “Replace text with closest available font”
   • e.g. OpenSans gets matched to Open Sans and it works out usually
   • the full list of fonts is shown in the import window
2. Select the part I want to crop
   1. or create a shape over it and select that shape or something
3. Export selection only as PDF; play with export settings as well if needed but for me the defaults worked this time

And either way

… test the results by downloading the PDF from overleaf and opening it locally, because antialiasing w/ imported vector pictures can be broken from within the preview window.

Used Apps/Dia - GNOME Wiki! all my life and love it, but

News! 2011-Dec-18: Version 0.97.2 has been released.

Depends on python 2.7 and is very problematic to install now. I’ll miss it.

I hear https://draw.io is a good alternative.

I’ll add a list of options here next time I need this.

(But for older dia files I’ll have to edit maybe later: it could be installed on a Windows machine and export a broken .SVG which I could then edit in Inkscape.)

Closing the window with the kitty terminal through qtile/OS means often asks you whether you really want to, as sth is running there.

Kitty has Ctrl+Shift+W which closes the terminal no questions asked.

For yay, the cache is in

$HOME/.cache/yay/

If the package is there, then: sudo pacman -U ./quarto-cli-1.4.555-1-x86_64.pkg.tar.zst

Then pin the package in /etc/pacman.conf:

# Pacman won't upgrade packages listed in IgnorePkg and members of IgnoreGroup
IgnorePkg   = quarto-cli
#IgnoreGroup =

Then yay -Syu will ignore it as well:

:: Synchronizing package databases...
 endeavouros is up to date
 core is up to date
 extra is up to date
 multilib is up to date
:: Searching AUR for updates...
 -> quarto-cli: ignoring package upgrade (1.4.555-1 => 1.5.52-1)
:: Searching databases for updates...
 there is nothing to do

Refs: How do you downgrade an AUR package? : r/archlinux

For not-AUR, there’s the downgrade command: archlinux-downgrade/downgrade: Downgrade packages in Arch Linux

References

• Refs:
  • Revealjs Options – Quarto¹
  • Advanced Reveal – Quarto
  • Revealjs – Quarto
• Previously:
  • 240424-1923 Presentations with Quarto and Reveal.js has my list of tricks
• Code
  • Quarto reveal.scss: quarto-cli/src/resources/formats/revealjs/reveal/css/reveal.scss at 303c4bc9f52ea0b0c36c5eb5dfbade9cece2b100 · quarto-dev/quarto-cli
  • Quarto Reveal footer CSS: quarto-cli/src/resources/formats/revealjs/plugins/support/footer.css at main · quarto-dev/quarto-cli

Misc

Better preview at a specific port, handy for restarting after editing CSS. Similar to quarto project frontmatter preview: port: 4444 browser: false.

quarto preview slides.qmd --port 4444 --no-browser

Supported by default

In frontmatter:

• logo is in bottom-right+
  • footer-logo-link
• footer for all slides
  • overwrite with div of class footer
• Likely relevant for me: Numbering reveajs options for dynamic bits, shift-heading-level-by, number-offset, number-section, number-depth — will touch if I need this.

Headers

• Extension: shafayetShafee/reveal-header: A Quarto filter extension that helps to add header text and header logo in all pages of RevealJs slide²
  • Has absolutely awesome documentation!

In front matter:

• header: one header text for all presentation
• title-as-header,subtitle-as-header: if true, place the presentation frontmatter’s title/subtitle as header if one not provided (overwriting the header value)
  • use-case: I same repeating text everywhere w/o specifying it every time.
• hide-from-titleSlide: all/text/logo to hide it from title slide
• sc-sb-title: if true, h1/h2 slide titles will appear in the slide header automatically when slide-level is 2 or 3

Divs with classes:

• .header: slide-specific header

• Excellent example from its docu of dynamic per-(sub-)section headers:

  • code: reveal-header/example_all.qmd at main · shafayetShafee/reveal-header
  • demo: Quarto Presentations

Left/right/center blocks of text in header

³

• If one uses section/subsection titles, they go in the left third and right third of the header, with the normal header text in the middle. This is neat to have in general, w/o the section/subsection titles.
  • sc-title, on the left, is section title
  • sb-title, on the right, is subsection title
• How do I add arbitrary text there? And what would be a good interface for it?

The right way — extension code

• I could look at the extension, maybe fork it, and find a way to put text in these divs
• Relevant code:
  • This JS populates them in each slide: reveal-header/_extensions/reveal-header/resources/js/sc_sb_title.js at main · shafayetShafee/reveal-header
  • add_header js: reveal-header/_extensions/reveal-header/resources/js/add_header.js at main · shafayetShafee/reveal-header
  • Main lua: reveal-header/_extensions/reveal-header/reveal-header.lua at main · shafayetShafee/reveal-header

Ugly CSS hack

• (Learn sass in Y Minutes)
• CSS for the header bits: reveal-header/_extensions/reveal-header/resources/css/add_header.css at main · shafayetShafee/reveal-header

Since .s[c|b]-title is present always, text added that way will be present on the title slide regardless of settings. … and — adding text to a presentation through CSS is, well, ..

.reveal-header .sc-title {
  background-color: red;

	&::before {
		content: 'sc-title header content';
  }

}

Slightly better ugly hack: main header text split in three, with two aligned l/r correspondingly.

.header-right {
  // text-align: left;
  float: right;
  background-color: pink;
  display: inline-block;
}

.header-left {
  // text-align: left;
  float: left;
  background-color: yellow;
  display: inline-block;
}

::: header 
[left]{.header-left} and not left [and right]{.header-right}
:::

Result:

Extended ugly SCSS hack

Improved the above to support both footer and headers I have a hammer and everything is a nail SCSS can do mixins let’s use them

// L/R margins of the footer — if logo is wider than this, it may overlap right footer text.
// By default, logo max-height is 2.2em, width auto based on this.
$footer-margin: 5em;

// left or right column bits
%hfcol {
  display: inline-block;
}


/* L/R columns in header */
.reveal .reveal-header .rightcol {
  @extend %hfcol;
  float: right;
}

.reveal .reveal-header .leftcol {
  @extend %hfcol;
  float: left;
}

/* L/R columns in footer */
.reveal .footer .leftcol {
  @extend %hfcol;
  float: left;
  margin-left: $footer-margin;
}

.reveal .footer .rightcol {
  @extend %hfcol;
  float: right;
  margin-right: $footer-margin;
}

Usage:

## I am a slide  with custom headers

And With content 

::: footer 
[Left footer]{.leftcol} Center footer [Right footer]{.rightcol}
:::

::: header 
[Left header]{.leftcol} Center header [Right header]{.rightcol}
:::

Frontmatter usage works only for footer, likely header doesn’t support markup.

    footer: "Center footer [right]{.rightcol} [left]{.leftcol}"  # CHANGEME

Result:

PROBLEMS:

• asymmetrical if only one of the two is present. Likely fixable, but I don’t want to force any center div.

Footers through qmd cols

This works almost perfectly, including missing values:

::: footer 

::: {.columns}
:::: {.column width="20%"}
left
::::
:::: {.column width="50%"}
Center footer
::::
:::: {.column width="20%"}
right
::::
:::

:::

It even works inside frontmatter as multiline string (not that it’s a good idea):

    footer: | 
      ::: {.columns}
      :::: {.column width="20%"}
      left
      ::::
      :::: {.column width="50%"}
      FB5 – Informatik und Sprachen: Deep Learning (MDS)
      ::::
      :::: {.column width="20%"}

      ::::
      :::

This removes the margin placing it exactly in the same place as a normal footer:

.reveal .footer p {
  margin: 0 !important;
}

(but headers don’t work)

What is a good interface?

A filter that parses frontmatter and puts things in the correct places.

Misc

• Down the front-end rabbit hole | quartoand.me

Usual:

format: 
  revealjs:
    logo: logo.png
    theme: [default, custom.scss]

Adding after the first one, so no !important needed¹:

format: 
  revealjs:
    logo: logo.png
    header-includes: |
      <link href="custom.css" rel="stylesheet">

Inline w/o declaring classes²:

Some [red words]{style="color:#cc0000"}

or:

::: {style="font-size: 1.5em; text-align: center"}
styling an entire div
:::

• Main Quarto Reveal SCSS: quarto-cli/src/resources/formats/revealjs/reveal/css/reveal.scss at 303c4bc9f52ea0b0c36c5eb5dfbade9cece2b100 · quarto-dev/quarto-cli
• Quarto footer css: quarto-cli/src/resources/formats/revealjs/plugins/support/footer.css at main · quarto-dev/quarto-cli

Download GitHub directory: paste an URI to the directory, get .zip of that directory only. Handy for downloading only parts of datasets

• nnU-Net Revisited paper lists the following, in bold the ones it considers the best:
  • ACDC, KiTS, AMOS “most suitable for benchmarking”
  • BTCV, LiTS, BraTS

Datasets

ACDC

• Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved? | IEEE Journals & Magazine | IEEE Xplore
• cardiac diagnosis
• 150 (Cardiac) MRI recordings/patients from 5 different diagnosis groups
• 3 classes: LV+RV heart cavities and myocardium
• 4D (3D+time)
• 100/50 patients train/test split
• nifti/.nii
• For each patient:
  • raw+ground truth data for two frames
  • 4d beating raw heart
  • age etc.
• links:
  • Website: ACDC Challenge
  • Dataset official download link: Human Heart Project
  • Their python code to load/save images to nifti (nii) format and compute metrics: https://www.creatis.insa-lyon.fr/Challenge/acdc/code/metrics_acdc.py
• Dataset structure
  • nifti (.nii)
  • sample: https://humanheart-project.creatis.insa-lyon.fr/database/#collection/637218c173e9f0047faa00fb/folder/6372204873e9f0047faa160b
  • folders: training/patient101/ containing
    • Info.cfg
      • metadata about the patient
    • MANDATORY_CITATION.md
    • patient101_4d.nii.gz
      • 3d of the beating heart (+time), viewable animated in the brainbrowser viewer¹
    • patient101_frame01_gt.nii.gz
      • ground truth data only for frame 01
    • patient101_frame01.nii.gz
      • raw data only for frame 01
    • patient101_frame14_gt.nii.gz
    • patient101_frame14.nii.gz

KiTS-23

• existed in 2019 2021 and 2023
  • KiTS23 | The 2023 Kidney Tumor Segmentation Challenge²
  • The main proceedings of the conference/challenge: Kidney and Kidney Tumor Segmentation: MICCAI 2023 Challenge, KiTS 2023, Held … - Google Books
  • The latest publication is from the 2021 challenge
  • // (Interesting anno startegy: professional places markers around the region, non-professional makes it into a pretty segmentation shape)
• KITS2023 dataset repo: neheller/kits23: The official repository of the 2023 Kidney Tumor Segmentation Challenge (KiTS23)
  • they use postprocessing after annotations
  • Sample: kits23/dataset/case_00194 at main · neheller/kits23
  • annotating etc. was done online and the webapp is still live: Annotate | KiTS23
    • They used the ulabel anno tool: SenteraLLC/ulabel: A browser-based tool for image annotation
• Structure
  • raw images must be separately downloaded from servers!
  • nifti/nii
  • 489 train set instances released³ — mostly similar to the files from older challenges
  • 3 classes: kidney, tumor, cyst
  • main metadata for all patients in kits23.json: kits23/dataset/kits23.json at main · neheller/kits23
  • directories are case00000-case00588
    • segmentation.nii.gz is the ground truth as used in the challenge, after postprocessing, the one we need.
    • ./instances/ has annotations — are the raw things annotated by humans.⁴
      • break brainviewer but not brainbrowser
      • [kidney|tumor|cyst]_instance-[1|2|..?]_annotation-[1|2|3].nii.gz
      • N instances — e.g. most people have two kidneys — all annotations done by 3 diff annotators, then merged into the main segmentation file.

For later.

• SenteraLLC/ulabel: A browser-based tool for image annotation is used by the KiTS23 | The 2023 Kidney Tumor Segmentation Challenge for annotation.

• TIL Arch has no cron by default, and uses systemd timers¹.
• cron - ArchWiki

sudo pacman -S cronie
sudo systemctl enable cronie.service

# many minutes were lost due to me forgetting to start it as well...
sudo systemctl start cronie.service

• I like this website to generate stuff:Crontab Generator - Generate crontab syntax
• cronie - Timed Tasks - Documentation as good as any
• the script has to be executable
• crontab -e edits your own crontab, then you can omit the username.

Cron git backup script

#!/bin/bash

# https://stackoverflow.com/posts/1482133/
# get the directory where this file is located
DIRNAME="$( dirname -- "$( readlink -f -- "$0"; )"; )"

cd $DIRNAME &&
git add -A &&
git commit -m "Automatic backup at `date`" &&
git push origin master &&
echo "Done"

# and this is the crontab
# odd hours:
# * 1-23/2 * * *  /bin/bash $HOME/this-script-location.sh > /tmp/cronlog

EDIT: sometimes if git crashes an index file stays and then nothing runs — so the cronjob must be somehow monitored manually, todo.

TL;DR: fish easy version below works, but needs quotes when expression is complex: cc 2+2 but cc 'floor(2.3)'.

I’m continuing to move my useful snippets from zsh to fish (240620-2109 Fish shell bits), and the most challenging one was the CLI python calculator I really love and depend on, since it contained arguments with parentheses (which are fish expressions as well).

Basically: cc WHATEVER runs WHATEVER inside python, can do both easy math a la 2+2 and more casual statistics-y mean([2,33,28]).

Before in zsh this was the magic function:

cc() python3 -c "from math import *; from statistics import *; print($*);"
alias cc='noglob cc'

Fish, easy version:

function cc
  command python3 -c "from math import *; from statistics import *; print($argv);"
end

Works for easy cc 2+2 bits, but as soon as functions and therefore parentheses get involved (cc floor(2.3)) it starts to error out.

[I] sh@nebra~/t $ cc mean([2,4])
fish: Unknown command: '[2,4]'
in command substitution
fish: Unknown command
cc mean([2,4])
       ^~~~~~^
[I] sh@nebra~/t $ cc mean\([2,4]\)

>>> mean([2,4])
3
[I] sh@nebra~/t $

(But I REALLY don’t want to do cc mean\([2, 3]\))

In the zsh snippet, noglob meant basically “take this literally w/o expanding anything”, and it passed everything as-is to python, and this is what fails in my fish solution.

Noglob in fish is fun:

• The fish language — fish-shell 3.7.0 documentation on escaping characters
• Implement the noglob modifier · Issue #3504 · fish-shell/fish-shell:

  If you wish to use arguments that may be expanded somehow literally, quote them. echo ‘’ and echo “” both will print the literal.

• The fish language on quotes:
  • single quotes = no expansion of any kind
    • \' for literals inside single
  • double quotes = variable exp. ($TERM) & command substitution ($(command))
    • \" for literal "s inside double
  • within each other, no special meaning
  • Let’s test:
    • echo (ls) = ls output, one line
    • echo "$(ls)" = ls output, multiline
    • echo '(ls)' = (ls)
    • echo "(ls)" = "(ls)"

THEN

• command python3 -c "from math import *; from statistics import *; print($argv);"

  • cc ceil\(2\) +
  • cc ceil(2) -

• `command python3 -c “from math import *; from statistics import *; print(’$argv’);”

  • literally prints the passed thing w/o python eval, w/ same rules

• OK can I do a variable then?

  set pyc $argv
  echo $pyc
  command python3 -c "from math import *; from statistics import *; print($pyc);"

nope.

Bruteforcing the solution

(and learning to use fish loops mainly, of course there are better ways to do this.)

# list of simple, brackets, and parentheses + no, single, double quotes 
# no space between nums in brackets, python interpreter would add them. [2,3] — literal, [2, 3] — parsed by python
set cmds \
'2+2' \
'\'2+2\'' \
'"2+2"' \
'[2,3]' \
'\'[2,3]\'' \
'"[2,3]"' \
'floor(2.3)' \
'\'floor(2.3)\'' \
'"floor(2.3)"' 

function tcc
  set pyc $argv
  # command python3 -c "from math import *; from statistics import *; print" '(' "$pyc" ');'
  # command python3 -c "from math import *; from statistics import *; print($pyc);"
  command python3 -c "from math import *; from statistics import *; print($pyc);"
end


# loop through all test cases to see sth that works for all
for i in $cmds
  echo $i:
  echo "   $(tcc $i)"
end

At the end, no additional literal quotes + initial command didn’t error out, and we came full circle:

set cmds \
'2+2' \
'[2,3]' \
'floor(2.3)' 

# winner command!
function tcc
  command python3 -c "from math import *; from statistics import *; print($argv);"
end

[I] sh@nebra~/t $ ./test_cc.sh
2+2:
   4
[2,3]:
   [2, 3]
floor(2.3):
   2

• Double quotes in the python command mean only $pyc gets expanded
• $pyc in the working versions have no hard-coded quotes
• in CLI tcc floor(2.3) still fails — because like that it’s a command, not a string. In the file it was inside single quotes, as a string. So I can do this in the CLI as well.

So simple and logical at the end.

Final solution

function cc
  echo ">>> $argv"
  command python3 -c "from math import *; from statistics import *; print($argv);"
end

When using, quotes are needed only for complex bits (parentheses, * etc.).

[I] sh@nebra~/t $ cc 2+2
>>> 2+2
4

[I] sh@nebra~/t $ cc [2,3,4]
>>> [2,3,4]
[2, 3, 4]

# no quotes
[I] sh@nebra~/t $ cc mean([2,3,4])
fish: Unknown command: '[2,3,4]'
in command substitution
fish: Unknown command
cc mean([2,3,4])
       ^~~~~~~~^

# with quotes
[I] sh@nebra~/t $ cc 'mean([2,3,4])'
>>> mean([2,3,4])
3

So I literally had to follow the advice from the first link I found and used single quotes in my initial command:

If you wish to use arguments that may be expanded somehow literally, quote them. echo ‘’ and echo “” both will print the literal.

Still, I learned a lot about fish in the process and honestly am loving it.

TL;DR use Chromium

PDF, PS and DjVu - ArchWiki has a table, but it lies, in my tests:

• zathura explicitly no support
• evince didn’t work
• gnome document viewer didn’t work

And for the Nth time, I end up remembering about Chrome/Chromium PDF viewer, that does this reliably.

Run and disown

In my old zsh config I had this:

function dos() {
    # run_disowned and silenced
    nohup "$@" >/dev/null 2>&1 & disown
}

Emulating my old dos command in fish:

• disown - remove a process from the list of jobs — fish-shell 3.7.0 documentation:
  • firefox &; disown starts and disowns ffx.
• terminal - How do I get fish to swallow? or at least to detach itself from a program - Stack Overflow

Both together give this solution:

function dos
  echo "disowning $argv"
  command $argv  >/dev/null 2>&1 &
  disown
end

Shell notation for $(cmd)

shell - Fish equivalent of bash $(command) notation - Stack Overflow: - bash: $(command) - fish: (command)

which command

type - locate a command and describe its type — fish-shell 3.7.0 documentation: type commandname.

[I] sh@nebra~/me/me-qs $ type cc
cc is a function with definition
# Defined in /home/sh/.config/fish/config.fish @ line 184
function cc
  echo ">>> $argv"
  command python3 -c "from math import *; from statistics import *; print($argv);"
end

When doing a multilingual quarto website using profiles (240619-1425 Quarto multilanguage website), quarto preview/render --profile de allows previewing only one language, and the menu links (/de/xxx) break.

Solution to preview EVERYTHING, including menu links logic:

# Todo break on erorr
quarto render --profile=ua
quarto render --profile=en
quarto render --profile=de

And then serve the _site directory locally through a simple webserver, e.g. Python SimpleHTTPServer - Python HTTP Server | DigitalOcean, and see the result on http://localhost:9000:

python3 -m http.server -d _site 9000

TL;DR –no-clean is dangerous

quarto render --no-clean ... (or an interrupted render?) created many .html.md files in the same dirs as their .qmd/.md sources:

papers/11-bank2011using/index.md               papers/23-hanig2023nlp/index.md
papers/11-hanig2011benefits/index.html.md      papers/23-tumler2023virtual/index.html.md
papers/11-hanig2011benefits/index.md           papers/23-tumler2023virtual/index.md
papers/11-hanig2011knowledge/index.html.md     papers/24-hamotskyi2024fincorpus/index.html.md
papers/

Then the custom listing for */*.md went crazy because the html.md files didn’t have the needed fields. (also .jsons/.html etc.)

Quarto has no quarto clean but one can just re-render everything without --no-clean

Fish globs

To find all these files arbitrarily many levels deep:

ls **/*.html.md

ERROR: TypeError: title.trim is not a function quarto

happens for me when in front-matter I do

---
# title: "Publications and Awards"
title: {\{< var test >}}

instead of QUOTED

---
# title: "Publications and Awards"
title: "{\{< var test >}}"

ALSO, interestingly, when I save the wrong version while previewing the error is better:

ERROR: Validation of YAML front matter failed.
ERROR: In file publications.qmd
(line 3, columns 8--24) Field "title" has value {\{< var test >}}, which must insteadbe a string
2: # title: "Publications and Awards"
3: title: {\{< var test >}}
         ~~~~~~~~~~~~~~~~~
4: css: ./posts_publications/pub_style.css
✖ The value {\{< var test >}} is of type object.
ℹ The error happened in location title.

ERROR: Render failed due to invalid YAML.

So: quarto errors are more detailed when previewing instead of when compiling from zero? Interesting. Okay.

• No native support

• R package exists, but I don’t feel like digging into R at all: Renders a Multilingual Quarto Book • babelquarto

• The other approaches I found are all based on project profiles and conditional output

  • oooo12 - Multi-language Blog with Quarto - Guide
  • Mario Angst - A multi-language (German/ English) Quarto website
    • only difference is that it’s /de/ /en/ instead of ENG being /en/
    • nice and more detailed about interesting corner cases

• Document Language – Quarto¹ for setting one language in the page and doing per-language changes to the template texts

  • quarto-cli/src/resources/language at main · quarto-dev/quarto-cli
  • quarto-cli/src/resources/language/_language.yml at main · quarto-dev/quarto-cli

• Conditional Content – Quarto

• Project Profiles – Quarto

Config files

_quarto-profilename.yml ONLY, the rest won’t get parsed

profile:
  default: en
  # mutually exclusive group: you can do only one
  # (otherwise `--profile one,two` does multiple)
  group: 
    - [en, de, uk]

Page content

# `unless-profile` exists as well
::: {.content-visible when-profile="en"}
This content will only appear in the advanced version.
:::

Links

Links are going to be interesting!

Currently /de is German, / is English.

Main home page is .. from DE lang, or /de from EN.

Menu items:

• ENG as-is. (`href: whatever.qmd)
• DEU: href: ../de/lehre.html — note the HTML bit!

But when previewing DEU, all of these pages are at / — ergo menu items don’t work, as they lead to a non-existing ../de/...

ALSO: marioangst.de shows nicely how one can link to other languages from the menu!

- icon: book
href: publications.qmd
text: Publikationen
- href: ../en/blog
text: Blog (englisch)

Site language

Website Options – Quarto tells me I can do this in each _quarto-de.yml etc. profile:

format:
  html:
    lang: de
	#lang: ua
	#lang: en

This changes the interface to follow the corresponding quarto-cli/src/resources/language/_language.yml at main · quarto-dev/quarto-cli

Unsolved

How do I change front matter (e.g. title)?

Not dealt with in any of the approaches: quarto’s native Document Language¹ thing

So: - How do I do different post titles per language? - How do I change site language, so _languages.yml, conditionally?

• Project Basics – Quarto discusses the various approaches to metadata

  • And I can conditionally include stuff

• … I could literally do a bash script that puts a _metadata.yaml, builds with a proflie, then removes that file. Oh this would be painful

• Skimming Website Options – Quarto doesn’t really help

  • except that I can set html format lang: de from within profiles! NICE

Do variables magic

• title gets taken from variable title[currnet-profile-name] or something?
• _variables.yml works for var shortcodes (240619-1845 Quarto error title.trim() is not a function), and shortcodes can do metadata too

# works
title-en: "Publications and Awards"
title: "{\{< meta title-en >}}"

If only I could do per-language attributes as shown in the docu²:

language:
  en:
    title-block-published: "Updated"
  fr:
    title-block-published: "Mis à jour"

It would be so cool if one could overwrite the other variables

language:
  de: 
    title: german post title

The above would nicely get a language from the profile _quarto-lang.yml and automatically change the things. Can I do this for titles and front-matter?

• I’m not alone:
  • Is there a way to insert variables in other shortcodes with Quarto? - Stack Overflow

I can get the current profile from the env variable

profile: {\{< env QUARTO_PROFILE >}}

If I could just

title: vars['titles']['postname'][QUARTO_PROFILE]

• Asked this on Github: How do I do conditional YAML frontmatter, e.g. based on profile? · quarto-dev/quarto-cli · Discussion #10064
  • Mea culpa, discussed already: Conditional metada for different profiles on a website · Issue #4516 · quarto-dev/quarto-cli Conditional metada for different profiles on a website · quarto-dev/quarto-cli · Discussion #4455 no idea how I missed this.

Use scripts

• Project Scripts – Quarto
  • A python script that sets title based on QUARTO_PROFILE

Quarto lua filters

OK let’s do this. No choice.

Learn Lua in Y Minutes

First³ attempt to write anything in lua:

function Meta(m)
  local profiles = quarto.project.profile
  local profile = profiles[1]
  if profile then
    print("Profile: " .. profile)
    m.active_profile = profile
  end

  if profile and m.titles and m.titles[profile] then
    cleantitle = pandoc.utils.stringify(m.titles[profile])
    oldtitle = pandoc.utils.stringify(m.title)
    m.title = cleantitle
    print("Profile:" .. profile)
    print("Old title:" .. oldtitle)
    print("New title:" .. cleantitle)
  end

  return m
end

I’d need to make it more robust:

• multiple profiles? Not now
• No titles set in titles — use the default one

So:

• Always require a title
• Optionally add array of titles, indexed by language=profile name
• If present use one of these.

function Meta(m)
  local profiles = quarto.project.profile
  if not profiles then
    -- TODO: missing YAML key? Empty YAML key?..
    -- TODO even more later: filter multiple profiles to use the language one
    return m
  end

  local profile = profiles[1]
  -- If we have a named profile, save it, otherwise return
  if profile then
    print("Profile: " .. profile)
    m.active_profile = profile
  else
    return m
  end

  if m.titles then
    local titles = m.titles
    if titles[profile] then
      newtitle = pandoc.utils.stringify(titles[profile])
      oldtitle = pandoc.utils.stringify(m.title)
      -- log both if they differ
      if newtitle ~= oldtitle then
        m.title = newtitle
        -- print("Old title:" .. oldtitle)
        -- print("New title:" .. newtitle)
        print(oldtitle .. " => " .. newtitle)
      end
    else
      print("Title for profile " .. profile .. " not found among ")
      for lang, title in pairs(titles) do -- Table iteration.
        print("    " .. lang .. ": " .. pandoc.utils.stringify(title))
      end
    end
  end

  return m
end

Main problems:

• listing pages use the old title anyway
• supports only title, not e.g. description (used, again, in listings)
  • and side menus!

All from javascript - How can I comment the EJS code (JS node) without getting an error - Stack Overflow:

// Neat multiline comment thing
<%if(false) {%>  
  <!-- single row of inline icons for pdf. etc instead of buttons -->
<%} %>

// hard to grep for and no syntax highlight, so maybea adding COMMENT works as my bad idea
<%if(false) {%>  
  <!-- COMMENT 
	single row of inline icons for pdf. etc instead of buttons 
  -->

// Documentation says: 
<%# comment %> 


// Not documentation, multiline comment:
<%/* comment */%> 

I had issues putting HTML code w/ EJS snippets in the latter one, so iffalse it is. For nested things, SO suggests these (both bad imo):

<!--label for="<%#= user.id %>" style="background-color: <%#= user.color %>;"-->
<!--label for= <%#=`${user.id}` %> style= <%#=`background-color: ${user.color};`%> -->

Given: quarto website page with publications. Previously touched upon in the messy 240605-2055 Quarto website creation notes.

Anchor links in (quarto) markdown

This works:

[UNLP paper](publications.qmd#hamotskyi2024unlp)

Anchor targets

• Only correct solution: ¹ <a name="title"></a>
  • (markdown - How to link to a named anchor in Multimarkdown? - Stack Overflow says to use the name=, not id=, attribute, but this doesn’t work for me in quarto.)

Template

// Get anchor id somehow from paper path
<% let y= item.path.split('/'); y.pop(); let dirname = y.pop(); let citation_key = dirname.split('-').pop() %>

// Overwrite with paper front-matter if there's one
<% if (item.citation_key) { %> 
	<% citation_key = item.citation_key %>
<% } %>       

// Add to paper listing thing 
<a id="<%= citation_key %>"></a>

Old way with systemd

/etc/systemd/system/user-suspend@.service:

[Unit]
Description=Lock the screen
Before=sleep.target

[Service]
User=%I
Type=forking
Environment=DISPLAY=:0
ExecStart=/usr/bin/xlock -usefirst -echokeys -description -modelist swarm,starfish,mandelbrot,polyominoes,fadeplot,matrix,lisa,life3d,life1d,kumppa,grav,flow


[Install]
WantedBy=sleep.target 

To enable¹: sudo systemctl enable user-suspend@myusername.service

Arch wiki (Session lock - ArchWiki) omits @myusername which leads to this error:

Failed to enable unit: Destination unit sleep.target is a non-template unit.

Also: xlock (xlockmore) is neat, here’s a list of modes: XlockMore modes

(Also: XScreenSaver versus XLock)

Offtopic but cool: Cool, but obscure X11 tools

New way with xss-lock

Added this to qtile autostart:

xss-lock -- xlock -usefirst -echokeys -modelist qix,lisa & 

Also

Auto-lock after 300 seconds:

xset s on xset s 300

Also

xlock is insecure, which is why it’s absent from Ubuntu repos, but especially jarring when having an external monitor — on my install, it used to show the lock screen on the external monitor, but show a still image of the desktop and my open programs on the internal one. Damn.

xscreensaver ftw.

Added ddg’s backslash-feeling-lucky to qb search engines. Encoded the backslash as URI, it worked!

https://duckduckgo.com/?q=%5C{}

Full list:

c.url.searchengines = {
        'DEFAULT': 'https://search.brave.com/search?q={}',
        'b': 'https://search.brave.com/search?q={}',
        'g': 'https://google.com/search?q={}', 
        's': 'https://scholar.google.de/scholar?hl=de&as_sdt=0%2C5&q={}&btnG=', 
        'ddg': 'https://duckduckgo.com/?q={}', 
        'l': 'https://duckduckgo.com/?q=%5C{}', 
        'c': 'http://dict.cc/?s={}', 
        'y': 'https://youtube.com/results?search_query={}',
        'd': 'https://en.wiktionary.org/wiki/Special:Search?search={}',
        'w': 'https://en.wikipedia.org/wiki/Special:Search?search={}',
        'wa': 'http://wolframalpha.com/input?i={}', 
}

maim is supposed to be the “better scrot”¹

# screenshot and put into clipboard
 maim -s | xclip -selection clipboard -t image/png -i

In qtile you can’t do that, because | is a shell construct².

Solution: do it in a shell:

CommandSet(commands={
	#...
	"S/clipboard": "bash -c \"maim -s | xclip -selection clipboard -t image/png -i\"",

A minimal citation style(for grant proposals) – anton.cromba.ch

Obyde needs 3.8 and fails otherwise, new OS maybe time for new ways to convert. There are many actually. Some active mantained and expandable.

For later:

• devidw/obsidian-to-hugo: Process Obsidian notes to publish them with Hugo. Supports transformation of Obsidian wiki links into Hugo shortcodes for internal linking. looks especially nice.
• Publishing Obsidian vault with Hugo - Sagar Behere lists all the other ways and is really detailed
• (Also on hugo A Hugo Survival Guide is a neat thing I’ll read sometime)

So

obsidian-to-hugo doesn’t support assets/images :( Leaving only ukautz/obsidian-meets-hugo: Command line tool to export Obsidian Vault into Hugo published website for me.

• preserves Obsidian directory structure — nice
• not recursive by default
• doesn’t create _index.md files, and converts existing ones in Obsidian to -index.md
  • pchr8/obsidian-meets-hugo: Command line tool to export Obsidian Vault into Hugo published website forked and did my first go programming to fix this
• doesn’t support folders, but one can filter by tags — good that I used both since the beginning!
  • for multiple tags a “list” is possible — and apparently that’s -i tag1 -i tag2 -i tag3¹

Current CLI:

go run cmds/omh/main.go --obsidian-root=../public_obs/ -R --hugo-root=../dtb/ --sub-path= -i=uni -i=zc/it -i=zc/rl

Onwards

• 3 refs not found
  • I don’t understand why hugo says “ref … page not found” - support - HUGO:

    Is linksphoto.md unpublished (draft, expired, or future)? Or, to put it another way, what happens when you run hugo server -DEF?

  • fixed for me! hardcoded hugo-format refs in obsidian broke because now different names — converted to real obsidian links and this fixed the remaining ones
• cyrillic names now unsupported?
  • 220407-2246 Чебуреки etc.
  • damn.
  • It’s the same

    var insane = regexp.MustCompile(`[^a-zA-Z0-9\-]`)
    func Sanitize(in string) string {
    	return insane.ReplaceAllString(in, "")
    }
    

  • w/ chatGPT, changed regex to [^a-zA-Z0-9\-\p{Cyrillic}] — now it does upper+lowercase cyrillics — so now it’s 220407-2246-Чебуреки.md in the md filename, but the URI has it lowercased.
  • Relevant: disablePathToLower in config² — not changing because cool URIs don’t change and this was the default for years on this website
• Images are broken
  • ![2024-05-13-182935_1304x442_scrot.png](//assets/2024-05-13-182935-1304-x-442-scrot.png)
  • … //assets alrighty…
  • Docu: Static files | Hugo

    • By default, the static/ directory in the site project is used for all static files (e.g. stylesheets, JavaScript, images). The static files are served on the site root path (eg. if you have the file static/image.png you can access it using http://{server-url}/image.png, to include it in a document you can use ![Example image](/image.png)).

    • you can have multiple such directories
  • Ah — maybe it’ll work when uploaded to my website, where /xxx will refer to the website and not my local install
  • If I manually fix //assets/.. to /assets (one slash) then it shows up nicely locally
  • In omh.go this happens: return fmt.Sprintf("[%s](/%s/%s)", title, c.SubPath, target)
  • AH it’s because of my empty subpath directory argument — if I pass something then everything works. Oh GodDAMN it.
  • OK this fixes it. And I hope creates no more problems.

    if c.SubPath == "" {
        return fmt.Sprintf("[%s](%s/%s)", title, c.SubPath, target)
      } else {
    return fmt.Sprintf("[%s](/%s/%s)", title, c.SubPath, target)
    

  } ```
• Debugging a notes called _index I realize that it takes Hugo frontmatter title from note title, which in turn is the filename, NOT the obsidian file frontmatter title :(
  • problem because my layouts rely on a magic constant in some directories…
  • I hope it’s my last fix.

  // Keep title in Obsidian front-matter as note title if it's there (a la obyde), 
  // otherwise use Obsidian filename for this (standard behaviour). 
  oldTitle := hugo["title"]
  if oldTitle==nil {
    log.Warn("No title in front-matter, using filename in ", note.Title)
    // must have title
    hugo["title"] = note.Title
  } else {
    log.Warn("Using frontmatter title for ", oldTitle)
  }

• Quarto – Document Language is related but different.

• R package for this, but R: ropensci-review-tools/babelquarto: Renders a Multilingual Quarto Project (Book or Website)

• Mario Angst - A multi-language (German/ English) Quarto website

  • Profiles + visible-when — all langs in one file, looks kinda ugly but why not…
  • Quarto – Project Profiles

• oooo12 - Multi-language Blog with Quarto - Guide profiles as well.

• Approach for now — write them in separate files, and optionally at some point include them in the visible-when option?..

quarto create project 

Listing pages

• Quarto – Document Listings
  • Quarto – Creating a Blog
• Dates are awkward: Quarto – Quarto Dates and Date Formatting

---
title: "Blog"
listing:
- id: test-l1
  contents: "blog_posts/*"
  sort: "date desc"
  type: table
  categories: true
- id: test-l2
  contents: "blog_posts/*"
  sort: "date desc"
  type: default
  categories: true
- id: test-l3
  contents: "blog_posts/*"
  sort: "date desc"
  type: grid
  categories: true
---

This will be a test listing page.


## Table listing
::: {#test-l1}
:::


## Default listing
::: {#test-l2}
:::


## Grid listing
::: {#test-l3}
:::

• Configs
  • each directory can have it’s own _metadata.yml with configs that will be applied to all files in that directory

Publications

To get something like Drew Dimmery - Research or our old wowchemy thingy some magic will be needed.

Doing

---
title: "Publications and Awards"
bibliography: "./publications/papers.bib"
nocite: |
  @*
---

results in a list in a certain CSL format, which is limited — no ways to link videos/slides/… etc.

So likely it’ll be yet another listings view, or how Drew Dimmery - Quarto for an Academic Website did it — papers to YAML with ALL the metadata, then python script (inside quarto qmds, first nice case I see for this!) to convert it into the on-screen form.

And if code — then maybe it’s a conveter package from wowchemy yaml thingy?

• Quarto – Custom Listings together with “listing from yaml” may help?.. - Supporting category like feature for listing authors (e.g. filter by author like categories) · Issue #6246 · quarto-dev/quarto-cli TL;DR complex tags work
  • EJS – Embedded JavaScript templates is used for templating — looks kinda similar to Hugo
  • Meghan Hall describes this and links to Quarto’s own listing templates: quarto-cli/src/resources/projects/website/listing/listing-table.ejs.md at main · quarto-dev/quarto-cli
  • (Quarto – Article Templates for the general thing)

OK, then:

• Concatenate all wowchemy publication files into one large yaml (one-time python thing?)
• do EJS template for it

(Alternatively — just use the dirs as-is and do no yaml)

Onwards

• Albert Rapp - The ultimate guide to starting a Quarto blog

  • cool tutorial about the basics, including:
    • many ways to use listings to create blogs, archive pages, etc. + per-directory options
    • layouts to create cool pages e.g. contacts
    • include-after-body footers
    • styling, incl. scss rules
    • page-layouts and complex reactive grids

• Quarto – Article Layout is my friend — columns, margins, overflows etc.

• Quarto – Page Layout

• Quarto – HTML Options

• Code/Other links text can be changed here: quarto-cli/src/resources/language/_language.yml at main · quarto-dev/quarto-cli

• Quarto glob syntax can do a lot: Quarto – Quarto Glob Syntax

• cool pic sources:

  • LoremFlickr: free placeholder images

• sample of including a style in the qmd: quarto-web/docs/gallery/index.qmd at main · quarto-dev/quarto-web

Creating a publications view

• This convets bibtex into directory+md: GetRD/academic-file-converter: 📚 Import Bibtex publications and Jupyter Notebook blog posts into your Markdown website or book. 将Bibtex转换为Markdown网站

• Hugo Blox template? hugo-blox-builder/modules/blox-bootstrap/layouts/publication/single.html at main · HugoBlox/hugo-blox-builder

  • all fields: hugo-blox-builder/modules/blox-bootstrap/archetypes/publication/index.md at main · HugoBlox/hugo-blox-builder

• Gallery example: https://github.com/quarto-dev/quarto-web/blob/main/docs/gallery/gallery.ejs

  • Ex. includes using quarto syntax w/ ::: in a template: quarto-web/ejs/links.ejs at main · quarto-dev/quarto-web

• Would be cool to have the format consistent with the existing quarto infra: quarto-web/docs/journals/authors.qmd at main · quarto-dev/quarto-web

• EJS

  • EJS – Embedded JavaScript templates
  • EJS — Embedded javascript tutorial | by Simuratli | May, 2024 | Level Up Coding:
  • Introduction to EJS: A Guide to Building Dynamic Web Applications - DEV Community

	<% for (let i = 0; i < item['authors'].length; i++) { %>
<%= item['authors'][i] %>,
    <% } %>

• Problem: full HTML templates don’t accept markdown bits: Markdown not parsed in listings with custom EJS · Issue #8756 · quarto-dev/quarto-cli

Changes in the paper mds

• Remove markdown from all existing paper md, HTML-only
• Remove “image” key
• date is publishing date of the paper, not of its page — publishDate doesn’t exist
• publicationType: maybe at some point change the int representation into str, as per latest hugo blocks behaviour
• doi: no URI
• publication: no In *journal*, just journal

This and only this will be supported:

title: 'Title'
authors:
  - TODO
  - TODO
date: '2010-10-20T00:00:00Z'
doi: 

# Publication name and optional abbreviated publication name.
publication: 'Proceedings of the World Congress on Engineering and Computer Science. Vol. 1'
publication_short: 'WCeCS 2010'

abstract: 'Long abstract'
links:
  - name: TODO Anthology
    url: https://aclanthology.org/L14-1240

url_pdf:
slides:
video:

tags:
  - paper-tag

EDIT: more fields here: hugo-blox-builder/modules/blox-bootstrap/archetypes/publication/index.md at main · HugoBlox/hugo-blox-builder

url_pdf:
url_code:
url_dataset:
url_poster:
url_source:
url_project:
url_slides:
url_video:

Process

Parsing date year

Datetime formatting / customization in ejs - Stack Overflow describes ways to do things with dates in EJS/JS

<%= new Date().getFullYear();%>

OK so I can use JS?

// Works
<%= new Date(item.date).getFullYear() %>

Iconify Icons

I can’t seem to use shortcodes inside html EJS (same as markdown problem I guess?)

But I can use the CSS (and ofc just download the PNG files)

Accessing file path to find its Bibtex

.. in a file inside same dir as paper markdown.

• Problem: no access to filename being rendered!

• item.filename is the name, not path. I can’t do “link to file in the same directory as the one being listed”

• Find the name of the current file being rendered in Quarto - General - Posit Community

• At some point you could use shortcodes in frontmatter: Variables in categories (in frontmatter) do not parse properly when categories is set to true in listing · Issue #5578 · quarto-dev/quarto-cli

  • metadata is the one in the listing page, not the page being listed :(

• [FR] New Lua utility function to access qmd input filename · Issue #2249 · quarto-dev/quarto-cli

  • mcanouil/quarto-lua-env: lua-env is an extension for Quarto to provide access to LUA objects as metadata.
  • extension to get access to some vars
  • works for not-frontmatter

• Just realized that for listings item.path (=location of page) is just what I need — I just need to change the last element

  • Split Path String - DEV Community for js path operations

Most horrible thing I’ve ever written but seems to work:

<%= item.path %><br>              
<% let x= item.path.split('/') %> 
<% x.pop() %>                     
<%- x.join('/') %>                
<%- x.join('/') %>/cite.bib       

// ---

<% let x= item.path.split('/'); x.pop(); let biburi =  x.join('/')+'/cite.bib' %>
<a href="<%- biburi %>">
	<%= biburi %>
</a>

(I should just do a lua filter or something at this point)

Anchors

Idea: link from elsewhere directly to the paper in papers

• Text fragments
  • not supported in firefox
  • could write a trivial style and use quarto’s mechanisms for citations but that doesn’t matter because not widely supported
• anchors
  • markdown - How to link to a named anchor in Multimarkdown? - Stack Overflow
  • completed with anchors, taken from dirname and optionally overwritten by a paper’s front-matter

Final system: described in 240618-1448 Quarto publications page and adding anchors

Wanted to do Manjaro, after googling found out it has a very bad reputation (arindas/manjarno: Reasons for which I don’t use Manjaro anymore) — I’ll go with EndeavourOS. OpenSUSE Tumbleweed will be my next choice if this fails.

Strategy:

• /opt will have executable programs
  • TG, hugo(?), pycharm, kitty?..
  • qtile, qutebrowser
• home encrypted, but separate swap+etc. partitions of size of ram
  • or not, since encrypted home/swap/… and hibernation are hard — so only encryption
• FS: BTRFS
  • …because snappshotting for if one randomly rms /etc¹².
    • SysadminGuide - btrfs Wiki
    • I like this: Btrfs - Manjaro
    • Reasons against exist³.
      • df will incorrectly report space usage
      • no native encryption support
      • complexity
  • F2FS is the fastest for SSDs but can lose data if battery dies
• Dotfiles
  • radically simplify everything, no need to source global ones and /home/me etc. — I’ll just use the same user for everything

OK, EndeavourOS+btfrs it is.

• Post-install steps
  • Average User’s To Do List after doing a fresh install of Endeavour OS with KDE Plasma : r/EndeavourOS
  • https://forum.endeavouros.com/t/coming-from-manjaro-to-endeavouros-installation-advice-for-btrfs-snapper-btrfs-assistant/34003/2 setting up snapshots

• Not supported out of the box:
  • https://inkscape.org/forums/questions/bulletpoints-and-more-advanced-text-controls/
• https://alpha.inkscape.org/vectors/www.inkscapeforum.com/viewtopic0f39.html?t=9647:

  • I usually clone a circle and distribute it evenly next to the text object

  • <C-u-2022> adds a bullet point symbol

https://www.kaggle.com/code/datark1/what-are-dcm-and-nii-files-and-how-to-read-them

• NII files are typical for MRI data

• This library can open them: https://nipy.org/nibabel/gettingstarted.html

  • Read and write access to common neuroimaging file formats, including: ANALYZE (plain, SPM99, SPM2 and later), GIFTI, NIfTI1, NIfTI2, CIFTI-2, MINC1, MINC2, AFNI BRIK/HEAD, ECAT and Philips PAR/REC. In addition, NiBabel also supports FreeSurfer’s MGH, geometry, annotation and morphometry files, and provides some limited support for DICOM.

• Cool viewers exist

  • online
    • https://socr.umich.edu/HTML5/BrainViewer/
      • for annotated files, 3d opacity+threshold help seeing the annotated areas VS raw image
      • drop multiple files (e.g. raw + ground truth) to have them as multiple layers
    • https://brainbrowser.cbrain.mcgill.ca/volume-viewer
      • can do 4d (3d+time) viewing! E.g. beating heart from ACDC dataset
    • https://www.fmrib.ox.ac.uk/ukbiobank/group_means/index.html may or may not be papaya
    • https://niivue.github.io/niivue-ui/ neat w/ probe1.nii.gz
    • https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.6279.6001.224985459390356936417021464571?seriesInstanceUID=1.2.276.0.7230010.3.1.3.0.57823.1553343864.578877,1.3.6.1.4.1.14519.5.2.1.6279.6001.273525289046256012743471155680 a cool viewerKKJJjjkk

• list of others: https://docs.facebase.org/docs/nifti-files/

• https://github.com/niivue/niivue even better list of others

• Other

  • Maternal Brain Project - OpenNeuro OpenNeuro has many datasets

Quarto’s roughnotation works either in speaker view or in the presentation, separately from each other.

Very brittle, but:

• jitsi share my presentation window, “show me what I’m sharing” and then right click picture-in-picture
• drag that picture over the presentation in speaker view :)

.. yeah.

A particularly complex file after a crash became cursed: two boxes would export to PNG fine, but not to PDF.

After closing and opening these two boxes became just as in the PDF export.

Solution: delete the problematic elements from the PDF and recreate them.

So I learned that poster videos are a thing, 10 years ago I’d have used Prezi but not now

And TIL inkscape has neat keybindings for zooming.

So.

• Inkscape keyboard and mouse reference | Inkscape

• View -> Zoom has a list as well

  • I should have looked at it earlier — it has neat things e.g. X-ray etc.:

• 3 zooms in to the selected element

• 1-6 are all zooming things (from the docu):

  • 

• 3-Layout-extraction-1 is a 6min description I have not watched but could be nice.

Context: 240423-1912 Better Posters notes

Grids

• Two — once in document properties, and there seems to be a tool for this as well.
• units are important — doing math in mm and then creating a grid in px/pc/pt/… may create not-obvious errors
• Snapping to an invisible grid still works — neat for a sub-grid that would create too much garbage but is needed (e.g. margins in a column)

Selecting

• You can select items by color/stroke/type/… in Edit -> Select…
  • E.g. select all texts, all blue rectangles etc.

Resizing boxes

You can disable scaling the stroke size together with the box in the upper-right toolbar! (The same one where locking aspect ratios is)

Text

• You can draw a rectangle and then add text to it — use the text tool to draw this rectangle.
• Text -> Flow into frames can do what it says on the label:
• If it looks weird or different from other text but font and size and the rest match, the text likely has a stroke.

Tracking invisible items

For things into which text “flows” etc. — if you have a white background making them white works for keeping them selectable when needed. (And removing the background makes them visible).

Smile for the screenshot

Damn I love doing random vector stuff.

Basics

• Quarto – Revealjs

• Quarto – Advanced Reveal

• Oh it has another page with a clean reference! Quarto – Revealjs Options

• Default presentation size is 1050x700

Presenting

    slide-number: true
    hash-type: number

Hide slide: ## Slide Title {visibility="hidden"}

Slides themselves

Title slide

If you exclude title and author from frontmatter, no title slide will be created and you can create your own

Asides

Asides exist: :::{.aside}

Format

• Many bits from Quarto – Markdown Basics etc. apply for presentations as well!

• Comments are HTML comments.

  • Other options exist but it’s still the best one. How to comment out some contents in .qmd files? · quarto-dev/quarto-cli · Discussion #3330

• [this is a div]{.to-which .i-can add="stuff"}

• For slides w/o titles you can still do this

  # {background-image="https://upload.wikimedia.org/wikipedia/commons/2/2b/Ouroboros-Abake.svg" background-position="center" background-size="contain" .center}
  

Centering stuff

• TODO
• Vertical/Horizontal
• Vertically and horizontally centered content in slides (revealjs) · quarto-dev/quarto-cli · Discussion #2951
• css - Vertical Align of images in Quarto Presentations - Stack Overflow

Complex layouts are possible with layouts:

::: {layout="1],[-1,1,1,1,-1"}
![](./images/logos/logo_chatgpt.png){.nostretch width="500px" fig-align="center"}

![](./images/logos/l2.png){.nostretch width="200px"}

![](./images/logos/l1.png){.nostretch width="200px"}

![](./images/logos/l3.png){.nostretch width="200px"}
:::

Classes

## {.classname}

::: {.classname}
div with class=classname
:::

::: {}
div with no class — we still need the {} for it to be a div
:::

::: {.one-div}
:::: {.inside-another}
But the number of : doesn't matter as long as it's >3 — they aren't even matching by count, it's just divs inside divs inside divs, the number of : is just for readability
::::
:::

Plugins

Attribution

quarto-ext/attribution: Display attribution text sideways along the right edge of Revealjs slides.

format:
	revealjs: ...
revealjs-plugins:
  - attribution
---

## attribution
![testatt](./images/logos/logo_chatgpt.png){.nostretch width="600px" fig-align="center"}

::: {.attribution}
Photo courtesy of [@ingtotheforest](https://unsplash.com/@ingtotheforest)
:::

Roughnotation

EmilHvitfeldt/quarto-roughnotation: An extension that uses the roughnotation javascript library to add animated annotations to revealjs documents.

R to run.

Sample presentation: RoughNotation; its source: quarto-roughnotation/example.qmd at main · EmilHvitfeldt/quarto-roughnotation

---
title: Simple roughnotation setup
filters:
   - roughnotation
---

- [type]{.rn rn-type=circle}
- [animate]{.rn rn-animate=false}
- [animationDuration]{.rn rn-animationDuration=20000}
- [color]{.rn rn-color=blue}
- [strokeWidth]{.rn rn-strokeWidth=3}
- [multiline multiline multiline multiline multiline multiline multiline multiline multiline multiline]{.rn rn-multiline=true}
- [iterations]{.rn rn-iterations=1}
- [rtl]{.rn rn-rtl=false}

also

{.rn rn-type=underline}

Key bits:

And this will be [circled]{.rn rn-type=circle rn-color=orange}
and [underlined]{.rn rn-type=underline rn-color=orange rn-animate=false}
and [boxed]{.rn rn-type=box rn-color=blue rn-animate=false}
and [crossed]{.rn rn-type=crossed-off rn-color=blue rn-animate=false}
and [crossed again]{.rn rn-type=strike-through rn-color=blue rn-animate=false}