serhii.net

Download GitHub directory: paste an URI to the directory, get .zip of that directory only. Handy for downloading only parts of datasets

TL;DR use Chromium

PDF, PS and DjVu - ArchWiki has a table, but it lies, in my tests:

• zathura explicitly no support
• evince didn’t work
• gnome document viewer didn’t work

And for the Nth time, I end up remembering about Chrome/Chromium PDF viewer, that does this reliably.

• TIL Arch has no cron by default, and uses systemd timers¹.
• cron - ArchWiki

sudo pacman -S cronie
sudo systemctl enable cronie.service

# many minutes were lost due to me forgetting to start it as well...
sudo systemctl start cronie.service

• I like this website to generate stuff:Crontab Generator - Generate crontab syntax
• cronie - Timed Tasks - Documentation as good as any
• the script has to be executable
• crontab -e edits your own crontab, then you can omit the username.

Cron git backup script

#!/bin/bash

# https://stackoverflow.com/posts/1482133/
# get the directory where this file is located
DIRNAME="$( dirname -- "$( readlink -f -- "$0"; )"; )"

cd $DIRNAME &&
git add -A &&
git commit -m "Automatic backup at `date`" &&
git push origin master &&
echo "Done"

# and this is the crontab
# odd hours:
# * 1-23/2 * * *  /bin/bash $HOME/this-script-location.sh > /tmp/cronlog

EDIT: sometimes if git crashes an index file stays and then nothing runs — so the cronjob must be somehow monitored manually, todo.

TL;DR: fish easy version below works, but needs quotes when expression is complex: cc 2+2 but cc 'floor(2.3)'.

I’m continuing to move my useful snippets from zsh to fish (240620-2109 Fish shell bits), and the most challenging one was the CLI python calculator I really love and depend on, since it contained arguments with parentheses (which are fish expressions as well).

Basically: cc WHATEVER runs WHATEVER inside python, can do both easy math a la 2+2 and more casual statistics-y mean([2,33,28]).

Before in zsh this was the magic function:

cc() python3 -c "from math import *; from statistics import *; print($*);"
alias cc='noglob cc'

Fish, easy version:

function cc
  command python3 -c "from math import *; from statistics import *; print($argv);"
end

Works for easy cc 2+2 bits, but as soon as functions and therefore parentheses get involved (cc floor(2.3)) it starts to error out.

[I] sh@nebra~/t $ cc mean([2,4])
fish: Unknown command: '[2,4]'
in command substitution
fish: Unknown command
cc mean([2,4])
       ^~~~~~^
[I] sh@nebra~/t $ cc mean\([2,4]\)

>>> mean([2,4])
3
[I] sh@nebra~/t $

(But I REALLY don’t want to do cc mean\([2, 3]\))

In the zsh snippet, noglob meant basically “take this literally w/o expanding anything”, and it passed everything as-is to python, and this is what fails in my fish solution.

Noglob in fish is fun:

• The fish language — fish-shell 3.7.0 documentation on escaping characters
• Implement the noglob modifier · Issue #3504 · fish-shell/fish-shell:

  If you wish to use arguments that may be expanded somehow literally, quote them. echo ‘’ and echo “” both will print the literal.

• The fish language on quotes:
  • single quotes = no expansion of any kind
    • \' for literals inside single
  • double quotes = variable exp. ($TERM) & command substitution ($(command))
    • \" for literal "s inside double
  • within each other, no special meaning
  • Let’s test:
    • echo (ls) = ls output, one line
    • echo "$(ls)" = ls output, multiline
    • echo '(ls)' = (ls)
    • echo "(ls)" = "(ls)"

THEN

• command python3 -c "from math import *; from statistics import *; print($argv);"

  • cc ceil\(2\) +
  • cc ceil(2) -

• `command python3 -c “from math import *; from statistics import *; print(’$argv’);”

  • literally prints the passed thing w/o python eval, w/ same rules

• OK can I do a variable then?

  set pyc $argv
  echo $pyc
  command python3 -c "from math import *; from statistics import *; print($pyc);"

nope.

Bruteforcing the solution

(and learning to use fish loops mainly, of course there are better ways to do this.)

# list of simple, brackets, and parentheses + no, single, double quotes 
# no space between nums in brackets, python interpreter would add them. [2,3] — literal, [2, 3] — parsed by python
set cmds \
'2+2' \
'\'2+2\'' \
'"2+2"' \
'[2,3]' \
'\'[2,3]\'' \
'"[2,3]"' \
'floor(2.3)' \
'\'floor(2.3)\'' \
'"floor(2.3)"' 

function tcc
  set pyc $argv
  # command python3 -c "from math import *; from statistics import *; print" '(' "$pyc" ');'
  # command python3 -c "from math import *; from statistics import *; print($pyc);"
  command python3 -c "from math import *; from statistics import *; print($pyc);"
end


# loop through all test cases to see sth that works for all
for i in $cmds
  echo $i:
  echo "   $(tcc $i)"
end

At the end, no additional literal quotes + initial command didn’t error out, and we came full circle:

set cmds \
'2+2' \
'[2,3]' \
'floor(2.3)' 

# winner command!
function tcc
  command python3 -c "from math import *; from statistics import *; print($argv);"
end

[I] sh@nebra~/t $ ./test_cc.sh
2+2:
   4
[2,3]:
   [2, 3]
floor(2.3):
   2

• Double quotes in the python command mean only $pyc gets expanded
• $pyc in the working versions have no hard-coded quotes
• in CLI tcc floor(2.3) still fails — because like that it’s a command, not a string. In the file it was inside single quotes, as a string. So I can do this in the CLI as well.

So simple and logical at the end.

Final solution

function cc
  echo ">>> $argv"
  command python3 -c "from math import *; from statistics import *; print($argv);"
end

When using, quotes are needed only for complex bits (parentheses, * etc.).

[I] sh@nebra~/t $ cc 2+2
>>> 2+2
4

[I] sh@nebra~/t $ cc [2,3,4]
>>> [2,3,4]
[2, 3, 4]

# no quotes
[I] sh@nebra~/t $ cc mean([2,3,4])
fish: Unknown command: '[2,3,4]'
in command substitution
fish: Unknown command
cc mean([2,3,4])
       ^~~~~~~~^

# with quotes
[I] sh@nebra~/t $ cc 'mean([2,3,4])'
>>> mean([2,3,4])
3

So I literally had to follow the advice from the first link I found and used single quotes in my initial command:

If you wish to use arguments that may be expanded somehow literally, quote them. echo ‘’ and echo “” both will print the literal.

Still, I learned a lot about fish in the process and honestly am loving it.

Run and disown

In my old zsh config I had this:

function dos() {
    # run_disowned and silenced
    nohup "$@" >/dev/null 2>&1 & disown
}

Emulating my old dos command in fish:

• disown - remove a process from the list of jobs — fish-shell 3.7.0 documentation:
  • firefox &; disown starts and disowns ffx.
• terminal - How do I get fish to swallow? or at least to detach itself from a program - Stack Overflow

Both together give this solution:

function dos
  echo "disowning $argv"
  command $argv  >/dev/null 2>&1 &
  disown
end

Shell notation for $(cmd)

shell - Fish equivalent of bash $(command) notation - Stack Overflow: - bash: $(command) - fish: (command)

which command

type - locate a command and describe its type — fish-shell 3.7.0 documentation: type commandname.

[I] sh@nebra~/me/me-qs $ type cc
cc is a function with definition
# Defined in /home/sh/.config/fish/config.fish @ line 184
function cc
  echo ">>> $argv"
  command python3 -c "from math import *; from statistics import *; print($argv);"
end

TL;DR –no-clean is dangerous

quarto render --no-clean ... (or an interrupted render?) created many .html.md files in the same dirs as their .qmd/.md sources:

papers/11-bank2011using/index.md               papers/23-hanig2023nlp/index.md
papers/11-hanig2011benefits/index.html.md      papers/23-tumler2023virtual/index.html.md
papers/11-hanig2011benefits/index.md           papers/23-tumler2023virtual/index.md
papers/11-hanig2011knowledge/index.html.md     papers/24-hamotskyi2024fincorpus/index.html.md
papers/

Then the custom listing for */*.md went crazy because the html.md files didn’t have the needed fields. (also .jsons/.html etc.)

Quarto has no quarto clean but one can just re-render everything without --no-clean

Fish globs

To find all these files arbitrarily many levels deep:

ls **/*.html.md

When doing a multilingual quarto website using profiles (240619-1425 Quarto multilanguage website), quarto preview/render --profile de allows previewing only one language, and the menu links (/de/xxx) break.

Solution to preview EVERYTHING, including menu links logic:

# Todo break on erorr
quarto render --profile=ua
quarto render --profile=en
quarto render --profile=de

And then serve the _site directory locally through a simple webserver, e.g. Python SimpleHTTPServer - Python HTTP Server | DigitalOcean, and see the result on http://localhost:9000:

python3 -m http.server -d _site 9000

ERROR: TypeError: title.trim is not a function quarto

happens for me when in front-matter I do

---
# title: "Publications and Awards"
title: {\{< var test >}}

instead of QUOTED

---
# title: "Publications and Awards"
title: "{\{< var test >}}"

ALSO, interestingly, when I save the wrong version while previewing the error is better:

ERROR: Validation of YAML front matter failed.
ERROR: In file publications.qmd
(line 3, columns 8--24) Field "title" has value {\{< var test >}}, which must insteadbe a string
2: # title: "Publications and Awards"
3: title: {\{< var test >}}
         ~~~~~~~~~~~~~~~~~
4: css: ./posts_publications/pub_style.css
✖ The value {\{< var test >}} is of type object.
ℹ The error happened in location title.

ERROR: Render failed due to invalid YAML.

So: quarto errors are more detailed when previewing instead of when compiling from zero? Interesting. Okay.

• No native support

• R package exists, but I don’t feel like digging into R at all: Renders a Multilingual Quarto Book • babelquarto

• The other approaches I found are all based on project profiles and conditional output

  • oooo12 - Multi-language Blog with Quarto - Guide
  • Mario Angst - A multi-language (German/ English) Quarto website
    • only difference is that it’s /de/ /en/ instead of ENG being /en/
    • nice and more detailed about interesting corner cases

• Document Language – Quarto¹ for setting one language in the page and doing per-language changes to the template texts

  • quarto-cli/src/resources/language at main · quarto-dev/quarto-cli
  • quarto-cli/src/resources/language/_language.yml at main · quarto-dev/quarto-cli

• Conditional Content – Quarto

• Project Profiles – Quarto

Config files

_quarto-profilename.yml ONLY, the rest won’t get parsed

profile:
  default: en
  # mutually exclusive group: you can do only one
  # (otherwise `--profile one,two` does multiple)
  group: 
    - [en, de, uk]

Page content

# `unless-profile` exists as well
::: {.content-visible when-profile="en"}
This content will only appear in the advanced version.
:::

Links

Links are going to be interesting!

Currently /de is German, / is English.

Main home page is .. from DE lang, or /de from EN.

Menu items:

• ENG as-is. (`href: whatever.qmd)
• DEU: href: ../de/lehre.html — note the HTML bit!

But when previewing DEU, all of these pages are at / — ergo menu items don’t work, as they lead to a non-existing ../de/...

ALSO: marioangst.de shows nicely how one can link to other languages from the menu!

- icon: book
href: publications.qmd
text: Publikationen
- href: ../en/blog
text: Blog (englisch)

Site language

Website Options – Quarto tells me I can do this in each _quarto-de.yml etc. profile:

format:
  html:
    lang: de
	#lang: ua
	#lang: en

This changes the interface to follow the corresponding quarto-cli/src/resources/language/_language.yml at main · quarto-dev/quarto-cli

Unsolved

How do I change front matter (e.g. title)?

Not dealt with in any of the approaches: quarto’s native Document Language¹ thing

So: - How do I do different post titles per language? - How do I change site language, so _languages.yml, conditionally?

• Project Basics – Quarto discusses the various approaches to metadata

  • And I can conditionally include stuff

• … I could literally do a bash script that puts a _metadata.yaml, builds with a proflie, then removes that file. Oh this would be painful

• Skimming Website Options – Quarto doesn’t really help

  • except that I can set html format lang: de from within profiles! NICE

Do variables magic

• title gets taken from variable title[currnet-profile-name] or something?
• _variables.yml works for var shortcodes (240619-1845 Quarto error title.trim() is not a function), and shortcodes can do metadata too

# works
title-en: "Publications and Awards"
title: "{\{< meta title-en >}}"

If only I could do per-language attributes as shown in the docu²:

language:
  en:
    title-block-published: "Updated"
  fr:
    title-block-published: "Mis à jour"

It would be so cool if one could overwrite the other variables

language:
  de: 
    title: german post title

The above would nicely get a language from the profile _quarto-lang.yml and automatically change the things. Can I do this for titles and front-matter?

• I’m not alone:
  • Is there a way to insert variables in other shortcodes with Quarto? - Stack Overflow

I can get the current profile from the env variable

profile: {\{< env QUARTO_PROFILE >}}

If I could just

title: vars['titles']['postname'][QUARTO_PROFILE]

• Asked this on Github: How do I do conditional YAML frontmatter, e.g. based on profile? · quarto-dev/quarto-cli · Discussion #10064
  • Mea culpa, discussed already: Conditional metada for different profiles on a website · Issue #4516 · quarto-dev/quarto-cli Conditional metada for different profiles on a website · quarto-dev/quarto-cli · Discussion #4455 no idea how I missed this.

Use scripts

• Project Scripts – Quarto
  • A python script that sets title based on QUARTO_PROFILE

Quarto lua filters

OK let’s do this. No choice.

Learn Lua in Y Minutes

First³ attempt to write anything in lua:

function Meta(m)
  local profiles = quarto.project.profile
  local profile = profiles[1]
  if profile then
    print("Profile: " .. profile)
    m.active_profile = profile
  end

  if profile and m.titles and m.titles[profile] then
    cleantitle = pandoc.utils.stringify(m.titles[profile])
    oldtitle = pandoc.utils.stringify(m.title)
    m.title = cleantitle
    print("Profile:" .. profile)
    print("Old title:" .. oldtitle)
    print("New title:" .. cleantitle)
  end

  return m
end

I’d need to make it more robust:

• multiple profiles? Not now
• No titles set in titles — use the default one

So:

• Always require a title
• Optionally add array of titles, indexed by language=profile name
• If present use one of these.

function Meta(m)
  local profiles = quarto.project.profile
  if not profiles then
    -- TODO: missing YAML key? Empty YAML key?..
    -- TODO even more later: filter multiple profiles to use the language one
    return m
  end

  local profile = profiles[1]
  -- If we have a named profile, save it, otherwise return
  if profile then
    print("Profile: " .. profile)
    m.active_profile = profile
  else
    return m
  end

  if m.titles then
    local titles = m.titles
    if titles[profile] then
      newtitle = pandoc.utils.stringify(titles[profile])
      oldtitle = pandoc.utils.stringify(m.title)
      -- log both if they differ
      if newtitle ~= oldtitle then
        m.title = newtitle
        -- print("Old title:" .. oldtitle)
        -- print("New title:" .. newtitle)
        print(oldtitle .. " => " .. newtitle)
      end
    else
      print("Title for profile " .. profile .. " not found among ")
      for lang, title in pairs(titles) do -- Table iteration.
        print("    " .. lang .. ": " .. pandoc.utils.stringify(title))
      end
    end
  end

  return m
end

Main problems:

• listing pages use the old title anyway
• supports only title, not e.g. description (used, again, in listings)
  • and side menus!

Given: quarto website page with publications. Previously touched upon in the messy 240605-2055 Quarto website creation notes.

Anchor links in (quarto) markdown

This works:

[UNLP paper](publications.qmd#hamotskyi2024unlp)

Anchor targets

• Only correct solution: ¹ <a name="title"></a>
  • (markdown - How to link to a named anchor in Multimarkdown? - Stack Overflow says to use the name=, not id=, attribute, but this doesn’t work for me in quarto.)

Template

// Get anchor id somehow from paper path
<% let y= item.path.split('/'); y.pop(); let dirname = y.pop(); let citation_key = dirname.split('-').pop() %>

// Overwrite with paper front-matter if there's one
<% if (item.citation_key) { %> 
	<% citation_key = item.citation_key %>
<% } %>       

// Add to paper listing thing 
<a id="<%= citation_key %>"></a>