After enabling “strict” newlines for markdown/hugo conformity I had to decide whether newline would be two trailing space or a single backspace (Line breaks in markdown)

Backspaces didn’t work out, so whitespaces it is - how to make them visible when editing?

Obsidian forum¹ provided this wonderful snippet:

.cm-trailing-space-new-line, .cm-trailing-space-a, .cm-trailing-space-b, .cm-tab{
  font-size: 0;
}
.cm-trailing-space-a::before, .cm-trailing-space-b::before, .cm-trailing-space-new-line::before, .cm-tab::before{
  content:'·';
  color:var(--text-faint);
  font-size: initial;
}
.cm-trailing-space-new-line::before {
  content:'↵';  
}
.cm-tab::before {
  content:'⟶'
}

Works!

(And shows tabs as bonus, perfect.)

I seem to keep googling this. … and this is not final and magic and I should actually understand this on a deeper level.

Not today.

So.

TL;DR for copypaste

Reading lines in a file:

while IFS="" read -r p || [ -n "$p" ]
do
  printf '%s\n' "$p"
done < peptides.txt

For outputs of a command:

while read -r p; do
	echo $p;
done < <(echo "one\ntwo")

Easy option with cat

Otherwise: Easy option that I can memorize, both for lines in command and in file that will will skip the last line if it doesn’t have a trailing newline:

for word in $(cat peptides.txt); do echo $word; done

Same idea but with avoiding this bug:

cat peptides.txt | while read line || [[ -n $line ]];
do
   # do something with $line here
done

Correct option without cat

1. Same as first cat option above, same drawbacks, but no use of cat:

   while read p; do
     echo "$p"
   done <peptides.txt
   

   ``

2. Same as above but without the drawbacks:

   while IFS="" read -r p || [ -n "$p" ]
   do
     printf '%s\n' "$p"
   done < peptides.txt
   

3. This would make command read from stdin, 10 is arbitrary:

   while read -u 10 p; do
     ...
   done 10<peptides.txt
   

(All this from the same SO answer¹).

In general, if you’re using “cat” with only one argument, you’re doing something wrong (or suboptimal).

Wanted to rename all tasks belonging to a certain project from a certain timeframe.

TL;DR

• Use filters to select tasks within a timeframe
• If you use hierarchical projects (pro:w.one.two) heavily and want to keep the children names:
  • Export them and use JSON parsing magic to get a unique list of project names
  • Bash loop to manually rename each of these project

Final command I used:

for p in $(task export "\(pro.is:w or pro:w.\) entry.after:2019-04-30 entry.before:2021-12-31"  | jq ".[].project" -r | sort | uniq);
	do task entry.after:2019-04-30 entry.before:2021-12-31 pro:$p mod pro:new_project_name$p;
done

Longer version

The task¹

Used project:w for work, now new work, makes sense to rename the previous one for cleaner separation.

To list all tasks created in certain dates (task all to cover tasks that aren’t just status:pending as by default):

task all pro:w entry.after:2019-04-30 entry.before:2021-12-31

1213 tasks. Wow.

Remembering when I was using sprints and renaming them at the end, pro:w covers pro:w.test and pro:whatever.

I was disciplined but wanted to cover all pro:w and pro:w.whatever but not pro:whatever just in case, so tested this, same result:

task all "\(pro.is:w or pro:w.\) entry.after:2019-04-30 entry.before:2021-12-31"

How to modify them?

The problem

Okay, got them. How to modify? Complexity: I need to change part of the project, so pro:w.one -> pro:old_w.one instead of changing all tasks' project to pro:old_w

Attempts

Commands

There’s prepend ² but seems to work only for descriptions.

There’s t mod /from/to/ syntax³, couldn’t get it to work part of the project.

There’s regex⁴, but works only for filters if enabled

There’s json export but I don’t feel like parsing JSON, feels too close to day job :)

Listing projects

You can list projects like this:

# currently used
task projects

# all
task rc.list.all.projects=1 projects

This gives hope, if I get the list of projects I can just iterate through them and rename all of them individually.

Can’t find this documented, but task rc.list.all.projects=1 projects pro:w filters the projects by ones starting with w.

Format parses the hierarchy sadly

Project       Tasks
w              1107
  a               1
  aan             1

Can I rename the character used for hierarchy so that I get them as list of separate tags with dots in them? Not exposed through config from what I can see

…alright, JSON export it is

JSON export

It exists, and of course it accepts filters <3

task export "\(pro.is:w or pro:w.\) entry.after:2019-04-30 entry.before:2021-12-31" | wc -l

1215 lines - about the same ballpark as the number of tasks.

JSON output is an array of these objects:

  {
    "id": 0,
    "description": "write attn mechanism also on token features",
    "end": "20191016T143449Z",
    "entry": "20191016T120514Z",
    "est": "PT1H",
    "modified": "20200111T094548Z",
    "project": "w",
    "sprint": "2019-41",
    "status": "completed",
    "uuid": "d3f2b2ac-ec20-4d16-bd16-66b2e1e568f9",
    "urgency": 2
  },

Okay

> task export "\(pro.is:w or pro:w.\) entry.after:2019-04-30 entry.before:2021-12-31"  | jq ".[].project" | uniq


"w.lm"
"w.l.p"
"w.lm"
"w.lm"
"w.l.py"
"w.lm"
"w"

Proud that I wrote that from the first try, as trivial as it is. Thank you ExB for teaching me to parse JSONs.

The quotes - jq -r returns raw output⁵, so same as above but without quotes.

Final command to get the list of projects:

task export "\(pro.is:w or pro:w.\) entry.after:2019-04-30 entry.before:2021-12-31"  | jq ".[].project" -r | sort | uniq

(Remembering that uniq works only after sort)

And let’s make it a loop, final command:

for p in $(task export "\(pro.is:w or pro:w.\) entry.after:2019-04-30 entry.before:2021-12-31"  | jq ".[].project" -r | sort | uniq);
	do task entry.after:2019-04-30 entry.before:2021-12-31 pro:$p mod pro:new_project_name$p;
done

Nice but forgotten stuff:

• Taskwarrior - Context are neat, useless for me here, but neat.
• task summary
• Nice post about projects: Taskwarrior Projects | Random Geekery

jq -r $stuff instead of quoted ‘correct’ values like

"one"
"two"
"three"

would return

one
two
three

Redirecting stuff

Had /dtb/days/day122.md-type posts, the older ones, and /dtb/days/1234-1234-my-title.md-type newer posts. They lived both in the same directory on disk, /content/dtb/days/.... The latter were converted from Obsidian, which meant (among other things) that deleting a page in Obsidian wouldn’t automatically delete the corresponding converted one in Hugo, and I couldn’t just rm -rf ..../days before each conversion because that would delete the older day234.md posts.

I wanted to put them in different folders on disk in ./content/, but keep the url structure serhii.net/dtb/post-name/ for both of them.

Solution was making all /dtb posts (incl. pages) use the section (dtb) in the permalink in config.yaml:

permalinks:
    dtb: '/:section/:filename'

Now they do, regardless of their location on disk.

Then I moved the old posts into ./content/dtb/old_days, kept the new ones in ./content/dtb/days

Lastly, this removes all converted posts (= all .mds except _index.md) before conversion so that no stray markdown posts are left:

find $OLD_DAYS | grep -v _index.md | xargs  rm 

Unsolved problems

Google still has serhii.net/dtb/days/... pages cached, and currently they’re available both from there and from /dtb/.... I can’t find a way to redirect all of the /dtb/days/... to /dtb/... except manually adding stuff to the frontmatter of each. I have scripts for that, but still ugly.

.htaccess is our friend.

" RewriteRule ^d/dtb(.*)$ /dtb$1 [R=301,NC,L]
RewriteRule ^dtb/days(.*)$ /dtb$1 [R=301,NC,L]

This is getting more and more bloated.

Generally, I see absolutely no reason not to rewrite this mess of build scripts in Python. obyde is a Python package, handling settings, file operations etc. is more intuitive to me in Python.

Instead I keep re-learning bash/zsh escape syntax every time, and I’m procrastinating doing error handling for the same reasons.

The only non-native thing would be rsync and git, which can be handled through a subprocess.

pytest-datafiles · PyPI allows copying files to a temporary directory, then they can be modified etc. Really neat!

Sample:

ASSETS_DIR = Path(__file__).parent / "assets"
PROJ_DIR = ASSETS_DIR / "project_dir"

konfdir =  pytest.mark.datafiles(PROJ_DIR)

@konfdir
def test_basedir_validity(datafiles):
	assert directory_is_valid(datafiles)

Also love this bit:

Note about maintenance: This project is maintained and bug reports or pull requests will be addressed. There is little activity because it simply works and no changes are required.

SADLY this means that returned path is py.path, I’m not the only one complaining about that¹

Pytest has newer native fixtures that use Pathlib (Temporary directories and files — pytest documentation) but datafiles hasn’t been moved to them.

A conftest.py file gets imported and run before all the other ones.

Pytest resolves all imports at the very beginning, I used conftest.py it to import a package so that it’ll be the one used by the imports in files that are imported in the tests (seeing that there’s a mypackage already imported, subsequent import mypackages are ignored)

(Can I think of this as something similar to an __init__.py?)

pytest-datafiles · PyPI is nice but returns a py.path instead of pathlib.Path.

Tried to write something to make it convert automatically.

ASSETS_DIR = Path(__file__).parent / "assets"

@pytest.fixture
def pfiles(datafiles):
    # Fixture that converts pytest-datafiles' py.path into a pathlib.Path
    return Path(str(datafiles))

@pytest.mark.datafiles(PROJ_DIR)
def test_read_meta_json(pfiles):
	assert do_sth_with_file(pfiles)

First nontrivial fixture I write, maybe a really bad idea to do it like that. This feels like a general use case and someone had to have had this problem

This looks really interesting! It’s not about the syntax, but about the basic design philosophies + examples of packages that use it.

What’s init for me? Designing for Python package imports | Towards Data Science

Other stuff I learned about __init__.py:

• You can use it to enforce import order ¹
• You can use it to declare package variables
• Automatically import modules from a package²

Stuff I discovered:

• You can set a breakpoint in pdb physically into an __init__.py, and for example look at the stack of what called it with w

Today, I ran this:

git commit -m "TICKETNAME Export of X generated with `name-of-some-utility`"

Commit message on gitlab was

"TICKETNAME Export of X generated with (Starting the export of data, wait till it downloads...)"

Clear but fascinating way it can break.

Do I want to get a clear picture of all the various levels of escaping, including globs, backticks, backslashes etc. happening in the shell?

Why doesn’t the # in git commit -m "Ticket #1231" result in a string with the 1234 commented out and a syntax error? I know it doesn’t but I wouldn’t be able to predict that behaviour without this knowledge. Would single quotes change much? How to actually comment the rest of the line this way?

What are the rules that decide whether a * gets expanded by the shell or passed to, say, scp as-is? Etc. etc. etc.

It’s all knowable and learnable, but I was never sure whether the ROI was worth it for me. Till now trial and error always worked in the rare instances I have to do something complex with bash scripts, but this is the first time it bites me in real life in an unexpected way.

I find this approach¹ brilliant (and of course it works with everything split in separate functions a la my last post: 211124-1744 argparse notes):

import argparse
import logging

parser = argparse.ArgumentParser()
parser.add_argument(
    '-d', '--debug',
    help="Print lots of debugging statements",
    action="store_const", dest="loglevel", const=logging.DEBUG,
    default=logging.WARNING,
)
parser.add_argument(
    '-v', '--verbose',
    help="Be verbose",
    action="store_const", dest="loglevel", const=logging.INFO,
)
args = parser.parse_args()    
logging.basicConfig(level=args.loglevel)

And TIL about dest= that will make my life much easier too by outsourcing more logic to argparse.

Connected an external screen, it was dark, googled for a solution after resetting redshift settings didn’t work.

So, there are a lot of ways to change brightness (SO¹).

xbacklight works with hardware-level brightness for the devices that support it.

For the others, software-level changing of gamma values is what’s usually needed, and what I did with a lot of different programs before. This worked this time:

xrandr --output LVDS1 --brightness 0.5

(As a bonus, it uses the already well-know and well-loved xrandr.)

Sad that arandr can’t do brightness though, but there are reasons (missing –brightness features (#35) · Issues · arandr / ARandR · GitLab)

From there I learned that ddcondrol is the way to change brightness for external monitors on hardware level, and that Jolmberg/wmbright: Monitor brightness control dockapp is a back-end that tries to do everything.

TODO, this look really really really good. Explanation of the relationship between python logging root logger and other loggers

(+ Love the way it’s split into separate .py files)

pytest took seconds at the “Collecting…” stage.

I had a directory with a lot of tiny files (./data_1234/) in the poetry package folder, and blamed it initially.

SO¹ told me that the syntax to ignore a folder is

[tool:pytest]
norecursedirs = subpath/*

Wildcards are nice and data*/* was the first attempt.

Nothing.

Then I without success tried this:

testpaths="tests"

After a one-hour saga, I found that the culprit was a package that I was using. The tests imported my package, which imported the slow package, and it takes seconds to do so.

‘Collecting’ seems not to be only “find test files”, but it reads them and imports them and all their dependencies.

Waiting time went back to normal as soon as I commented out importing my package from the test.

If you use a directory structure like this:

resources/
src/project_name/
tests/
[...]

then you get these directories in the same order regardless of the name of the project! Then it’s always uniform, muscle memory has a chance, etc.

From within an issue, use the dropdown left of “Create merge request” -> Create branch, will create a branch with the format “issue_n-issue_title”, for example 3-this-is-issue-number-three.

<Ctrl-C> of a program running inside pdb (python3 -m pdb myscript.py or whatever) doesn’t kill the program, but drops you in the debugger!

Useful when you suspect there’s an infinite loop somewhere, and want to see what exactly is the program doing when it starts using 120% of your CPU

Use-case - using limited mobile internet.

vnstat is nice. sudo apt install vnstat, service has to be started/enabled through systemctl as usual.

Logs traffic with 5-minute granularity, so for the first 5 minutes after install will say that there’s not enough information :)

vnstat -5 returns the last hours in 5-minute interval, -h/-d/-m is hourly/daily/monthly.

-i selects the interface (otherwise all existing non-zero ones will be shown).

Installed noisetorch, it complained about CAP_SYS_RESOURCE like the last time and I fixed it by installing polkit like the last time, didn’t work though.

Issue seems to be that by default Mint has the home partition mounted with nosetuid¹, confirmed by doing mount.

Fix was to put the binary in /opt, the prompt is the same but after entering the password it works and I see the expected interface.

Saw this in the python pandoc cookbook¹

holder[index:index+1] = split_home(elt)

Wow.

Never thought I could assign multiple elements to a slice!

pdbpp is a drop-in replacement for pdb, and I like it more than ipdb for some reason.

Installing it makes it the default one imported when importing pdb (incl. by pytest, python’s breakpoint() etc!)

Really nice tutorial: pdb++, a drop-in replacement for pdb (the Python debugger) | PythonRepo

Vanilla-pdb cheatcheet: Python Debugger Cheat Sheet - Kapeli

Features not present in pdb that I love:

• ll outputs the text of the current function
• sticky updates the function listing with each new line, giving a nice interactive visual feeling to the debugging process

pytest -s works to make it play nice with the stdouts generated by pdbpp.

– The same procedure as last year?
– The same procedure as every year!

Now playing: “И вновь продолжается бой, и сердцу тревожно в груди…"

Досягнення року: майже повністю вийшов зі сфери сервісів Гугл! + Звільнився з (по суті першої) роботи і перейшов на другу¹.
Настрій року: постійного неясного стресу, тиску і відповідальності; WORK HARD and STAY IN BED THE REST OF THE TIME, UNABLE TO MOVE; нездатності знайти баланс між потребою в спілкуванні, нездатністю це робити онлайн, відсутністю сил/контекстів щоб це робити ІРЛ, і перманентним фоновим відчуттям вини за все вищезгадане
Зустріч року: “Дядя Сережа, Давайте вы выберите покемона на день рождения а я его нарисую как только смогу”
Подія року: Маша подарувала велосипед!²
Жах року: Вкрали велосипед :( А якщо серйозно - пожежа в³ костелі під час якої згорів⁴ орган⁵
Країна року: Україна
Місто року: Лейпциг!!!! Цього року з трьома чотирма окличними знаками!
Слово року: Corona-Warn / Дій вдома / тій фОтме ⁶
Подорож року: літня, осіння і особливо зимня поїздка додому
Веб-сервіс року: Fastmail + Coronavirus charts⁷
Колір року: приємний помаранчево-кремовий
Запах року: спирту, сливовиці, бензину, алое вера, у кожного магазину свій
Новина року: про одну конкретну чергову можливу довготермінову поїздку
Книга року: The Culture Map (Erin Meyer) + Permutation City
Фільм/серіал року: Star Trek Enterprise
Media N.O.S.: Дуже багато подкастів! Lore, Old Gods of Appalachia, SCP Archives
Пісня року: Voltaire - The Trouble with Tribbles (Song Only) - YouTube; just one snack - YouTube; Let the Sunshine In - Hair - YouTube (конкретно те відео)
Заклад року: кафе через парк неподалік від квартири в Лейпцигу
Напій року: Чай, заварений в молочнику (або в паперовому пакетику, куди його можна насипати). На другому місці “чай” з хризантеми
Їжа року: свіжеспечені булочки + Lemon curd salmon recipe
Транспорт року: мій власний (другий за цей рік…) велосипед!

Побажання собі на 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 рік:
Знайти свій sustainable дзен з роботою, стресом, сном, кофеїном, ритмом дня або його відсутністю. Знайти свій sustainable дзен з людьми та спілкуванням з ними. Знайти свій sustainable дзен з енергією, її джерелами, та способами її витрачати.

Ціль на наступний рік:

• Більше писати/створювати не важливо чого (пости, бібліотеки на Github, малюнки, вірші), але make an effort, умовно пости про PKM в блог⁸ а не короткі нариси про окремі деталі⁹ (хоча краще вони, ніж взагалі нічого).
• Навчитися підтримувати спілкування з людьми навіть при жорсткій зміні shared context (how do we talk to each other if there’s no watercooler anymore?…). Згадати дзен Мерзебургу і організовувати речі. Більше спілкуватися з людьми особисто та через відеодзвінки (.. або хоча б просто по телефону), менше тексту.
• Більше свіжого повітря, подорожей, спонтанності, легкості, не економити свою енергію з ціллю потім витратити її на лежання в кроваті

Ну і вічнозелене: Продовжити сон, спорт, медитацію – ЦЕ ПРАЦЮЄ. (x4)

(У)