This goes into “things you’re allowed to do” (Previously: List of good things - serhii.net) territory, but:

• previously, screen/tmux’s use case was “ssh into a server far away and let things run even when your SSH session disconnects”
• had two terminals open on a remote server, had to edit the exact two files every time, over days and disconnections
• just realized that I can just have a screen session open with vim and the files I edit, and just attach to it next time I’d doing something on that server, whenever that is!

cloudflared:
image: cloudflare/cloudflared:latest
command: tunnel run
environment:
  - TUNNEL_TOKEN=my-super-secred-tunnel-token
restart: unless-stopped
network_mode: "host"

Then whatever can run in its network with bridge driver:

networks:
  nextcloud:
    driver: bridge
....
services:
  nextcloud:
    networks:
      - nextcloud
	ports:
	  - "1234:80"

And then in the cloudflare zero trust UI add a tunnel from localhost:1234.

Neat thing is that tunnel type HTTP refers to the connection to the host running cloudflared, but the thing is accessible through cloudflare’s servers as both http and https. No need to manually do any certs stuff!

• General:
  • https://fleet.linuxserver.io/ has a list of docker images of things, not necessarily for self-hosting
  • List of docker-compose examples for various self-hosting-relevant apps: petersem/dockerholics: Apps and examples from the Dockerholics group.
• Specific services:
  • bitwarden-rs / vaultwarden:
    • Their example w/ caddy for domain name: Using Docker Compose · dani-garcia/vaultwarden Wiki
    • Generally their wiki is excellent: Home · dani-garcia/vaultwarden Wiki

Wanted to run frp’s client frpc with docker to forward the SSH port.

Main issue was binding to a port already open on the host, and one not controlled by a docker thing.

My first attempt led to this: “: Error starting userland proxy: listen tcp4 0.0.0.0:22: bind: address already in use”

After looking around the Internet, found a solution.

Docker’s docker-compose.yml:

services:
  frpc:
    image: chenhw2/frp
    restart: unless-stopped
    environment:
      - ARGS=frpc
    volumes:
      - ./conf/frpc.ini:/frp/frpc.ini
    network_mode: "host"
    ports:
      - "22:22"

The key being the “nertwork_mode” part.

Neither frp server nor client configs needed anything special.

Strangely , I didn’t even need to set any capabilities like I did for dns:

services:
  dns:
    restart: always
    image: strm/dnsmasq
    volumes:
      - ./conf/dnsmasq.conf:/etc/dnsmasq.conf
    ports:
      - "53:53/udp"
    cap_add:
      - NET_ADMIN

• Allegedly happens when the network is misconfigured.
  • Since a black screen issue I religiously md5sum the ISOs, otherwise that would’ve been the prime suspect
• In my case I had port forwarding and DMZ and ipv6 configured in the router, disabling all of that fixed the installation issues
• To debug installation issues, <Ctrl-Shift-F2> to go to the tty and cat /var/log/syslog
  • less is not installed but nano is
  • tty4 has live running logs
    • that are nice for non-graphical install and “is it doing anything now?”

Relevant: 5.4. Troubleshooting the Installation Process

I always look in zsh history for this string:

sudo dd if=/path/to/debian-live-11.5.0-amd64-cinnamon.iso of=/not/dev/sda bs=1M status=progress

/dev/sda is the usb drive, will be ofc. deleted fully; not a partition like /dev/sdaX but the actual /dev/sda disk itself.

I specifically added /not/dev/sda at the beginning for systems where I have not set up unset zle_bracketed_paste and that might press enter on paste or after I edit the .iso but not of. That way I’m forced to think when editing of.

For functions/callables, Callable is not the entire story: you can annotate the arguments and returns values of these callables!

From mypy documentation:

The type of a function that accepts arguments A1, …, An and returns Rt is Callable[[A1, ..., An], Rt]."

You can only have positional arguments, and only ones without default values, in callable types

If your @abstractmethod should also be a @staticmethod, you can happily blend both, as long as the @staticmethod (or other) decorator comes first.

In other words, @abstractmethod should always be the innermost decorator.¹

The Yunohost documentation adds checkmarks to articles you already read, I love this. Not to track progress, but to quickly parse the list and find the 4 articles I keep reading.

User Guide — Certbot 1.30.0 documentation

Needed to manually get a cerificate.

`` Needed to manually get a cerificate, as opposet to ‘get and install automatically’. `

• sudo certbot certonly --manual -d *.my.domain
• It gave me a TXT record to add to DNS records
• It gave me this link to check if adding the record worked
• I could prolly copy them to yunohost or something, but it wouldn’t be managed by it

The reason I’m doing this is weird DNS configuration.

Let’s try getting around it: Certificate | Yunohost Documentation

yunohost domain cert-install your.domain.tld --self-signed --force

if the certificate installation still doesn’t work, you can disable the checks with --no-checks after the cert-install command.

Oh nice! Let’s try with non self-signed:

admin@me:~$ sudo yunohost domain cert install sub.do.main --no-checks

Works! Even if the web interface complains of DNS issues, this works as long as it’s actually accessible from outside - say, with one of the 220924-2043 Options to access a host from behind NAT and firewall or something.

Adding domains through CLI is also much faster than using the GUI:

admin@me:~$ sudo yunohost domain add my.domain.another.one

And the certificate bit accepts lists of domains. Okay!

admin@me:~$ sudo yunohost domain add b.my.doma.in && sudo yunohost domain add g.my.doma.in && sudo yunohost domain add n.my.doma.in
admin@me:~$ sudo yunohost domain cert install n.my.doma.in b.my.doma.in g.my.doma.in --no-checks

• Except that I don’t see the added domains in the web interface :(
• And no adding through the web interface doesn’t work anymore.
• BUT after I added a domain

admin@me:~$ sudo yunohost log
usage: yunohost log {list,show,display,share} ... [-h]
yunohost log: error: the following arguments are required: {list,show,display,share}

• list
• log
• display
• share

Interesting different commands doing different things!

Bitwarden-rs in now called vaultwarden.

Second time I find setting it up on Yunohost hard, so documenting.

“Create account” from main page with the yh email doesn’t work because the user allegedly exists.

1. Install it
2. You get an email with a link to the admin page to the Yunohost email
3. Open it, you’ll find the admin panel, you can invite users
4. Invite yourself
5. Check your email again
6. Find invitation there to the Vaultwarden group
7. Click it -> “create account”
8. After this, log in to your account and click ‘verify email’
9. Check email, click linkss
10. Done

Have a vodafone router and a real ASUS router that does everything better, and I connect the vodafone router to it and then use the ASUS router for everything else.

Was debugging stuff and set it to AP mode - wanted to go back, but I couldn’t access the ASUS admin panel anymore at the usual 192.168.2.1.

It had a different IP, one I could find in the Vodafone router control panel, and through that found the ASUS router admin interface.

Was diagnosing an intermittent internet failure, and for logging when it disappears - ping -D 8.8.8.8. -D prints the timestamps:

[1664029219.968932] 64 bytes from 8.8.8.8: icmp_seq=27 ttl=115 time=17.1 ms
[1664029220.971096] 64 bytes from 8.8.8.8: icmp_seq=28 ttl=115 time=18.0 ms
[1664029222.100859] 64 bytes from 8.8.8.8: icmp_seq=29 ttl=115 time=147 ms
[1664029222.973428] 64 bytes from 8.8.8.8: icmp_seq=30 ttl=115 time=19.4 ms
[1664029223.973696] 64 bytes from 8.8.8.8: icmp_seq=31 ttl=115 time=18.1 ms
[1664029224.990894] 64 bytes from 8.8.8.8: icmp_seq=32 ttl=115 time=33.9 ms
[1664029225.973556] 64 bytes from 8.8.8.8: icmp_seq=33 ttl=115 time=15.4 ms
[1664029226.978178] 64 bytes from 8.8.8.8: icmp_seq=34 ttl=115 time=18.5 ms
[1664029227.980347] 64 bytes from 8.8.8.8: icmp_seq=35 ttl=115 time=19.0 ms
[1664029228.989004] 64 bytes from 8.8.8.8: icmp_seq=36 ttl=115 time=26.4 ms
[1664029230.091472] 64 bytes from 8.8.8.8: icmp_seq=37 ttl=115 time=127 ms
[1664029230.982869] 64 bytes from 8.8.8.8: icmp_seq=38 ttl=115 time=18.3 ms

Here and later, ‘host’ is the thingy hidden behind NAT.

• Mullvad allows forwarding ports. Then the host connects to mullvad, and I connect to its forwarded ports through public mullvad IP
  • forwarded port is dynamically generated
  • the host has to connect to the same exact server with the forwarded port
• anderspitman/awesome-tunneling: List of ngrok alternatives and other ngrok-like tunneling software and services. Focus on self-hosting.
  • “For most people, I currently recommend CloudFlare Tunnel.”
• Cloudflare Tunnel · Cloudflare Zero Trust docs
  • Had an acc there
  • Moved one domain there
  • Set up cloudflared on the server
    • Really easy install
  • No luck
  • Oh wait it works!
    • Sometimes
    • May debug later
• https://boringproxy.io/
  • needs 80/445, wasn’t able to do non-standard ports on an already existing server w/ certs through dir
  • didn’t get it to work :(
• https://tunnel.pyjam.as
  • worked like a charm!
  • Quick etc., but using someone else’s domain is not what I need
• fatedier/frp: A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.
  • Worked immediately for SSH
  • Worked almost immediately for HTTP (no ‘S’, but that was the issue w/ boringproxy - acme errors)
    • Doesn’t force you to use HTTPS
  • Neat intuitive documented config and config files!
  • Really neat dashboard with statistics etc!

ssh -v localhost is a quick way to get the versions of everything.

How to see ping requests being recieved on the destination machine? - Super User:

Wireshark is too heavy duty for something so simple. Just use tcpdump -nn icmp. Add and host 1.2.3.4 if you want to limit it to packets coming from 1.2.3.4.

I religiously do .realpath() pretty much every time I get a path from user input. Naively believing it also expands ~ etc.

Once I forgot and once I entered a non-expanded path myself: ~/this/

Then was tracking it as a bug, and found this bundle of joy:

/home/sh/me/dir~/me/dir/Checkpoints/checkpoint_288

It is in fact not illegal to create a directory called ~ in Unix.

And the things that used it as-is where there, and the things that were using it after a realpath were using another directory.

OK, I resolve()-d it - still the same.

TIL Path.resolve() takes care of symlinks and ..-like components, but not ~. So it should be Path.expanduser().resolve() from now on.

jq’s to_entries allows parsing key names as values/fiels:

``s__` jq ‘to_entries’ Input {“a”: 1, “b”: 2} Output [{“key”:“a”, “value”:1}, {“key”:“b”, “value”:2}]

Goal: log everything to file, but show only part of the info on the screen. Previously: 220914-2249 Python logging change level through context manager and operator magic

My current understanding:

format = "[%(asctime)s %(name)s:%(lineno)s %(levelname)s]: %(message)s"

# Set it up, no handlers -> no default StreamHandler
# this loglevel is the one handlers will have access to!
logging.basicConfig(
	level=logging.DEBUG,
	handlers=[]
)
# Format, if we don't do this will be literally none
fmtr = logging.Formatter(fmt=format)

sh = logging.StreamHandler()
fh = logging.FileHandler("debug.log")

fh.setFormatter(fmtr)
sh.setFormatter(fmtr)

# Screen output set to whatever we want, fh to debug
sh.setLevel(loglevel)
fh.setLevel(logging.DEBUG)

# Add both handlers to root, both get propagated to logger etc.
logging.getLogger('').addHandler(sh)
logging.getLogger('').addHandler(fh)

Even though i did logger = logging.getLogger(__package__) at the very top of the file before the above bits, I can do logger.debug() etc. and it follows these settings. Nice.

Documented worse than I’d like to.

Filters allow to do things to the records (structs that make up a log message later), be it change them in place or don’t let them pass.

You can pass a function in place of a Filter, it should:

• get a logging.LogRecord
• optionally change it in place
• decide whether to let it pass
• return 0 for no, non-zero for yes

The fields of a LogRecord are the same ones we name when doing formatting: name, lineno, msg and friends.

If your Filter tries to log something in a way that it’ll get filtered through it, you get recursion.

Sample of a filter that removes specific matches and gets added to a Handler:

def filter(record: logging.LogRecord) -> int:
	"""Filters away log records containing annoying stuff."""
	blacklist_condition = (
		(
			record.name == "lib_sdk.data"
			and "not available on your" in record.msg
		)
		or (
			record.name == "lib_sdk.data"
			and record.levelno == logging.WARNING
			and "which is legacy" in record.msg
		)
		or (
			record.name == "lib_sdk.data"
			and record.levelno == logging.WARNING
			and "created but without information" in record.msg
		)
	)
	if blacklist_condition:
		return 0
	else:
		return 1

sh = logging.StreamHandler()
sh.addFilter(filter)

Much better than what I had before (220914-2249 Python logging change level through context manager and operator magic).

One can go crazy here with regexes etc. but I shan’t.

My standard logging setup is logger=logging.getLogger(__package__) in my main runner file and .getLogger(__name__) for all other files.

I wanted to temporarily change the loglevel of a specific logger of a library. Logical thing is to use a context manager, and such things exist:

• Context-dependent logger in Python
• Logging Cookbook — Python 3.10.7 documentation

I liked the second one, but what I wanted is to change the loglevel of another logger.

Usage:

# inside somelib.data...
liblogger = logging.getLogger(__name__)

logger.info("Stuff")
liblogger.info("Stuff from the lib")
with LoggingContext(
	"somelib.data",
	level=logging.ERROR
):
	# very deep inside somelib.data...
	liblogger.warning("Useless warning")

liblogger.warning("Not useless warning")
logger.info("Stuff")

Idea:

• While inside the context, the loglevel of the logger used inside the library gets set to ERROR
  • I see only ERRORs from inside the library
  • I don’t see their useless warnings that would be logger.debug()s in my world
• Other loggers are unchanged
• On end of context everything goes back to normal

Second draft with operators!

But if I’m debugging I want these useless warnings!

After doing level=logging.ERROR if logger.level != logging.DEBUG else logging.getLogger('somelib_data').level oneliners I decided that I want the context manager to be flexible.

Ended up with this:

class LoggingContext:
    """Temporarily change the loglevel of a logger based on loglevels of
    other loggers or arbitrary conditions."""

    def __init__(
        self,
        logger_name: str,
        level_true: int,
        level_false: Optional[int] = None,
        l1: Union[logging.Logger, int] = logger,
        l2: Optional[int] = None,
        comp_fn: Optional[Callable] = lambda x, y: True,
    ):
        """Temporarily change logging level of a logger, optionally dependent
        on another logger's level.

        :param logger_name: Change the level of a logger with this name
            if None, the `level` new logger level will be used
        :param callable_for_unchanged: if set, will be used to compare
            main_logger_level to comparison logger level
            and if True, will leave everything unchanged.
        :param level_true: which loglevel to set in logger if condition is True
        :param level_false: loglevel to set if condition is False
            None means "don't change anything"
        :param l1: main logger whose effective loglevel we'll use, or a loglevel
            if None the global `logger` will be used
        :param l2: loglevel to compare l1 with
            if None will compare to the loglevel `level_true`
        :param comp_fn: callable taking two params, loglevels/ints l1 and l2,
            returning a boolean. Can be a lambda function or `operators` library
            operators (eq,neq etc.)
            If None will return True, ergo setting level_true always
        """
        self.other_logger = logging.getLogger(logger_name)

        # If it's a logger, get its effective level, if int - use that
        main_level = (
            l1.getEffectiveLevel() if isinstance(l1, logging.Logger) else l1
        )

        # Compare to l2 if it's there, otherwise to level_true
        effective_comparison_level = l2 if l2 else level_true

        # If callable is True, leave everything unchanged
        comparison_result = comp_fn(main_level, effective_comparison_level)

        # If we have no level_false, interpret it as "don't change anything"
        if comparison_result:
            self.level = level_true
        else:
            # 'None' here is a magic value "don't change anything"
            self.level = level_false

        logger.debug(
            f"{logger_name=}, {l1=}, {l2=}, "
            f"{level_true=}, {level_false=}, {comp_fn=}"
        )
        logger.debug(
            f"{self.other_logger=}, {self.level=}, {main_level=}, "
            f"{effective_comparison_level=}, {comparison_result=}"
        )

        if self.level is not None:
            logger.debug(f"Changing {logger_name=} to loglevel {self.level}")
        else:
            logger.debug(f"Leaving {logger_name=} unchanged.")

    def __enter__(self):
        if self.level is None:
            return None

        self.old_level = self.other_logger.level
        self.other_logger.setLevel(self.level)

    def __exit__(self, et, ev, tb):
        if self.level is None:
            return None
        else:
            self.other_logger.setLevel(self.old_level)

This changes the idea completely and brings some VERY non-intuitive dynamics with default values, not sure yet if it’s worth doing it like that for the sake of brevity but we’ll see.

• level_true, level_false are levels to use based on condition
• l1, l2 are the two loglevels we compare
• cond_fn is a Callable/lambda/… that does the condition and returns a boolean.
• Non-intuitive dynamics and default values. If omitted:
  • level_false means “no change to status quo”
  • l1 takes the global logger, which is probably a child of the logger we care about and inherits its effective loglevel
  • l2 becomes level_true
    • For cases like “change loglevel to X only if X is more/less/equal than/to our l1”

EXAMPLE SCENARIOS

• temporarily silence useless warnings of a library’s logger ‘other’:
  with LoggingContext('other', logging.ERROR):
• temporarily change loglevel of ‘other’, only if they’ll still be visible to me afterwards (=level higher than current one):
  with LoggingContext('other', logging.INFO, comp_fn=operators.le):
• temporarily change loglevel of ‘other’ to shut it up unless we’re in debug mode, in which case I want to see everything:

  with LoggingContext('other', logging.ERROR,
   l2=logging.DEBUG, comp_fn=operators.eq):
  

• if we’re at loglevel INFO or less, change ‘other’ to WARNING, if not - otherwise change it to ERROR

  from operators import le as less_or_equal
  
  with LoggingContext('other', level_true=logging.WARNING,
  level_false=logging.ERROR,
  l1=logger.level,  # just as demo, it's implicit everywhere
  l2=logging.INFO, comp_fn=less_or_equal):`
  

Initially it was lambdas, but I kept wishing for “can I just pass <= as a function?” and lo and behold - yes, through the operator library!

Fazit

That was fun, and TIL about operators. In any case - another function for my small library of snippets.

Best of all, my favourite python blog has an article about the topic:The Unknown Features of Python’s Operator Module | Martin Heinz | Personal Website & Blog

Let’s see if I end up using this utility function more than once.

Bonus

Another similar-ish snippet I wrote once and still love. You get pretty progress bars only if you have enough elements in your sequence for it to make sense:

def _tqdm(list_like: Sequence, iflarge: bool = False, lsize: int = 100, **kwargs):
    """Use tqdm if it's on, optionally based on length of list.

    Args:
        list_like: thing to iterate.
        iflarge (bool): If on, will use tqdm only for large lists
        lsize (int): anything more than this is 'large'
        **kwargs: get passed to tqdm as-is
    """

    if USE_TQDM:
        if not iflarge:
            return tqdm(list_like, **kwargs)
        else:
            # Count size only if it doesn't mean iterating an iterator
            if isinstance(list_like, Sequence) and len(list_like) > lsize:
                return tqdm(list_like, **kwargs)

    return list_like

Then, if the global USE_TQDM is true:

• for x in _tqdm(sth) is a vanilla tqdm
• for x in _tqdm(sth, True) becomes a tqdm only if we’re iterating through something larger than 100 elements.
• _tqdm(sth, True, 50, desc="DOCS") tqdm on 50+ elements with a label (how cool is that?)

And on the same topic:

def log(msg) -> None:
    """Use loglevel.debug if tqdm is used, loglevel.info otherwise."""
    if USE_TQDM:
        logger.debug(msg)
    else:
        logger.info(msg)

logger.info() destroy tqdms, so - if we’re using TQDM, log it as logger.debug(). We’ll still see it on that loglevel if we want to (or maybe we’re logging it to a file, who knows).

TODO

• I think the RIGHT way to solve this would be a logging.Filter object. Later.
• I want a stable workflow that logs everything to a logfile but shows only a subset on screen. This means setting loglevel DEBUG, and adding a handler of loglevel INFO for stdout and a FileHandler of same DEBUG level for a file.

Python has a property function/decorator: Built-in Functions — Python 3.10.7 documentation.

Basically - you have a field and you want getter/setter functions on it.

Seen first in konfuzio_sdk, sample from there:

@property
def number_of_lines(self) -> int:
	"""Calculate the number of lines in Page."""
	return len(self.text.split('\n'))

Then you can run document.number_of_lines and it runs the function.

In .ideavimrc I added these two:

nmap <leader><leader> :action CloseContent<cr>
nmap <C-S-T> :action ReopenClosedTab<cr>

First equal to my vim settings, second equal to the usual binding for it in “normal” browsers.

From OmegaConf source:

def fail() -> None:
	raise ValueError("Input list must be a list or a tuple of strings")

if not isinstance(dotlist, (list, tuple)):
	fail()

for arg in dotlist:
	if not isinstance(arg, str):
		fail()

I don’t know if I like this or not, but it’s interesting. But I did write similar things with a parametrized fail()

Intro

My keyboard setup has always been weird, and usually glued together with multiple programs. Some time ago I decided to re-do it from scratch and this led to some BIG improvements and simplifications I’m really happy about, and want to describe here.

Context: I regularly type in English, German, Russian, Ukrainian, and write a lot of Python code. I use vim, qutebrowser, tiling WMs and my workflows are very keyboard-centric.

TL;DR: This describes my current custom keyboard layout that has:

• only two sub-layouts (latin/cyrillic)
• the Caps Lock LED indicating the active one
• Caps Lock acting both as Ctrl and Escape
• things like arrow keys, backspace accessible without moving the right hand
• Python characters moved closer to the main row

It looks like this¹:

and is available on Github.

How I got into custom keyboard layouts

First, one long summer, I switched to the Dvorak keyboard layout² and loved it.

Then I saw xkcd’s Randall Munroe’s Mirrorboard: A one-handed keyboard layout for the lazy – xkcd. The idea is that it’s easy to repeat with your left hand movements that you do with your right if they are mirrored. This works for blind typing too - if you type l with your right pinky finger, probably your left pinky finger ‘knows’ that reflex as well.

I loved the idea. My right hand usually has either a mouse or a cup of tea in it, and casual left-hand typing without needing to learn an entirely new layout sounded really interesting.

I decided to create such a mirrored layout for Dvorak.

This led me to the topic of customizing xkb keyboard layouts (the Arch wiki describes it very well).

At the end I did create a Dvorak Mirrorboard layout and used it more than expected (for example, image editing is easier if you don’t need to move your hand from the mouse).

But almost immediately I realized the potential of editing layouts and started to add things I needed, like Enter/Backspace, ümläüts and ß etc. - still mirrored, but now not a generic layout anymore. Needing a new name I decided on Pchr8board.

There were N iterations, here’s an old post about one of them: Pchr8board - a mirrored left-hand keyboard layout for Dvorak - serhii.net.

Then I kept adding stuff, in the process abandoning most left-hand features. Slowly we converged to a layout I liked.

Non-xkb weirdness

I had other non-standard changes I was really attached to:

• Since forever I have my Caps Lock key remapped to Ctrl, which I strongly recommend to literally everyone. Ctrl is used often and that position is much easier to reach, and no one needs Caps Lock. An ugly xmodmap hack on autostart remapped both keys.
• Caps-Lock-but-now-Ctrl, if released quickly, it acted as Escape. Incredibly neat, for vim especially. I used xcape³ for this.

It all worked but not flawlessly

All together the setup was a net positive, but was very brittle.

Xcape is clearly abandoned, and neither xmodmap nor xcape work in Wayland. But there are far worse problems.

You need to run it manually on startup (could never get it to run automatically in a reliable way, believe me I tried) and every time you connect a new keyboard.⁴

Sometimes it took multiple attempts to get all parts working. And every time you run xmodmap it resets your layout and you need to re-run setxkbmap.

Not a hypothetical scenario

…Which you may not be able to do, because you’re stuck with a broken layout or Caps Lock on or a Ukrainian layout and no way to change it, because you can’t open a terminal and type a command to do that.

Also all the WM keybindings relying on the former Ctrl key are broken in the process.

You can GUESS where the Alt key is now and then try to get into a tty. Then your Ctrl and Esc are not where you are used to.

Long story short - it was worth the pain, but the pain was there. The setup was band aids on top of other band aids, some applied at the very beginning when I had no idea what I was doing but hey, it works.

One keyboard layout to rule them all

Then I finally did it all from scratch and for real.

The layout in all its glory

Only the changed keys are labelled, with the exception of the default characters when they help me to identify a key.

The keys are read like this:

The key change is that the Left Alt button becomes a modifier key that makes more symbols/actions available. “Latch” is <Level3> and is located on Left Alt⁵.

For example, to get Ä you press <Shift+Latch+a>. For ä you just press <Latch+a>. A needs only Shift.

In the layout definition itself this is represented like this:

key <AC01> { [	    a,	A, adiaeresis,	Adiaeresis]	};

The Right Alt key still works like a normal Alt.

Notation

Notation that I made up⁶:

• Written in full (‘Shift’ or ‘Left Alt’) or capitalized (<LALT>) are physical keys on the keyboard. Given as:
  • keylabel / what xkb calls them inside the layout file (<RCTRL>)
  • default Dvorak value (<c> refers to the key that produces an i in QWERTY or ш in Ukrainian/Russian).
• Shift/LALT are the logical thing after all remappings and modifiers are applied
• <Shift+q> are keybindings, with modifiers given by their logical/remapped values (Shift can be located anywhere on the keyboard as long as it works as a Shift), and the letters are the usual/normal/Level1 unchanged ones (<Shift+q> produces a Q, but alone the key <q> would be a lowercase q).

So <LCTRL> would be “left physical key on the keyboard with Ctrl written on it”, <Shift-Latch-g> would be “Press whatever keys / pedals / mouse buttons that are your Shift and Latch modifiers, then the key on the keyboard that in dvorak on normal systems results in a q appearing on the screen”.

Two languages instead of four and one LED

I created two layouts, v6 the latin one with umlauts and everything else, and ruua, that contains both Russian and Ukrainian characters in the same layout.

Pressing the right Control key once changes the layout:

	key  <RCTL> {	[ISO_Next_Group]	};

Having only two layouts means you never have to guess which one comes next or set up indicators in the taskbar. You just let your muscle memory automatically do its thing.⁷

But unlocking the laptop was still a pain. You don’t know the language you were typing in when you locked it, and things like i3lock don’t tell you the layout by default - and you never know if a wrong password is a typo or a wrong layout.

The grp_led option allows you to use keyboard LEDs as indicators.

setxkbmap -option -option 'grp_led:caps' vv,ruua

Now anytime I’m typing in Russian or Ukrainian the Caps Lock LED is on, regardless of what is shown on the display. Pressing RCTRL changes the layout and makes the LED turn off, and you know it worked.

Custom modifiers defined in the layout itself

No xmodmap anymore! Caps Lock is now Ctrl, with Latch it becomes an Escape (and the former Ctrl button is a new modifier key Hyper_L, guaranteed not to collide with anything).

That took time to get right, the key was making Caps Lock a four-level key⁵, then we can define what happens to it with Level3/Latch/<LALT>:

key <CAPS> { type[Group1] = "FOUR_LEVEL", symbols[Group1] = [ Control_L, Control_L, Escape, NoSymbol] };
modifier_map Control { <CAPS> };

For more, look into modifier_map⁸ and real/virtual modifiers⁹ on the Arch Linux Wiki.

Arrow keys and Backspace easy to reach

Shown in purple, directly in the right hand resting position:

• Arrow keys (CHTN is the new WASD!).¹⁰
• <Backspace> and <Delete>!

key <AD10> { [	    l,	L, BackSpace, Delete		]	};
key <AC07> { [	    h,	H,	Left,	Left		]	};
key <AC08> { [	    t,	T,	Down,	Down   ]	};
key <AC09> { [	    n,	N,	Right,	Right		]	};

Being able to quickly delete text with my ring finger without stopping to type to reach the backspace key feels as good as it sounds.

Best thing, all this works with keyboard shortcuts! <Ctrl-Alt-R> deletes the entire previous word, etc.

Programming features

Mostly the improvements cluster in two areas:

• Move all brackets closer to resting position.
  • ALL OF THEM
• No Shift for frequent characters
  • Python and programming:+,-,=
  • Vim and vim-like things: :!
    • ; now needs a Shift, a sacrifice I’m ready to make.

Left Alt as modifier key works very well for them - it’s easier to reach for my left thumb than Shift was for any finger ever.

Some redundancy and left-hand features

There are two additional Enter keys, one on Space and the other one under Escape. Both closer than the real one, and the latter needs only the left hand. (I found I need a left-hand Enter more often than any other.)

On that same tilde key there’s a Compose key too, which allows to type some exotic characters that are used too rarely to get their own key.

There is also more than one way to do slashes, this mostly has to do with old layouts I had and still remember if I’m tired or stressed.

Installing

After you read ArchWiki’s Precautions and preparations and assuming you need both the latin and cyrillic layouts:

• Copy the source files to /usr/share/X11/xkb/symbols/. (Or maybe create a symlink to a version-controlled version of the layout, then you can do your modifications and test them more easily.)
• Name them something reasonable, the file name will be the name used by setxkbmap to refer to the layout.¹¹

For the full experience

1. Assuming the layouts are in /usr/share/X11/xkb/symbols/v6 and /usr/share/X11/xkb/symbols/ruua, run:

   setxkbmap -option -option 'grp_led:caps' v6,ruua
   

2. If it works, add that command to autostart, it should work.

Light-mode experience

If you don’t want to go all-out:

1. Run setxkbmap -option us, now you have it in your terminal history
2. setxkbmap -option -option 'grp_led:caps' v6,us would give you the new layout and on <RCTRL> you get a standard QUERTY one.
3. If something goes wrong, use the arrow keys to find the command setxkbmap -option us and press Enter to run it, and you’re back in known territory.

The layouts definitions

The sources, keyboard-layout-editor.com .json and the pictures are all available on Github. Pasting them below too for completeness and redundancy.

// My current layout, no connection to dvorak-mirrorboard anymore

default  partial alphanumeric_keys modifier_keys
xkb_symbols   "sh" {

	name[Group1] = "SH Custom layout";

	// Using L-Alt as modifier instead of Caps lock.
	key <LALT> { type[Group1] = "ONE_LEVEL", symbols[Group1] = [ ISO_Level3_Shift ] };

	//// TAB AND FRIENDS  
	// Mod+Space is return
	// TODO
	key <SPCE> { [ space, space, Return ] };

	// Bsp, Enter, **Compose Key **
	key <TLDE> {	[     BackSpace,	Multi_key,	Return,	 NoSymbol]	};

	// Tab, LTab, /, b\

	key  <TAB> {	[ Tab,	backslash, slash, NoSymbol]	};

	// Switch groups by RCTL
	key  <RCTL> {	[ISO_Next_Group]	};

	// Caps is Ctrl, ? <Escape> ?
	// Mapping Escape to Caps+Shift doesn't work for some reason
	key <CAPS> { type[Group1] = "FOUR_LEVEL", symbols[Group1] = [ Control_L, Control_L, Escape, NoSymbol] };
    modifier_map Control { <CAPS> };

	key <LCTL> { type[Group1] = "ONE_LEVEL", symbols[Group1] = [Hyper_L] };
	modifier_map Mod3 { Hyper_L };

	////

	//// FIRST ROW 
	// '"`?
	key <AD01> { [  apostrophe,	quotedbl, quoteleft, NoSymbol] };
	// ,<[?
	key <AD02> { [	comma,	less,   bracketleft, NoSymbol] };
	// .>]?
	key <AD03> { [      period,	greater, bracketright, NoSymbol] };
	////

	key <AD04> { [	    p,	P, asciitilde, NoSymbol		]	};
	key <AD05> { 
		[y,	Y, f, F], 
		[a, a, a, a] 
	};

	// Umlauts
	key <AC01> { [	    a,	A, adiaeresis,	Adiaeresis]	};
	key <AC02> { [	    o,	O, odiaeresis,	Odiaeresis]	};
	key <AC03> { [	    e,	E, ediaeresis,	Ediaeresis]	};
	key <AC04> { [	    u,	U, udiaeresis,	Udiaeresis]	};
	key <AC05> { [	    i,	I, d, D		]	};

	key <AB01> { [   colon,	semicolon,z, Z] };
	key <AB02> { [	    q,	Q, v, V		]	};
	key <AB03> { [	    j,	J, w, W		]	};
	key <AB04> { [	    k,	K, m, M		]	};
	key <AB05> { [	    x,	X, b, B		]	};

	key <AE01> {	[	  1,	exclam,		NoSymbol,	NoSymbol	]	};

	// 2@<{
	key <AE02> {	[	  2,	at,		less,	NoSymbol	]	};
	// 3#>}
	key <AE03> {	[	  3,	numbersign,	greater,	NoSymbol	]	};
	key <AE04> {	[	  4,	dollar,		EuroSign,	NoSymbol	]	};
	key <AE05> {	[	  5,	percent,	NoSymbol,	NoSymbol	]	};

	//// Backspace, arrow keys, ...
	// TODO 
	// key <AD07> { [	    g,	G, Prior, NoSymbol		]	};
	key <AD07> { [	    g,	G, parenleft, braceleft		]	};
	key <AD08> { [	    c,	C,	Up,	 Up	]	};
	key <AD09> { [	    r,	R,	parenright,	braceright		]	};
	// key <AD09> { [	    r,	R,	Next,	Next		]	};
	key <AD10> { [	    l,	L, BackSpace, Delete		]	};
	key <AC07> { [	    h,	H,	Left,	Left		]	};
	key <AC08> { [	    t,	T,	Down,	Down   ]	};
	key <AC09> { [	    n,	N,	Right,	Right		]	};

	key <AD06> { [	    f,	F  		]	};
	// Slash and Backslash
	key <AD11> { [	slash,	question, backslash, NoSymbol	]	};
	key <AD12> { [	equal,	plus		]	};


	// TODO
	key <AC06> { [	    d,	D, NoSymbol, NoSymbol		]	};
    key <AC10> { [	    s,	S,	ssharp,	ssharp		]	};
	key <AC11> { [	minus,	underscore	]	};

	key <AB06> { [	    b,	B		]	};
	key <AB07> { [	    m,	M		]	};
	key <AB08> { [	    w,	W		]	};
	key <AB09> { [	    v,	V		]	};
	key <AB10> { [	    z,	Z		]	};

	// +|\? - the key that by default has only backslash+bar
	
	key <BKSL> { [  plus,  bar, backslash, NoSymbol             ]       };


	key <AE06> {	[	  6,	asciicircum	]	};
	key <AE07> {	[	  7,	ampersand	]	};
	key <AE08> {	[	  8,	asterisk	]	};
	key <AE09> {	[	  9,	parenleft	]	};
	key <AE10> {	[	  0,	parenright	]	};
	key <AE11> {	[     bracketleft,	braceleft	]	};
	key <AE12> {	[     bracketright,	braceright		]	};

};

// Keyboard layouts for Russia.
// AEN <aen@logic.ru>
// 2001/12/23 by Leon Kanter <leon@blackcatlinux.com>
// 2005/12/09 Valery Inozemtsev <shrek@altlinux.ru>
// 2018/07/15 @a13 (a.k.a. @dbvvmpg) and Stepanenko Andrey <ftvkyo2011@yandex.ru>
// 2021 - Adapted to contain Ukrainian characters - serhii.net

// Windows layout
default  partial alphanumeric_keys
xkb_symbols "winkeys" {

    include "ruua(ruua)"
    name[Group1]= "Russian";

    key <AE03> { [           3,  numerosign  ] };
    key <AE04> { [           4,   semicolon  ] };
    key <AE05> { [           5,     percent  ] };
    key <AE06> { [           6,       colon  ] };
    key <AE07> { [           7,    question  ] };
    key <AE08> { [           8,    asterisk, U20BD  ] };

    key <AB10> { [      period,       comma  ] };

    // SH -- now adding the bksp and stuff and removing the Enter thing.
 
	key <SPCE> { [ space] };
	// Mod+Tab gives a slash, which I use often (searching etc.) 
	// Mod+Shift+Tab gives an umlaut on the next character
};

hidden partial alphanumeric_keys
xkb_symbols "ruua" {

    key <AE01> { [           1,      exclam  ] };
    key <AE02> { [           2,    quotedbl  ] };
    key <AE03> { [           3,  numbersign  ] };
    key <AE04> { [           4,    asterisk  ] };
    key <AE05> { [           5,       colon  ] };
    key <AE06> { [           6,       comma  ] };
    key <AE07> { [           7,      period  ] };
    key <AE08> { [           8,   semicolon  ] };
    key <AE09> { [           9,   parenleft  ] };
    key <AE10> { [           0,  parenright  ] };
    key <AE11> { [       minus,  underscore  ] };
    key <AE12> { [       equal,        plus  ] };
    key <BKSL> { [   slash,         backslash  ] };

    key <AB10> { [       slash,    question  ] };
    key <LSGT> { [       slash,         bar  ] };

    key <TLDE> { [       Cyrillic_io,	apostrophe,	U02BC,       Cyrillic_IO  ] };
    key <AD01> { [   Cyrillic_shorti,   Cyrillic_SHORTI  ] };
    key <AD02> { [      Cyrillic_tse,      Cyrillic_TSE  ] };
    key <AD03> { [        Cyrillic_u,        Cyrillic_U  ] };
    key <AD04> { [       Cyrillic_ka,       Cyrillic_KA  ] };
    key <AD05> { [       Cyrillic_ie,       Cyrillic_IE] };
    key <AD06> { [       Cyrillic_en,       Cyrillic_EN  ] };
    key <AD07> { [      Cyrillic_ghe,      Cyrillic_GHE  ] };
    key <AD08> { [      Cyrillic_sha,      Cyrillic_SHA  ] };
    key <AD09> { [    Cyrillic_shcha,    Cyrillic_SHCHA  ] };
    key <AD10> { [       Cyrillic_ze,       Cyrillic_ZE  ] };
    key <AD11> { [       Cyrillic_ha,       Cyrillic_HA  ] };
    key <AD12> { [ Cyrillic_hardsign,	Cyrillic_HARDSIGN,	Ukrainian_yi,	Ukrainian_YI] };

    key <AC01> { [       Cyrillic_ef,       Cyrillic_EF  ] };
    key <AC02> { [     Cyrillic_yeru,     Cyrillic_YERU,	Ukrainian_i,	Ukrainian_I] };
    key <AC03> { [       Cyrillic_ve,       Cyrillic_VE  ] };
    key <AC04> { [        Cyrillic_a,        Cyrillic_A  ] };
    key <AC05> { [       Cyrillic_pe,       Cyrillic_PE  ] };
    key <AC06> { [       Cyrillic_er,       Cyrillic_ER  ] };
    key <AC07> { [        Cyrillic_o,        Cyrillic_O  ] };
    key <AC08> { [       Cyrillic_el,       Cyrillic_EL  ] };
    key <AC09> { [       Cyrillic_de,       Cyrillic_DE  ] };
    key <AC10> { [      Cyrillic_zhe,      Cyrillic_ZHE  ] };
    key <AC11> { [        Cyrillic_e,        Cyrillic_E,	Ukrainian_ie,	Ukrainian_IE] };

    key <AB01> { [       Cyrillic_ya,       Cyrillic_YA  ] };
    key <AB02> { [      Cyrillic_che,      Cyrillic_CHE  ] };
    key <AB03> { [       Cyrillic_es,       Cyrillic_ES  ] };
    key <AB04> { [       Cyrillic_em,       Cyrillic_EM  ] };
    key <AB05> { [        Cyrillic_i,        Cyrillic_I] };
    key <AB06> { [       Cyrillic_te,       Cyrillic_TE  ] };
    key <AB07> { [ Cyrillic_softsign, Cyrillic_SOFTSIGN  ] };
    key <AB08> { [       Cyrillic_be,       Cyrillic_BE  ] };
    key <AB09> { [       Cyrillic_yu,       Cyrillic_YU  ] };

    include "kpdl(comma)"
};

Parting thoughts

Custom keyboard layouts for the win

Tweaking to your purposes something as fundamental as a keyboard layout is strangely empowering. And adapting to a new layout is like learning a foreign language - if you did it at least once in your life, the next ones are much easier. Especially if it’s small things like moving a key, or just adding more symbols to the existing layout.

Wouldn’t recommend it to everyone, though.

One thing I would recommend to everyone without exception is switching the Ctrl and Caps Lock keys. It can be easily done on any OS, including Linux, where too it can be done without editing any layout files¹².

Interesting resources on topic

• X keyboard extension - ArchWiki (of course)
• Custom Keyboard in Linux/X11 | Daniel Paul O’Donnell (“As an early medieval English philologist, I make daily use of a number of characters that are not found on the standard US computer keyboard. These include characters and accented letters from various European languages (e.g. ß, é, ä, ø), IPA symbols (e.g. ŋ, ә, etc.), Middle English ȝ, and various letters and forms used in Old English (æ, þ, ċ, ġ, ƿ, ū, ȳ, etc.).") - dated but very cool
• tonyaldon/keyboard-layout: keyboard-layout pools all the needed files to set up my custom XKB keyboard layout (takbl) on Linux Ubuntu.
• A simple, humble but comprehensive guide to XKB for linux | by Damiano Venturin | Medium

Thank you for reading!