SH | Diensttagebuch

Pycharm presentation mode and font size

Previously, I had to manually increase font sizes in Pycharm when presenting stuff in meeting, and couldn’t automate it.

Today I realized that I can change the script resolution to a lower one, giving the same results, and easily automatable through randr and a shell script!

Pycharm moving functions

“Right click -> Refactor” works not just for renaming files/folders, but also for moving functions to different files!

Miroboard moving

Holding <space> makes the mouse move the view, not the content

Logging in Python

logging — Logging facility for Python — Python 3.9.7 documentation

logger.exception() exists! Exception info is written as given by the exception handler.

Exceptions handling in Python

Was looking for a strategy to handle errors in a complex-ish applications, with logging, different levels etc.

• Three options how to deal with exceptions:¹

  • Swallow it quietly (handle it and keep running).
  • Do something like logging, but re-raise the same exception to let higher levels handle.
  • Raise a different exception instead of the original.

• Defining your custom exception¹

  class SuperError(Exception):
      def __init__(self, message):
          Exception.__init__(message)
          self.when = datetime.now()
  
  raise SuperError('Something went wrong')
  

• Re-raising the same exception after handling it ¹

  def invoke_function(func, *args, **kwargs):
      try:
          return func(*args, **kwargs)
      except Exception as e:
          print type(e)
          raise
  

• Ways to clean stuff up in try..catch blocks:²

  • try: - execute this
  • except: execute this if there’s an exception
  • else: - execute if no exceptions
  • finally: - always run this code

• Context managers

  • Alternative to finally, standard with ... syntax

• Logging best practice¹

  import logging
  logger = logging.getLogger()
  def f():
      try:
          flaky_func()
      except Exception:
          logger.exception()
          raise
  

  If you re-raise, make you sure you don’t log the same exception over and over again at different levels.¹

  The simplest way to do it is to let all exceptions propagate (unless they can be handled confidently and swallowed earlier) and then do the logging close to the top level of your application/system.¹

• Error logger decorator for the above¹

  def log_error(logger)
      def decorated(f):
          @functools.wraps(f)
          def wrapped(*args, **kwargs):
              try:
                  return f(*args, **kwargs)
              except Exception as e:
                  if logger:
                      logger.exception(e)
                  raise
          return wrapped
      return decorated
  

  And usage:

  import logging
  logger = logging.getLogger()
  
  @log_error(logger)
  def f():
      raise Exception('I am exceptional')
  

• If there are multiple decorators, that one should be the immediate next one to the function! When I did it wrong, I got an exception (ha) about “‘staticmethod’ object is not callable”.

  The correct way is:

  @staticmethod
  @return_trainer_exception(logger=None)
  

Cherry-picking commits from pycharm

Messed up merging/rebasing branches from branches from branches, but needed to merge literally a couple of commits.

So I created a clean branch from master. Then:

• Check out the target branch, the one you’re cherry-picking to
• Open the git log
• Select the commits you want to cherry-pick, right click, “cherry-pick”
• Done!

As usual, docs exist¹ and are quite readable.

PEP8 max line width of 80 characters

… is the best thing since sliced bread, I was skeptical at first but makes editing code in multiple windows so much better!

Installing python3.8 on Ubuntu 18.04 LTS

• Needed a third-party PPA² that has all the newer python versions:

    sudo add-apt-repository ppa:deadsnakes/ppa
    sudo apt install python3.8
    sudo apt install python3.8-dev
  

• Needed sudo apt-get install python3.8-venv³
• Needing to reinstall all packages for it, haha.
  • Set up locally a venv38 for this; if I source venv38/bin/activate python3 becomes python3.8 by default.
• Needed to install

python3.8-dev was added after an error I had⁴ when installing pycocotools, it didn’t find python.h when building.

Installing python locally

This describes the process well: Install python3.6+ for local user on remote machine without root access - ~/web/logs

The official documentation: 2. Using Python on Unix platforms — Python 3.9.7 documentation

Basically make altinstall is a safer version that doesn’t overwrite system-wide stuff:

make install can overwrite or masquerade the python3 binary. make altinstall is therefore recommended instead of make install since it only installs exec_prefix/bin/pythonversion.

TL;DR:

• Download the source tarball
• Then:

  ./configure --prefix=whatever
  make
  make altinstall
  

• Add the prefix to $PATH:

  export PATH=$PATH:/data/sh/tools/python3.8/bin
  

Hugo auto-reload and CSS

Just now remembered that when doing CSS stuff it’s sometimes cached, and one needs to <Shift-R> or sth similar. Hugo’s automatic reloading reloads the content/templates/…, but not the CSS!

Explains a lot of what happened the last two days.

Hugo Templating

Copypasting from the docu⁵:

• Parameters for functions are separated using spaces
• Dot-notations for methods and fields ({{ .Params.bar }})
• Things can be grouped via parentheses:
  • {{ if or (isset .Params "alt") (isset .Params "caption") }} Caption {{ end }}
• A Single Statement Can be Split over Multiple Lines:

  {{ if or 
    (isset .Params "alt") 
    (isset .Params "caption")
  }}
  

Setting directory-specific settings in vim

Given that Hugo’s markdown considers code as part of a bullet-point if it’s indented two spaces more than the *-bulletpoint’s level, and that I have a tabwidth of 4 and tabs everywhere else and two spaces were a pain…

To apply settings only within a specific directory, add this to ~/.vimrc⁶:

autocmd BufNewFile,BufRead /home/me/ntb/* set tabstop=4 softtabstop=4 shiftwidth=4 expandtab foldmethod=marker

Notably, for me it didn’t work when the path contained a symlink, had to write it explicitly.

Another option from that SO questiont was placing a ~/.vimrc in that directory⁷, allowing vim to use it by default, and sourcing the usual global one from the first line. Has security implications, may lead to issues with paths/plugins, didn’t try it.

vim tabs and spaces and indentation settings

Looking for indentation stuff for the above lead me here: Tab settings in Vim. Summary: | by Ari Sweedler | Medium

It has this description, copying verbatim:

• tabstop: display-only, how many spaces does one \t equal visually?
• shiftwidth: how many spaces does one level of indentation equal? (shifting commands, formatting, behaviour).
• softtabstop: how much whitespace to add/remove when pressing tab/backspace?
  • Disabled by default; if using tabs - we create as much whitespace as needed to get to the next tabstop
  • but when using spaces for indentation, we don’t want backspace to delete one space, then this is needed
• expandtab: should pressing <Tab> on the keyboard create spaces or a tab character?

highlight indentation levels in vim, if indentation is done with spaces

Highlighting tab-indents is easy, and I had these settings for that:

set listchars=tab:\:\ 
set listchars+=trail:◦

For spaces it’s harder.

Tried the indentLine plugin⁸, due to it using the conceal setting I couldn’t see my json-quotes and _ underscores anymore. Setting conceallevel to 1 from 2 helped only for the latter. May get fixed by colorscheme/syntax files with less concealed stuff?

Setting let g:indentLine_concealcursor = '' (by default inc) helps - text is not concealed at all in the cursor line in any of the modes. I see all concealed text and don’t see the guides. I can kinda live with that.

In any case replacing the 's in json is ugly.

Then found this excellent SO answer. set cursorcolumn cursorline highlight the entire column/row where the cursor is. Which is why I want indentation highlighting 99% of the time!

With my newfound vim knowledge, added this to ~/.vimrc:

autocmd filetype python set cursorcolumn cursorline

But this didn’t satisfy me for the dtb and I kept looking.

Then I found vim-indent-guides⁹ that changes just the background color. Settings I ended up using:

let g:indent_guides_enable_on_vim_startup = 1
let g:indent_guides_auto_colors = 0
let g:indent_guides_start_level = 2
let g:indent_guides_guide_size = 4
" autocmd VimEnter,Colorscheme * :hi IndentGuidesOdd  guibg=darkgrey  ctermbg=233
autocmd VimEnter,Colorscheme * :hi IndentGuidesEven guibg=blue ctermbg=233

ctermbg=233is one of the darkest black-like vim colors, there’s a nice vim colors reference¹⁰ online.

At the end, wrapped everything related to DTB and indentst in one nice startup function:

fun! SetDTB()
	set tabstop=4  shiftwidth=2 expandtab 
	foldmethod=marker
	set nocursorline nocursorcolumn 
	let g:indent_guides_auto_colors = 0
	let g:indent_guides_start_level = 1
	let g:indent_guides_guide_size = 1
	autocmd VimEnter,Colorscheme * :hi IndentGuidesEven guibg=blue ctermbg=236
endfu

autocmd BufNewFile,BufRead /home/me/ntb/* :call SetDTB()

1001th post in Hugo!

Set up Hugo for DTB and partly sth I’ll call NTB, which is non-work stuff.

So far Hugo is 110/10.

Old one for now is here.

Jekyll to Hugo

TODO:

• Aliases/redirects from old posts to new ones (serhii.net/day123.html -> serhii.net/day123)
  • uglyurls: true in config does exactly this!
  • …but breaks lists/indexes somehow :(
• Look through master file for formatting issues
• Better black-background syntax highlighting if no language specified
  • Ideally make them indistinguishable from styled ones
  • And remove ghost ones like day 996
    • The problem was with my markdown syntax, apparently *I need a two space indentation from the * for it to be parsed correctly. Another reason to revisit my vim tab settings?
  • using '''text seems like a workaround:

    This is text
    No syntax highlighting
    

    And:

    This is text
    No syntax highlighting _at all_
    

• Randomized footers
• Set up Atom feed on home page
  • Or actually - I could move the entire website to Hugo, and have the index-index as a template and /dtb for the posts stuff?
• Strikethrough
  • Markdown strikethrough is ~~strikethrough~~ ¹
• Fix code listings' width influencing the width of entire Day.

tree including hidden files

I love how intuitive it is - needed a dotfile in tree, tried tree -a, it worked.

Python unittest

setUp() and tearDown() methods in unittests get executed before/after each test method!

Unregistering Detectron2 datasets for unittests

The dictionary with the datasets is a global dictionary, which means that you can’t register_coco_instances() in separate unittests in the same file!

This worked:

if Constants.TRAIN_DATASET_NAME in MetadataCatalog.data:
    MetadataCatalog.remove(Constants.TRAIN_DATASET_NAME)
    MetadataCatalog.remove(Constants.TEST_DATASET_NAME)
    DatasetCatalog.remove(Constants.TRAIN_DATASET_NAME)
    DatasetCatalog.remove(Constants.TEST_DATASET_NAME)

Pycharm / Intellij idea visual guides for character limit

Through IDE settings one can configure whether one or multiple visual guides are shown, and the actual number of characters is configured through Settings -> Code Style.

Random

Jupyter notebooks + RISE + Reveal.js + a makefile: cornell-cs5785-2021-applied-ml/Makefile at main · kuleshov/cornell-cs5785-2021-applied-ml

TODO - Git - squashing multiple commits into one

Squash commits into one with Git - Internal Pointers (link by SO):

# Merge the last 7 commits into one
git rebase --interactive HEAD~[7]
# Merge the commits from that commit hash
git rebase --interactive 6394dc

In the latest one, the commit hash is “the hash of the commit just before the first one you want to rewrite from.”

Practically, assuming I want to squash together the a ones, I’d do git rebase --interactive B as it’s the one immediately following the ones I need.

commit a1 (latest/newest)
commit a2
commit a3
commit B
commit C

When actually doing the squashing, set squash in front of the commit lines to squash. In the next screen, leave only the commit message(s) needed.

I love how it uses vim for this! Very interesting way to do an interface.

Python PEP8 / black / flake8 / style

flake8 file.py shows issues;

black file.py applies black. black --line-length=79 file.py applies the line length as per PEP8.

Pycharm uses 119 characters as limit, coming from Intellij I think; officially PEP8 recommends 79.

German / Words

Blau sein = be drunk (heard at work)

Hugo the static site generator

My blog takes minutes to be generated, this DTB is not far from it either. I heard Hugo is fast, and I dislike most of what’s on my blog, the logical thing seems to burn it to the ground and start from zero using Hugo.

• Quick Start | Hugo
• Single posts are markdown too, with very similar font matter!
• A theme I like: Ink | Hugo Themes
• Installing themes:

cd themes
git submodule add https://github.com/chollinger93/ink-free
cd ..
echo theme = \"ink-free\" >> config.toml

• Creating a post:

hugo new posts/my-first-post.md

puts the file in ./content/posts/my-first-post.md

• Building:
  • Starting the local server: hugo server -D
  • REALLY fast, and reloaded live in my browser!
  • Building the site: hugo -D
• Configs
  • config.toml supports #comments
  • con-fig and con-tent in the same folder make my tab completion sad.
  • Configure Hugo | Hugo
  • It supports .yaml, .json and .toml configs and config directories!
• Directory structure: Directory Structure | Hugo
  • Site structure in inferred by directories: Content Sections | Hugo
    • They still need to be added to config to be in the menu
    • Nevertheless accessible when typing the URL directly
    • A subdirectory is a navigable section only if it has an _index.md
  • hugo new content/pages/one/two/test-page.md
    • The command only from parent of config
    • It generates the boilerplate, I don’t need to write a script for that! It even gets the title from the filename!
    • If there’s an archetype with the same name it’ll use that!
• Writing / content creation
  • This seems to describe everthing: Content Management | Hugo

Python typing cheatsheet & other stuff

Nice cheatsheet, not mypy-specific: Type hints cheat sheet (Python 3) — Mypy 0.910 documentation

Otherwise:

• Functions that may return None:
  • x = 1 if True else None, x would be Optional[int]
• Iterables / Sequences:
  • Iterable is anything usable inside a for
  • Sequence is anything that supports len()
  • For example:

    def f(ints: Iterable[int]) -> List[str]:
    return [str(x) for x in ints]
    

python unittests through CLI

From docu ¹:

python -m unittest test_module1 test_module2
python -m unittest test_module.TestClass
python -m unittest test_module.TestClass.test_method

When I’m in the directory with the test_xy.py files, running python3 -m unittest runs all of them. I can also do python3 -m unittest test_xy for that file, and python3 -m unittest test_xy.TestXY.test_specific_thing.

Debugging python from CLI through breakpoints

Found this, and it’s freaking awesome: Debugging by starting a REPL at a breakpoint is fun

Sample from there:

def make_request():
    result = requests.get("https://google.com")
    import ipdb; ipdb.set_trace()

There’s the default pdb, there’s ipdb that has to be installed.

Adding

import ipdb; ipdb.set_trace()

anywhere in the code launches a typical debug window that can be used to look into the vars etc.

Just used this for the first time to debug a python program that was running on a remote server and failing, but not locally.

SO much better than print(f"XXX {i}") and friends!

Nice tutorial about its usage: Better Python Debugging With IPDB

• n - next line in current method (=“step over”)
• s - next line of executable code anywhere (=“step into”)
• c - continue till next breakpoint
• r - continue till function returns (would be nice to learn how to do this in pycharm btw!)
• a - args - print arguments current function received
• b - adds breakpoint to locations
  • b filename.py:234
  • b <function>
  • b 123 - line in current file

Full documentation here: 26.2. pdb — The Python Debugger — Python 2.7.18 documentation

Python serializing Enums by declaring them as subclass of str

My main issue with Enum classes was that serialization is weird, especially if you’re dumping parameters. Tried again, found this: python - Serialising an Enum member to JSON - Stack Overflow

TL;DR class EnumDerivedClass(str, Enum)

import json
from enum import Enum

class LogLevel(str, Enum):
    DEBUG = 'DEBUG'
    INFO = 'INFO'

print(LogLevel.DEBUG)
print(json.dumps(LogLevel.DEBUG))
print(json.loads('"DEBUG"'))
print(LogLevel('DEBUG'))

will output

LogLevel.DEBUG
"DEBUG"
DEBUG
LogLevel.DEBUG

Google Presentations work in progress slides

“Folie überspringen” is a much better way to do what I did with setting a yellow background color - easy to see and worst case scenario it’ll just get skipped

Tensorboard and no data because wrong input folder

If you run tensorboard on a non-existing folder, you’ll get no feedback about it anywhere?.. No data on Tensorboard itself, nothing useful in CLI.

Pycharm / Intellij idea local history - for files and directories!

After some ill-fated undoing of commits, couldn’t find the work of an hour or so.

Guess what: Using Local History to Restore Code Fragments or Deleted Files | The IntelliJ IDEA Blog

I knew about local history for a file, but you can do the same for a directory, through its right-click menu in the Projects view!

Nvidia GPU/eGPU drivers blues

I already thought I had set up nvidia-smi and friends (Day 850 | Diensttagebuch (Work journal)), then didn’t use it for months, now when I tried it didn’t work anymore, nvidia-smi said “No devices found”

boltctl showed the device as connected and authorized, prime-select said nvidia was selected, modprobe showed that the correct drivers were used and dkms status had said the correct drivers were installed.

(11:53:23/10181)~/$ dkms status
nvidia, 460.73.01, 5.4.0-73-generic, x86_64: installed
nvidia, 460.73.01, 5.4.0-74-generic, x86_64: installed

(11:53:49/10182)~/$ boltctl
[snip]
 ● Lenovo ThinkPad Thunderbolt 3 Dock #2
   ├─ type:          peripheral
   ├─ name:          ThinkPad Thunderbolt 3 Dock
   ├─ vendor:        Lenovo
   ├─ uuid:          xxx
   ├─ status:        authorized
   │  ├─ domain:     domain0
   │  └─ authflags:  none
   ├─ authorized:    Mo 20 Sep 2021 09:41:16 UTC
   ├─ connected:     Mo 20 Sep 2021 09:41:16 UTC
   └─ stored:        no

 ● GIGABYTE GV-N1070IXEB-8GD
   ├─ type:          peripheral
   ├─ name:          GV-N1070IXEB-8GD
   ├─ vendor:        GIGABYTE
   ├─ uuid:          xxx
   ├─ status:        authorized
   │  ├─ domain:     domain0
   │  └─ authflags:  none
   ├─ authorized:    Mo 20 Sep 2021 09:42:35 UTC
   ├─ connected:     Mo 20 Sep 2021 09:42:35 UTC
   └─ stored:        Mo 20 Sep 2021 09:31:09 UTC
      ├─ policy:     manual
      └─ key:        no

(11:54:54/10188)~/$ lsmod
Module                  Size  Used by
nvidia_uvm           1015808  0
nvidia_drm             57344  1
nvidia_modeset       1228800  1 nvidia_drm
nvidia              34123776  17 nvidia_uvm,nvidia_modeset

(11:55:54/10192)~/$ sudo prime-select query
nvidia

What didn’t work:

• prime-select cycling to default and then back to nvidia and rebooting
• power-cycling the CPU
• Connecting it directly, not through the dock, exact same setup I had in when it was working (link above)

What worked:

• Honestly no idea
• logging into gnome, opening the driver config window, logging back into i3, rebooting?…

Offtopic, when I was googling these issues I found my own serhii.net link above on the first page of Google for the key '“nvidia-smi “no devices were found” authorized', which is both nice and sad at the same time :)

EDIT: the next morning it didn’t work again. None of the same magic steps in all possible orders. I think it might be an issue with the eGPU or dock or something of that level. The best way to check this would be to do the nuclear option, uninstall all drivers, and install from the beginning, but I think my monthly quota of GPU stuff is full five times over now.

Diensttagebuch / Meta

We’re on day 993 (!) of Diensttagebuch! Freaking awesome.

python pip “advanced” requirements.txt creation

Was creating a requirements.txt for detectron2, official install instructions were:

python -m pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.9/index.html

Answer specificalyl about this: python - How to format requirements.txt when package source is from specific websites? - Stack Overflow:

requirements.txt format is:

[[--option]...]
<requirement specifier> [; markers] [[--option]...]
<archive url/path>
[-e] <local project path>
[-e] <vcs project url>

<requirements specifier> is:

SomeProject
SomeProject == 1.3
SomeProject >=1.2,<2.0
SomeProject[foo, bar]
SomeProject~=1.4.2

The –option (such as the -f/–find-links) is the same as the pip install options you would use if you were doing pip install from the command line.

Therefore, in requirements.txt it ended up literally as this:

--find-links https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.9/index.html detectron2

And by the way, detectron2’s own requirements.txt demonstrates nicely part of the above.

My own requirements.txt for CUDA 11.1:

opencv-python==4.2.0.32

# torch 1.9 for cuda 10.2 (for this config https://pytorch.org/get-started/locally/ has no versions in the command
# getting both exact versions from pip freeze
-f https://download.pytorch.org/whl/torch_stable.html
torch==1.9.0+cu111
torchvision==0.10.0+cu111
#torch==1.7.1
#torchvision==0.8.2

# python -m pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.9/index.html
-f https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.9/index.html
detectron2

grep/ag

Best part about ag is that I don’t need to escape anything with its default settings:

pip freeze | ag "(detectron|torch)"
pip freeze | grep "\(detectron\|torch\)"

pycharm test “teamcity” output bug

Suddenly stopped getting readable output. Fix is to add the env variable JB_DISABLE_BUFFERING, without any value, to the env of the test. teamcity - no output in console for unittests in pycharm 2017 - Stack Overflow

Detectron2 parameters train/eval/checkpoint etc

The documentation about default confg covers all the parameters' meanings and can be used as reference for that! detectron2.config — detectron2 0.5 documentation

And me dreaming up cfg.MODEL.CHECKPOINT_PERIOD is exactly what they wanted to avoid by disallowing the creation of new keys.

Highlights:

# Number of images per batch across all machines. This is also the number
# of training images per step (i.e. per iteration). 
_C.SOLVER.IMS_PER_BATCH = 16

Phone disk space & Telegram cache

For the second time, discovered that Telegram Cache takes 40gb of disk space.

In the phone’s own menus related to disk space, this was shown as “Pictures” taking 40gb, not the Telegram app and its cache. But this info is exposed through Telegram’s own menus.

timewarrior track and :fill

Who knew you could combine commands! This is how you start tracking tag1,tag2 starting from the end of the previous span:

$ w track :fill tag1,tag2

Backfilled to 2021-09-15T12:21:41
Tracking "tag1,tag2"
  Started 2021-09-15T12:21:41
  Current               23:47
  Total               0:02:06

Running DUC with sshfs (excluding files and filesystems)

TL;DR:

duc index ~/ --fs-exclude fuse.sshfs

duc is about disk space, before running it the index should be built/updated. Usually similar to duc index ~/.

If I have a sshfs mounted somewhere, the process never ends as it tries to index the folder where it’s mounted.

Found some solutions:

• To exclude entire filesystems, duc index ~/ --fs-exclude fuse.sshfs
  • According to the man page, this would be a comma-separated list of filesystems as found in fstab, like ext3,ext4.
  • My /etc/fstab didn’t have the sshfs filesystem, but mount called it fuse.sshfs and this worked!
• To exclude individual files, duc index ~/ -e "*somefilename*"
  • doesn’t seem to work with folders in all variations I could think of (*folder\/file* etc).
  • So no way to exclude a folder? Except using its name and praying no other folders share it

Bonus: -p shows progress during indexing.

Now I have a new alias in ~/.zshrc:

ducindex() {
	duc index "$1" -p --fs-exclude fuse.sshfs 
}

cdd CLI alias for CD-ing to directory containing a file

I copypaste a lot the locations of the files from pycharm/intellij to run them from CLI or something similar. Easiest way, because they are focused and I don’t need to focus on the files/project view for that. I can’t find an Action in pycharm/intellij to copypaste only the directory.

Yet another alias for today:

cdd() {
	$(dirname "$1")
}

dirname gives the directory, dirname .. | cd and dirname ... | xargs cd don’t work (TODO - why?), so I’m using the zsh thing about “cd to the directory if it’s in a command by itself”.

Now cdd /home/me/wahtever/test.py takes me to /home/me/whatever/ which will saved tens of seconds per year!

Concatenating/splitting tiffs

Of course tiffsplit¹ has a sister tiffcp! Ubuntu Manpage: tiffcp - copy (and possibly convert) a TIFF file

Concatenate N pages into a result.tif:

tiffcp xaaa.tif xaab.tif xabr.tif result.tif

pycharm highlights comments if they’re immediately below a TODO one and indented

# TODO - this is highlighted yellow
# And this is not

# ... BUT!

# TODO - this is highlighted yellow
#  This too, because it's indented one space and logically belongs to the comment above!

Random / vim / TODO

I often do <CR>ddkkp or d$kp as a reverse-enter, moving what’s to the right of the cursor on the line above the current one. I’m sure something like this already exists in vim.

Detectron2 and Yacs / YAML config / CfgNode; allow adding new keys

Detectron’s Yacs has a github repo with documentation and examples, much better than detectron’s own: rbgirshick/yacs: YACS – Yet Another Configuration System

This works:

comp_cfg.set_new_allowed(True)
comp_cfg['key'] = 'value'

Interesting bit about why it’s not like this by default:

We typically don’t use this so that typo in config file can be noticed. ²

Additionally, this is set per leaf, not per config - you can allow adding stuff to the root but not to its existing children.

And, still, even with comp_cfg.set_new_allowed(True), why can’t I merge_from_list etc. for non-existing keys? (TODO)

Detectron’s logger and log.txt

log.txt is nice and colorful on CLI, I don’t remember how to let vim interpret the CLI colors but less log.txt works magnificently.

cfg.merge_from_file() doesn’t work with new keys · Issue #2082 · facebookresearch/detectron2