In the middle of the desert you can say anything you want
flake8 file.py shows issues;
black file.py applies black.
black --line-length=79 file.py applies the line length as per PEP8.
Pycharm uses 119 characters as limit, coming from Intellij I think; officially PEP8 recommends 79.
Blau sein = be drunk (heard at work)
My blog takes minutes to be generated, this DTB is not far from it either. I heard Hugo is fast, and I dislike most of what’s on my blog, the logical thing seems to burn it to the ground and start from zero using Hugo.
cd themes
git submodule add https://github.com/chollinger93/ink-free
cd ..
echo theme = \"ink-free\" >> config.toml
hugo new posts/my-first-post.md
puts the file in ./content/posts/my-first-post.md
hugo server -Dhugo -Dconfig.toml supports #comments_index.mdhugo new content/pages/one/two/test-page.md
Nice cheatsheet, not mypy-specific: Type hints cheat sheet (Python 3) — Mypy 0.910 documentation
Otherwise:
x = 1 if True else None, x would be Optional[int]forlen()def f(ints: Iterable[int]) -> List[str]:
return [str(x) for x in ints]
From docu 1:
python -m unittest test_module1 test_module2
python -m unittest test_module.TestClass
python -m unittest test_module.TestClass.test_method
When I’m in the directory with the test_xy.py files, running python3 -m unittest runs all of them. I can also do python3 -m unittest test_xy for that file, and python3 -m unittest test_xy.TestXY.test_specific_thing.
Found this, and it’s freaking awesome: Debugging by starting a REPL at a breakpoint is fun
Sample from there:
def make_request():
result = requests.get("https://google.com")
import ipdb; ipdb.set_trace()
There’s the default pdb, there’s ipdb that has to be installed.
Adding
import ipdb; ipdb.set_trace()
anywhere in the code launches a typical debug window that can be used to look into the vars etc.
Just used this for the first time to debug a python program that was running on a remote server and failing, but not locally.
SO much better than print(f"XXX {i}") and friends!
Nice tutorial about its usage: Better Python Debugging With IPDB
n - next line in current method (=“step over”)s - next line of executable code anywhere (=“step into”)c - continue till next breakpointr - continue till function returns (would be nice to learn how to do this in pycharm btw!)a - args - print arguments current function receivedb - adds breakpoint to locations
b filename.py:234b <function>b 123 - line in current fileFull documentation here: 26.2. pdb — The Python Debugger — Python 2.7.18 documentation
My main issue with Enum classes was that serialization is weird, especially if you’re dumping parameters. Tried again, found this: python - Serialising an Enum member to JSON - Stack Overflow
TL;DR class EnumDerivedClass(str, Enum)
import json
from enum import Enum
class LogLevel(str, Enum):
DEBUG = 'DEBUG'
INFO = 'INFO'
print(LogLevel.DEBUG)
print(json.dumps(LogLevel.DEBUG))
print(json.loads('"DEBUG"'))
print(LogLevel('DEBUG'))
will output
LogLevel.DEBUG
"DEBUG"
DEBUG
LogLevel.DEBUG
“Folie überspringen” is a much better way to do what I did with setting a yellow background color - easy to see and worst case scenario it’ll just get skipped
If you run tensorboard on a non-existing folder, you’ll get no feedback about it anywhere?.. No data on Tensorboard itself, nothing useful in CLI.
After some ill-fated undoing of commits, couldn’t find the work of an hour or so.
Guess what: Using Local History to Restore Code Fragments or Deleted Files | The IntelliJ IDEA Blog
I knew about local history for a file, but you can do the same for a directory, through its right-click menu in the Projects view!
I already thought I had set up nvidia-smi and friends (Day 850 | Diensttagebuch (Work journal)), then didn’t use it for months, now when I tried it didn’t work anymore, nvidia-smi said “No devices found”
boltctl showed the device as connected and authorized, prime-select said nvidia was selected, modprobe showed that the correct drivers were used and dkms status had said the correct drivers were installed.
(11:53:23/10181)~/$ dkms status
nvidia, 460.73.01, 5.4.0-73-generic, x86_64: installed
nvidia, 460.73.01, 5.4.0-74-generic, x86_64: installed
(11:53:49/10182)~/$ boltctl
[snip]
● Lenovo ThinkPad Thunderbolt 3 Dock #2
├─ type: peripheral
├─ name: ThinkPad Thunderbolt 3 Dock
├─ vendor: Lenovo
├─ uuid: xxx
├─ status: authorized
│ ├─ domain: domain0
│ └─ authflags: none
├─ authorized: Mo 20 Sep 2021 09:41:16 UTC
├─ connected: Mo 20 Sep 2021 09:41:16 UTC
└─ stored: no
● GIGABYTE GV-N1070IXEB-8GD
├─ type: peripheral
├─ name: GV-N1070IXEB-8GD
├─ vendor: GIGABYTE
├─ uuid: xxx
├─ status: authorized
│ ├─ domain: domain0
│ └─ authflags: none
├─ authorized: Mo 20 Sep 2021 09:42:35 UTC
├─ connected: Mo 20 Sep 2021 09:42:35 UTC
└─ stored: Mo 20 Sep 2021 09:31:09 UTC
├─ policy: manual
└─ key: no
(11:54:54/10188)~/$ lsmod
Module Size Used by
nvidia_uvm 1015808 0
nvidia_drm 57344 1
nvidia_modeset 1228800 1 nvidia_drm
nvidia 34123776 17 nvidia_uvm,nvidia_modeset
(11:55:54/10192)~/$ sudo prime-select query
nvidia
What didn’t work:
What worked:
Offtopic, when I was googling these issues I found my own serhii.net link above on the first page of Google for the key '“nvidia-smi “no devices were found” authorized', which is both nice and sad at the same time :)
EDIT: the next morning it didn’t work again. None of the same magic steps in all possible orders. I think it might be an issue with the eGPU or dock or something of that level. The best way to check this would be to do the nuclear option, uninstall all drivers, and install from the beginning, but I think my monthly quota of GPU stuff is full five times over now.
We’re on day 993 (!) of Diensttagebuch! Freaking awesome.
Was creating a requirements.txt for detectron2, official install instructions were:
python -m pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.9/index.html
Answer specificalyl about this: python - How to format requirements.txt when package source is from specific websites? - Stack Overflow:
requirements.txt format is:
[[--option]...]
<requirement specifier> [; markers] [[--option]...]
<archive url/path>
[-e] <local project path>
[-e] <vcs project url>
<requirements specifier> is:
SomeProject
SomeProject == 1.3
SomeProject >=1.2,<2.0
SomeProject[foo, bar]
SomeProject~=1.4.2
The –option (such as the -f/–find-links) is the same as the pip install options you would use if you were doing pip install from the command line.
Therefore, in requirements.txt it ended up literally as this:
--find-links https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.9/index.html detectron2
And by the way, detectron2’s own requirements.txt demonstrates nicely part of the above.
My own requirements.txt for CUDA 11.1:
opencv-python==4.2.0.32
# torch 1.9 for cuda 10.2 (for this config https://pytorch.org/get-started/locally/ has no versions in the command
# getting both exact versions from pip freeze
-f https://download.pytorch.org/whl/torch_stable.html
torch==1.9.0+cu111
torchvision==0.10.0+cu111
#torch==1.7.1
#torchvision==0.8.2
# python -m pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.9/index.html
-f https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.9/index.html
detectron2
Best part about ag is that I don’t need to escape anything with its default settings:
pip freeze | ag "(detectron|torch)"
pip freeze | grep "\(detectron\|torch\)"
Suddenly stopped getting readable output.
Fix is to add the env variable JB_DISABLE_BUFFERING, without any value, to the env of the test.
teamcity - no output in console for unittests in pycharm 2017 - Stack Overflow
The documentation about default confg covers all the parameters' meanings and can be used as reference for that! detectron2.config — detectron2 0.5 documentation
And me dreaming up cfg.MODEL.CHECKPOINT_PERIOD is exactly what they wanted to avoid by disallowing the creation of new keys.
Highlights:
# Number of images per batch across all machines. This is also the number
# of training images per step (i.e. per iteration).
_C.SOLVER.IMS_PER_BATCH = 16
For the second time, discovered that Telegram Cache takes 40gb of disk space.
In the phone’s own menus related to disk space, this was shown as “Pictures” taking 40gb, not the Telegram app and its cache. But this info is exposed through Telegram’s own menus.
Who knew you could combine commands! This is how you start tracking tag1,tag2 starting from the end of the previous span:
$ w track :fill tag1,tag2
Backfilled to 2021-09-15T12:21:41
Tracking "tag1,tag2"
Started 2021-09-15T12:21:41
Current 23:47
Total 0:02:06
TL;DR:
duc index ~/ --fs-exclude fuse.sshfs
duc is about disk space, before running it the index should be built/updated. Usually similar to duc index ~/.
If I have a sshfs mounted somewhere, the process never ends as it tries to index the folder where it’s mounted.
Found some solutions:
duc index ~/ --fs-exclude fuse.sshfs
ext3,ext4./etc/fstab didn’t have the sshfs filesystem, but mount called it fuse.sshfs and this worked!duc index ~/ -e "*somefilename*"
*folder\/file* etc).Bonus: -p shows progress during indexing.
Now I have a new alias in ~/.zshrc:
ducindex() {
duc index "$1" -p --fs-exclude fuse.sshfs
}
cdd CLI alias for CD-ing to directory containing a fileI copypaste a lot the locations of the files from pycharm/intellij to run them from CLI or something similar. Easiest way, because they are focused and I don’t need to focus on the files/project view for that. I can’t find an Action in pycharm/intellij to copypaste only the directory.
Yet another alias for today:
cdd() {
$(dirname "$1")
}
dirname gives the directory, dirname .. | cd and dirname ... | xargs cd don’t work (TODO - why?), so I’m using the zsh thing about “cd to the directory if it’s in a command by itself”.
Now cdd /home/me/wahtever/test.py takes me to /home/me/whatever/ which will saved tens of seconds per year!
Of course tiffsplit1 has a sister tiffcp! Ubuntu Manpage: tiffcp - copy (and possibly convert) a TIFF file
Concatenate N pages into a result.tif:
tiffcp xaaa.tif xaab.tif xabr.tif result.tif
# TODO - this is highlighted yellow
# And this is not
# ... BUT!
# TODO - this is highlighted yellow
# This too, because it's indented one space and logically belongs to the comment above!
I often do <CR>ddkkp or d$kp as a reverse-enter, moving what’s to the right of the cursor on the line above the current one. I’m sure something like this already exists in vim.
Detectron’s Yacs has a github repo with documentation and examples, much better than detectron’s own: rbgirshick/yacs: YACS – Yet Another Configuration System
This works:
comp_cfg.set_new_allowed(True)
comp_cfg['key'] = 'value'
Interesting bit about why it’s not like this by default:
We typically don’t use this so that typo in config file can be noticed. 2
Additionally, this is set per leaf, not per config - you can allow adding stuff to the root but not to its existing children.
And, still, even with comp_cfg.set_new_allowed(True), why can’t I merge_from_list etc. for non-existing keys? (TODO)
log.txt is nice and colorful on CLI, I don’t remember how to let vim interpret the CLI colors but less log.txt works magnificently.
cfg.merge_from_file() doesn’t work with new keys · Issue #2082 · facebookresearch/detectron2
If you are inside a screen, and need to write the entire contents to a file (not just the ones currently visible), this will work:
<C-a> :hardcopy -h <filename>.
In the context of a raised hand in google Hangouts meeting: “Do you have a question or an opinion?” (heard at work)
…TIL at work in a remote meeting. Makes the window with the code full-screen, hides the other windows, and increases the font size. Neat!
Yet another chapter of “python stdlib implementing most things I need better than me”, to create an infinite iterator itertools.count() is better than stuff like iter(range(100500)) (from AA’s comment in a PR)