General DVC notes

• Access:
  • Can directly get stuff from a repo when not inside a dvc project environment
    • Such as from within ML or code
    • Git repo has to be accessible ofc
  • DVC import - same as above, but also gets the metadata
    • Needs to be inside a DVC repo
      • Or have to do git init & dvc init first
  • Python bindings exist
• Stages:
  • Nice and neat
  • parameters.yaml
  • See parametrization below for maybe easier ways to pass parameters
  • Otherwise you just have your script read parameters.yaml, and version parameters.yaml too

DVC parametrization

Parametrization · iterative/dvc Wiki is an experimental feature.

Allows to call parameters directly, such as:

stages:
  build:
    foreach: ${models}
    do:
      cmd: >- 
          python script.py
          --out ${item.filename}
          --thresh ${item.thresh}
      outs:
          - ${item.filename}

as opposed to getting your program to read parameters.yaml

Ipset ipv6 ranges; online subnet ip calculators

IPSet set structures: wiki.ipfire.org - IPset To create an ipv6 ipset that supports domain ranges, we need the hash:net one:

ipset create my6 hash:net family inet6

Nice subnet calculators:

• IPv6 Subnet Calculator - Vultr.com
• IP Calculator / IP Subnetting

iptables doesn’t do ipv6, but ip6tables does, seems to be installed by default along with vanilla iptables. Commands seem to be identical.

Iptables persistent

• iptables-save > some_output_file to save them to a file (this alone doesn’t make it persist reboots)
• The packageiptables-persistent does what is says on the label,¹ for rules being saved in:
  • /etc/iptables/rules.v4
  • /etc/iptables/rules.v6

Ipset save and restore

ipset save > output_file
ipset save -f output_file

ipset restore -f output_file
ipset restore < output_file

The output files it generates seem to be the exact commands without the leading ipset ?

iptables and ipset persistence on yunohost

Looked into yunohost’s recommendations, there’s a best practice.² Created a shell script that does ipset restore -f file and then runs the iptables commands, put it into /etc/yunohost/hooks.d/post_iptable_rules/99-specific_rules. Survived a reboot, mission accomplished.

mktemp for temporary files

> mktemp /tmp/somescript.XXXX
/tmp/somescript.6Zxi

mktemp creates random files with a set format, replacing the XXX with random characters, and returns the filename (+ can also create directories). Cool!