serhii.net

Just some really quick notes on this, it’s pointless and redundant but I’ll need these later

• lm-sys/FastChat: An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
  • FastChat/docs/model_support.md at main · lm-sys/FastChat
• LiteLLM Proxy Server (LLM Gateway) | liteLLM

LiteLLM

Create .yaml with a github models model:

model_list:
  # - model_name: github-Llama-3.2-11B-Vision-Instruct # Model Alias to use for requests
  - model_name: minist # Model Alias to use for requests
    litellm_params:
      model: github/Ministral-3B
      api_key: "os.environ/GITHUB_API_KEY" # ensure you have `GITHUB_API_KEY` in your .env

After setting GITHUB_API_KEY, litellm --config config.yaml

FastChat

• Install FastChat, start the controller:
  • python3 -m fastchat.serve.controller
• FastChat model config:

{
    "minist": {
        "model_name": "minist",
        "api_base": "http://0.0.0.0:4000/v1",
        "api_type": "openai",
        "api_key": "whatever",
        "anony_only": false
    }
}

• Run the webUI
  • python3 -m fastchat.serve.gradio_web_server_multi --register-api-endpoint-file ../model_config.json
  • Direct chat now works!

TIL about 1C when I had to move it from one windows laptop to another. First and last windows post here hopefully

Ref: 1С – как перенести базу на другой компьютер

Long story short:

• 1C v 7.7 from 1999 can be literally copied to another laptop and it works, windows is good at this.
• the database is also a directory, which is usually shown when you start 1C and choose the DB to open
• that directory can be copied
  • I THINK it’s good karma if the path on both computers is identical, but am not certain
• if the version matches that should be it
• no idea how to check if it worked :)

command line - How to do: underline, bold, italic, strikethrough, color, background, and size in Gnome Terminal? - Ask Ubuntu:

echo -e '\e[1mbold\e[22m'
echo -e '\e[2mdim\e[22m'
echo -e '\e[3mitalic\e[23m'
echo -e '\e[4munderline\e[24m'
echo -e '\e[4:1mthis is also underline (since 0.52)\e[4:0m'
echo -e '\e[21mdouble underline (since 0.52)\e[24m'
echo -e '\e[4:2mthis is also double underline (since 0.52)\e[4:0m'
echo -e '\e[4:3mcurly underline (since 0.52)\e[4:0m'
echo -e '\e[4:4mdotted underline (since 0.76)\e[4:0m'
echo -e '\e[4:5mdashed underline (since 0.76)\e[4:0m'
echo -e '\e[5mblink (since 0.52)\e[25m'
echo -e '\e[7mreverse\e[27m'
echo -e '\e[8minvisible\e[28m <- invisible (but copy-pasteable)'
echo -e '\e[9mstrikethrough\e[29m'
echo -e '\e[53moverline (since 0.52)\e[55m'

echo -e '\e[31mred\e[39m'
echo -e '\e[91mbright red\e[39m'
echo -e '\e[38:5:42m256-color, de jure standard (ITU-T T.416)\e[39m'
echo -e '\e[38;5;42m256-color, de facto standard (commonly used)\e[39m'
echo -e '\e[38:2::240:143:104mtruecolor, de jure standard (ITU-T T.416) (since 0.52)\e[39m'
echo -e '\e[38:2:240:143:104mtruecolor, rarely used incorrect format (might be removed at some point)\e[39m'
echo -e '\e[38;2;240;143;104mtruecolor, de facto standard (commonly used)\e[39m'

echo -e '\e[46mcyan background\e[49m'
echo -e '\e[106mbright cyan background\e[49m'
echo -e '\e[48:5:42m256-color background, de jure standard (ITU-T T.416)\e[49m'
echo -e '\e[48;5;42m256-color background, de facto standard (commonly used)\e[49m'
echo -e '\e[48:2::240:143:104mtruecolor background, de jure standard (ITU-T T.416) (since 0.52)\e[49m'
echo -e '\e[48:2:240:143:104mtruecolor background, rarely used incorrect format (might be removed at some point)\e[49m'
echo -e '\e[48;2;240;143;104mtruecolor background, de facto standard (commonly used)\e[49m'

echo -e '\e[21m\e[58:5:42m256-color underline (since 0.52)\e[59m\e[24m'
echo -e '\e[21m\e[58;5;42m256-color underline (since 0.52)\e[59m\e[24m'
echo -e '\e[4:3m\e[58:2::240:143:104mtruecolor underline (since 0.52) (*)\e[59m\e[4:0m'
echo -e '\e[4:3m\e[58:2:240:143:104mtruecolor underline (since 0.52) (might be removed at some point) (*)\e[59m\e[4:0m'
echo -e '\e[4:3m\e[58;2;240;143;104mtruecolor underline (since 0.52) (*)\e[59m\e[4:0m'

Kitty was really slow to start and this has been bugging me. Especially setting up a new system, using the default gnome-terminal, and seeing it appear instantly.

Kitty’s single instance mode (also -1) decreased start from 400ms to 300, still too much.

time alacritty -e bash -c exit
time gnome-terminal -e "bash -c exit"
time kitty --single-instance bash -c exit

New best friend: Alacritty

Saw Alacritty mentioned and it’s awesome. Has everything I wanted from or set up for kitty. Kitty is more configurable (I think), but I’m not missing anything at all so far.

Spawn instance in same directory

I used to have a separate command for that!

ArchWiki:

[keyboard]
bindings = [
   { key = "Return", mods = "Control|Shift", action = "SpawnNewInstance" }
]

Hints

Updated the default config to copy instead of launch, use better letters, and do file paths together with URIs:

[hints]
alphabet = "aoeusndh"

hints.enabled
action = "Copy"
# command = "xdg-open" # On Linux/BSD
hyperlinks = true
post_processing = true
persist = false
mouse.enabled = true
binding = { key = "N", mods = "Control|Shift" }

# adds file paths as well
regex = '(?:(?:ipfs:|ipns:|magnet:|mailto:|gemini://|gopher://|https?://|news:|git://|ssh:|ftp:|file:)[^\u0000-\u001F\u007F-\u009F<>"\s{}\-\^⟨⟩`\\]+|(?:(?:\.\.?/)+|/)[^\u0000-\u001F\u007F-\u009F<>"\s{}\-\^⟨⟩`\\]+)'
# regex = "(ipfs:|ipns:|magnet:|mailto:|gemini://|gopher://|https://|http://|news:|file:|git://|ssh:|ftp://)[^\u0000-\u001F\u007F-\u009F<>\"\\s{-}\\^⟨⟩`\\\\]+"

Vim mode

<C-S-Space> runs a vim-ish mode on the text, one can then copy etc. with all usual movements!

In kitty I had to do vim as scrollback pager etc. (old code below, prolly broken, didn’t use it because too complex)

For later

• alacritty/alacritty-theme: Collection of Alacritty color schemes

Previously

Some old configs from kitty, for reference:

## Bindings

# https://sw.kovidgoyal.net/kitty/index.html#kittens
map kitty_mod>f1 launch --stdin-source=@screen_scrollback --stdin-add-formatting less +G -R

## Hints
# File paths
map kitty_mod+n>f kitten hints --type path --program @

# IPs (+ with ports)
map kitty_mod+n>i kitten hints --type regex --regex [0-9]+(?:\.[0-9]+){3} --program @
map kitty_mod+n>p kitten hints --type regex --regex [0-9]+(?:\.[0-9]+){3}:[0-9]+ --program @

# CLI Commands
# map kitty_mod+n>c kitten hints --type regex --regex "(\$|>)(.+)(?:\n|\s*$)?" --program @
# This version copies up to the vim mode indicator
# map kitty_mod+n>c kitten hints --type regex --regex "(\$|>)(.+?)(?:\n|\s+$|\s+(?:INS|VIS|REP|SEA))" --program @
# map kitty_mod+n>c kitten hints --type regex --regex "\$(.+)" --program @

# Linenum
map kitty_mod+n>l kitten hints --type line --program @

Scrollback/vim:

# https://sw.kovidgoyal.net/kitty/index.html#kittens
map kitty_mod>f1 launch --stdin-source=@screen_scrollback --stdin-add-formatting less +G -R
scrollback_pager vim - -c "w! /tmp/kitty_scrollback_sh" -c "term ++curwin cat /tmp/kitty_scrollback_sh"

• After running a command (e.g. ag whatever, ^ag^rg re-runs it substituting the first part with the second — so rg whatever)
  • Bash History: Correct / Repeat The Last Command With a Substitution - nixCraft
• Fish doesn’t (want to) support this: Frequently asked questions — fish-shell 4.0.1 documentation

Ty BB for this

Getting Started | Task

Jan-Piet Mens :: A shell command to create JSON: jo

• Lua-guide - Neovim docs
  • tl;dr vim.opt.xxx = yy is the most conv. way to set all local global options.
  • vim.wo etc. exist for easier access

vim.wo.relativenumber = true
vim.wo.cursorcolumn = true
vim.wo.cursorline = true
vim.wo.colorcolumn = "80"

-- :set tabstop=8 shiftwidth=4 softtabstop=4 expandtab shiftround
vim.g.tabstop = "8"
vim.g.softtabstop = "8"
vim.g.shiftwidth = "4"
vim.g.expandtab = true
vim.g.smarttab = true

General info

• Roughly, it’s evaluating
  • context — how relevant/correct are the retrieved chunks
  • answer — how good are the generated claims
  • (with interplay inbetween — e.g. whether the answer comes from the context, regardless of whether both are relevant to the query)
• Or: generator metrics, retriever metrics, and overall metrics as used in RAGChecker (see RAGChecker picture later below).

Sources / libs

• Evaluating - LlamaIndex

  • Especially Usage Pattern (Response Evaluation) - LlamaIndex
    • Faithfullness: context(=source)+answer => whether the answer comes from the context available (=whether the answer was hallucinated)
    • Relevancy: query+context+answer => whether the answer AND CONTEXT were relevant for the specific query
  • All LlamaIndex’s eval modules: Modules - LlamaIndex
    • CorrectnessEvaluator: compare query+answer to a reference answer: Correctness Evaluator - LlamaIndex w/ another LLM
    • Embedding Similarity Evaluator - LlamaIndex response+reference semantic similarity via embeddings

• Deepeval’s metrics as given in the llamaindex docs:

  from deepeval.integrations.llama_index import ( DeepEvalAnswerRelevancyEvaluator, DeepEvalFaithfulnessEvaluator, DeepEvalContextualRelevancyEvaluator, DeepEvalSummarizationEvaluator, DeepEvalBiasEvaluator, DeepEvalToxicityEvaluator, )
  

RAGChecker

• RAGChecker: RAGChecker: A Fine-grained Framework For Diagnosing RAG
  • LlamaIndex integration docs: RAGChecker: A Fine-grained Evaluation Framework For Diagnosing RAG - LlamaIndex

• https://blog.edward-li.com/tech/advanced-python-features/
  • Protocols
  • Python slots to make accessing fields of a class faster
  • for-else statements to avoid if x is not None(): ...
  • etc., TODO a lot of cool stuff
• https://news.ycombinator.com/item?id=43770494

* did you know __init__.py is optional nowadays?

* you can do relative imports with things like "from ..other import foo"

* since 3.13 there is a @deprecated decorator that does what you think it does

* the new generics syntax also works on methods/functions: "def method[T](...)" very cool

* you can type kwargs with typeddicts and unpack: "def fn(*kwargs: Unpack[MyKwargs])"

* dataclasses (and pydantic) support immutable objects with: "class MyModel(BaseModel, frozen=True)" or "@dataclass(frozen=True)"

* class attributes on dataclasses, etc. can be defined with "MY_STATIC: ClassVar[int] = 42" this also supports abstract base classes (ABC)

* TypeVar supports binding to enforce subtypes: "TypeVar['T', bound=X]", and also a default since 3.13: "TypeVar['T', bound=X, default=int]"

* @overload is especially useful for get() methods to express that the return can't be none if the default isn't None

* instead of Union[a, b] or Optional[a] you can write "a | b" or "a | None" nowadays

* with match you can use assert_never() to ensure exhaustive matching in a "case _:" block

* typing has reveal_type() which lets mypy print the type it thinks something is

* typing's "Self" allows you to more properly annotate class method return types

* the time package has functions for monotonic clocks and others not just time()