In the middle of the desert you can say anything you want
By default, <Esc>
— bad idea for the same reason in vim it’s a bad idea.
AND my xkeymap-level keyboard mapping for Esc doesn’t seem to work here.
Default-2 is <C-]> which is impossible because of my custom keyboard layout.
Will be <C-=>
.
{
"command": "vim:leave-insert-mode",
"selector": ".jp-NotebookPanel[data-jp-vim-mode='true'] .jp-Notebook.jp-mod-editMode",
"keys": [
"Ctrl =",
]
}
(I can’t figure out why ,l
etc. don’t work in jupyterlab for this purpose)
(<leader>
is ,
)
"Insert mode mappings
" Leave insert mode
imap <leader>l <Esc>
imap qj <Esc>
" Write, write and close
imap ,, <Esc>:x<CR>
map ,. :w<CR>
… I will have an unified set of bindings for this someday, I promise.
Copypasting this (still draft version) here in full, before radically shortening it for my master thesis.
L’Ukraine a toujours aspiré à être libre
“Ukraine has always aspired to be free.” Voltaire, 1731 1
This section describes the bilingual nature of Ukraine’s society and the impact of historical state policies on the modern development of the language.
The ongoing Russian invasion is viewed by many as a continuation of a long-standing historical pattern, rather than an isolated incident.
This section doesn’t attempt to justify or challenge any particular position regarding the events described, nor is meant to be a definitive account of the history of the language.
But I believe this perspective is important to understanding the current linguistic landscape in Ukraine, as well as the linguistic challenges and phenomena that had a direct relevance on this thesis. (TODO mention how and which tasks are impacted by this)
In Ukraine itself, the status of Ukrainian (its only official language) varies widely, but for a large part of Ukrainians the question was never too much on the foreground (until recently, that is).
A significant number of people in Ukraine are bilingual (Ukrainian and Russian languages), and almost everyone can understand both Russian and Ukrainian.2
The reasons for this include Ukraine’s geographical and cultural proximity to Russia, and was to a large extent a result of consistent policy first of the Russian empire and the Soviet Union.
In the Russian Empire, the broader imperial ideology sought to assimilate various ethnicities into a single Russian identity (with Russian as dominant language), and policies aimed at diminshing Ukrainian national self-consciousness were a facet of that3. TODO source
Ukrainian (then officially called little Russian language/малорусский язык) was stigmatized as a (uncultured town folks’) dialect of Russian, unsuited for ‘serious’ literature or poetry — as opposed to the great Russian language (not editorializing, it was literally called that; these phrasing applied to the names of ethnicities as well, Russia as great Russia and Ukraine as little Russia; the extent to which this referred broader cultural attitudes is a discussion out of scope of this Thesis). (TODO footnote to ‘War and Punishment’ for more on this)
The history of Ukrainian language bans is long enough to merit a Wikipedia page itemizing all the attempts, 4 with the more notable ones in the Russian Empire being the 1863 Valuev Circular (forbidding the use of Ukrainian in religious and educational printed literature) and the Ems Ukaz, a decree by Emperor Alexander II banning the use of the Ukrainian language in print (except for reprinting old documents), forbidding the import of Ukrainian publications and the staging of plays or lectures in Ukrainian (1876). (TODO sources for both)
The 1928 grammar reform (sometimes called Skrypnykivka after the minister of education Skrypnyk) passed during this period, drafted by a commitee of prominent Ukrainian linguists, writers, and teachers synthetized the different dialects into a single orthography to be used across the entire territory.
The Ukrainian writers and intellectuals of that period became known as “the executed Renaissance”: most of them were purged in the years to follow, after the Soviet Union took a sharp turn towards Russification in the late 1920s and in the multiple waves of purges that followed. (Most prominent members of committee behind Skrypnykivka were repressed as well; Skrypnyk himself committed suicide in 1933.)
A new ‘orthographic’ reform was drafted in 1933. It had the stated goal of removing alleged burgeoise influences of the previous one. Andriy Khvylia5, the chairman of the new Orthography Commission described in his 1933 book “Eradicate, Destroy the Roots of Ukrainian Nationalism on the Linguistic Front” (TODO source) how the new reform eliminates all “deadly conservative norms established by nationalists” that “focused the Ukrainian language on the Polish and Czech borgeois cultures (…) and set a barrier between the Ukrainian and Russian language”.
In practice the reform brought the Ukrainian language much closer to Russian in many ways:
Many Ukrainian writers, poets and dissidents kept using the ‘old’ orthography, as well as the Ukrainian community outside the Soviet Union.
After the fall of the Soviet Union, there were many proposals for restoring the original orthography, but only the letter ґ was restored. In 2019 a new version of the Ukrainian orthography was approved, which restored some of the original rules as ’legal’ variants but without mandating any of them.
TODO format citation Debunking the myth of a divided Ukraine - Atlantic Council citing Oeuvres complètes de Voltaire - Voltaire - Google Books ↩︎
While the two languages are mutually intelligible to a large extent, knowing one doesn’t automatically make understand the other - most Russians can’t understand Ukrainian nearly as well as Ukrainians undestand the Russian language, for example. ↩︎
(by no means the only one — but the stories of other victims of Russia’s imperialism are best told elsewhere, and for many ethnicities, especially ones deeper inside Russia’s borders, there’s no one left to tell the story) ↩︎
Later repressed for nationalism. ↩︎
EDIT: this is becoming a more generic thingy for everything I’d ever need to refer to when writing a paper, later I’ll clean this mess.
Resources – DREAM Lab links to https://dream.cs.umass.edu/wp-content/uploads/2020/04/Tips-and-Best-Practices.pdf. Until I set up a system to save PDF info, I’ll paste it as screenshots here:
ChatGPT summarized the relevant pages of the PDF file thus, but didn’t do it well, mostly rewriting myself:
multi-discipli\-nary
\begin{sloppypar}...
for paragraphs where latex goes over the margin.\begin{figure}[t]
\centering
for aligning tables and figures.sth~\cite{whatever}
\emph
over bold or \textit
.\newcommand{\system}{SQuID\xspace}
\xspace
here adds as space unless end of sentence. Package \usepackage{xspace}
\smallskip
, \medskip
, and \bigskip
, instead of \vspace
\linewidth
or \textwidth
.\resizebox
with appropriate dimensions.Compression hacks (see pics)
Paper writing hacks:
best practices - When should I use non-breaking space? - TeX - LaTeX Stack Exchange lists ALL the places where Knuth wanted people to put nonbreaking spaces, incl:
1)~one 2)~two
Donald~E. Knuth
1,~2
Chapter~12
Less obvious and not from him:
I~am
Also:
ALSO
ChatGPT says that citations should come before footnotes to prioritize the scholarly source over unimportant info. So this [32] 3 and not this3 [32]. Basically footnotes after all punctuation and citations. OK
EDIT: NOT PARENTHESES, THEY SHOULD BE WITHIN PARENTHESES.4 DAMN
ALSO
I sometimes write and around ~50%
forgetting that ~
is a nbsp — hard to catch when reading the text.
ALSO
As when writing code I like to add some assert False
(or a failing test) so that I know where I stopped the last time,
\latexstopcompiling here
is a neat way to make sure I REALLY finis ha certain line I started but not finished.
Rounding.
Previously: 211018-1510 Python rounding behaviour with TL;DR that python uses banker’s rounding, with .5th rounding towards the even number.
Floor/ceil have their usual latex notation as \rceil
, \rfloor
(see LaTeX/Mathematics - Wikibooks, open books for an open world at ‘delimiters’)
“Normal” rounding (towards nearest integer) has no standard notation: ceiling and floor functions - What is the mathematical notation for rounding a given number to the nearest integer? - Mathematics Stack Exchange
let XXX denote the standard rounding function
Bankers’ rounding (that python and everyone else use for tie-breaking for normal rounding and .5) has no standard notation as well
Let $\lfloor x \rceil$ denote "round half to even" rounding (a.k.a. "Banker's rounding"), consistent with Python's built-in round() and NumPy's np.round() functions.
Require/Ensure is basically Input/Output and can be renamed thus1:
\floatname{algorithm}{Procedure}
\renewcommand{\algorithmicrequire}{\textbf{Input:}}
\renewcommand{\algorithmicensure}{\textbf{Output:}}
\usepackage{algorithm}
\usepackage{algpseudocode}
% ...
\begin{algorithm}
\caption{Drop Rare Species per Country}
\label{alg:drop}
\begin{algorithmic}
\Require $D_0$: initial set of occurrences
\Ensure $D_1$: Set of occurrences after filtering rare species
\State $D_1 \gets$ \emptyset
\For{each $c$ in Countries}
\For{each $s$ in Species}
\If {$|O_{c,s} \in D_0| \geq 10$} % if observations of species in country in D_0 have more than 10 entries; || is set cardinality
\State{$D_1 \gets D_1 \cup O_{c,s}$}
\EndIf
\EndFor
\EndFor
\end{algorithmic}
\end{algorithm}
from pathlib import Path
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
INTERACTIVE_TABLES=False
USE_BLACK = True
# 100% width table
from IPython.core.display import display, HTML
display(HTML("<style>.container { width:100% !important; }</style>"))
if INTERACTIVE_TABLES:
from itables import init_notebook_mode
init_notebook_mode(all_interactive=True, connected=True)
# black formatting
if USE_BLACK:
%load_ext jupyter_black
# column/row limits removal
pd.set_option("display.max_columns", None)
pd.set_option('display.max_rows', 100)
# figsize is figsize
plt.rcParams["figure.figsize"] = (6, 8)
plt.rcParams["figure.dpi"] = 100
# CHANGEME
PATH_STR = "xxxxx/home/sh/hsa/plants/inat500k/gbif.metadata.csv"
PATH = Path(PATH_STR)
assert PATH.exists()
List of all map providers, not all included in geopandas and some paid, nevertheless really neat: https://xyzservices.readthedocs.io/en/stable/gallery.html
For the 231024-1704 Master thesis task CBT task of my 230928-1745 Masterarbeit draft, I’d like to create an ontology I can use to “seed” LMs to generate ungoogleable stories.
And it’s gonna be fascinating.
I don’t know what’s the difference between knowledge graph, ontology etc. at this point.
I want it to be highly abstract - I don’t care if it’s a forest, if it’s Cinderella etc., I want the relationships.
Let’s try. Cinderella is basically “Rags to riches”, so:
…
Or GPT3’s ideas from before:
"Entities": {
"Thief": {"Characteristics": ["Cunning", "Resourceful"], "Role": "Protagonist"},
"Fish": {"Characteristics": ["Valuable", "Symbolic"], "Role": "Object"},
"Owner": {"Characteristics": ["Victimized", "Unaware"], "Role": "Antagonist"}
},
"Goals": {
"Thief": "Steal Fish",
"Owner": "Protect Property"
},
"Challenges": {
"Thief": "Avoid Detection",
"Owner": "Secure Property"
},
"Interactions": {
("Thief", "Fish"): "Theft",
("Thief", "Owner"): "Avoidance",
("Owner", "Fish"): "Ownership"
},
"Outcomes": {
"Immediate": "Successful Theft",
"Long-term": "Loss of Trust"
},
"Moral Lessons": {
"Actions Have Consequences",
"Importance of Trust",
"Greed Leads to Loss"
}
Here’s it generating an ontology based on the above graph: https://chat.openai.com/share/92ed18ce-88f9-4262-9dd9-f06a07d06acc
And more in UKR: https://chat.openai.com/share/846a5e85-353e-4bb5-adbe-6da7825c51ed
In bold bits I’m not sure of. In decreasing order of abstraction, with the first two being the most generic ones and the latter ones more fitting for concrete stories.
Characteristics
:
Role
: CHARACTER ROLEEntity
: ENTITYGoal
: main goal of entity in this contextSHORT-TERM
: plaintext descriptionLONG-TERM
: plaintext descriptionRemaining issues:
Here’s ChatGPT applying that to Shrek: https://chat.openai.com/share/d96d4be6-d42f-4096-a18f-03f786b802c6
Modifying its answers:
“Using this ontology for abstract fairy tale description, please create a generalized graph structure for THE FIRST HARRY POTTER MOVIE. Focus on the overarching themes and character roles without specific names or unique settings. The graph should include key plot points, character roles, entities, goals, interactions, outcomes, and moral lessons, all described in a manner that is broadly applicable to similar stories.”
<
Context:
> pandoc 230928-1745\ Masterarbeit\ draft.md -o master_thesis.pdf
# unicode magic
Try running pandoc with --pdf-engine=xelatex.
# thank you
> pandoc 230928-1745\ Masterarbeit\ draft.md -o master_thesis.pdf --pdf-engine=xelatex
# a volley of...
[WARNING] Missing character: There is no о (U+043E) in font [lmroman10-italic]:mapping=tex-text;!
Exporting Hugo to PDF | akos.ma looks nice.
build/pdf/%.pdf: content/posts/%/index.md
$(PANDOC) --write=pdf --pdf-engine=xelatex \
--variable=papersize:a4 --variable=links-as-notes \
--variable=mainfont:DejaVuSans \
--variable=monofont:DejaVuSansMono \
--resource-path=$$(dirname $<) --out=$@ $< 2> /dev/null
Let’s try:
pandoc 230928-1745\ Masterarbeit\ draft.md -o master_thesis.pdf --pdf-engine=xelatex --variable=links-as-notes \
--variable=mainfont:DejaVuSans \
--variable=monofont:DejaVuSansMono
Better but not much; HTML is not parsed, lists count as lists only after a newline it seems.
pandoc 230928-1745\ Masterarbeit\ draft.md -o master_thesis.pdf --pdf-engine=xelatex --variable=links-as-notes \
--variable=mainfont:DejaVuSans \
--variable=monofont:DejaVuSansMono \
--from=markdown+lists_without_preceding_blankline
Better, but quotes unsolved:
Markdown blockquote shouldn’t require a leading blank line · Issue #7069 · jgm/pandoc
pandoc 230928-1745\ Masterarbeit\ draft.md -o master_thesis.pdf --pdf-engine=xelatex --variable=links-as-notes \
--variable=mainfont:DejaVuSans \
--variable=monofont:DejaVuSansMono \
--from=markdown+lists_without_preceding_blankline
#+blank_before_blockquote
ACTUALLY, - f gfm
(github-flavour) solves basically everything. commonmark
doesn’t parse latex, commonmark_x
(‘with many md extensions’) on first sight is similar to gfm
.
I think HTML is the last one.
Raw HTML says it’s only for strict:
--from=markdown_strict+markdown_in_html_blocks
msword - Pandoc / Latex / Markdown - TeX - LaTeX Stack Exchange suggest md to tex and tex to pdf, interesting approach.
6.11 Write raw LaTeX code | R Markdown Cookbook says complex latex code may be too complex for markdown.
This means this except w/o backslashes:
\```{=latex}
$\underset{\text{NOUN-NOM}}{\overset{\text{man}}{\text{чоловік-}\varnothing}}$ $\underset{\text{PST}}{\overset{\text{saw}}{\text{побачив}}}$ $\underset{\text{NOUN-ACC}}{\overset{\text{dog}}{\text{собак-у}}}$.
\```
Then commonmark_x can handle that.
EDIT: --standalone
!
I don’t need HTML, I need <sub>
.
pandoc md has a syntax for this: Pandoc - Pandoc User’s Guide
Options
--from=markdown+lists_without_preceding_blankline+blank_before_blockquote
? :(ChatGPT tried to create a filter but nothing works, I’ll leave it for later: https://chat.openai.com/share/c94fffbe-1e90-4bc0-9e97-6027eeab281a
This produces the best HTML documents:
> pandoc 230928-1745\ Masterarbeit\ draft.md -o master_thesis.html \
--from=gfm --mathjax --standalone
NB If I add CSS, it should be an absolute path:
It’d be cool to wrap examples in the same environment!
https://forum.obsidian.md/t/rendering-callouts-similarly-in-pandoc/40020:
-- https://forum.obsidian.md/t/rendering-callouts-similarly-in-pandoc/40020/6
--
local stringify = (require "pandoc.utils").stringify
function BlockQuote (el)
start = el.content[1]
if (start.t == "Para" and start.content[1].t == "Str" and
start.content[1].text:match("^%[!%w+%][-+]?$")) then
_, _, ctype = start.content[1].text:find("%[!(%w+)%]")
el.content:remove(1)
start.content:remove(1)
div = pandoc.Div(el.content, {class = "callout"})
div.attributes["data-callout"] = ctype:lower()
div.attributes["title"] = stringify(start.content):gsub("^ ", "")
return div
else
return el
end
end
Makes:
> [!NOTE]- callout Title
>
> callout content
into
::: {.callout data-callout="note" title="callout Title"}
callout content
:::
.callout {
color: red; /* Set text color to red */
border: 1px solid red; /* Optional: add a red border */
padding: 10px; /* Optional: add some padding */
/* Add any other styling as needed */
}
Then this makes it pretty HTML:
pandoc callout.md -L luas/obsidian-callouts.lua -t markdown -s | pandoc --standalone -o some_test.html --css luas/callout-style.css
<div class="callout" data-callout="note" title="callout Title">
<p>callout content</p>
</div>
For PDF: .. it’s more complex, will need such a header file etc. later on. TODO
\usepackage{xcolor} % Required for color definition
\newenvironment{callout}{
\color{red} % Sets the text color to red within the environment
% Add any other formatting commands here
}{}
--css /abs/tufte.css
!Copied executables to /home/sh/.local/bin/:
aha so that’s where you put your filters, inside $PATH
Damn! Just had to replace index.md with my thesis, then make all
and it just …worked. Wow.
Apparently to make it not a sidenote I just have to add -
to the footnote itself. Would be trivial to replace with an @
etc., then I get my inital plan - citations as citations and footnotes with my remarks as sidenotes.
I can add --from gfm --mathjax
to the makefile command and it works with all my other requirements!
pandoc \
--katex \
--section-divs \
--from gfm \
--mathjax \
--filter pandoc-sidenote \
--to html5+smart \
--template=tufte \
--css tufte.css --css pandoc.css --css pandoc-solarized.css --css tufte-extra.css \
--output docs/tufte-md/index.html \
docs/tufte-md/index.md
I wonder if I can modify it to create latex-style sidenotes, it should be very easy: pandoc-sidenote/src/Text/Pandoc/SideNote.hs at master · jez/pandoc-sidenote
![Caption](file.ext){#fig:label}
$$ math $$ {#eq:label}
Section {#sec:section}
TODO figure out, and latex as well.
TODO