serhii.net

Copypasting this (still draft version) here in full, before radically shortening it for my master thesis.

Historical context and bilingualism in the modern Ukrainian language

L’Ukraine a toujours aspiré à être libre
“Ukraine has always aspired to be free.” Voltaire, 1731 ¹

This section describes the bilingual nature of Ukraine’s society and the impact of historical state policies on the modern development of the language.

The ongoing Russian invasion is viewed by many as a continuation of a long-standing historical pattern, rather than an isolated incident.

This section doesn’t attempt to justify or challenge any particular position regarding the events described, nor is meant to be a definitive account of the history of the language.

But I believe this perspective is important to understanding the current linguistic landscape in Ukraine, as well as the linguistic challenges and phenomena that had a direct relevance on this thesis. (TODO mention how and which tasks are impacted by this)

A historical overview

• todo: more synonyms for ‘policy’
• todo: better title
• todo: sources for everything-everything-everything
• sources
  • 1987 book about the entire topic [^@krawchenko1987social]
  • Article The Executed Renaissance: The Book that Saved Ukrainian Literature from Soviet Oblivion | Article | Culture.pl
  • Keeping a record is the best book on this [^@1130282272476965120]

In Ukraine itself, the status of Ukrainian (its only official language) varies widely, but for a large part of Ukrainians the question was never too much on the foreground (until recently, that is).

A significant number of people in Ukraine are bilingual (Ukrainian and Russian languages), and almost everyone can understand both Russian and Ukrainian.²
The reasons for this include Ukraine’s geographical and cultural proximity to Russia, and was to a large extent a result of consistent policy first of the Russian empire and the Soviet Union.

The suppression of Ukrainian in the Russian Empire

In the Russian Empire, the broader imperial ideology sought to assimilate various ethnicities into a single Russian identity (with Russian as dominant language), and policies aimed at diminshing Ukrainian national self-consciousness were a facet of that³. TODO source

Ukrainian (then officially called little Russian language/малорусский язык) was stigmatized as a (uncultured town folks’) dialect of Russian, unsuited for ‘serious’ literature or poetry — as opposed to the great Russian language (not editorializing, it was literally called that; these phrasing applied to the names of ethnicities as well, Russia as great Russia and Ukraine as little Russia; the extent to which this referred broader cultural attitudes is a discussion out of scope of this Thesis). (TODO footnote to ‘War and Punishment’ for more on this)

The history of Ukrainian language bans is long enough to merit a Wikipedia page itemizing all the attempts, ⁴ with the more notable ones in the Russian Empire being the 1863 Valuev Circular (forbidding the use of Ukrainian in religious and educational printed literature) and the Ems Ukaz, a decree by Emperor Alexander II banning the use of the Ukrainian language in print (except for reprinting old documents), forbidding the import of Ukrainian publications and the staging of plays or lectures in Ukrainian (1876). (TODO sources for both)

The redefinition of Ukrainian in the Soviet Union

• TODO [^@marshall2002post] has many sources for this! The first decade of Soviet Union brought Ukrainisation as part of a new Soviet nationalities policy, and the use of Ukrainian in different areas was promoted. The reason for this was chiefly ideological — the ghosts of the old empire were considered a danger and encouraging formerly-suppressed cultures was one way to fight them — but nevertheless, this period led to a short-lived period of flourishing for Ukrainian literature and culture in general.

The 1928 grammar reform (sometimes called Skrypnykivka after the minister of education Skrypnyk) passed during this period, drafted by a commitee of prominent Ukrainian linguists, writers, and teachers synthetized the different dialects into a single orthography to be used across the entire territory.

The Ukrainian writers and intellectuals of that period became known as “the executed Renaissance”: most of them were purged in the years to follow, after the Soviet Union took a sharp turn towards Russification in the late 1920s and in the multiple waves of purges that followed. (Most prominent members of committee behind Skrypnykivka were repressed as well; Skrypnyk himself committed suicide in 1933.)

A new ‘orthographic’ reform was drafted in 1933. It had the stated goal of removing alleged burgeoise influences of the previous one. Andriy Khvylia⁵, the chairman of the new Orthography Commission described in his 1933 book “Eradicate, Destroy the Roots of Ukrainian Nationalism on the Linguistic Front” (TODO source) how the new reform eliminates all “deadly conservative norms established by nationalists” that “focused the Ukrainian language on the Polish and Czech borgeois cultures (…) and set a barrier between the Ukrainian and Russian language”.

In practice the reform brought the Ukrainian language much closer to Russian in many ways:

• Grammatically, by introducing relatively major changes in declension paradigms, plurals, especially targeting grammatical structures absent in Russian. The letter ґ (absent in the Russian alphabet) was dropped.
• Changes in the vocabulary were also massive: the spelling of foreign-derived words and proper names were changed to fit Russian patterns. For example, the German diphthong ei (IPA: [aɪ], approx. English “eye”), in Ukrainian ай with the same sound, became the Russian ей (IPA: [ɛj], approx. in English “they”), changing surnames like Einstein / Айнштайн (IPA: [ˈajnʃtajn])[^44] to Ейнштейн (IPA: [ˈɛjnʃtɛjn]). Genders of words were changed.
• Terminology: there was an effort to eliminate Ukrainian-specific vocabulary, much of which was just replaced with the (sometimes completely different) Russian equivalents. This was quite explicit, to the point that publishing houses regularly received lists of words to avoid. This increased the dependence of Ukrainian on Russian science.

Many Ukrainian writers, poets and dissidents kept using the ‘old’ orthography, as well as the Ukrainian community outside the Soviet Union.

After the fall of the Soviet Union, there were many proposals for restoring the original orthography, but only the letter ґ was restored. In 2019 a new version of the Ukrainian orthography was approved, which restored some of the original rules as ’legal’ variants but without mandating any of them.

Require/ensure

Require/Ensure is basically Input/Output and can be renamed thus¹:

\floatname{algorithm}{Procedure}
\renewcommand{\algorithmicrequire}{\textbf{Input:}}
\renewcommand{\algorithmicensure}{\textbf{Output:}}

Sample algpseudocode algorithm float

\usepackage{algorithm}
\usepackage{algpseudocode}

% ...

\begin{algorithm}
\caption{Drop Rare Species per Country}
\label{alg:drop}
\begin{algorithmic}
\Require $D_0$: initial set of occurrences
\Ensure $D_1$: Set of occurrences after filtering rare species
\State $D_1 \gets$ \emptyset
\For{each $c$ in Countries}
    \For{each $s$ in Species}
        \If {$|O_{c,s} \in D_0| \geq 10$} % if observations of species in country in D_0 have more than 10 entries; || is set cardinality
            \State{$D_1 \gets D_1 \cup O_{c,s}$} 
        \EndIf
    \EndFor
\EndFor
\end{algorithmic}
\end{algorithm}

List of all map providers, not all included in geopandas and some paid, nevertheless really neat: https://xyzservices.readthedocs.io/en/stable/gallery.html

For the 231024-1704 Master thesis task CBT task of my 230928-1745 Masterarbeit draft, I’d like to create an ontology I can use to “seed” LMs to generate ungoogleable stories.

And it’s gonna be fascinating.

I don’t know what’s the difference between knowledge graph, ontology etc. at this point.

Basics

I want it to be highly abstract - I don’t care if it’s a forest, if it’s Cinderella etc., I want the relationships.

Let’s try. Cinderella is basically “Rags to riches”, so:

• Character roles
  • Protagonist
    • Underprivileged protagonist (Cinderella)
  • Benefactor
  • …
• Key plot points
  • Hardship
  • Opportunity
  • Transformation
  • Achievement
• Thematic elements
  • Transformation
  • Reward?..

…

Or GPT3’s ideas from before:

"Entities": {
	"Thief": {"Characteristics": ["Cunning", "Resourceful"], "Role": "Protagonist"},
	"Fish": {"Characteristics": ["Valuable", "Symbolic"], "Role": "Object"},
	"Owner": {"Characteristics": ["Victimized", "Unaware"], "Role": "Antagonist"}
},
"Goals": {
	"Thief": "Steal Fish",
	"Owner": "Protect Property"
},
"Challenges": {
	"Thief": "Avoid Detection",
	"Owner": "Secure Property"
},
"Interactions": {
	("Thief", "Fish"): "Theft",
	("Thief", "Owner"): "Avoidance",
	("Owner", "Fish"): "Ownership"
},
"Outcomes": {
	"Immediate": "Successful Theft",
	"Long-term": "Loss of Trust"
},
"Moral Lessons": {
	"Actions Have Consequences",
	"Importance of Trust",
	"Greed Leads to Loss"
}

• ENTITIES have a:
  • ROLE
    • Protagonist
  • CHARACTERISTICS
    • Poor

Awesome chats with ChatGPT

Here’s it generating an ontology based on the above graph: https://chat.openai.com/share/92ed18ce-88f9-4262-9dd9-f06a07d06acc

And more in UKR: https://chat.openai.com/share/846a5e85-353e-4bb5-adbe-6da7825c51ed

Tale ontology v0.0.1

In bold bits I’m not sure of. In decreasing order of abstraction, with the first two being the most generic ones and the latter ones more fitting for concrete stories.

• KEY PLOT POINTS:
  • Hardship
  • Disruption
  • Opportunity
  • Transformation
  • Achievement
• CHARACTER ROLES:
  • Description: most abstract roles for the characters
  • Values:
    • Protagonist
    • Antagonist
    • Object
    • Benefactor
    • Goal object
    • Ally
    • etc.
• ENTITIES
  • Description:
    • can be characters, important objects, whatever
  • Attributes:
    • Characteristics:
      • … basically adjectives.
      • e.g. Resourceful, motivated, hungry
    • Role: CHARACTER ROLE
• GOALS
  • Description: what drives the character/entity
  • Attributes:
    • Entity: ENTITY
    • Goal: main goal of entity in this context
  • Example:
    • Thief: steal fish
    • Owner: protect property
• INTERACTIONS
  • Plaintext description of what’s up between pairs of ENTITIES
  • Example: (’thief’,‘fish’): theft
• OUTCOMES
  • Attributes:
    • SHORT-TERM: plaintext description
    • LONG-TERM: plaintext description
  • Example: “loss of trust”
  • Optional
• MORAL LESSONS:
  • Description: short plaintext description
  • Optional
  • examples:
    • “Actions Have Consequences”, “Importance of Trust”, “Greed Leads to Loss”

Remaining issues:

• Can I generalize to remove “fish” and stuff? Maybe call it object?
• Two versions of the ontology with different levels of abstractions?

Test 1:

Here’s ChatGPT applying that to Shrek: https://chat.openai.com/share/d96d4be6-d42f-4096-a18f-03f786b802c6

Modifying its answers:

“Using this ontology for abstract fairy tale description, please create a generalized graph structure for THE FIRST HARRY POTTER MOVIE. Focus on the overarching themes and character roles without specific names or unique settings. The graph should include key plot points, character roles, entities, goals, interactions, outcomes, and moral lessons, all described in a manner that is broadly applicable to similar stories.”

Revised Ontology for Abstract Fairy Tale Description

Character Roles

• Protagonist: Central character facing a challenge.
• Antagonist: Character or force opposing the protagonist.
• Goal Object: The objective or person the protagonist seeks.
• Ally: A supportive character aiding the protagonist.

Entities

• Description: Characters or significant elements in the story.
• Attributes:
  • Characteristics: Abstract adjectives describing the entity.
  • Role: One of the defined character roles.

Goals

• Description: Motivations driving the entities.
• Attributes:
  • Entity: Reference to an entity.
  • Objective: Abstract goal of the entity.

Interactions

• Description: General interactions between pairs of entities.
• Format: (‘Entity1’, ‘Entity2’): Type of interaction.

Outcomes

• Short-term: Immediate result of the story’s climax.
• Long-term: Lasting changes or lessons from the story.

Moral Lessons

• Description: Universal themes or morals derived from the story.

Attempt 2

<

Tools

• Tutorial: Build a Knowledge Graph using NLP and Ontologies - Developer Guides

Context:

• 230928-1745 Masterarbeit draft,
• 231225-2240 Glosses markdown magic, 231226-1702 Ideas for annotating glosses in my Masterarbeit

> pandoc 230928-1745\ Masterarbeit\ draft.md -o master_thesis.pdf
# unicode magic
Try running pandoc with --pdf-engine=xelatex.
# thank you

> pandoc 230928-1745\ Masterarbeit\ draft.md -o master_thesis.pdf --pdf-engine=xelatex
# a volley of...
[WARNING] Missing character: There is no о (U+043E) in font [lmroman10-italic]:mapping=tex-text;!

• Ugly:
  • 
  • 

Makefile magic etc

Exporting Hugo to PDF | akos.ma looks nice.

build/pdf/%.pdf: content/posts/%/index.md
	$(PANDOC) --write=pdf --pdf-engine=xelatex \
		--variable=papersize:a4 --variable=links-as-notes \
		--variable=mainfont:DejaVuSans \
		--variable=monofont:DejaVuSansMono \
		--resource-path=$$(dirname $<) --out=$@ $< 2> /dev/null

Let’s try:

pandoc 230928-1745\ Masterarbeit\ draft.md -o master_thesis.pdf --pdf-engine=xelatex  --variable=links-as-notes \
--variable=mainfont:DejaVuSans \
--variable=monofont:DejaVuSansMono

Better but not much; HTML is not parsed, lists count as lists only after a newline it seems.

Pandoc’s Markdown requires a newline after a paragraph for a list to render · Issue #6590 · jgm/pandoc

pandoc 230928-1745\ Masterarbeit\ draft.md -o master_thesis.pdf --pdf-engine=xelatex  --variable=links-as-notes \
--variable=mainfont:DejaVuSans \
--variable=monofont:DejaVuSansMono \
--from=markdown+lists_without_preceding_blankline

Better, but quotes unsolved:

Markdown blockquote shouldn’t require a leading blank line · Issue #7069 · jgm/pandoc

pandoc 230928-1745\ Masterarbeit\ draft.md -o master_thesis.pdf --pdf-engine=xelatex  --variable=links-as-notes \
--variable=mainfont:DejaVuSans \
--variable=monofont:DejaVuSansMono \
--from=markdown+lists_without_preceding_blankline
#+blank_before_blockquote

ACTUALLY, - f gfm (github-flavour) solves basically everything. commonmark doesn’t parse latex, commonmark_x (‘with many md extensions’) on first sight is similar to gfm.

I think HTML is the last one.

Raw HTML says it’s only for strict:

--from=markdown_strict+markdown_in_html_blocks

msword - Pandoc / Latex / Markdown - TeX - LaTeX Stack Exchange suggest md to tex and tex to pdf, interesting approach.

6.11 Write raw LaTeX code | R Markdown Cookbook says complex latex code may be too complex for markdown.

This means this except w/o backslashes:

\```{=latex}
$\underset{\text{NOUN-NOM}}{\overset{\text{man}}{\text{чоловік-}\varnothing}}$ $\underset{\text{PST}}{\overset{\text{saw}}{\text{побачив}}}$ $\underset{\text{NOUN-ACC}}{\overset{\text{dog}}{\text{собак-у}}}$.
\```

Then commonmark_x can handle that.

EDIT: --standalone!

More on HTML sub/sup to PDF

I don’t need HTML, I need <sub>.

• pandoc md has a syntax for this: Pandoc - Pandoc User’s Guide

  • …but I’m not using pandoc md :(

• Options

  • Can I replace my tags w/ that with yet another filter?
  • Ignore that obsidian/hugo can’t parse them and use pandoc syntax?.. and do --from=markdown+lists_without_preceding_blankline+blank_before_blockquote? :(
  • export to HTML w/ mathjax and from it PDF?
  • just use latex syntax everywhere? :(

ChatGPT tried to create a filter but nothing works, I’ll leave it for later: https://chat.openai.com/share/c94fffbe-1e90-4bc0-9e97-6027eeab281a

• I’m not alone:
  • Pandoc style superscript and subscript - Feature requests - Obsidian Forum

HTML

This produces the best HTML documents:

> pandoc 230928-1745\ Masterarbeit\ draft.md -o master_thesis.html \
--from=gfm --mathjax --standalone

NB If I add CSS, it should be an absolute path:

Later

Convert Markdown to PDF

• 6.2 Pandoc options for LaTeX output | R Markdown Cookbook
  • This has config options for latex, incl. fonts Pandoc - Pandoc User’s Guide
• 6.4 Include additional LaTeX packages | R Markdown Cookbook what a cool place

Callouts

It’d be cool to wrap examples in the same environment!

https://forum.obsidian.md/t/rendering-callouts-similarly-in-pandoc/40020:

-- https://forum.obsidian.md/t/rendering-callouts-similarly-in-pandoc/40020/6
--
local stringify = (require "pandoc.utils").stringify

function BlockQuote (el)
    start = el.content[1]
    if (start.t == "Para" and start.content[1].t == "Str" and
        start.content[1].text:match("^%[!%w+%][-+]?$")) then
        _, _, ctype = start.content[1].text:find("%[!(%w+)%]")
        el.content:remove(1)
        start.content:remove(1)
        div = pandoc.Div(el.content, {class = "callout"})
        div.attributes["data-callout"] = ctype:lower()
        div.attributes["title"] = stringify(start.content):gsub("^ ", "")
        return div
    else
        return el
    end
end

Makes:

> [!NOTE]- callout Title
>
> callout content

into

::: {.callout data-callout="note" title="callout Title"}
callout content
:::

.callout {
    color: red; /* Set text color to red */
    border: 1px solid red; /* Optional: add a red border */
    padding: 10px; /* Optional: add some padding */
    /* Add any other styling as needed */
}

Then this makes it pretty HTML:

pandoc callout.md -L luas/obsidian-callouts.lua -t markdown -s | pandoc --standalone -o some_test.html --css luas/callout-style.css

<div class="callout" data-callout="note" title="callout Title">
<p>callout content</p>
</div>

For PDF: .. it’s more complex, will need such a header file etc. later on. TODO

\usepackage{xcolor} % Required for color definition
\newenvironment{callout}{
  \color{red} % Sets the text color to red within the environment
  % Add any other formatting commands here
}{}

Unrelated

Footnotes

• What if I put the footnotes in the text margins and do a HTML-first thesis?
• margin notes!
  • LaTeX/Footnotes and Margin Notes - Wikibooks, open books for an open world:
  • Margin notes like footnotes?
  • discussion with css: Writing a side note on R markdown html #2 - R Markdown - Posit Community

Tufte CSS with pandoc

• A Tufte Handout Example
• tufte-css/tufte.css at gh-pages · edwardtufte/tufte-css
  • Which I can add as --css /abs/tufte.css!
• even better: jez/tufte-pandoc-css: Starter files for using Pandoc Markdown with Tufte CSS
  • hard install process though
  • very hard
• It uses this: jez/pandoc-sidenote: Convert Pandoc Markdown-style footnotes into sidenotes
  • Copied executables to /home/sh/.local/bin/: aha so that’s where you put your filters, inside $PATH

Damn! Just had to replace index.md with my thesis, then make all and it just …worked. Wow.

Apparently to make it not a sidenote I just have to add - to the footnote itself. Would be trivial to replace with an @ etc., then I get my inital plan - citations as citations and footnotes with my remarks as sidenotes.

I can add --from gfm --mathjax to the makefile command and it works with all my other requirements!

pandoc \
	--katex \
	--section-divs \
	--from gfm \
	--mathjax \
	--filter pandoc-sidenote \
	--to html5+smart \
	--template=tufte \
	--css tufte.css --css pandoc.css --css pandoc-solarized.css --css tufte-extra.css \
	--output docs/tufte-md/index.html \
	docs/tufte-md/index.md

I wonder if I can modify it to create latex-style sidenotes, it should be very easy: pandoc-sidenote/src/Text/Pandoc/SideNote.hs at master · jez/pandoc-sidenote

Numering references etc

• lierdakil/pandoc-crossref: Pandoc filter for cross-references
  • pandoc-crossref(1) | pandoc-crossref

![Caption](file.ext){#fig:label}
$$ math $$ {#eq:label}
 Section {#sec:section}

TODO figure out, and latex as well.

Citations

TODO

Current best:

eng: the man^NOM.SG saw the dog^NOM.SG
ukr: чоловік^man-NOM.SG побачив^saw-PST собакy^dog-ACC.SG

Ideas

I’d love to integrate the usual UD feats bits but they take a lot of space, and it’s either latex magic or one word per line.

• ukr: чоловік^{(man): Case=Nom|Number=Sing} побачив^(saw) собакy^{(dog): Case=Acc|Number=Sing}

• $чоловік^{man}_{Case=Nom|Number=Sing}$

• ${\underset{man}{чоловік}}^{Case=Nom|Number=Sing}$

• $\underset{Case=Nom|Number=Sing}{чоловік^{man}}$

• $\underset{NOM.SG}{чоловік^{man}}$

• ${\underset{man}{чоловік}}^{Case=Nom|Number=Sing}$

• ${\underset{man}{чоловік}}^{NOM.SG}$

• ${\underset{man}{чоловік}}^{NOM.SG}$ ${\underset{saw}{побачив}}$ ${\underset{dog}{собаку}}^{GEN.PL}$

я I _{Case=Nom|Number=Sing}
побачив saw
собаку saw  _{Animacy=Anim|Case=Acc|Gender=Masc|Number=Sing}

ukr: чоловік^man-NOM.SG побачив^saw-PST собакy^dog-GEN.PL

${\underset{man}{чоловік}}$ _{Case=Nom|Number=Sing} ${\underset{man}{чоловік}}$ _{Case=Nom|Number=Sing}

I think this is cool! But hell to write and parse:

$\underset{\text{NOUN.NOM}}{\overset{\text{man}}{\text{чоловік-}\varnothing}}$ $\underset{\text{PST}}{\overset{\text{saw}}{\text{побачив}}}$ $\underset{\text{NOUN-ACC}}{\overset{\text{dog}}{\text{собак-у}}}$.

$\underset{\text{NOUN.NOM}}{\overset{\text{man}}{\text{чоловік-}\varnothing}}$ $\underset{\text{PST}}{\overset{\text{saw}}{\text{побачив}}}$ $\underset{\text{NOUN-ACC}}{\overset{\text{dog}}{\text{собак-у}}}$.

Let’s play more with it:

$\underset{\text{Case=Nom|Number=Sing}}{\overset{\text{man }}{\text{чоловік}}}$ $\underset{\text{}}{\overset{\text{saw}}{\text{побачив}}}$ $\underset{\text{Case=Acc|Number=Sing}}{\overset{\text{dog}}{\text{собаку}}}$.

I can split it in diff lines: $\underset{\text{Case=Nom|Number=Sing}}{\overset{\text{man }}{\text{чоловік}}} \underset{\text{}}{\overset{\text{saw}}{\text{побачив}}} \underset{\text{Case=Acc|Number=Sing}}{\overset{\text{dog}}{\text{собаку}}}$.

$$\underset{\text{Case=Nom|Number=Sing}}{\overset{\text{man }}{\text{ЧОЛОВІК}}} \underset{\text{}}{\overset{\text{saw}}{\text{ПОБАЧИВ}}} \underset{\text{Case=Acc|Number=Sing}}{\overset{\text{dog}}{\text{СОБАКУ}}}$$

Splitting by morphemes

ukr: використовуватимуться ^{Aspect=Imp|Number=Plur|Person=3}

1 використовуватимуться використовуватися VERB _ Aspect=Imp|Mood=Ind|Number=Plur|Person=3|Tense=Fut|VerbForm=Fin 0 root _ SpaceAfter=No

ukr: використовуватимуть-ся^VERB-REFL

ukr: використовуватимуть^VERB -ся^REFL

$\underset{\text{NOM.SG}}{\overset{\text{man }}{\text{чоловік}}}$ $\underset{\text{PST}}{\overset{\text{saw}}{\text{побачив}}}$ $\underset{\text{SG-ACC}}{\overset{\text{dog}}{\text{собак-у}}}$.

This is one of the cooler ones, I’ll use it if I ever need to: Examples — graphviz 0.20.1 documentation

It’s also supported by HackMD! How to use MathJax & UML - HackMD

I’ll need something like overleaf for my markdown thesis.

5 Best Collaborative Online Markdown Editors - TechWiser

• HackMD
  • Markdown quick start guide - HackMD is nice!
  • The comments are part of the markdown code itself
    • but no way to highlight a certain phrase, just add a comment in a place of the text
  • you can share your doc to make it editable to not-logged-in users
  • Seems to hold my current thesis length quite well
  • I think I can stop looking

Interlinear Glosses

… are a way to annotate grammar bits of a language together with translation: Interlinear gloss - Wikipedia

The Leipzig Glossing Rules are a set of rules to standardize interlinear glosses. They are focused less on understandability and more on consistency.

• Markdown
  • cysouw/pandoc-ling: Pandoc Lua filter for linguistic examples
    • Using pandoc-ling
  • gunnarnl/pangb4e: Pandoc filter for gb4e support
    • Ref: Gunnar Lund - Markdown for linguists
  • parryc/doctor_leipzig: Leipzig, MD - glossing for Markdown - one can always do <span style="font-variant:small-caps;">Hello World</span>[^1]
• Python
  • Tutorial — pyigt 2.1.1.dev0 documentation

Using interlinear glosses

I’m writing my thesis in Obsidian/Markdown, synced to Hugo, later I’ll use sth like pandoc to make it into a PDF, with or without a latex intermediate step.

EDIT: newer technical part lives now here 231226-1702 Ideas for annotating glosses in my Masterarbeit

Pandoc-ling

cysouw/pandoc-ling: Pandoc Lua filter for linguistic examples

> pandoc --lua-filter=pandoc_ling.lua 231225-2240\ Glosses\ markdown\ magic.pandoc.md -o test.pdf
Error running filter pandoc_ling.lua:
pandoc_ling.lua:21: attempt to call a nil value (method 'must_be_at_least')
stack traceback:
	pandoc_ling.lua:21: in main chunk

:::ex
| Dutch (Germanic)
| Deze zin is in het nederlands.
| DEM sentence AUX in DET dutch.
| This sentence is dutch.
:::

:::ex | Dutch (Germanic) | Deze zin is in het nederlands. | DEM sentence AUX in DET dutch. | This sentence is dutch. :::

.. it was the pandoc version. Updated. No error, but no luck either.

Digging into the examples I think this is happening:

Code is code. Using that formatting without code makes it be interpreted as a line, and that doesn’t survive the obsidian’s pandoc extensions’ conversion to pandoc markdown.

The original docu generation had this script:

function addRealCopy (code)
  return { code, pandoc.RawBlock("markdown", code.text) }
end

return {
  { CodeBlock = addRealCopy }
}

It changes code blocks into code blocks and the content of the code block. Then the ::: block is put after the code but like normal markdown text, and it gets correctly changed by the pandoc-ling filter.

> pandoc 231225-2240\ Glosses\ markdown\ magic.pandoc.md -t markdown -L processVerbatim.lua -s --wrap=preserve | pandoc -L pandoc_ling.lua -o my.html

I can drop exporting extensions and just manually convert bits?..

This works:

> pandoc "garden/it/231225-2240 Glosses markdown magic.md" -t markdown -L pandoc_ling.lua  -s
> pandoc "garden/it/231225-2240 Glosses markdown magic.md" -L pandoc_ling.lua -o my.html

and is OK if my masterarbeit file will have no complexities at all.

(Can i add this as parameter to the existing bits?)

YES!

-L /home/sh/t/pandoc/pandoc_ling.lua added as option to the pandoc plugins, together with “from markdown” (not HTML) option, works for getting this parsed right!

(Except that it’s ugly in the HTML view but I can live with that)

And Hugo. Exporting to Hugo through obyde is ugly as well.

I colud write sth like this: A Pandoc Lua filter to convert Callout Blocks to Hugo admonitions (shortcode).

We’lll o

Obsidian link gloss plugin

Mijyuoon/obsidian-ling-gloss: An Obsidian plugin for interlinear glosses used in linguistics texts.

Pandoc export from HTML visualizes them quite well.

\gla Péter-nek van egy macská-ja
\glb pe:tɛrnɛk vɒn ɛɟ mɒt͡ʃka:jɒ
\glc Peter-DAT exist INDEF cat-POSS.3SG
\ft Peter has a cat.

\gla Péter-nek van egy macská-ja
\glb pe:tɛrnɛk vɒn ɛɟ mɒt͡ʃka:jɒ
\glc Peter-DAT exist INDEF cat-POSS.3SG
\ft Peter has a cat.

\set glastyle cjk
\ex 牆上掛著一幅畫 / 墙上挂着一幅画
\gl 牆 [墙] [qiáng] [wall] [^[TOP]
	上 [上] [shàng] [on] [^]]
	掛 [挂] [guà] [hang] [V]
	著 [着] [zhe] [CONT] [ASP]
	一 [一] [yì] [one] [^[S]
	幅 [幅] [fú] [picture.CL] []
	畫 [画] [huà] [picture] [^]]
\ft A picture is hanging on the wall.

Maybe a solution

• Do all my glosses using pandoc-ling format
• put them into code blocks belonging to a special class
• write a very similar filter to processVerbatim but that operates only on code blocks of this class
• when outputting to Hugo they’ll stay as preformatted code
• when exporting to pandoc run them through it first, then pandoc-ling, leading to pretty glosses in the final exported option

function addRealCopy (code)
  -- return { code, pandoc.RawBlock("markdown", code.text) }
	if code.classes[1] == "mygloss" then
		return { pandoc.RawBlock("markdown", code.text) }
	else
		return { code }
	end
end

return {
	{ CodeBlock = addRealCopy }
}

Should parse:

:::ex
| Dutch (Germanic)
| Deze zin is in het nederlands.
| DEM sentence AUX in DET dutch.
| This sentence is dutch.
:::

Should stay as code:

:::ex
| Dutch (Germanic)
| Deze zin is in het nederlands.
| DEM sentence AUX in DET dutch.
| This sentence is dutch.
:::

pandoc "/... arden/it/231225-2240 Glosses markdown magic.md" -L processVerbatim.lua -t markdown -s | pandoc -L pandoc_ling.lua -o my.html

It works!

But not this:

> pandoc "/home231225-2240 Glosses markdown magic.md" -L processVerbatim.lua -L pandoc_ling.lua -o my.html

Likely because both require markdown and the intermediate step seems to break.

Maybe I’m overcomplicating it and I can just use the UD I can use superscripts!

Just use superscripts

The inflectional paradigm of Ukrainian admits free word order: in English the Subject-Verb-Object word order in “the man^man-NOM.SG saw the dog^dog-NOM.SG” (vs “the dog ^man-NOM.SG saw the man^man-NOM.SG “) determines who saw whom, while in Ukrainian (“чоловік^man-NOM.SG побачив^saw-PST собакУ^dog-GEN.PL”) the last letter of the object (dog) makes it genetive, and therefore the object.