WT recommended Streamlit • The fastest way to build and share data apps

“Streamlit turns data scripts into shareable web apps in minutes. All in pure Python. No front‑end experience required.”

Sample demos:

• BERT Keyword Extractor · Streamlit
• Time series annotations · Streamlit

Other examples are in the Gallery • Streamlit

Awesome Streamlit is freaking awesome.

Connects well to explorables etc., and would replace about 30% of my use-cases for jupyter notebook. Especially random small demos, ones I don’t do because I don’t want to mess with interactive graphs in Jupyterlab or re-learn d3.

Speaking of d3 - I should rewrite Flappy Words in it!

Wrote this small wrapper script that (if a global USE_TQDM parameter is set) uses pretty tqdm lists on lists that have enough elements where it matters. I think I’ll be reusing it.

So when enabled, it will tqdm a list of 150 elements but won’t tqdm a list of 99 elements.

To use:

for el in _tqdm(whatever_list_thing):
	do_stuff_to(el)

Function:

def _tqdm(list_like: Sequence, iflarge: bool = False, lsize: int = 100):
    """Use tqdm if it's on, optionally based on length of list.
    Args:
        list_like: thing to iterate.
        iflarge (bool): If on, will use tqdm only for large lists
        lsize (int): anything more than this is 'large'
    """
    if USE_TQDM:
        if not iflarge:
            return tqdm(list_like)
        else:
            # Count size only if it doesn't mean iterating an iterator
            if isinstance(list_like, Sequence) and len(list_like) > lsize:
                return tqdm(list_like)
    return list_like

git - GitLab: You cannot push commits for . You can only push commits that were committed with one of your own verified emails - Super User

Setting is per-project and lives in push rules:

I set the credentials to the right ones the usual ways:

git config user.email "my@verified-ema.il"

But the commits were still using the old identity.

Solution to fix the last commit by only setting the author to the new / current one:

git commit --amend --reset-author --no-edit

Hugo summaries are weird.

.Summary returns whatever summary it has, which is either the .. more .. tag, then everything before it gets returned including formatting, or whatever is set in the settings as summary length, while removing markdown formatting.

There was no easy way to get an auto-summary with preserved formatting, except manually adding stuff.

What I really wanted is to truncate posts manually when needed, and leave the rest in full by default while preserving formatting.

Setting the limit to infinite made .Summary returned the full post with stripped formatting.

(I needed this for footnotes in multiple posts all on the home page, they got mixed up and there were no clean solutions. The blackfriday renderer could fix this, but not the default goldmark, which I’m using for some layout issues it does better.)

After googling for better ways to truncate with preserved formatting, found Summary .Render · Scott Willsey

It has this code for a better summarization:

    {{ if gt ( sub (len (plainify .Content)) (len .Summary)) 10 }}
    {{ .Content | replaceRE "<sup.+>.+</sup>" "" | safeHTML | truncate (len .Summary) }}
    <p><i>(<a href="{{ .RelPermalink }}">Read More</a>)</i></p>
    {{ else }}
    {{ .Content | safeHTML }}
    {{- end -}}
    {{- if .Params.linkurl -}}
    <p><a href="{{ .RelPermalink }}"><i class="fas fa-level-down-alt fa-xs"></i>&ensp;Permalink</a></p>
    {{- end -}}

First up is an if statement that checks to see if the post even needs to be truncated into a summary or not, or whether it’s short enough to just show the whole post.

this works nice, but I wanted no summarization for

{{ if .Truncated}}
{{ .Summary }}
<p><i>(<a href="{{ .RelPermalink }}">Read More</a>)</i></p>
{{ else }}
{{ .Content | safeHTML }}
{{- end -}}
{{- if .Params.linkurl -}}
<p><a href="{{ .RelPermalink }}"><i class="fas fa-level-down-alt fa-xs"></i>&ensp;Permalink</a></p>
{{- end -}}

and setting the summary limit to infinite.

What this does is:

1. If Hugo thinks that the post is .Truncated, return its summary. This means that the POST IS TRUNCATED ONLY IF I MANUALLY ADD THE MORE TAG, because the auto-summary limit is set to a big number.
2. If hugo doesn’t think so (= no more tag explicitly added by me), then return the usual content. I didn’t change that part at all and the safeHTML is prolly not needed there but whatever.

When downloading a Google Colab (and prolly a classic Jupyter Notebook) as .py it preserves the plain-text cells as python comments!

From No more disk space: How can I find what is taking up the space? - Ask Ubuntu, run this as root:

du -cha --max-depth=1 / | grep -E "M|G"

The grep is to limit the returning lines to those which return with values in the Megabyte or Gigabyte range.

Next one would be /var etc.

Then there’s ncdu and friends too.

From SO’s credentials - How can I save username and password in Git? - Stack Overflow:

git config --global credential.helper store

Then on the next git pull the credentials entered will be saved in plain text on disk.

Wow. WOW.

Wrote a program accepting a LONG --yes_delete_all_data_completely, without a short version, to make sure no one does an error and deletes everything.

Today I mistyped a --y parameter, it started in the mode above.

Then I learned that argparse does prefix matching.

python - How to share global variables between tests? - Stack Overflow:

import pytest

def pytest_configure():
    pytest.my_symbol = MySymbol()

allows then to use pytest.my_symbol elsewhere, it’s a part of global pytest namespace now.

That said, fixtures are still the preferred way it seems (todo - how are they shared between files?)

Playing with
Spacy and it’s as nice and I thought it’d be.

Interesting bits and general dump of first impressions:

• NER @ CLI: Custom-named entity recognition with spaCy in four lines: spacy can:
  • Convert NER datasets from conll
  • while outputting nice status info
• Has a “Debug data” tool that allows to validate train data (and other stuff): Command Line Interface · spaCy API Documentation
• Can do rule-based matching, linguistic features, Rule-based matching · spaCy Usage Documentation
• Some support for Transformers, including allegedly all HuggingFace ones!
• Both Doc and Span are heavily token-based, including for NER stuff. Can’t set a sub-token entity, for example.
• But Doc.char_span() supports creating a Span based on characters and various alignment methods! Doc · spaCy API Documentation
  • And of course we can get the character offsets from the span itself
• You can merge/split tokens: Linguistic Features · spaCy Usage Documentation
• The Example class for individual training instances can do neat stuff with BIO mapping, aligning of NER tokens etc: Example · spaCy API Documentation

Okay, this blew my mind. Taskwarrior can have lowercase +t tags, along with the +T-uppercase ones I’ve been using my entire life.

Wow.

When writing a function requiring a --yes_I_know_what_this_means_delete_everything and writing a warning message with tens of exclamation points, I decided that ASCII art is the better way to go.

Found this: Caution Text Art (Copy & Paste) - textart.sh

Allows even changing backgrounds from spaces to _s etc.!

textart.sh has a lot of topics and allows basic customisation of the arts themselves.

(Can’t find a single ASCII art piece with an artists' signature though, which kinda worries me. And the dynamic scrolling without a way to see a list of all results…)

“pic"related:

                                                                                        
                ░░░░                                                                    
                                                                                        
                                            ██                                          
                                          ██░░██                                        
  ░░          ░░                        ██░░░░░░██                            ░░░░      
                                      ██░░░░░░░░░░██                                    
                                      ██░░░░░░░░░░██                                    
                                    ██░░░░░░░░░░░░░░██                                  
                                  ██░░░░░░██████░░░░░░██                                
                                  ██░░░░░░██████░░░░░░██                                
                                ██░░░░░░░░██████░░░░░░░░██                              
                                ██░░░░░░░░██████░░░░░░░░██                              
                              ██░░░░░░░░░░██████░░░░░░░░░░██                            
                            ██░░░░░░░░░░░░██████░░░░░░░░░░░░██                          
                            ██░░░░░░░░░░░░██████░░░░░░░░░░░░██                          
                          ██░░░░░░░░░░░░░░██████░░░░░░░░░░░░░░██                        
                          ██░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░██                        
                        ██░░░░░░░░░░░░░░░░██████░░░░░░░░░░░░░░░░██                      
                        ██░░░░░░░░░░░░░░░░██████░░░░░░░░░░░░░░░░██                      
                      ██░░░░░░░░░░░░░░░░░░██████░░░░░░░░░░░░░░░░░░██                    
        ░░            ██░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░██                    
                        ██████████████████████████████████████████                      
                                                                                        
                                                                                        
                                                                                        
                                                                                        
                  ░░

That which can be destroyed by the truth should be.

(Litany of Hodgell) ¹

“Almost no one is evil; almost everything is broken”.

(Litany of Jai)

простите за очевидное, но все-таки по поводу «плохих людей не бывает, бывают несчастные/те, кому плохо/озлобленные/которые не знают правду» и так далее.

так вот:

ПЛОХИЕ
ЛЮДИ
БЫВАЮТ

(@lizafocht on Twitter)

As I write this, it’s 1 month 11 days 22 hours 24 minutes 44 seconds since war started.

If you asked me two months ago about the worst period in my life, I’d have had an answer. There was a very weird year, the worst time in my life and the best time in my life. It’s a miracle I’m alive, but it was a formative year, beautiful, scary, hard, surreal, it left scars and created the strongest memories I have.

If you asked this question now, it’s a tie between that time and the last month. And about this month, any positive sides I can find are either exceptions to problems the war itself created (“my family and friends are all alive, only few of them got seriously physically hurt”) or don’t pass the smell test.

Ukraine is united as never before - nice. Would I take the “Ukraine is as divided, fragmented and unstable as ever, but there is no war” option? Yes, in a heartbeat.

As I write this, the consensus on Twitter is that the Battle of Kyiv™ is over. I like this summary by Tomi T Ahonen, good storytelling and mostly factually correct.

As the danger to Kyiv diminished about 2 weeks ago, I started analyzing less and breathing more.

My brain started to work through a backlog of emotions and questions like “What does it mean now?”

This post is not about a specific video. This post is about how researching it made me realize what should’ve been clear weeks ago.

At some point you have to stop and think whether gathering more information is likely to be useful.

This post is how about I missed that point.

Moral fashions², moral judgements

Since this blog post floats into Rationalist / Rationality community / Lesswrong / SSC territory, why not go all in and emulate the whole aesthetic:
Epistemic status³: High about my feelings, medium about the rest (didn’t make an effort to check), medium-to-low about the soundness of the reasoning employed (I did my best, but no way to double-check)
Epistemic effort: First stream-of-consciousness draft with light editing afterwards

List of reasons critical thinking is hard

Both Putin, the Russian army, Russian soldiers, the people in Russia who support this (50%-80% depending which poll you pick) are really easy to hate now. Europe/USA, most media I read, people I follow are of the opinion that of course Russia is the agressor. The stated reasons for the attack are fake, the attack is unjustified.

If my side is obviously right, the other side is the devil, if it feels pleasant to hate them – all my alarm bells start ringing, here be dragons biases, and I have to be more vigilant than usual.

There are at least 70 millions of people⁴ in Russia feeling as certain about the opposite.

In this specific case it’s even harder: I’m clearly being manipulated by both sides⁵ and, well, I walked on the streets that are now being bombed.

But my judgement of the facts and of responsibility/blame should not change because of that.⁶

Two core values and an emotional backlog

The two core values most important to me, the ones I see as defining my identity, are:

1. Being curious, comfortable with unknowns and probabilities, do my best to rationally come to the truth and then follow it wherever it leads me
2. Being open-minded and understanding

I’m confident that the first one is with me, no matter the emotional stress I’m under. The second one, open-mindedness and understanding, is having a much harder time.

Can I blame people in Russia for believing what the TV says (for instance)? If yes - then how can I blame them for feeling and acting according to what they think is true? How does this translate to Russian soldiers on the ground?⁷

Right as I was intellectualizing away my pain reflecting about how justified are negative emotions directed towards Russians (in the context of probabilities, availability of information, collective responsibility, reliability of polls, …) the pictures from Irpin' and Bucha started appearing on Twitter.

This video/tweet⁸ from April 1:

PSYOPs and looking for truth in a sea of propaganda

My priors say that the Ukrainian Army is much less likely to lie, as measured by the times I could prove they did. And if RU and UA directly contradict each other, going with the UA version is usually the better choice.

That said, propaganda everywhere and with differing patterns.

Ukrainian false and false-y statements

A short tour through the Zaporizhzhia NPP and Ukrainian propaganda

If anyone ever attacks me for even questioning official UA messaging, I mention how official Ukrainian accounts tweeted screenshots from games and how the Ukrainian army successfully took cities it had never lost in the first place. Both happened at least twice and I have the screenshots to prove it.

But the main sin has been to blow things out of proportion.

When the Russians were attacking the Zaporizhzhia nuclear power plant, the minister of foreign affairs tweeted this:

Russian army is firing from all sides upon Zaporizhzhia NPP, the largest nuclear power plant in Europe. Fire has already broke out. If it blows up, it will be 10 times larger than Chornobyl! Russians must IMMEDIATELY cease the fire, allow firefighters, establish a security zone!"⁹

Demonstrably false already then, but his job as MFA is not to give people up to date info about NPPs. It’s getting people outraged about Russia and using that to help Ukraine’s cause.

Then Українська Правда’s Sources™®©¹⁰ said their plan is to rig the NPP with explosives to blackmail Europe. I mean why yes of course that’s the most likely sequence of events.

… then I’m stuck to reading screenshots of random people’s Facebook comments on Telegram to make sense of how dangerous this actually is, looking at third-party websites to measure radiation levels, and seeing whether the city closest to the NPP will get evacuated.

All while watching the NPP burn on the livestream, thinking .. what a glorious time to be alive! I’m watching what might become the defining event of this decade, as a Livestream, with 69k other people.

As I was watching their credibility in my eyes erode in the first days/weeks of the war, I was sad because “are we falling on their level?" and because I couldn’t rely on my gov’t for any factual information.

How I stopped worrying and started to love the propaganda for internal and external consumption

…. But I kept noticing the positive effects.

Uplifting and calming things were what most Ukrainians needed the most at the beginning, and we mostly got it.

We all hate Russia but did you know that everyone else hates Russia too? We are winning and everyone is on our side. Most Russian soldiers have never seen a washing machine.

Don’t panic and carry on, we got this. We all got this.

At the same time, for the rest of the world:

Russia is Satan and you’re next. We’re fighting and dying for democracy and freedom. You ignored (or financed) Putin before, this is your fault. We are really brave and cool and destroy Russian drones with pickled tomatoes, and steal their tanks with tractors. Imagine what we could do with better weapons. You want to be seen supporting us.

Externally, USA/UK/EU news were doing whan news usually do - amplifying some of the extreme views (voices of reason won’t get shared as much), with predictable effects.

I accepted that lying and especially misrepresenting how good/bad stuff is borderline acceptable if a war is going on.

And I resigned myself to the fact that internal VS external messaging is now a thing I’ll have to track in my own media too.

That the truth might be as subtle as not evacuating the city closest to the NPP, while the external messaging is framing the (already awful and irresponsible) attack on an Ukrainian NPP as an ATTACK ON EUROPEs security by knowingly shooting at a reactor.

Extraordinary claims require extraordinary evidence, especially if you have a story of making up extraordinary claims to for sympathy and weapons

And, of course, given the drama they create every time something happens, and given that Russia is Satan and the ultimate evil, of course I’ll believe without additional proofs that Russians systematically kill civilians en masse and leave the bodies on the street for weeks!

Russian false and false-y statements

I’m skipping Russian propaganda because A LOT has been written about it, I can’t add much to the conversation there.

There’s the country saying that the NPP exploding will be 10 times worse than Chernobyl, and there’s the country we all watched, live, shooting at the NPP.

Dissecting news of the first country is much more interesting (the second country is just “we didn’t shoot at it, and if we did, it was not dangerous at all”), but I mean, we know on whose side I’m on.

Back to Bucha

April 1, I see the video from Bucha.

All the eyewitness reports I’ve read and suspended judgement on instantly come to mind.

Nevertheless, before I allow myself to be too angry or sad, I need to be sure that:

1. it happened
2. one party is clearly to blame for the deaths. (I mean, It’s been a war zone for weeks. Could it be unlucky civilians caught in the crossfire?)
3. That party isn’t Ukraine, by intent or by error

There were allegations that the video was staged because of a hand that allegedly moves, they were happily and quickly debunked¹¹.

Okay, now I’m certain that it’s dead people, not actors. But dead people on the street (or mass graves) in a war zone doesn’t prove much by itself.

I kept watching that video again and again. I couldn’t think of any realistic scenario that would explain everything I see.

What situation could create such a video? Why so many dead people in civilian clothes in the middle of the street? When did they die, and how was the weather back then?¹² Did they die together? How?

How does one die suddenly, riding a bike in the middle of a large street?

… So many people dead on that street implies they felt safe enough to be there in the open? Not just safe, but “riding on a bike in the middle of the street”-safe.

What happened in Bucha?

On Telegram, the usual Ukrainian and pro-Ukrainian English-speaking sources were all certain the Russians killed them and were asking too few questions.

Russian places were silent. (Days later, RU propaganda got their talking points in order and the Russian MFA made them official¹³, but on April 1 no one knew much.)

I found only one post in a pro-Russian group touching and trying to explain the topic¹⁴:

После того как наши войска отошли от Бучи, ВСУ начали накидывать по городу артиллерией.

Итог данного дебилизма- убитые мирные жители и разбитые дома! Утырки воюют со своими же гражданами. Но в их сми это будет звучать так-Русские отходя убивали мирных.

Allegedly, it was UA artillery that was hitting Bucha after the Russian Army left.

Okay, this is something I haven’t heard before. A sudden artillery attack would explain a lot of people on the street feeling safe all dying together. Some directly because of it, some killed by shrapnel.

Apparently firing your ammo to cover your retreat is a thing.¹⁵, so not trying to do attribution now, but I can try to figure out whether it was artillery at least.

As I was watching that video on 0.25x speed for the Nth time, trying to see damage consistent with an artilery strike (impact craters? Burned stuff?), it hit me.

How many videos with corpses do I have to watch?

Why am I doing this?

I’m doing this to make sure I believe beyond a reasonable doubt that the Russian army did this.

Why?

To make sure my pain and my anger directed at Russia is 100% justified. I mean I know the last time I did this it ended up being Russia’s fault, and generally this is a rule at this stage, but what if …

Why do I feel the need to prove to every single crime they do, before allowing myself to be angry at them?

Just whom am I protecting?

Of course not all Russians support Putin, and of course the horror of this war is not entirely caused by the Russian army.

But since the beginning, the evidence is inexorably pointing in one direction.

Most of the people I’m afraid to hate unjustly wouldn’t question the established narrative and watch a video with dead people to see if it looks like artillery.

At which point can I just be angry?

I never seriously expected to be wrong here, but I did my homework and my due diligence. Just in case it’s my government slaughtering people I want to know this.

I could justify analyzing the hell of everything when I gave it 80/20 chances. Now at 99 / 1, are the chances that something changes worth the emotional strain and trauma I go through to gather more evidence?

… at that specific instant, I decided that it’s enough.

Brave new world when you believe Ukrainian propaganda by default

Reading news at a high abstraction level is more than enough for me from now on.

Anyone who gives Russia the benefit of the doubt by default at this point is either stupid, has no morals, or is so deep in the rabbit hole I’m not going to save them.

How are my priors doing?

I know which side I’m on, so for the sake of balance I was especially critical on the news sources I “expected” to believe. I think I overdid it. I should have trusted Ukrainian mainstream sources more. (And by blindly trusting them, I’d have saved myself a lot of heartache while coming to the same conclusions, but I’m not going to think about that.)

I completely ignored the bits about the Russian army conscripts suffering from frostbite (Russian army unable to fight in … Winter? Really?) till someone noticed sneakers in an unrelated video.¹⁶

I didn’t trust the UA gov’t when it claimed it shot down at least one IL-76 but released no pictures on that day or ever¹⁷ but maaybe the wreckage was in RU-controlled woods till now?¹⁸ Still no pictures or anything, assuming this didn’t happen.

Even the wild official Ukrainian numbers about Russian losses don’t seem so wild anymore.

People I know personally that had first-hand knowledge about civilian corpses in the woods, clearly taken there to be executed. I chalked it up to isolated incidents. … There were a lot of eyewitness accounts about horrors happening in cities near Kyiv on RU-controlled territory, too horrible and propaganda-like to believe, so I didn’t.

Guess what.

What now?

What now?

I don’t know.

I’m not used to feel so much hate and anger at anyone for so long, ever. And to feel after all of this time that it’s honestly justified.

I don’t have the energy to give Russia the benefit of the doubt, and if I had it - I’d find better ways to use it.

I spent more energy “doing my own research” than most of them have spent doubting themselves, I’m emotionally burned out. My decisions took me here, but the energy and emotions I do have left are best spent not looking into fakes that are easier to create than to debunk.

And about the normal Russian people, who too are victims of Putin’s war - … Guilty by assumption from now on, statistically likely to support this war, and if not - it’s their job to spend energy to make me think otherwise.

I’ve always hated the phrase “Putin’s war” for obvious reasons. It was Putin who massacred the people in Bucha, Irpin', and countless smaller unnamed towns and untold tragedies, it’s Putin who drops the bombs, it’s Putin who organizes demos (Mehrere hundert Fahrzeuge: Autokorso mit russischen Fahnen fährt durch Berlin - Berlin - Tagesspiegel).

People are dying in Ukraine, and I have no energy or empathy for the Russians who are feeling disadvantaged or discriminated against.

I have no energy or empathy left for anyone now

EDIT: Saw the satellite pictures confirming that the bodies actually were there for weeks, throughout the entire Russian occupation.

My God.

If I were to pick the one claim of UA Gov’t that I thought the most likely to be false, it’d be “they don’t pick up the bodies, leave them on the street for days”, and I’m looking at satellite pictures proving exactly that.

“I couldn’t think of any realistic scenario that would explain…” All those people died at different times and were just not picked up. This is one scenario I have just not considered. Explains the video perfectly.

Pure, immense and heartless evil. Stronger evil than I could imagine existing, and as good of a proof as I can imagine.

I was not ready for this. Neither it existing at all, nor it existing 20km from my home.

EDIT2: The father of a friend lived in a village around Bucha during the Russian occupation. He confirmed that in his town, the Russians' behaviour was identical. As in he saw them doing exactly the kind of stuff they allegedly did in Bucha. Killing families with children that were trying to escape, randomly shooting people, not allowing to bury the dead, all that.

Just leaving this here just as a reminder for myself to be thankful that that’s the closest confirmation I have.