Digital Garden

My Digital Garden - after an idea from so many others. Because I could not express it in any better way, I quote Joel Hooks

While not everybody has or works in a dirt garden, we all share a familiarity with the idea of what a garden is.

A garden is usually a place where things grow.

Gardens can be very personal and full of whimsy or a garden can be a source of food and substance.

(...)

Like with real gardens, our digital gardens are a constant ebb and flow towards entropy.

Weeds take over. Left untended the Earth will reclaim what belongs to it.

The same is true for our digital gardens here on the internet.

I try to work with my garage door up, so feel free to have a look.

Picture of me (non-cc) This is me, in case you wondered.

Except where otherwise noted ("non-cc"), is this work by Raphael Sprenger licensed under CC BY-NC 4.0

Algorithms

The first entry on this page should be a link to Nick's damn cool algorithms. Because algorithms can be damn cool. The rest of links on this page come in no specific order (for now).

Artificial Intelligence

These are mostly reading notes from the wonderful book Artificial Intelligence by Melanie Mitchell.

Monte Carlo Tree

The path of a tree is followed from the root towards one leaf. The path is chosen at random. At every branching node, a random branch is chosen. This approach is useful for huge decision trees, for example in the game of Go. Evaluating every path is computationally not feasible. When evaluating a big amount of paths, it is likely to find a good enough solution. In contrast, when evaluating all paths, the optimum solution will be found.

Genetic Algorithms

Genetic algorithms can find a good solution in a non-gradient environment. This is possible whenever a program can follow a rule set. Rules are changed for every iteration and the outcome is evaluated. Rules have to have a binary representation, so that a binary String can be genetically modified. The general pattern is:

  1. Initialisation with a random rule set
  2. Select best performing rules
  3. Apply genetic operators such as Crossover and Mutation
  4. Terminate under a condition, for example after a number of rounds or at a cost-threshold

Q-Tables

First a word about what Q-Tables are good for. Whenever a program has to reach one single goal, and the way to achieve this goal is unknown, Q-Tables can do the trick. An intuitive example is to find the exit of a maze: One final reward without the evaluation of intermediate steps.

There are States the program can be in, the rows of the Q-Table. For any given State, there are Options the algorithm can choose from, the columns. To stay with the maze example, if the player is in a State of being placed in the middle of an open field, then the options are to move in any direction (North, East, South, …).

At first, the moves start from a random location and happen randomly in order to find the reward. Only the action that led to the reward gets stored with a high number of points in the Q-Table (for example 100 points. This is up to the fine tuning of the use-case). For example, the State is “One step straight ahead towards the exit” and the chosen Option was “Move straight”, which unlocks the reward, will lead to a high number in this State-Option combination.

In the second round, not finding the reward counts, as the solution was already found. More important is, how does the algorithm find the State that led to the reward. In the example, what comes before standing right in front of the exit? If this pre-reward step was found, another high number is stored in the Q-Table for the respective State-Option combination.

This pattern of finding the pre-step to every previous round continues. There is one exception: With a low chance, the algorithm can choose an Option that is not associated with a (high) number of points. In that way, it will find potentially better solutions and try moves it never tried before.

Deep (convolutional) networks

Whenever the output for an input of n values is known, this algorithm strikes. That means the input needs to be quantifiable and the output has to be known already, at least for a training set. Between Input and Output is a “deep” or “hidden” layer. This layer multiplies the Inputs and forwards the result to the Outputs. Every node in the hidden layer is connected to every Input and every Output.

When training the network, the multipliers at the hidden layer(s) are gradually adapted to best match all Inputs to their expected result.

This technique is conceptually easy to understand, but extremely hard to master, as it requires fine-tuning and a good intuition on how to apply changes.

Reenforced learning

As well known as “self learning”. Good behaviour is rewarded, while bad behaviour is ignored. An example will give an intuition on how reenforced learning works in practice. AlphaZero, the Go playing machine, uses a combination of deep neural networks and Monte Carlo trees (at least one of the earlier versions of AlphaZero did. The approach was changed later). The deep neural network suggests, which Monte Carlo trees to try. The best performing Monte Carlo tree is fed back into the neural network, so that in a next iteration, the neural network can make even better suggestions which Monte Carlo trees to try for a similar situation. This technique only works if the output of the round can be clearly scored, so that the neural network only trains itself with good inputs.

Natural language processing

A language is natural if humans speak it.

Recurring neural networks

An early approach for natural language processing was the use of recurring neural networks. A recurring neural network feeds back its own values into the network, together with another input. Let’s say, every word in the dictionary gets a value. A sentence, now formed of a String of values, is fed into the recurring neural network, one at a time. To keep track of the sentence’ state, the output from the previous words are fed back into the network together with the value of the current word. This technique is also called one-hot input, as only always one word is active at a time. After years of experimentation, the results are not as good as with other techniques for language processing.

Long short-term memory

Unfortunately not furhter explained in Mitchell's book and only briefly referenced in Translation. The LSTM addresses the deficites of sequential inputs in recurring neural networks. The cell state is the information transportation system from step to step. The information can stay the same over time or gradually change. It is used to generate unit output together with other operations explained below. LSTMs have four hidden layers. The first layer decides about which information to "forget" from cell state. The next two layers figure out, what to add to the cell state. The fourth layer renders the output from the cell state and the cell's input for this particular step.

Word vectors

Before explaining how word vectors are generated, I start with a motivating example: Word vectors give the similarity of meaning between to words. Words that are more closely related have a shorter distance than words that have very different meanings. A surprising property is the distance between word correlations. Measuring the distance Man -> Prince yields almost the same distance as Woman -> Princess. To make use of this property, the word cloud can answer questions. Asking Fish -> Water; Bird -> ? will give the answer Air. This is possible by measuring the distance Fish -> Water and then checking what lays in the same distance from the word Bird. Google released a 300 dimensional word cloud called word2vec. How did they create it?

Start with a neural network with one hidden layer containing 300 (hidden) units. The input and output layers have a unit for every word from the dictionary, so it is a rather huge input/output layer. From a sentence, feed adjacent words into the network. For example, feed Burger and train for Restaurant. Also train for the opposite case. Once the training is done, extract the word vector. To do that, every word from the dictionary is “lit up” in the network. The “illumination” of the hidden layer will mark the position in the 300-dimensional space.

It is possible to project a higher-dimensional space into two or three dimensions, so that humans can visualise and inspect them more easily.

Translation

Traditionally, translation systems were composed of human made rules. Google drastically changed this in 2016 with the release of the machine learning translation system.

The outline of this idea is to have an encoder- and a decoder neural network. The encoder is fed with a sentence, using the one-hot approach (see recurring neural networks) . At the end, a stop sequence is used and the activation of units in the network is extracted. This activation is fed into the decoder network for the target language, which converts the sequence of values back into a sequence of words. In contrast to simple neural networks, the units in these encoder/decoder networks are made from “long short-term memory” LSTM units. These units account for inputs that come over time and autonomously decide which inputs to generously “forget”. The length of the sentence from the original language can be different from the length of the resulting language.

Understanding text

How to evaluate that a machine can really understand the contents of a sentence, in contrast to simply react on it in a trained way? A powerful approach is to ask a question like “The couch did not fit through the hallway, because it was too narrow. What was too narrow?”. A machine neither understanding what a couch or a hallway is nor knowing its dimensions and relations, will never be able to answer this question. This test has an infinite amount of possibilities as new questions can simply be created: “Water was filled from the bottle into the glass until it was empty. What was empty?”. By creating enough questions in this pattern, only a machine that clearly understands text can score at about 100%, otherwise it will be roughly 50% for random choices.

References

Long short-term memory (LSTM)

Aviation

VFR flight preparation

Automation

Digital Garden

One of my needs is an easy setup for gardening. I need to be able to write notes on all of my devices, including my phone. gitea has a built-in Markdown editor, so I can edit text and work with git at the same time. The content is rendered by mdBook.

Using git hooks

Git hooks are run on the server while commits are being applied to a repository. This is a low resource approach because it only runs when the garden received an update. It is quite limited to the machine that hosts the git repository though.

Gitea has an option to manage git hooks. I read they only run if Gitea manages the SSH keys. I didn't test with local SSH keys. To enable git hooks, change the app.ini and add DISABLE_GIT_HOOKS = false, default is true. Restart gitea. Caution! : Users who can access git hooks can run arbitrary commands on the server. Only allow this option if you can fully trust the user.

Go to the repo settings and define the post-receive git hook:

#!/bin/sh
echo "removing old deployment"
targetdir=/yourwebdir
cd $targetdir
rm -rf *

echo "running deployment"
git --work-tree=$targetdir --git-dir=/path-to-your/garden.git checkout -f

/usr/local/bin/mdbook build
rm *
rm -rf src/
cp -r book/* .
rm -rf book/
echo "new mdbook built and deployed"

The directory /yourwebdir needs to have read/write permissions for the user running gitea which is usually git.

Using deploy-keys and a cron job

On my server I created a user

useradd --system --create-home --home-dir /var/lib/mdbook --shell /usr/sbin/nologin --comment "mdbook Digital Garden" mdbook

For this user, I create a new ssh-key. When asked where to store it, I choose /var/lib/mdbook/.ssh

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

In gitea, I use the option for deploy-keys and add above key. Deploy keys are read only for a specific repository.

To /var/lib/mdbook I add a build.sh script

#!/bin/bash
cd /var/lib/mdbook/garden
/usr/bin/git pull -q
/usr/bin/mdbook build -d /usr/share/garden /var/lib/mdbook/garden

And a cronjob in /etc/crontab

* * * * *  mdbook  /var/lib/mdbook/build.sh

Books I read

No particular order yet

Bedside table

Currently reading

  • Blutorangen - Verena Boos

English

Science

  • A Philosophy of Software Design - John Ousterhout
  • Artificial intelligence - Melanie Mitchell (my reading notes)
  • Six impossible things (The 'Quanta Solace' and the Mysteries of the Subatomic World) - John Gribbin
  • Designing Data-Intensive Applications (The big ideas behin reliable, scalable, and maintainable systems) - Martin Kleppmann
  • Complexity (A guided tour) - Melanie Mitchell (See also Machine Learning’s ‘Amazing’ Ability to Predict Chaos)
  • Game feel (A game designer's guide to virtual sensation) - Steve Swink

Fiction

  • The sound of rain - Paul Honkani
  • House of leaves - Mark Z. Danielewski
  • The garden of bad dreams - Christopher Hope

German

Science

  • Grenzen der Demokratie (Teilhabe als Verteilungsproblem) - Stephan Lessenich
  • Ethik in KI und Robotik - Chrisoph Bartneck, Christoph Lütge, Alan Wagner, Sean Welsh
  • Der Sprachverführer (Die deutsche Sprache: was sie ist, was sie kann) - Thomas Steinfeld
  • Lean Brain Management (Erfolg und Effizienzsteigerung durch Null-Hirn) - Gunter Dueck
  • Schwarmdumm (So blöd sind wir nur gemeinsam) - Gunter Dueck
  • Eine kurze Geschichte der Zeit - Stephen Hawking
  • Eine kurze Geschichte von fast allem - Bill Bryson
  • Das Neue und seine Feinde (Wie Ideen verhindert werden und wie sie sich trotzdem durchsetzen) - Gunter Dueck
  • Keine Ahnung von der Materie (Physik für alle!) - Hans Graßmann
  • Die Philosophie des Abendlandes - Bertrand Russel

Fiction

  • Ein ganzes Leben - Robert Seethaler
  • Der Sonntag, an dem ich Weltmeister wurde - Friedrich Christian Delius
  • Die Besteigung der Eiger-Nordwand unter einer Treppe - Max Scharnigg
  • Die Entdeckung der Langsamkeit - Sten Nadolny
  • Der Kampf geht weiter! (Nicht weggeschmissene Briefe) - Harry Rowohlt
  • Pu der Bär - A. A. Milne (Audiobook - Harry Rowohlt)
  • Mir kocht die Blut! (Die wunderbare Welt der Querulanten und Sonderlinge) - Roger Willemsen (Audiobook - Anke Engelke, Roger Willemsen)
  • Der leidenschaftliche Zeitgenosse (Zum Werk von Roger Willemsen) - Insa Wilke
  • Die Enden der Welt - Roger Willemsen
  • Momentum - Roger Willemsen
  • Wer wir waren - Roger Willemsen
  • Afghanische Reise - Roger Willemsen
  • Deutschlandreise - Roger Willemsen
  • Bangkok Noir - Roger Willemsen
  • Die wundersamen Irrfahrten des William Lithgow - publisher Roger Willemsen
  • Das süße Gift der Sünde - publisher Roger Willemsen
  • Auf Schwimmen-zwei-Vögel - Flann O'Brien
  • Die Hauptstadt - Robert Menasse
  • Das Muschelessen - Birgit Vanderbeke
  • Der satanarchäolügenialkohöllische Wunschpunsch - Michael Ende

Dutch

Fiction

  • Het smelt - Lize Spit

Publisher

Cheat Sheets

Complexity

Notes from the book A Philosophy of Software Design, which is about managing complexity in software designs.

Causes of complexity: Accumulation of dependencies and obscrurities. A single occurence of these doesn't make a system complex. It is always the accumulation of these factors.

Results of complexity:

  • Change amplification (A small change requires a big refactoring)
  • High cognitive load
  • Unknown unknowns (sometimes you don't even know what you need to know to safely apply a change)

Which lead to:

  • More code modifications for new features
  • requiring more time to gather information
  • risk of modification

Digital Gardens

Digital Gardens live somewhere in the space of personal wikis and experimental knowledge systems, as Mappletons describes it in a twitter thread. This garden runs on gitea and mdbook.

As we may think

As we may think is the title of an article published by The Atlantic in July 1945, written by Vannevar Bush. Bush makes a prediction about how we will use technology in the future as an external thinking device. He elaborates lengthy on the feasability of his suggestion, something that would not be necassary today, as we all know that microelectronic storage of information, search-algorithms, text-to-speech and speech-to-text, cameras and displays are part of the daily life for many. In Bush's oppinion, mature thought can't be replaced. In contrast, he believes that creative thought and repetitive thought can be supported by mechanical aids. (I am sure, the word mechanical was used in lack of foreseeing an electronic revolution, enabled by the invention of transistors.) He calls such a mechanical aid a Memex. A machine that is equipped with research papers, encyclopedias and so forth. Furthermore it can be fed with own input like writing, pictures and voice recodings. The user can build so called trails of information. That is, whenever they see two references on the screen, the user hits a button and joins the two articles. These joins can grow to a trail. Bush even speaks about branching out, whenever a trail needs to follow a sub-topic.

Managing digital stuff

The Science of Managing Our Digital Stuff is a talk given by Ofer Bergman at Microsoft. The way it is presented is not really easy to follow due to sound and image quality. The content is dense and interesting though. Bergman argues that managing data inside file hierarchies is not the best approach. Files get hidden in nested folder structures. Files that could logically be sorted into multiple places can only take one place. There are no conventions for folder structures and people tend to forget even their own structures after a few weeks. Modern operating systems offer better systems than folder structures. The two prominent ones are quick full text search and file tagging. In his research, Bergman found that most people still prefer navigating through a folder structure, even when they were free to choose one of the other systems. Is it just habit? Bergman found that formulating a search query requires a context switch whereas navigation activates a different part in the brain. The context switch makes people forget what they worked on initially. His research is backed by fMRI scans where he could prove the activation of different parts of the brain. Bergman concludes that we need improved navigation instead of improved search and tagging. He developed prototypes to showcase how it could look like. An address book on a phone would show the most dialed contacts on top and keep all other contacts below. A shared drive would highlight the subjectively most important documents on top and render all other files below, slightly greyed out.

Gardens

Patterns

  • Open Transclude Pattern for including iframe previews with plain HTML/CSS/JS
  • Andy Matuschak Thoughts about knowledge and thinking (he has a nice garden too, see above)
  • Freeing the web from the browser with open hypermedia
  • Book review: How to Take Smart Notes? gives an overview of the book's concepts and linking it to academic writing, including criticism of academic writing like the increasing pattern of "publish or perish", quoting a study that the academic output doubles every 5 years.
  • Digital Tools I wish existed Jonathan has some thoughts about his workflow of processing information, and what he would need to do so more efficiently.

Private Garden

There are some notes, ideas, thoughts, ... that I want to keep private and others, for example e-books, that I have to keep private for legal reasons.

I think about creating a private garden with the same setup as this public garden, but behind a basic auth and under a different sub-domain. That way, I can cross-reference my notes.

Unfortunately, this would lead to a lot of "not for public"-links. Something, I don't want my visitors encounter too often.

Distributed Systems

The cloud native maturity model

Cloud native maturity model

Consistency Models

(Summary of Strong consistency models)

Concurrent Histories

With several processes, I write to one common storage, that can be spread across multiple nodes.

Light Cones

There is a time slot for the task to write to acknowledge the write. Same for read access.

Linearizability

If a write task was acknowledged, the result is visible for all reading clients.

Sequential Consistency

If something was given to the system in a certain sequence, the visibility will be in the same sequence. Example: posts (A, then B) to a social network are not visible immediately. But if A becomes visible, only after that, B will become visible.

Causal Consistency

Only if operations are dependent from another, they will be given in the correct sequence. For example can read access only be allowed after certain conditions are met.

Serializable Consistency

Is weak because reads and writes can travel to the past and to the future. Is strong because it requires a linearization and certain conditions. The reading history is determined.

Food, Restaurants, Cafes

Cafe

Food

Cola I found in Oslo, listing all of its ingredients, and a recepy for 90 liters of it

Cola

Mayo I like

Mayo

Recepies

FreeBSD

Install notes for IBM Thinkpad T60

I am new to FreeBSD so I am not saying this is the way to do it, but it works for me:

  1. Install pkg drm_kmod
  2. load radeon drivers on boot with rc.conf 'kld_list="radeonkms"' (don't use the recommended option of loading /boot/modules/radeonkms.ko, as this crashes during boot with a page fault)
  3. adduser - add an unprivileged user. While running this command, answer the question to join additional groups with "video" to make use of a graphical user interface and "wheel" to allow the command "su".
  4. Install pkg xorg, xinit and xfce
  5. Enable dbus in rc.conf 'dbus_enable="YES"' as required by xfce
  6. Start xfce with "startxfce4" command

Btw sound worked out of the box, which I not always had with a fresh Linux.

Unsolved problems

Running sway: It works when started from whithin xfce. Started outside, it fails with a DRM warning message.

WIFI: It seems there are compatibility problems between FreeBSD and OpenWRT. Using an old "cheap plastic router" works. Also, DHCP is not always responsive and takes two or three trials. I also had the feeling I had to switch off "powerd" on FreeBSD first (which I enabled during the FreeBSD install process) in order to get WIFI working at all, even with the plastic router. This behaviour is not confirmed though and might be coincidence.

Thinkpad Trackpad: The trackpoint (little red dot) works for mouse interactions, but the trackpad is dead

Change keyboard layout

kbdmap or in rc.conf with keymap="de.kbd"

Jail

man jail is a great resource including examples on how to get started.

  1. Dedicate a directory (/usr/jail/myjail; /data/jail/myjail etc) or a new zfs dataset (zfs create zroot/myjail) to the new jail.
  2. Load the directory with FreeBSD files by running bsdinstall jail /usr/jail/myjail.
  3. Start a simple jail with jail -c path=/usr/jail/myjail mount.devfs host.hostname=testhostname ip4.addr=192.0.2.100 command=/bin/sh

It is possible to copy a binary into the jail filesystem and execute it from whithin the jail.

Delete a jail directory

When doing rm -rf on the jail directory, even as root, it fails to execute on a few files. The reason is that some files are marked with the "system immutable" flag. To remove it run chflags -R noschg /jaildir and try rm again. It could still fail if devices got mounted. Check mount and unmount in that case.

Go

Private Repository

Not all packages are public but go mod kind of expects that. There is a way around it.

  1. Modify .gitconfig (convince git to use ssh (and its key) instead of https)
[url "ssh://git@git.example.com/"]
        insteadOf = https://git.example.com/
  1. Set these environment variables, so go knows to not verify checksums for the private repo against the public checksum API; not use the go package proxy
GOPROXY=direct;GOPRIVATE=git.example.com

GOOS and GOARCH

Type go tool dist list or check Go (Golang) GOOS and GOARCH · GitHub

String formatting cheat sheet

fmt.Printf formatting tutorial and cheat sheet

String formatting with padding (Christopher Oezbek CC BY-SA 4.0)

Use the Printf function from the fmt package with a width of 6 and the padding character 0:

fmt.Printf("%06d", 12) // Prints to stdout '000012'

Setting the width works by putting an integer directly preceeding the format specifier ('verb'):

fmt.Printf("%d", 12)   // Uses default width,                          prints '12'
fmt.Printf("%6d", 12)  // Uses a width of 6 and left pads with spaces, prints '    12'

The only padding characters supported by Golang (and most other languages) are spaces and 0:

fmt.Printf("%6d", 12)   // Default padding is spaces, prints '    12'
fmt.Printf("%06d", 12)  // Change to 0 padding,       prints '000012'

It is possible to right-justify the printing by prepending a minus -:

fmt.Printf("%-6d", 12)   // Padding right-justified, prints '12    '

Beware that for floating point numbers the width includes the whole format string:

fmt.Printf("%6.1f", 12.0) // Prints '0012.0' (width is 6, precision is 1 digit)

It is useful to note that the width can also be set programmatically by using * instead of a number and passing the width as an int parameter:

myWidth := 6
fmt.Printf("%0*d", myWidth, 12) // Prints '000012' as before

This might be useful for instance if the largest value you want to print is only known at runtime (called maxVal in the following example):

myWidth := 1 + int(math.Log10(float64(maxVal)))
fmt.Printf("%*d", myWidth, nextVal)

Last, if you don't want to print to stdout but return a String, use Sprintf also from fmt package with the same parameters:

s := fmt.Sprintf("%06d", 12) // returns '000012' as a String

Movies

  • Victoria - One-shot movie, playing in Berlin

  • Blau ist eine warme Farbe (orig. La vie d’Adèle) - Love story of two girls

  • Nocturnal Animals by Tom Ford - Intensive colors and costumes

Optimization

Designing systems that optimize a set of metrics subject to constraints.

The optimization process

minimize f(x) subject to x ∈ X

Minimize f(x) can be replaced by maximize -f(x)

                                           +------+
                                       +---+Change<-----+
                                       |   |Design|     |no
                                       |   +------+     |
                   +---------+   +-----v-----+       +--+--+
Design         +--->  Initial+--->Evaluate   +------->Good?|
Specifications     |  Design |   |Performance|       |     |
                   +---------+   +-----------+       +--+--+
                                                        |
                                                        |yes
                                                        v
                                                      Final
                                                      Design

Optimize with respect to data, as intuition can be misleading.

Translating real world problems

There are some books describing the process to transform real world optimization problems to optimization problems

  • Optimization: Algorithms and Applications (R.K. Arora)
  • Optimization Concepts and Applications in Engineering (2nd edition, A. Keane, P. Nair)
  • Computational Approaches for Aerospace Design (P.Y. Papalambros, D.J. Wilde)
  • Principals of Optimal Design (Cambridge University Press, 2017)

Constraints

Constraints can be numerical (for example x ⋝ 4) but should always include the boundary (in the example 4). Excluding it (x > 4), the solution can move infinitely close to 4 without ever reaching it, which means no solution can be found.

Critical Points

A function f(x) may have a global minimum but may have multiple local minima. A zero derivative is a necessary condition for a local minimum but not a sufficient condition. The second derivative has to be >0, so the point is at the bottom of the bowl.

Podcasts

Usually available (sometimes exclusively) on Spotify

German

  • Fest & Flauschig
  • Baywatch Berlin
  • Hotel Matze
  • Codestammtisch
  • Sprechen wir über Mord?! - Der SWR2 True Crime Podcast
  • Podschalk
  • Wie war der Tag, Liebling? - Anke Engelke, Kristian Thees
  • Alles gesagt?
  • Apokalypse und Filterkaffee
  • Cui Bono: WTF happened to Ken Jebsen?
  • JOKES mit Till Reiners
  • Podcasts - der Podcast
  • Wirecard: 1,9 Milliarden Lügen

Dutch

  • De Brand in het Landhuis - (Documentary about a wealthy Dutch man who died in a fire in his villa under mysterious circumstances)

Projects

The things I work(ed) on

(Mostly) Code Snippets

Bash

Create a random string cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 8 | head -n 1

General Software

Reserved usernames

Git

Multi-line commit messages with -m "..." -m "...". Bash (and other shells) allow for typing -m "... ⏎ ..."

Checkout a file from a branch with checkout <branch> -- <file or directory> to get the state of a branch's file or directory into the current branch.

Gitea

Access to an account over https

Create an access token for this user and use it for git like https://user:token@git...

Big files over https

Gitea supports ssh access and has no file size limitations that I am aware of. Access via an Nginx proxy can lead to a 413 status code. Nginx has to accept bigger bodies client_max_body_size 100M; or any other reasonable size.

Java Web Start

JNLP files can be executed under linux with icedtea-web.

Linux

Table of System Calls

grep

grep -E -o ".{0,5}pattern.{0,5}" file.txt shows 5 characters before and after the found pattern.

Intel 3945AGB WiFi adapter

An old laptop has the 3945 integrated. On CentOS 8, the installation is as follows:

  • Make sure the 3945 firmware is installed (search with dnf. Was installed for me by default)
  • Enable the ElRepo, which contains the package kmod-iwlegacy

Explanation iwlegacy: Usually the kernel module iwlwifi contains the drivers for the chipset. Support for 3945 was removed not too long ago, so a lot of documentation still refers to the iwlwifi package. Only iwlegacy still supports the chipset.

  • Install kmod-iwlegacy
  • Install package crda, which contains the 'regulatory.db' file, so the wifi chip knows which local regulations to follow.
  • Install NetworkManager-wifi or other preferred way of handling wifi connections

If things don't work, check lspci, dmesg, journalctl -u NetworkManager -e and other logs for hints.

PXE Boot OpenWRT

How to install Debian via PXE using an OpenWRT router only: First it's handy to have more storage on a USB drive attached to the router. My drive was formatted with NTFS, so I had to install the ntfs-3g package to be able to mount the drive on the router opkg install ntfs-3g (then mount /dev/sda /mnt).

Next step is to enable PXE boot on dnsmasq. The GUI has a tab for TFTP. Enable the TFTP server and configure the mounted USB drive as TFTP root.

Debian has a handy package ready for downloading called "netboot". After unpacking, it reveals the pxelinux.0 file and a folder structure that is preconfigured to PXE boot Debian. Only make sure the pxelinux.0 resides in the TFTP root folder together with everything else that was included in the netboot archive. Link to the pxelinux.0 from the GUI. Now PXE is ready in the network.

Miniflux

My RSS reader miniflux supports Auth Proxy, which means any header can be used to pass an (existing) username. I use an internal PKI anyway, so I wanted to authenticate myself with a certificate on my server.

Nginx config snippet

ssl_client_certificate /etc/nginx/client_certs/ca.crt;
ssl_verify_client on;

Inside the location section I put proxy_set_header X-Forwarded-For $ssl_client_s_dn;

The problem here is that $ssl_client_s_dn extracts the Common Name (CN) from the client certificate like this CN=username which miniflux does not understand. To solve this, I wrote a variable mapping for Nginx

map  $ssl_client_s_dn  $ssl_client_s_dn_cn {
        default "";
        ~CN=(.*) $1;
}

And used the new variable $ssl_client_s_dn_cn inside the location section. What the map does:

  1. Take the original CN=... string
  2. If anything goes wrong, return "" by default
  3. Match a regex ~, capture everything after the CN= in a group and return the first group $1

Miniflux only needs one configuration parameter AUTH_PROXY_HEADER="X-Forwarded-For"

OSX

Remove quarantine attribute from executable xattr -rd com.apple.quarantine <executable-file>

PostgreSQL

Streaming

Stream like a CTO Very professional, expensive setup for streaming. I like the tooling advise. And once money is very little object, this home office setup seems to be a lot of fun, including camera, microfone, UPS, screens, Lenovo ThinkStation and what so not.

VPN

SSH

Use SSH SOCKS proxy ssh -D 1337 -q -C -N user@server (-D Socks, -q quiet, -C compress -N no output (-i private key))

Certificates

Convert .pfx / .p12 to .pem openssl pkcs12 -in client.pfx -out client.pem, as used with anyconnect or openconnect.

Or the other way round, crt to pfx openssl pkcs12 -export -out certificate.pfx -inkey privateKey.key -in certificate.crt -certfile more.crt

Swift Learning

Location Services

There are three types of location services

  1. Visits location
  2. Significant Change
  3. Standard

The first two don’t rely on GPS location and are power-efficient. They are not meant for real-time applications or precision. Every location service needs authorisation.

Authorization

Apple Developer Documentation Add the required Description to the Info.plist file. The description does not need to contain the App’s name. It should be friendly in tone and possibly give an example why the data is needed.

In the code, create a user dialog and ask for permission with CLLocationManager().requestWhenInUseAuthorization.

Standard location service

Create a Class implementing NSObject, CLLocationManagerDelegate. NSObject is required. I am not sure at the moment why, but what I read is that the underlying Objective-C of the Apple-provided imports needs this declaration as well. Link the delegation handler to an instance of CLLocationManager. clman.delegate = myDelegatedInstance

Swift Language

Attributes

Attributes have the @ operator. Attributes provide additional information about a declaration or type. A widely used case for an attribute is the @State in SwiftUI.

Delegation

Delegation is a design pattern to hand off responsibilities to an instance of another class. To use this pattern, a few steps are necessary:

Define a protocol

protocol MyDelegate {
    func myfunc()
}

Use this protocol for an assignment in another Class

class MyClass {
   var delegate:MyDelegate?
}

So delegate is ready to receive any class that conforms to the MyDelegate protocol. Now implement this Class

class MyDelegationClass: MyDelegate {
    func myfunc() {
        // function implementation
    }
}

The delegation can be linked with the calling Class

let caller = MyClass()
let delegater = MyDelegationClass()
caller.delegate = delegater

Opaque Type

The Opaque Type was introduced in Swift 5.1 and comes into play whenever the concrete type is unknown but conforms to the protocol or protocol composition.

Optional

As in Rust, Swift knows Optionals that are enums consisting of a none and a some type. It is used if the presence of a variable is optional.

# Declare Optional
var optInt: Int? # Short-hand declaration
var optInt: Optional<Int> # Complete declaration

An Optional can be unwrapped with the unsafe short-hand optInt! which results in runtime error if it can’t be unwrapped, for example if the variable contains nil. The complete declaration is Optional<optInt>

Variables

A constant value is declared with let whereas variables are declared with var.

SwiftUI

To describe a view, structs are used. Swift uses modifiers for structs which can programatically change them, which in turn changes the layout. Updates happen asynchronously. States will not always propagate immediately.

struct ContentView: View {
    var body: some View {}
}

The struct implements the protocol View with the required variable body.

some declares an Opaque Type. The structure View can have as many different implementations as there are layouts. That’s why a some is given, so that any structure conforming to the View protocol can be used.

@State Attribute

The State attribute is used for variables that define the single source of truth for a state in a _View_. A State property should only be accessed by the View itself or accessed by a function that’s called from inside the View. Calling states is thread safe. Whenever a State variable changes, the body of the View is recomputed. A State is a means of reading and writing a value and is not the value itself. Projecting a value works by prepending a $. It will pass the value to another View.

WindowGroup

The WindowGroup is a container. It can contain more than one View. It declares a hierarchy of Views. Each Window allocates its own State. Depending on the platform, the WindowGroup can allow to present the different Windows simultaneously, for example on macOS and iPadOS.

References

Things to think

  • Engineering productivity can be measured is a bold statement, after so many years of trying to quantify software development. In this article, the author states that most metrics you try to get from a team, can - and will be challenged by engineers. The most simple example is to measure lines of code. Engineeres will find ways to improve the number of lines of code tremendously, without affecting effectivity. On the other hand, not measuring at all won't help either. The bottom line of the article is to measure blockers:

Engineering should instead be about effectiveness: "How able is this engineer to effect positive impact?"

  • Quality of developer tools
  • Frequency and quality of internal activities (like meetings or code reviews)
  • Focused maker time (free from disruptive meetings)
  • Easy access to documentation
  • Psychological safety on the team
  • Work-life balance
  • Presence of other high-performers
  • A fair system of rewards

Screenshot from a talk Gunter Dueck gave about Intrapreneuring

Intrapreneur's Commandments

Documentation done right

There are four categories of documentation:

  • Tutorials (learning oriented)
  • How-to guides (problem oriented)
  • Reference (information oriented)
  • Discussions (understanding oriented)

These categories need structurally be kept apart.

Tutorials

Tutorials are supposed to be the hardest part of a documentation to create. Let a beginner do an exercise to learn using the software. The point is practical knowledge, not theoretical knowledge. Tutorials need to be concrete and not abstract. There should be no explanation. There should be no options or choices of the path. The tutorial writer is in charge of that path. The tutorial must be reproducable under all circumstances.

How-to guides

Recipes to solve a problem to achieve an outcome. In contrast to a tutorial, the learner would not even be able to formulate a problem to solve. How-to guides do not need to start at the beginning. They can assume basic knowledge of the domain. Again, no explanations. They get in the way of actions. Choose practicability over completeness.

Reference

References are purely descriptive. They are supposed to describe the machinery. A reference should not explain what can be assumed as general knowledge of the topic. The reference’s structure should resemble the code base. The wording should be consistent.

Discussions / Background material / Explanation

Clarify and illuminate a topic. Give context and discuss alternatives. Outline conflicting opinions. Make connections. It should not contain guides or technical reference.

Reference

Why incentive plans cannot work

This is an article by Alfie Kohn published in 1993 in the Harvard Business Review Magazine. It cought my interest because of its relevance today.

Kohn claims that many companies and their managers believe in a reward system to motivate employees for better performance. He gives a number of reasons, based on scientific research, for rewards to miss the point every time. Rewards are for example

piece-work pay for factory workers, stock options for top executives, special privileges accorded to Employees of the Month, and commissions for salespeople

Rewards only achieve one thing: temporary compliance. They will lead to movement but not motivation. So the question is, do we strive for excellence? For long term growth? Or just for a quick, short-term goal.

Rewards do not create a lasting commitment. They merely, and temporarily, change what we do.

At least two dozen studies have shown that rewards will not lead to better outcomes. A study by McKinsey could not find any difference in return for shareholders among 90 companies that use rewards and those which don't.

  • People doing excellent work are not driven by money. They are excellent because they intrinsically love what they do.
  • Rewards and Punishment are two sides of the same coin. They are manipulative. Being controlled has a punitive character over time. Not receiving an expexted reward has the same effect as a punishment. This ultimately leads to a controlled workplace and not one that empowers learning, exploration and progress.
  • Rewards only know winners and losers. They divide what was used to be a team into rewarded and punished people.
  • Rewards hide underlying reasons. It is not wise to introduce rewards if there are causes for bad performance. Reasons for bad performance, among others, are too strict hierarchies, workers unable to collaborate, inadequatly prepared workers for the job, sacrifice long-term growth for short-term goals.
  • Rewards discourage risk taking. With "This for that", people facing a reward will focus on the "this", not the "that".
  • Any form of pay-per-performance makes people less enthusiastic. There are a few theories trying to explain this observed behaviour.

The number one casualty of rewards is creativity. As the late John Condry put it, rewards are the “enemies of exploration.”

UI / UX

Aviation

A typical action sequence is

  1. Reformulation of the mission task
  2. Access desired inputs
  3. Format data for proper input
  4. Find the place where to insert data
  5. Verify and monitor the process

The workload and thus the perceived complexity to perform a task is a function of the volume of memorized action sequences. To keep workload low, it is adviced to perform a mission task analysis as the starting point of the design process. This will enable pilots to access functions easily, which are sometimes not directly implemented in FMS, for example Descent to crossing restriction, Change of departure runway etc. Pilots are well trained for tasks in the pattern "aviate / navigate / communicate" and "manual / tactical / strategical control of the aircraft". Still, these patterns hardly appear as a visual breakdown in FMS.

Spacial Orientation

Experienced computer game players can perform worse than non-players on tasks in virtual environments that do not resemble typical tasks in computer games.

Web

References

  • Designing user-interfaces for the cockpit: Five common design errors and how to avoid them (Lance Sherry, Peter Polson, Michael Feary)

creative commons

Attribution-NonCommercial 4.0 International

Creative Commons Corporation (“Creative Commons”) is not a law firm and does not provide legal services or legal advice. Distribution of Creative Commons public licenses does not create a lawyer-client or other relationship. Creative Commons makes its licenses and related information available on an “as-is” basis. Creative Commons gives no warranties regarding its licenses, any material licensed under their terms and conditions, or any related information. Creative Commons disclaims all liability for damages resulting from their use to the fullest extent possible.

Using Creative Commons Public Licenses

Creative Commons public licenses provide a standard set of terms and conditions that creators and other rights holders may use to share original works of authorship and other material subject to copyright and certain other rights specified in the public license below. The following considerations are for informational purposes only, are not exhaustive, and do not form part of our licenses.

  • Considerations for licensors: Our public licenses are intended for use by those authorized to give the public permission to use material in ways otherwise restricted by copyright and certain other rights. Our licenses are irrevocable. Licensors should read and understand the terms and conditions of the license they choose before applying it. Licensors should also secure all rights necessary before applying our licenses so that the public can reuse the material as expected. Licensors should clearly mark any material not subject to the license. This includes other CC-licensed material, or material used under an exception or limitation to copyright. More considerations for licensors.

  • Considerations for the public: By using one of our public licenses, a licensor grants the public permission to use the licensed material under specified terms and conditions. If the licensor’s permission is not necessary for any reason–for example, because of any applicable exception or limitation to copyright–then that use is not regulated by the license. Our licenses grant only permissions under copyright and certain other rights that a licensor has authority to grant. Use of the licensed material may still be restricted for other reasons, including because others have copyright or other rights in the material. A licensor may make special requests, such as asking that all changes be marked or described. Although not required by our licenses, you are encouraged to respect those requests where reasonable. More considerations for the public.

Creative Commons Attribution-NonCommercial 4.0 International Public License

By exercising the Licensed Rights (defined below), You accept and agree to be bound by the terms and conditions of this Creative Commons Attribution-NonCommercial 4.0 International Public License ("Public License"). To the extent this Public License may be interpreted as a contract, You are granted the Licensed Rights in consideration of Your acceptance of these terms and conditions, and the Licensor grants You such rights in consideration of benefits the Licensor receives from making the Licensed Material available under these terms and conditions.

Section 1 – Definitions.

a. Adapted Material means material subject to Copyright and Similar Rights that is derived from or based upon the Licensed Material and in which the Licensed Material is translated, altered, arranged, transformed, or otherwise modified in a manner requiring permission under the Copyright and Similar Rights held by the Licensor. For purposes of this Public License, where the Licensed Material is a musical work, performance, or sound recording, Adapted Material is always produced where the Licensed Material is synched in timed relation with a moving image.

b. Adapter's License means the license You apply to Your Copyright and Similar Rights in Your contributions to Adapted Material in accordance with the terms and conditions of this Public License.

c. Copyright and Similar Rights means copyright and/or similar rights closely related to copyright including, without limitation, performance, broadcast, sound recording, and Sui Generis Database Rights, without regard to how the rights are labeled or categorized. For purposes of this Public License, the rights specified in Section 2(b)(1)-(2) are not Copyright and Similar Rights.

d. Effective Technological Measures means those measures that, in the absence of proper authority, may not be circumvented under laws fulfilling obligations under Article 11 of the WIPO Copyright Treaty adopted on December 20, 1996, and/or similar international agreements.

e. Exceptions and Limitations means fair use, fair dealing, and/or any other exception or limitation to Copyright and Similar Rights that applies to Your use of the Licensed Material.

f. Licensed Material means the artistic or literary work, database, or other material to which the Licensor applied this Public License.

g. Licensed Rights means the rights granted to You subject to the terms and conditions of this Public License, which are limited to all Copyright and Similar Rights that apply to Your use of the Licensed Material and that the Licensor has authority to license.

h. Licensor means the individual(s) or entity(ies) granting rights under this Public License.

i. NonCommercial means not primarily intended for or directed towards commercial advantage or monetary compensation. For purposes of this Public License, the exchange of the Licensed Material for other material subject to Copyright and Similar Rights by digital file-sharing or similar means is NonCommercial provided there is no payment of monetary compensation in connection with the exchange.

j. Share means to provide material to the public by any means or process that requires permission under the Licensed Rights, such as reproduction, public display, public performance, distribution, dissemination, communication, or importation, and to make material available to the public including in ways that members of the public may access the material from a place and at a time individually chosen by them.

k. Sui Generis Database Rights means rights other than copyright resulting from Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended and/or succeeded, as well as other essentially equivalent rights anywhere in the world.

l. You means the individual or entity exercising the Licensed Rights under this Public License. Your has a corresponding meaning.

Section 2 – Scope.

a. License grant.

  1. Subject to the terms and conditions of this Public License, the Licensor hereby grants You a worldwide, royalty-free, non-sublicensable, non-exclusive, irrevocable license to exercise the Licensed Rights in the Licensed Material to:

    A. reproduce and Share the Licensed Material, in whole or in part, for NonCommercial purposes only; and

    B. produce, reproduce, and Share Adapted Material for NonCommercial purposes only.

  2. Exceptions and Limitations. For the avoidance of doubt, where Exceptions and Limitations apply to Your use, this Public License does not apply, and You do not need to comply with its terms and conditions.

  3. Term. The term of this Public License is specified in Section 6(a).

  4. Media and formats; technical modifications allowed. The Licensor authorizes You to exercise the Licensed Rights in all media and formats whether now known or hereafter created, and to make technical modifications necessary to do so. The Licensor waives and/or agrees not to assert any right or authority to forbid You from making technical modifications necessary to exercise the Licensed Rights, including technical modifications necessary to circumvent Effective Technological Measures. For purposes of this Public License, simply making modifications authorized by this Section 2(a)(4) never produces Adapted Material.

  5. Downstream recipients.

    A. Offer from the Licensor – Licensed Material. Every recipient of the Licensed Material automatically receives an offer from the Licensor to exercise the Licensed Rights under the terms and conditions of this Public License.

    B. No downstream restrictions. You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, the Licensed Material if doing so restricts exercise of the Licensed Rights by any recipient of the Licensed Material.

  6. No endorsement. Nothing in this Public License constitutes or may be construed as permission to assert or imply that You are, or that Your use of the Licensed Material is, connected with, or sponsored, endorsed, or granted official status by, the Licensor or others designated to receive attribution as provided in Section 3(a)(1)(A)(i).

b. Other rights.

  1. Moral rights, such as the right of integrity, are not licensed under this Public License, nor are publicity, privacy, and/or other similar personality rights; however, to the extent possible, the Licensor waives and/or agrees not to assert any such rights held by the Licensor to the limited extent necessary to allow You to exercise the Licensed Rights, but not otherwise.

  2. Patent and trademark rights are not licensed under this Public License.

  3. To the extent possible, the Licensor waives any right to collect royalties from You for the exercise of the Licensed Rights, whether directly or through a collecting society under any voluntary or waivable statutory or compulsory licensing scheme. In all other cases the Licensor expressly reserves any right to collect such royalties, including when the Licensed Material is used other than for NonCommercial purposes.

Section 3 – License Conditions.

Your exercise of the Licensed Rights is expressly made subject to the following conditions.

a. Attribution.

  1. If You Share the Licensed Material (including in modified form), You must:

    A. retain the following if it is supplied by the Licensor with the Licensed Material:

    i. identification of the creator(s) of the Licensed Material and any others designated to receive attribution, in any reasonable manner requested by the Licensor (including by pseudonym if designated);

    ii. a copyright notice;

    iii. a notice that refers to this Public License;

    iv. a notice that refers to the disclaimer of warranties;

    v. a URI or hyperlink to the Licensed Material to the extent reasonably practicable;

    B. indicate if You modified the Licensed Material and retain an indication of any previous modifications; and

    C. indicate the Licensed Material is licensed under this Public License, and include the text of, or the URI or hyperlink to, this Public License.

  2. You may satisfy the conditions in Section 3(a)(1) in any reasonable manner based on the medium, means, and context in which You Share the Licensed Material. For example, it may be reasonable to satisfy the conditions by providing a URI or hyperlink to a resource that includes the required information.

  3. If requested by the Licensor, You must remove any of the information required by Section 3(a)(1)(A) to the extent reasonably practicable.

  4. If You Share Adapted Material You produce, the Adapter's License You apply must not prevent recipients of the Adapted Material from complying with this Public License.

Section 4 – Sui Generis Database Rights.

Where the Licensed Rights include Sui Generis Database Rights that apply to Your use of the Licensed Material:

a. for the avoidance of doubt, Section 2(a)(1) grants You the right to extract, reuse, reproduce, and Share all or a substantial portion of the contents of the database for NonCommercial purposes only;

b. if You include all or a substantial portion of the database contents in a database in which You have Sui Generis Database Rights, then the database in which You have Sui Generis Database Rights (but not its individual contents) is Adapted Material; and

c. You must comply with the conditions in Section 3(a) if You Share all or a substantial portion of the contents of the database.

For the avoidance of doubt, this Section 4 supplements and does not replace Your obligations under this Public License where the Licensed Rights include other Copyright and Similar Rights.

Section 5 – Disclaimer of Warranties and Limitation of Liability.

a. Unless otherwise separately undertaken by the Licensor, to the extent possible, the Licensor offers the Licensed Material as-is and as-available, and makes no representations or warranties of any kind concerning the Licensed Material, whether express, implied, statutory, or other. This includes, without limitation, warranties of title, merchantability, fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. Where disclaimers of warranties are not allowed in full or in part, this disclaimer may not apply to You.

b. To the extent possible, in no event will the Licensor be liable to You on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of this Public License or use of the Licensed Material, even if the Licensor has been advised of the possibility of such losses, costs, expenses, or damages. Where a limitation of liability is not allowed in full or in part, this limitation may not apply to You.

c. The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability.

Section 6 – Term and Termination.

a. This Public License applies for the term of the Copyright and Similar Rights licensed here. However, if You fail to comply with this Public License, then Your rights under this Public License terminate automatically.

b. Where Your right to use the Licensed Material has terminated under Section 6(a), it reinstates:

  1. automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or

  2. upon express reinstatement by the Licensor.

For the avoidance of doubt, this Section 6(b) does not affect any right the Licensor may have to seek remedies for Your violations of this Public License.

c. For the avoidance of doubt, the Licensor may also offer the Licensed Material under separate terms or conditions or stop distributing the Licensed Material at any time; however, doing so will not terminate this Public License.

d. Sections 1, 5, 6, 7, and 8 survive termination of this Public License.

Section 7 – Other Terms and Conditions.

a. The Licensor shall not be bound by any additional or different terms or conditions communicated by You unless expressly agreed.

b. Any arrangements, understandings, or agreements regarding the Licensed Material not stated herein are separate from and independent of the terms and conditions of this Public License.

Section 8 – Interpretation.

a. For the avoidance of doubt, this Public License does not, and shall not be interpreted to, reduce, limit, restrict, or impose conditions on any use of the Licensed Material that could lawfully be made without permission under this Public License.

b. To the extent possible, if any provision of this Public License is deemed unenforceable, it shall be automatically reformed to the minimum extent necessary to make it enforceable. If the provision cannot be reformed, it shall be severed from this Public License without affecting the enforceability of the remaining terms and conditions.

c. No term or condition of this Public License will be waived and no failure to comply consented to unless expressly agreed to by the Licensor.

d. Nothing in this Public License constitutes or may be interpreted as a limitation upon, or waiver of, any privileges and immunities that apply to the Licensor or You, including from the legal processes of any jurisdiction or authority.

Creative Commons is not a party to its public licenses. Notwithstanding, Creative Commons may elect to apply one of its public licenses to material it publishes and in those instances will be considered the “Licensor.” Except for the limited purpose of indicating that material is shared under a Creative Commons public license or as otherwise permitted by the Creative Commons policies published at creativecommons.org/policies, Creative Commons does not authorize the use of the trademark “Creative Commons” or any other trademark or logo of Creative Commons without its prior written consent including, without limitation, in connection with any unauthorized modifications to any of its public licenses or any other arrangements, understandings, or agreements concerning use of licensed material. For the avoidance of doubt, this paragraph does not form part of the public licenses.

Creative Commons may be contacted at creativecommons.org