Luc's blog

We all know we should have backups, but you often see ransomware groups also targeting backup storage. Luckily, there are some nice ways to make your backups-over-SSH more secure using a neat authorized_keys trick.

In short: You can add some options to your authorized_keys file to force connections with that key to run a specific command.

These tricks should also work on Hetzner storage boxes, and most other providers that allow you to upload an authorized_keys file.

Borg (non-immutable)

At Ferox, we use the following for onsite quick-access borg backups:

command="/usr/bin/borg serve --restrict-to-path /storage/mcsystembackups",restrict ssh-ed25519 AAAAC...

Since borg works over ssh, it tries to run borg serve on the remote host. This command basically replaces that borg serve command with one that is limited to a specific folder.

Borg (Immutable)

Keep in mind that this key will now not allow pruning or deleting backups, so you will have to schedule that from a trusted location/key.

command="borg serve --append-only ",restrict ssh-ed25519 AAAAC3NzaC1lZ...J borgmatic_offsite

rsync (Immutable)

command="rrsync -wo -no-del -no-overwrite DIR",restrict,from="fdba:000:1c1c:bff6::1" ssh-rsa AAAAB3Nza... root@box-to-backup

This requires the remote host to have rrsync, which we are unfortunately missing on hetzner’s storage boxes.

Age + SCP (Encrypted)

Age is a nice tool for simple encrypted files. Useful for things like offsite backups: https://github.com/FiloSottile/age

$ apt install age 
$ age-keygen -o backupkey.txt
Public key: agepublickkeythingy 
$ age --decrypt -i /opt/backupkey.txt data.tar.gz.age > data.tar.gz

cat /tmp/billingdbbackups/billingbackup-$(date +%d-%m-%Y).sql.gz | age -r agepublickkeythingy > /tmp/billingbackup.sql.gz.age
scp -P23 /tmp/billingbackup.sql.gz.age [email protected]:/home/agebackups/billingbackup-$(date +%d-%m-%Y).sql.gz.age

Here’s a quick list of IT events that may be fun to go to for Dutch people. May or may not be up to date. Any suggestions? let me know!

Some notes for myself written down publicly in the hopes that they might help some future person.

It is highly recommended to have the cluster on a vlan/vswitch!
Galera does not implement it’s own security for the cluster ports. See the vlan part at the bottom of this document if you do not have vlans setup yet.

In this example the following IPs are used:
Local machine: 192.168.100.1
Another cluster node: 192.168.100.2

Make sure mariadb is off before starting!

edit /etc/mysql/mariadb.conf.d/60-galera.cnf

[galera]
wsrep_on = ON
wsrep_provider = /usr/lib/galera/libgalera_smm.so
wsrep_cluster_address = "gcomm://192.168.100.1,192.168.100.2"
wsrep_provider_options="gmcast.listen_addr=tcp://192.168.100.1:4567;ist.recv_addr=192.168.100.1:4568"
binlog_format = row
wsrep_cluster_name="galeracluster"
default_storage_engine = InnoDB
wsrep_node_address = 192.168.100.1
innodb_autoinc_lock_mode = 2
wsrep_sst_receive_address=192.168.100.1:4444
bind-address = 192.168.100.1

If this is the first node in the cluster, or all nodes in the cluster are currently down, you have to run
galera_new_cluster to bootstrap the cluster again. Otherwise you can just start the mariadb service through systemd.

Now you’re good to go!
While everything should work now, if you want to read more about galera you can do so here.

Addendum: Vlans

You should keep the cluster on a vlan for security reasons. This can be done in netplan by editing /etc/netplan/(yourconfig).yaml and adding:

  vlans:
    enp0s31f6.4001:
      id: 4001
      link: enp0s31f6
      mtu: 1400
      addresses:
        - 192.168.100.1/24

Here enp0s31f6 is the interface name, 4001 is the vlan id, and 192.168.100.1 is the ip to assign to the local box.

Bit of an afterthought after seeing patrick winston’s talk

Mainly applicable to middle/high-school and smaller university classes. Depending on your country’s school system.

Minimize Distraction

• preferably blur out windows at eye height when sitting. Having something going on outside can completely destroy how information flows, however good your teaching is. This is often not something you can do something about as a teacher. But you can try to pick a room with as least distraction as possible.
• Have students take notes. The error that a lot of lecturers make is that they expect students to make notes by themselves, but if you start without a notebook in front of you you are in no way inclined to grab it when needed. Start the lecture with a short question “does everyone have their books in front of them?” and also give them some time to write down what you say. A subtle clue can also be to say that people to not have to write down x thing, implying that the rest must be written down.
• Talking people are the death of any lecture. The best way to minimze this is to make sure you are well audible and also to show that you (and all other students) can hear it when a student is talking. Preferably have semi-small groups of no more than ~20 students or a room that is setup such that sound seems amplified, how counter-intuitive it may seem.

Minimize cutoff

• Never, ever, have a slide full of text or even worse, equations to talk through. People will lose you, and probably wont catch on later either if you take too long. Break problems up into smaller sets, and preferably leave out duplicate information.
• Give people a chance to jump back on after a while. Give people something to think about for a short time and then start a new part-subject. Make it obvious that you are restarting from scratch so people have a chance to get on.
• Don’t be boring, but also not too hyper-active. A boring teacher will make his students bored, but a hyper-active teacher will probably also lose out on the energy of students really quickly. A teacher that likes their area is way more likely to generate interested students than one who seems to be bored. (if the teacher is bored then why would you ever want to learn more about it)
• Be sure of what you are saying. If you have to think a long time of what you are doing people will quickly lose trust, start talking about it, and dropping off.
• Mainly for math subjects: You do not have to rigorously prove complicated things in your lectures. Students don’t care and they sure as hell don’t have the time to take it all in during the 1 minute that you have your slide up. Students really like to know “why” something is true, but giving intuition is often a thousand times better than plomping down a complicated formula and just reading that off of the slides.
• Similar to last point. Don’t just show definitions. Show examples. A student cannot remember all your definitions, but they will remember examples. If needed show them many edgecases that implicitly or explicitly explain the definition.
• Again mainly for math: Where possible, use a blackboard or even digital drawing instead of slides. It is extremely important for students to grasp the steps of solving a problem, and doing it step by step allows you to explain your reasons, allows asking more specific questions from students, and better overall intuition. Blackboards are a lot of work, but what works similarly well is a drawing tablet projected on a screen. You don’t have to lose time and focus when clearing out the board, can go back to show how something correlates to a previous assignment, and share the results for later reference. Do make sure that you have a cheatsheet next to you so you don’t make any big mistakes. Or do and show how students can check themselves when they are solving problems.
• Lectures often build on eachother, but may be spread out. If you are handling a lot of information in a single lecture then it’s a good idea to have recitations after a day or two to help students remember.

Optimizing tests

• Tests are not for teaching, please do not include too much new information that might confuse students. Application of the actual course content is of course fine or even imperative.
• Give everyone their test back after they are graded. Show people where they made errors even if they do not want to redo it. Tests often give a great amount of different topics to see what you know, and re-reading a test is a great way to see what errors you made and where your ideas might differ from the teacher’s. (notationwise, etc). Don’t have students go somewhere to view their test or have them wait a long time. Stimulate them to actually take their time and take away any barriers.
• Offer a demo-test. This is a great way to prepare students so they have a nice checklist of what to learn and a way to test this. If you think most of your students aren’t making the demo-test include one of the demo-test questions on the real test. This will give them a good reason to use the demo-test when learning and will greatly improve their grades. (don’t re-use more questions, don’t be that guy)
• Make sure students have enough time. Your goal is to check if the student knows what you told them. Not how fast they can read and write
• Use a lesson to talk through items of the last test that didn’t go well, and add a small question on the next test with that same item for a couple of points. If you have no reason to want them to still learn it then why were you testing them in the first place?

If you have no reason to want them to still learn it then why were you testing them in the first place?

(Math) Textbooks

The following example is taken from a friend’s mathematical textbook, but any student who has done math at a decent level will recognize the kind of text. It being in romanian only helps prove the point.

Mixing math and normal language is funest for learning. Take mixing french and german in a text. You can’t read it easily even if you know both languages. Let alone remember the content. For most humans, the language processor cannot switch between such different languages efficiently enough (though i’m not aware of any scientific research on this). Books like this want to be mathematically rigorous, but end up being completely useless. If you look at for example MIT’s mathematics for CS book and this the difference is as day and night. MIT has formulas cleanly separated and focussed on their relation in written text, while other books often have this mixmatch of inline unreadable math that makes most people want to stop reading and just watch youtube videos instead.

Asking questions

This is something that many education systems struggle with, and it’s almost become a culture thing in a lot of countries.

Educators often ask themselves why students are not asking questions. Even though it’s obvious from the next test that they didn’t understand everything well. And it’s a sad by-product of mass education.

The amount of questions asked is often already decided in the first lecture by the speed and thoroughness of which an educator anwsers it. If a student has to wait a long time and the subject has already passed, the student (and for one question-asker many others) will already have missed the point, and having to go back feels like you are wasting everyone’s time.

If an explanation is too short, the student might not understand yet or feel dumb, which will stop them from asking more questions. If the explanation is too long the other students will quickly get distracted or the student will feel like it’s a waste of time. Try to keep your answers to questions to the global audience and make them part of your lecture, don’t just look the specific person in the eye and talk softly to them as you will surely lose all other students. The trick here is to gauge what level of understanding all students are on, but this is an extremely hard if not impossible task.

What my university did very well was create a discord group with specific channels for each subject for asking questions. Being able to ask them semi-anonymously (under your real name, but no face) helps a lot with getting people to ask questions, and you have a lot more tools like a LaTeX bot or images. It would help if there was a bit more of a culture of also anwsering those questions. Perhaps by having TA’s anwser some of them, but this fills a very important gap. Somes universities have specific question-hours but these are often not very productive as people are not actively working on the content at those moments. It’s better to allow asking these questions when people are working on the content at school or at home.

People have different ways of learning a new ecosystem or library. Some prefer to just start and look at the docs as they go, some like to read through all the examples that can be found online. I would like to add one thing to that list: Learning by porting an old program over to the newest version of your ecosystem.

But, what?

Programs get made and abandoned. Updates happen and things break. An old piece of software made for the same ecosystem that you are learning right now might not (won’t?) just compile.

The process of changing a piece of software made for one ecosystem to a similar but different one is called porting. This is often not as easy as it seems, even less so in low-level languages like C. (and if you go low enough, even C will seem “high-level”)

Why porting

When porting over a piece of software, you generally have to have a decent understanding of the software and ecosystem you are working with. When starting out you won’t have that.

But. When porting over a piece of software you will learn a tremendous amount about the tooling and common setups. When googling errors (which will absolutely come up) you will find the appropriate documentation and forums for your piece of software, and you will often find many common mistakes that you will now not have to make. An added bonus is that you will learn about both the old, and the new ecosystem. Learning the new ecosystem is obvious, but learning about the old ecosystem will also help a lot when you are working with older blog posts and pieces of software that you may or may not want to “steal” some code from.

As an added bonus, the open-source community will have a new updated version of the piece of software that you have worked on. Great!

DPDK

As an example i have recently been busy porting an old DPDK program over to a newer version of the library. I wanted to try and start by just writing out a basic program that would let me do what i wanted, but i immediately ran into trouble with outdated docs and undocumented missing libraries. After i got a simple version working i decided to try and port over an old program that i found on github. By doing this i was able to easily discern old functionality and the accompanying new improved version that i could also better understand the outdated documentation and use it to write newer programs.

Okay okay, i’ll give it a try.

This might not work for everyone, but i highly recommend you to try this method and see if it works for you. Good luck learning!