Planet NUUG

November 29, 2021

Peter Hansteen (That Grumpy BSD Guy)

The 'sextortion' Scams: The Numbers Show That What We Have Is A Failure Of Education

Subject: Your account was under attack! Change your credentials!
From: Melissa <chenbin@jw-hw.com>
To: adnan@bsdly.net

Hello!

I am a hacker who has access to your operating system.

I also have full access to your account.

I've been watching you for a few months now.

The fact is that you were infected with malware through an adult site that you visited.

Did you receive a message phrased more or less like that, which then went on to say that they have a video of you performing an embarrasing activity while visiting an "adult" site, which they will send to all your contacts unless you buy Bitcoin and send to a specific ID?

The good news is that the video does not exist. I know this, because neither does our friend Adnan here. Despite that fact, whoever operates the account presenting as Melissa appears to believe that Adnan is indeed a person who can be blackmailed. You're probably safe for now. I will provide more detail later in the article, but first a few dos and don'ts:

• Whatever some tempting web site tells you in a popup, unless you know what you are doing, do not install software on your devices from any other sources than the official ones. You do not need to install a new video viewer for that site or update your existing one, neither do you need to enter your administrator user name and password along with your credit card details into an unfamiliar-looking dialog box or web form.
   
• Unless you know what you are doing, stay away from Bitcoin or other cryptocurrencies. If that message is the first you've heard of Bitcoin, you do not know what you are doing, leave it alone. As assets go, there is not much difference between financial derivatives, toxic waste and cryptocurrencies like Bitcoin, in that they should be handled with equal care and only from a distance unless you are in fact an expert in the field.
   
• If you are not sure about either of the two bullet points before this one, please forget any shame over what you may or may not have done, and contact somebody you trust and who knows the subject better. This may be an adult such as a parent, teacher, social worker or other, a tech-savvy friend, or for that matter law enforcement such as your local police.

The important point is that you are or were about to be the victim of what I consider a very obvious scam, and for no good or even nearly valid reason. You should not need to become the next victim.

And this, dear policy makers and tech heads in general is our problem: A large subset of the general public simply do not know their way around the digital world we created for them to live in. We need to do better.

In that context I find it quite disturbing that people who should know better, such as the Norwegian Center for Information Security, in a recently issued report (also see Digi.no's article (both in Norwegian only, sorry)) predict that the sextortion attacks will become "more sophisticated and credible". Then again at some level they may technically be right, since this kind of activity starts out with a net negative credibility score.

A case in point: Some versions of the scam messages I have been able to study went as far as to claim that the perpetrators had not only had taken control of the target's device, they had even sent that very email message from there. That never happened, of course, and it would have been easy for anybody who had learned to interpret Received: headers to verify that the message was in fact sent from the great elsewhere. Unfortunately the skill of reading email headers is rarely, if ever, taught to ordinary users.

The fact that people do not understand those -- to techies -- obvious facts is a fairly central and burdening problem, and again we need to do better.

Now let me explain. Things get incrementally more technical from here, so if you came here only for the admonitions or practical advice and have no use for the background, feel free to wander off.

I know the message I quoted at the beginning here is a scam because I run my own mail service, and looking at just the logs there just now I see that since the last logs archiving rotation early Saturday morning, more than 3000 attempts at delivery of messages like the one for Adnan happened, aimed at approximately 200 non-existent recipients before my logs tell me they finally tried to deliver one to my primary contact address, never actually landing in any inboxes.

One of the techniques we use to weed out unwanted incoming mail is to maintain and publish a list of known bad and invalid email addresses in our domains. These known bad addresses have then in ways unknown (at least not known to us in any detail) made it into the list of addresses sold to spammers, and we at the receiving end can use the bad addresses as triggers to block traffic from the sending hosts (If you are interested, you can read elsewhere on this blog for details on how we do this, look for tags such as greylisting, greytrapping or antispam).

If it was not clear earlier, those numbers tell us something about the messages at hand. It should be fairly obvious that compromising videos of non-existent users could not, in fact, exist.

Looking back in archived logs from the same system I see that a variant of this message started appearing in late January 2018. The specifics of that message sequence will be interesting to revisit when the full history of sextortion (I still do not like the term, but my preferred alterantive is at risk of being filtered out by polite society-serving robots) will be written, but let us rather turn to the more recent data, as in data recorded earlier this week.

Mainly because I found the media coverage of the "sextortion" phenomenon generally uninformed and somewhat annoying, I had been been mulling writing an article about it for a while, but I was still looking for a productive angle when on Wednesday evening I noticed a slight swelling in the number of greytrapped hosts. A glance at my spamd log seemed to indicate that at least one of the delivery attempts had a line like

       I am a hacker who has access to your operating system.

Which was actually just what I had been pondering writing about.  

So I set about for a little research. I greped (searched) in my yet-unrotated spamd logs for the word hacker, which yielded lots of lines of the type

Feb 22 04:04:35 skapet spamd[8716]: 89.22.104.47: Body: I am a hacker who has access to your operating system.
Feb 22 04:17:04 skapet spamd[8716]: 5.79.23.92: Body: I am a hacker who has access to your operating system.
Feb 22 04:34:03 skapet spamd[8716]: 153.120.146.199: Body: I am a hacker who has access to your operating system.
Feb 22 04:40:30 skapet spamd[8716]: 45.181.93.45: Body: I am a hacker who has access to your operating system.
Feb 22 04:55:04 skapet spamd[8716]: 93.186.247.18: Body: I am a hacker who has access to your operating system.
Feb 22 05:09:39 skapet spamd[8716]: 123.51.190.154: Body: I am a hacker who has access to your operating system.
Feb 22 05:13:22 skapet spamd[8716]: 212.52.131.4: Body: I am a hacker who has access to your operating system.
Feb 22 05:38:02 skapet spamd[8716]: 5.79.23.92: Body: I am a hacker who has access to your operating system.
Feb 22 05:44:39 skapet spamd[8716]: 123.51.190.154: Body: I am a hacker who has access to your operating system.
Feb 22 06:00:30 skapet spamd[8716]: 45.181.93.45: Body: I am a hacker who has access to your operating system.
(the full result has been preserved here). Extracting the source addresses gave a list of 198 IP addresses (preserved here).

Extracting the To: addresses from the fuller listing yielded 192 unique email addresses (preserved here). Looking at the extracted target email addresses yielded some interesting insights:

1) The target email addresses were not exclusively in the domains my system actually serves, and

2) Some ways down the list of target email addresses, my own primary address turns up.

Of course 2) made me look a little closer, and only one IP address in the extract had tried delivery to my email address.

A further grep on that IP address turned up this result.

There are really no surprises to be had here, at least to a large subset of my supposed readers. The sender had first tried to deliver one of the sexstortion video messages to one of the by now more than quarter million spamtraps, and its IP address was still blacklisted by the time it finally tried delivery to a potentially deliverable address.

Doing a few spot checks on the sender IP addresses in recent and less recent logs it looks like the only two things could be mildly exciting about those messages. One is the degree the content was intended to be embarrasing to the recipient. The other is a possible indicator of the campaign's success: Looking back through the logs for the approximate year of known activity, it even looks like the campaign became multilingual, while retaining the word "hacker" in most if (possibly) not all language versions.

Other than that it is almost depressing how normal the sextortion campaign is: It uses the same spam sending infrastructure and the same low quality target address lists (the ones containing some subset of my spamtrap addresses) as the regular and likely not too successful spammers of every stripe. Nothing else stands out.

And as returning readers will notice, the logs indicate that the spambots are naive enough in their SMTP code that they frequently mistake spamd's delaying tactics for a slow, but functional open SMTP relay.

Now to recap the main points:

• Regular users: The sextortion messages are scams, the videos do not exist. If this quasi-random sample is representative, the scammers are seen to send to 200 non-existing, invalid addresses before lucking on a real one. This alone strongly indicates that no videos exist. There is no reason to send money, bitcoin or otherwise. Look instead to learning how your devices and the networks and services they connect to actually work.
• Competent mail admins: The tools to stop the flow of sextortion messages or at least slow to a manageable trickle are available today. You simply need to keep your antispam game up to speed with best practices and best of breed tools. If you are a user or someone who manages mail admins, check what your mail service does.
  
• Competent authorities: Please step up to the task of educating the public. Sane, fact based approaches to IT security work. While it is easy to get distracted by the potential presence of porn and users' feelings of shame over accessing that kind of material, assigning much weight to that side of the matter is counterproductive. Work to educate the public and please focus on real threats, not imagined ones like the present topic.

Whatever evolves next out of these rather hamfisted attempts at blackmail is unlikely to ever achieve any level of sophistication worthy of the name.

We would all be much better served by focusing on real threats such as, but not limited to, credential harvesting via deceptive content delivered over advertising networks, which themselves are a major headache security- and privacy-wise, or even harvesting via phishing email.

Both of the latter have been known to lead to successful compromise with data exfiltration and identity theft as possible-to-probable results.

To a large extent the damage could could have been significantly limited had the general public been taught sensible security practices such as using multi-factor authentication or at least actually good passwords combined with securely coded password management applications, and insisting that services encourage such practices.

Yes, I know you have been dying to ask: What is the thing about Adnan? According to my activity log, the address adnan@bsdly.net was added as a spamtrap on July 8th, 2017 after somebot had tried to log on as the user adnan, a user name not seen before at bsdly.net,

Jul  8 09:40:34 skapet sshd[34794]: Failed password for invalid user adnan from 118.217.181.8 port 41091 ssh2

apparently from a network in South Korea.

As always, there is more log material available to competent practitioners and researchers with a valid research agenda. Please contact me if you are such a person who could use the collected data productively.

---

Update 2020-02-29: For completeness and because I felt that an unsophisticated attack like the present one deserves a thorough if unsophisticated analysis, I decided to take a look at the log data for the entire 7 day period, post-rotation.

So here comes some armchair analysis, using only the tools you will find in the base system of your OpenBSD machine or any other running a sensibly stocked unix-like operating systen. We start with finding the total number of delivery attempts logged where we have the body text 'am a hacker' (this would show up only after a sender has been blacklisted, so the gross number actual delivery attempts will likely be a tad higher), with the command

zgrep "am a hacker" /var/log/spamd.0.gz | awk '{print $6}' | wc -l

which tells us the number is 3372.

Next up we use a variation of the same command to extract the source IP addresses of the log entries that contain the string 'am a hacker', sort the result while also removing duplicates and store the end result in an environment variable called lastweek:

 export lastweek=`zgrep "am a hacker" /var/log/spamd.0.gz | awk '{print $6}' | tr -d ':' | sort -u `

With our list of IP addresses tucked away in the environment variable go on to: For each IP address in our lastweek set, extract all log entries and store the result (still in crude sort order by IP address), in the file 2020-02-29_i_am_hacker.raw.txt:

 for foo in $lastweek ; do zgrep $foo /var/log/spamd.0.gz | tee -a 2020-02-09_i_am_hacker.raw.txt ; done

For reference I kept the list of unique IP addresses (now totalling 231) around too.

Next, we are interested in extracting the target email addresses, so the command

grep "To:" 2020-02-29_i_am_hacker.raw.txt | awk '{print substr($0,index($0,$8))}' | sort -u

finds the lines in our original extract containing "To:", and gives us the list of target addresses the sources in our data set tried to deliver mail to.

The result is preserved as 2020-02-29_i_am_hacker.raw_targets.txt, a total of 236 addresses, mostly but not all in domains we actually host here. One surprise was that among the target addresses one actually invalid address turned up that was not at that time yet a spamtrap. See the end of the activity log for details (it also turned out to be the last SMTP entry in that log for 2020-02-29).

This little round of armchair analysis on the static data set confirms the conclusions from the original article: Apart from the possibly titillating aspects of the "adult" web site mentions and the attempt at playing on the target's potential shamefulness over specific actions, as spam campaigns go, this one is ordinary to the point of being a bit boring.

There may well be other actors preying on higher-value targets through their online clumsiness and known peculiarities of taste in an actually targeted fashion, but this is not it.

A final note on tools: In this article, like all previous entries, I have exclusively used the tools you will find in the OpenBSD (or other sensibly put together unixlike operating system) base system or at a stretch as an easily available package.

For the simpler, preliminary investigations and poking around like we have done here, the basic tools in the base system are fine. But if you will be performing log analysis at scale or with any regularity for purposes that influences your career path, I would encourage you to look into setting up a proper, purpose-built log analysis system.

Several good options, open source and otherwise, are available. I will not recommend or endorse any specific one, but when you find one that fits your needs and working style you will find that after the initial setup and learning period it will save you significant time.

As per my practice, only material directly relevant to the article itself has been published via the links. If you are a professional practitioner or researcher with who can state a valid reason to need access to unpublished material, please let me know and we will discuss your project.

Update 2020-03-02: I knew I had some early samples of messages that did make it to an inbox near me squirreled away somewhere, and after a bit of rummaging I found them, stored here (note the directory name, it seemed so obvious and transparent even back then). It appears that the oldest intact messages I have are from December 2018. I am sure earlier examples can be found if we look a littler harder.

Update 2020-03-17: A fresh example turned up this morning, addressed to (of all things) the postmaster account of one of our associated .no domains, written in Norwegian (and apparently generated with Microsoft Office software). The preserved message can be downloaded here. 

Update 2020-05-10: While rummaging about (aka 'researching') for something else I noticed that spamd logs were showing delivery attempts for messages with the subject "High level of danger. Your account was under attack."  So out of idle curiosity on an early Sunday afternoon, I did the following:

$ export muggles=`grep " High level of danger." /var/log/spamd | awk '{print $6}' | tr -d ':' | sort -u`
$ for foo in $muggles; do grep $foo /var/log/spamd >>20200510-muggles ; done

and the result is preserved for your entertainment and/or enlightenment here. Not much to see, really other than that they sent the message in two language varieties, and to a small subset of our imaginary friends.

Update 2020-08-13: Here is another snapshot of activity from August 12 and 13: this file preserves the activity of 19 different hosts, and as we can see that since they targeted our imaginary friends first, it is unlikely they reached any inboxes here. Some of these campaigns may have managed to reach users elsewhere, though

Update 2020-09-06: Occasionally these messages manage to hit a mailbox here. Apparently enough Norwegians fall for these scams that Norwegian language versions (not terribly well worded) get aimed at users here. This example, aimed at what has only ever been an email alias made it here, slipping through by a stroke of luck during a time that IP address was briefly not in the spamd-greytrap list here, as can be seen from this log excerpt. It is also worth noting that an identically phrased message was sent from another IP address to mailer-daemon@ for one of the domains we run here.

Update 2021-01-06: For some reason, a new variant turned up here today (with a second message a few minutes later and then a third), addressed to a generic contact address here. A very quick check of logs here only turned up only this indication of anything similar (based on a search for the variant spelling PRONOGRAPHIC), but feel free to check your own logs based on these samples if you like.

Update 2021-01-16: One more round, this for my Swedish alter ego. Apparently sent from a poorly secured Vietnamese system.

Update 2021-01-18: A Norwegian version has surfaced, attempted sent to approximately 115 addresses in .no domains we handle, fortunately the majority of the addresses targeted were in fact spamtraps, as this log extract shows.

Update 2021-03-03: After a few quiet weeks, another campaign started swelling our greytrapped hosts collection, as this hourly count of IP addresses in the traplist at dump to file time shows:

Tue Mar  2 21:10:01 CET 2021 : 2425
Tue Mar  2 22:10:01 CET 2021 : 4014
Tue Mar  2 23:10:01 CET 2021 : 4685
Wed Mar  3 00:10:01 CET 2021 : 4847
Wed Mar  3 01:10:01 CET 2021 : 5759
Wed Mar  3 02:10:01 CET 2021 : 6560
Wed Mar  3 03:10:01 CET 2021 : 6774
Wed Mar  3 04:10:01 CET 2021 : 7997
Wed Mar  3 05:10:01 CET 2021 : 8231
Wed Mar  3 06:10:01 CET 2021 : 8499
Wed Mar  3 07:10:01 CET 2021 : 9910
Wed Mar  3 08:10:01 CET 2021 : 10240
Wed Mar  3 09:10:01 CET 2021 : 11872
Wed Mar  3 10:10:01 CET 2021 : 12255
Wed Mar  3 11:10:01 CET 2021 : 13689 
Wed Mar  3 12:10:01 CET 2021 : 14181
Wed Mar  3 13:10:01 CET 2021 : 15259
Wed Mar  3 14:10:01 CET 2021 : 15881
Wed Mar  3 15:10:02 CET 2021 : 17061
Wed Mar  3 16:10:01 CET 2021 : 17625
Wed Mar  3 17:10:01 CET 2021 : 18758
Wed Mar  3 18:10:01 CET 2021 : 19170
Wed Mar  3 19:10:01 CET 2021 : 20028
Wed Mar  3 20:10:01 CET 2021 : 20578
Wed Mar  3 21:10:01 CET 2021 : 20997

by Peter N. M. Hansteen (noreply@blogger.com) atNovember 29, 2021 06:20 AM

November 11, 2021

Ole Aamot GNOME Development Blog

Record Live Multiple-Location Audio immediately in GNOME Gingerblue 2.0.1

GNOME Gingerblue 2.0.1 is available and builds/runs on GNOME 41 systems such as Fedora Core 35.

It supports immediate, live audio recording in compressed Xiph.org Ogg Vorbis encoded audio files stored in the private $HOME/Music/ directory from the microphone/input line on a computer or remote audio cards through USB connection through PipeWire (www.pipewire.org) with GStreamer (gstreamer.freedesktop.org) on Fedora Core 34 (getfedora.org) as well as XSPF 1.0 playlist stored in the private $HOME/Music/GNOME.xspf playlist of the previous, latest recording.

See the GNOME Gingerblue project (www.gingerblue.org) for screenshots, Fedora Core 35 x86_64 RPM package and GNU autoconf installation package (https://download.gnome.org/sources/gingerblue/2.0/gingerblue-2.0.1.tar.xz) for GNOME 41 systems and https://gitlab.gnome.org/ole/gingerblue.git for the GPLv3 source code in my GNOME Git repository.

Gingerblue music recording session screen. Click “Next” to begin session.

The default name of the musician is extracted from g_get_real_name(). You can edit the name of the musician and then click “Next” to continue ((or “Back” to start all over again) or “Finish” to skip the details).

Type the name of the musical song name. Click “Next” to continue ((or “Back” to start all over again) or “Finish” to skip any of the details).

Type the name of the musical instrument. The default instrument is “Guitar”. Click “Next” to continue ((or “Back” to start all over again) or “Finish” to skip any of the details).

Type the name of the audio line input. The default audio line input is “Mic” ( gst_pipeline_new("record_pipe") in GStreamer). Click “Next” to continue ((or “Back” to start all over again) or “Finish” to skip the details).

Enter the recording label. The default recording label is “GNOME” (Free label). Click “Next” to continue ((or “Back” to start all over again) or “Finish” to skip the details).

Enter the Computer. The default station label is a Fully-Qualified Domain Name (g_get_host_name()) for the local computer. Click “Next” to continue ((or “Back” to start all over again) or “Finish” to skip the details).

Notice the immediate, live recording file. The default immediate, live recording file name falls back to the result of g_strconcat(g_get_user_special_dir(G_USER_DIRECTORY_MUSIC), "/", gtk_entry_get_text(GTK_ENTRY(musician_entry)), "_-_", gtk_entry_get_text(GTK_ENTRY(song_entry)), "_[",g_date_time_format_iso8601 (datestamp),"]",".ogg", NULL) in gingerblue/src/gingerblue-main.c

Click on “Cancel” once in GNOME Gingerblue to stop immediate recording and click on “Cancel” once again to exit the application (or Ctrl-c in the terminal).

The following Multiple-Location Audio Recording XML file [.gingerblue] is created in G_USER_DIRECTORY_MUSIC (usually $HOME/Music/ on American English systems):

<?xml version='1.0' encoding='UTF-8'?>
<gingerblue version='2.0.1'>
<musician>Wilber</musician>
<song>Gingerblue Track 0001</song>
<instrument>Piano</instrument>
<line>Mic</line>
<label>GNOME Music</label>
<station>streaming.gnome.org</station>
<filename>/home/wilber/Music/Wilber_-_Song_-_2021-07-12T21:36:07.624570Z.ogg</filename>
<album>/home/wilber/Music/GNOME</album>
<studio>file://streaming.gnome.org/</studio>
</gingerblue>

You’ll find the recorded Ogg Vorbis audio files along with the Multiple-Location Audio Recording XML files in g_get_user_special_dir(G_USER_DIRECTORY_MUSIC) (usually $HOME/Music/) on GNOME 41 systems configured in the American English language and can launch the latest recording by opening $HOME/Music/GNOME.xspf:

<?xml version="1.0" encoding="UTF-8"?>
<playlist version="1" xmlns="http://xspf.org/ns/0/">
<trackList>
<track>
<title>Song_-_2021-11-11T00:28:10.306448Z</title>
<location>file://streaming.gnome.org//home/wilber/Music/Wilber_-_Song_-_2021-07-12T21:36:07.624570Z.ogg</location>
</track>
</trackList>
</playlist>

by oleaamot atNovember 11, 2021 01:00 AM

November 07, 2021

Peter Hansteen (That Grumpy BSD Guy)

What every IT person needs to know about OpenBSD

How to have fun with the world’s most important free software project

"Functional, free and secure by default", OpenBSD remains a crucial yet largely unacknowledged player in the open source field. This talk aims to highlight the project's signature security features and development practices -- razor sharp focus on correct and secure code coupled with continuing code audit -- as well as the project's role as source of innovation in security practices and 'upstream' source for numerous widely used components such as OpenSSH, PF, LibreSSL and others.

If you only have a few minutes to spare, the highlights are:

• OpenBSD has been around for more than 25 years (started October 1995)
• OpenBSD is proactively secure with only 2 remote holes in default install in all those years
• OpenBSD pioneered use of strong cryptography, the first free system to ship with IPSec (entangling itself in US export regulations in the process)
• OpenBSD pioneered and is still leading in code audit, fixing similar bugs tree-wide when found
• OpenBSD has all security enhancements enabled by default and are hard going on impossible to disable
• OpenBSD is open source, free software and the project actively encourages independent verification of code quality and security.
• Today OpenBSD is in use in many network-centric roles, even though it is a general purpose operating system albeit with a particular emphasis on security.
• OpenBSD has a high profile quality image based on actual code quality and proven performance in real world use
• OpenBSD is upstream (origin) for several widely used pieces of software such as OpenSSH, OpenBGPD, PF, OpenSMTPd, LibreSSL, iked, mandoc and a number of others. For a complete list, please see the OpenBSD Innovations page on the OpenBSD website.
• OpenBSD has been ‘growing up in public’ with code generally accessible via anonymous CVS (the first of its kind) since 1995 – transparent process, development discussions on public tech@ mailing list
• Developers would do well to study high quality (mainly) C source and how the project runs a 6 month release cycle like clockwork (with only a few notable exceptions).

Note: If you are more of a slides person you will be happy to hear that indeed the presentation I used for this when given as a talk is available here with the main highlights and little to no geek jokes.

Now with that out of the way, let's step back to  where it all started.

OpenBSD: How it all started

OpenBSD's history is to a large extent the history of the Internet itself. You may have heard of the time back in the 1980s when the likes of IBM and Digital were slugging it out in the corporate IT sphere and the US department of defence paid for experiments in distributed, device independent networking.

That's when a loosely organized group of hackers somewhat coordinated by researchers at University of California's Berkeley campus rose to prominence with "BSD Unix", which by a sequence of happy accidents became the home of the reference implementation of the TCP/IP internet protocols.

By the early 1990s, commercialization of the Internet had started, and the Berkeley Computer Science Research Group (CSRG) that had coordinated the efforts was set to be disbanded. In addition to the net itself, the main tangible product out of Berkeley was the Berkeley Software Distribution (BSD), often distributed on tapes in the mail but also available on the net itself, which had started as a collection of software for AT & T's Unix but had over the years been extended become a full featured Unix operating system.

Several different groups wanted BSD to go on even if the CSRG did not, and several things happened in fairly rapid succession:
 

• Lynne and Bill Jolitz ported BSD to Intel x86 (actually 80386sx), creating 386BSD. This was chronicled in a series of articles in Dr Dobbs' Journal (also see a more condensed summary over at salon.com)
• Next up, hackers started sharing improvements to the 386BSD code as "patchkits", and two separate groups took the work further to form their projects: The FreeBSD group would be working on bringing the best possible BSD to PC-style hardware, while the NetBSD group's ambition was to make BSD run on any hardware they could get their hands on. [See Addendum at the end of the article.]
• A group of former CSRG employees formed BSDi Inc. and marketed their product BSD/386 with among other things a contact phone number "1-800-ITS-UNIX". The activities of an actual corporation in turn triggered a lawsuit from the owners of the UNIX trademark over code copyrights.

The lawsuit was eventually settled -- only six files of several thousand in the tree were 'potentially encumbered' and had to be replaced, leaving both NetBSD and FreeBSD with a rush to replace the code which if I remember correctly was at least in part fairly central to the virtual memory subsystem.

OpenBSD came into existence a couple of years later, from a fork of the NetBSD code base in October 1995, with the initial release in July 1996.

From the very start, the OpenBSD project has been running a code audit of the entire tree, focusing on code correctness and security. We want secure, correct code that makes up a usable system with sane defaults and complete and readable documentation. And we want that code to be available under a free license and actually available to the world as soon as it is committed to the project's version control.

One of the early achievements of the OpenBSD project was anonymous CVS, which makes it possible for anyone on the internet to get the code with changes in near real time. This was a major break with the normal practice of most projects of the time, which would typically work in relative isolation on private mailing lists and at quasi-random intervals issue a release as a tarball on an FTP server somewhere.

You are already an OpenBSD user!

It is probably useful at this point to reveal that even if you do not know it, you are more likely than not using code with an OpenBSD origin right now. Your Apple product, be it iPad, iPhone or Mac, your Android device, your Cisco router, Solaris, Linux or other Unix or even your Microsoft product has some or a lot of OpenBSD originated code in it. We will get back some detail on that later.
 

OpenBSD: Code audit and security evolution

But about the code audit. The activity runs roughly like this: 

Read the code, understand what it does.

Look for unsafe behaviors, assume a hostile environment.

When you find a bug and fix it, look for similar code elsewhere in the tree and fix everywhere.

You will be amazed how much different programmers think alike and make the same mistakes. Wash, rinse, repeat.

That may sound somewhat unexciting, but careful study of how the code actually performs in real life situations lead to a number of innovations over the years, with a strong slant for being proactively secure, making it harder for bugs to actually do damage:

• W^X -- memory can be writeable XOR executable
• Address space randomization (ASLR) so the jump targets and gaps will vary for each execution
• random-sized gaps inserted in the stack, again catching fixed-sized returns
• unreadable, unwriteable guard pages at the end of malloc()ed chunks to catch overruns
• privilege separation -- daemons run the bulk of their code as a non-privileged user, more likely than not in a shell-less chroot, coupled with privilege revocation, which means that daemons drop privilege as soon as possible
• the pledge(2) system call to declare a profile to restrict program behavior to only specified operations and resources
• the unveil(2) system call to restrict file system access to specified paths and permissions only
• A fairly user visible change came when OpenBSD 6.2 introduced kernel address randomized link, or KARL, which sees to it that the kernel is relinked to a new, randomized layout for each boot. Once again, introducing randomness where none had been before is seen as a way to mitigate possible exploits based on code loading at predictable addresses.

All of those features have been integrated in the OpenBSD source tree, and with the developers admonished to adhere to the rule

"where it is possible to spot damage, fail hard".

-which means that poorly written software will crash a lot more often on OpenBSD than elsewhere. That in itself should make the platform attractive to developers. Exposing your code to a hostile environment and see it perform or fail can be quite entertaining and enlightening.

Usable, portable and secure

To complete the picture, it is useful to keep in mind that OpenBSD runs on a total of fourteen platforms. All platforms are self-hosting. Cross-compiling is only used in the early phase of porting to a new platform.

And of course, in addition to purely maintaining existing code to run on diverse platforms, users and developers have real world needs that are addressed by developing new software, extending the features of existing programs, adding new functionality or even replacing programs or entire subsystems.

Security is a many-faceted topic. Early on, OpenBSD stood out as the system that included real crypto in the base system, to the extent that exporting OpenBSD source code from the United States was technically illegal under that country's munitions export restrictions as they were defined at the time. 

Fortunately for us, the project was always coordinated from Canada by undisputed project leader Theo de Raadt who lives in Canada. There is anecdotal evidence that US based developers would trek across the border for hackathons with clean slate equipment to install OpenBSD while in Canada and hack, that is, work on the system and would then legally bring the result back with them.

One early application of crypto in OpenBSD was when a full IPSEC stack was included in the system in 1997. OpenBSD was the first free system to include IPSec by default in its base install.

In a prime example of hacker humor of the time, a T-shirt featuring one of the early appearances of Puffy the blowfish that would become the project mascot touted the Blowfish password hashing algorithm which remains the default on OpenBSD both with the picture caption "So long and thanks for all the passwords" just below Puffy on the front, along with the full source code of the blowfish function on the back. 

The expectation was that the T-shirt would be illegal to re-export from the US.

In addition to attention to security and code correctness, one other important feature of OpenBSD is attention to intellectual integrity and insisting on clearly worded and unambiguous license to use and modify code and documentation that forms part of the system.

So why use OpenBSD? What is it like?

So what is OpenBSD like for a user or developer, and why is it better?

I'd say the short version is that it's a real Unix. Unlike the Linuxes of the world that spent years muddling through an evolutionary succession of init systems and have ended up more or less settling on the ever expanding systemd which seems to have tentacles into everything and is on a clear course to replacing most of what we have traditionally thought of as the base system, OpenBSD has stayed with and refined the traditional BSD init so can have both uncluttered services management and a base system that consists of programs that for the most part adhere to the classical Unix philosophy that every program should do exactly one thing, and do that thing well.

If you are a developer, you will also appreciate hearing that the base system of well designed programs that all have a readable and useful man page already contains basic Unix developer tools along with a C and C++ compiler -- clang where supported and gcc where necessary -- plus perl and a host of tools. Basically, everything that is needed to build the base system from a fresh checkout of the source code is contained in the base system on a default install.

Ported software goes under /usr/local

Once you have the thing installed on whatever hardware you have, keeping in mind that you can run a selection of 14 platforms ranging from fairly ancient kit to modern hardware, you will likely turn to installing ported third party software from packages, using pkg_add(8) which will suck in whatever you tell it to fetch from the same mirror you installed from or what appears to be the most local one.

More software is available on the more popular platforms than on the more, dare we say exotic ones.

For the OpenBSD 6.9 release, the most mainstream platform amd64 came with 11310 prebuilt and installable packages, while mips64 had only 8182 and the mips64el platform is marked as (still building).

Installing pre-built packages is almost always more convenient and is recommended in most cases, but if you for one reason or the other want to build your own from a cvs checkout of the ports tree, you are free to do so at the cost of your own time watching the process.

Whichever route you choose to go, you will see that installing packages does not land you with any files in the directories used by the base system outside of a few that drop their configuration files in subdirectories under /etc and add their combined startup/shutdown scripts to the collection in /etc/rc.d. Anything else ends up under /usr/local, and you can see why the installer by default sets up that file system on a separate and fairly roomy partition. My previous article You've installed it. Now what? Packages! is a few years old, but gives you the main motivations and some background along with some pointers on packages practice.

The installer was always good, got better

When I found OpenBSD more than twenty years ago, my main Unix exposure was from working with Linuxes and FreeBSD. What attracted me to OpenBSD and finally had me buy an OpenBSD 2.5 CD set was the strong focus on security and code correctness. When the CD set and the classic wireframe daemon T-shirt finally arrived in the mail, I set about at first to install it on whatever spare hardware I had lying around.

If I remember correctly, the first machine I tried installing OpenBSD on was an 80386/33MHz with 8MB RAM and I think a 100MB IDE hard disk. Which I can report sounded pretty crappy even then, but the thing did work.

The initial install was fairly straightforward, and when I started poking around I found two things about myself and the new system: Everything made sense, and everything I could think of had a readable man page. So the first change I am aware of that made the world better with OpenBSD was the decision to enforce the "No commit without documentation" rule, which came into being early in the project's life, probably roughly at the same time the OpenBSD developers gave us a real-time view of development via anonymous CVS. You can see things happening in almost real time.

It is worth mentioning that the installer has remained famously non-graphical, text only. The reason the installer remains text-only is that this is a major advantage that enables the developers and the users to handle the fairly diverse collection of hardware platforms that OpenBSD runs on with the same portable, familiar and compact code everywhere.

The installer was always scriptable and extensible, and over the years the installer has added automatic, repeatable and scriptable installs (dubbed autoinstall(8) which appeared in OpenBSD 5.5 in 2014) and the sysupgrade(8) extension (first found in OpenBSD 6.6 in 2019) that automates snapshot to snapshot or release to subsequent release upgrades for all not too hacked-up configurations. Each of these moments, or more specifically when the new code started appearing in snapshots, had me appreciate the OpenBSD system a bit more, and made me feel quality of life had improved.

Now something for your laptop - hardware support

Fast forward some twenty-plus years and the last article I published, and even got into Norwegian mainstream IT news site Digi.no, centers on a few moments involving new OpenBSD developments. It took some interaction with OpenBSD developers, but those interactions lead to my new laptop with an 11th generation Intel Core chipset working even better with OpenBSD. Yes, OpenBSD developers and a significant subset of their user base actually run OpenBSD on their laptops. I do use a Mac and a work-issued Thinkpad with Ubuntu Linux too, but life is not complete without an OpenBSD laptop.

Now to be honest, what I saw within the space of a few days was development that had me going from "Oh, sh*t, the SSD isn't recognized" -- the controller was set to a RAID-ish mode by default -- through this kernel panic:

-- to seeing it all fully supported.

The SSD problem turned out to be simple to fix: Simply find the "Advanced" BIOS option that turned the pseudo-raid feature off and let the operating system speak directly to the storage device.

For the rest there was a period of a couple of weeks I had to run with not yet commited patches in a home baked kernel built from checkouts from Jonathan Grey's git repo. When the code was committed to -current, I could resume my normal sysupgrade(8) routine, going from one development snapshot to the next.

The process, even with the need to build custom kernels for a while, was actually quite pleasant, and when the support code went into the main development branch, that too was a a moment when I felt my life had been improved by changes in OpenBSD. The hardware support is now in snapshots and will be in OpenBSD 7.0 which is set to be released on November 1st, 2021.

Why use OpenBSD? IPSEC

As I mentioned earlier, OpenBSD was the first free system to ship with IPSEC -- the tools for enabling encrypted network traffic -- in its base system. The first OpenBSD release with IPSEC was OpenBSD 2.1, which was released in 1997.

The tools worked of course, but in the early days the complaint was that IPSEC was hard and near impossible to debug from an almost-working to a fully working setup.

Further development took a while, but the tools got a major usuability upgrade in OpenBSD 3.8 with ipsecctl and its human-readable configuration file /etc/ipsec.conf to serve as a friendlier front end to the IPSEC tools.

A complete configuration for a minimal setup could look like this:

# Set up two flows:
# First between the machines 192.168.3.14 and 192.168.3.100
# Second between the networks 192.168.7.0/24 and 192.168.8.0/24
flow esp from 192.168.3.14 to 192.168.3.100
flow esp from 192.168.7.0/24 to 192.168.8.0/24 peer 192.168.3.12

This was a major step compared to other platforms. One famous example is preserved in a presentation by Mathieu Sauve-Frankel at AsiaBSDCon 2007, where he demonstrated that setting up the equivalent of the config shown here with Microsoft tools took the user through a sequence of no less than 36 dialog boxes, and still there was a distinct possibility that the configuration was not actually a working one.

The problem in both the Microsoft implementation and others was that the developers had not really given any thought to the user experience. The majority of the options the user was required to set had sensible defaults and could easily have been hidden from view.

The standards documents the developers had worked from were fairly unclear, so it is not entirely unreasonable that "it was hard to write, so it should be hard to use" was a factor in how the products ended up from a user experience perspective.

What defaults would be actually sensible would perhaps not be clear to the developer from the specification, and the only way to see what would be the sensible default would be experience with actual use in the field. The OpenBSD developers who wrote ipsecctl came with extensive experience of using IPSEC and decided that IPSEC did not in fact need to be so hard. The defaults should make sense.

The next milestone in IPSEC development on OpenBSD came with Internet Key Exchange (IKE) protocol support in OpenIKED in OpenBSD 3.8 with iked and ikectl. For convenience, ikectl is able to generate configurations for Windows and macOS clients too. That might save you the agony of clicking through dozens of dialog boxes at the client end.

The thing that lured me in

But I hear you ask, what made me turn into an almost all-in OpenBSD user?

Back in 2001 I was still only experimenting with OpenBSD, but my experience with Linux and iptables had made me long for a switch to a saner firewall. I had done some small experiments with the IPF firewall that was in OpenBSD until the 2.9 release. Then, as some of us will remember, the it was discovered that IPF's license was in fact not free, so it needed to be replaced.

There was a distinct rush, not quite a stampede, to replace IPF over the months that followed. Fortunately, the new code that replaced the previous packet filter proved to perform better. The OpenBSD Packet filter, dubbed PF for short, had been born and made its debut in OpenBSD 3.0 in December 2001. The release had originally been planned for November, but was pushed out a month to hack the "working prototype" packet filter into something usable.

Almost needless to say, this turn of events finally pushed me to take the final steps to replace the Linux gateways I had in place with OpenBSD ones. I was pleasantly surprised to find that not only did they perform well, but they also came with complete and reasonably well documented tools so I could understand what was going on. That's how I got started on the process that lead to among other things writing The Book of PF and taking that text through three editions so far. But more about that later.

It is worth noting that the IPFilter copyright episode spurred the OpenBSD developers to perform a license audit of the entire source tree and ports in order to avoid similar situations in the future. This activity ran for some months and uncovered a number of potential problems. Theo de Raadt summed up the effort in a message to the openbsd-misc mailing list on February 20th, 2003.

What they found when they started looking was that there was a significant number of files that were in fact not under a free license, much like the entire IPF subsystem had been. Those needed to be replaced. Other parts had either no license or no copyright stated. In some cases the developers gave explicit permission to continuing use, but quite a few things needed to be rewritten with a free license so OpenBSD and other free software would be able to move forward without copyright problems.

I later heard in a rather informal setting that among the no copyright and/or no license cases, it was usually possible to track down the developers via version control system logs or mailing list archives. In a large number of those cases, the initial reaction was along the lines "Say what? Are people still using that?".

SSH, open and better

PF was written from scratch to replace a subsystem that it turned out was illegal to use in an open source context. But it was not the first time the OpenBSD project had performed a nonlibreectomy, that is, taken on the task of replacing code for license reasons.

A few years earlier it had become clear that the original developer of the secure shell system ssh had commercial ambitions and the license for the software had changed in a proprietary direction. After a bit of deliberation on how to resolve the situation, the OpenBSD developers started digging around for earlier versions of the code that had been published with an acceptable license. Then they forked their version from the last version they found that still had free license. Next came an intensive period of re-introducing the features that were missing in the old code.

The result was introduced as OpenSSH in OpenBSD 2.6 in 1999. Over the next few years OpenSSH grew a portable version that started grabbing market share rapidly. The last I heard OpenSSH's market share is somewhere in the high nineties percent.

With a state of the art secure shell subsystem in place and growing all sorts of useful features, the time finally came to end unencrypted shell login sessions on OpenBSD. OpenBSD's telnetd was moved to the CVS attic in time for OpenBSD 3.8, which was released November 2005.

One other notable thing about OpenSSH is that it was the first daemon to be properly privilege separated, a model practice that debuted with the overhauled OpenSSH in OpenBSD 3.2 in March 2002. Since then privilege separation has been put in place in all daemons where it made sense to do so, and it is now a signature part of the secure by default stance of all newer OpenBSD daemons.

And yes, that packet filter

I mentioned PF, the OpenBSD packet filter, earlier. I must confess that PF has been an important part of my life in various contexts since the early noughties. Over the years, things I have written have contributed to creating the popular but actually wrong perception that OpenBSD was primarily a firewall operating system. There are a lot of useful and fun features that turned up in or in connection with PF over the years and were pioneered by OpenBSD. Some features were ported to or imitated in other systems, while others remain stubbornly OpenBSD only.

So I will touch on some of my favorite PF and PF-attached features, in quasi-random but almost chronological order.

Beating up spammers with OpenBSD spamd(8) since OpenBSD 3.3

When I started playing with OpenBSD in general and PF in particular way back when, I was already responsible for the SMTP mail service for my colleagues. My gateways by then ran OpenBSD, while the mail server rosalita, named after a Springsteen song, was not too badly specced server running FreeBSD with exim as the mail transfer agent that fed the incoming messages to spamassassin and clamav for content filtering before handing off to user mailboxes.

So when it dawned on me that I could set up spamd(8) the spam deferral daemon on the internet-facing gateway and save load on the poor suffering rosalita that was running hot with content filtering, I was quick to implement a setup that sucked in well known block lists.

Going grey, then trapping

The effect was obvious and immediate, the mail server's fans grew noticeably quieter. When greylisting was introduced in spamd soon after, I implemented that too, and witnessed yet another drop in pitch and intensity of the sound from rosalita's fans. Then a couple of releases later greytrapping -- the practice of adding IP addresses of incoming SMTP connections to blocklists if the attempted delivery is aimed at a known-bad address in the target domain -- was introduced, and that sounded like enough fun that I just went ahead and did it.

The idea of detecting spam senders by the bogus addresses they were already trying to deliver to just sounded too good to not try. And we knew that getting started would be pretty easy too. We had seen rejects for addresses that had never existed in our domains in our mail server logs for quite a while, so it was simply a matter of harvesting from a fairly bountiful source and adding stuff that we were sure would never ever be actually deliverable here to the spamtrap list. I think the first setup had only a couple of hundred entries in it, but I did not note the exact number at the time.

By July 2007 I had decided to publish both the list of spamtrap addresses and an hourly dump of the greytrapped addresses. Both remain free to download. The list of spamtraps, harvested from various log sources, by now numbers just over 270,000 imaginary friends, while the number of trapped hosts is typically in the 3000 to 5000 range. We occasionally see the list swell to 20,000 or more when high volume campaigns run with bad address lists fed to them. I am pretty sure it went over 100,000 at one point.

It's fun to watch, and it looks like a significant subset of the spamtraps have made it into the address lists of active spam operations. I frankly never thought I would still be collecting spam traps from logs all these years later. Yes, it all sounds a bit absurd, but it is effective for keeping our mailboxes largely spam free, even though it feels at times like running a weird found object-ish art project. Anyway, a summary of the lists we publish can be found in this article.

The brutes, the password gropers and the state tracking options

If you run an SSH service or really any kind of listening service with the option to log in, you will see some number of failed authentication attempts that generate noise in the logs. The password guessing, or as some of us say, password groping, turned out to be annoying enough that OpenBSD 3.6-current and later OpenBSD 3.7 introduced a set of features to use data that would anyway be available in the state table, to track the state of active connections, and to act on limits you define such as number of connections from a single host over a set number of seconds.

The action could be to add the source IP that tripped the limit to a table. Additional rules could then subject the members of that table to special treatment. Since that time, my internet-facing rule sets have tended to include variations on

table <bruteforce> persist
block quick from <bruteforce>
pass inet proto tcp from any to $localnet port $tcp_services \
        flags S/SA keep state \
	(max-src-conn 100, max-src-conn-rate 15/5, \
         overload <bruteforce> flush global)

which means that any host that tries more than 100 simultaneous connections or more than 15 new connections over 5 seconds are added to the table and blocked, with any existing connections terminated.

It is a good practice to let table entries in such setups expire eventually. At first I followed the spamd(8) defaults' example and set expiry at 24 hours, but with password gropers like those caught by this rule being what they are, I switched a few years ago to at four weeks at first, then upped again a few months later to six weeks. Groperbots tend to stay broken for that long. And since they target any service you may be running, state tracking options with overload tables can be useful in a lot of non-SSH contexts as well.

It is also worth noting that state tracking actions are useful for essentially all services. The article Forcing the password gropers through a smaller hole with OpenBSD's PF queues has a few suggestions on how to handle noise sources with various other services.

One final point I would like to make about the state tracking and actions is that much like the greytrapping feature of spamd, this feature gives you the tools to build a configuration that adapts to network conditions and learns from the traffic it sees. 

While this does not rise to the level of being an actual Artificial Intelligence or AI, this has enough buzzwordability potential that I remain to this day extremely puzzled that none of the other big names at least imitated those features in their own products and marketed for all it would be worth. 

I certainly know what I would have done in their position. But then I am more engineer than marketer and in the contexts where I call the shots, the best option is just to keep running OpenBSD.

We went to modern queueing

OpenBSD has had traffic shaping available in the ALTQ subsystem since the very early days. ALTQ was rolled into PF at some point, but the code was still marked experimental 15 years after it was written, and most people who tried to use it in anger at the time found the syntax inelegant at best, infuriating or worse at most times.

So Henning Brauer took a keen interest in the problem, and reached the conclusion that all the various traffic shaping algorithms were not in fact needed. They could all except one be reduced to mere configuration options, either as setting priorities on pass or match rules or as variations of the theme of the mother algorithm Hierarchical Fair Service Curve (HFSC for short).

Soon after, another not-small diff was making the rounds. The patch was applied early in the OpenBSD 5.5 cycle, and for the lifetime of that release older ALTQ setups were possible side by side with the new queueing system.

The feedback I get is that the saner syntax in the new queueing system lead to more users taking up traffic shaping. Here is the queue setup that I came up with for one of my sites:

queue rootq on $ext_if bandwidth 20M
        queue main parent rootq bandwidth 20479K min 1M \
                                    max 20479K qlimit 100
             queue qdef parent main bandwidth 9600K min 6000K \ 
                                    max 18M default
             queue qweb parent main bandwidth 9600K min 6000K \
                                    max 18M
             queue qpri parent main bandwidth 700K min 100K \
                                    max 1200K
             queue qdns parent main bandwidth 200K min 12K \
                                    burst 600K for 3000ms
        queue spamd parent rootq bandwidth 1K min 0K max 1K \
                                    qlimit 300

while tying the queues into the subsequent rules with a set of match rules just following that block.

This is what triggered the need to write the third edition of The Book of PF. The book includes descriptions of both the new and the old system as well as tips on how to make a smooth transition. The ALTQ code was removed from OpenBSD during the OpenBSD 5.6 cycle, but continues to live on in some form in FreeBSD and NetBSD.

And yes, if you think my queues setup punishes spammers a bit more in addtion to being subjected to spamd(8), you're right.

pflow(4) offers network insights lite

Everybody who has been tasked with looking after a network has at some point been at least a little curious about what actually moves around there. At times we will see situations where it is essential for troubleshooting purposes to see the traffic flows with data about endpoints, packets and bytes transferred, protocol and so forth.

If you do not need to see the data itself, but rather the metadata, the NetFlow standard and its close cousin IPFIX offers just that. Netflow tools existed as packages on OpenBSD already, but from OpenBSD 4.5 PF has the pflow state tracking option, paired with the pflow(4) virtual network interface which together offer a full netflow sensor package.

Set up one or more pflow interfaces to send data to one or more collectors, and add the pflow option to specific rules or as a state default and you have started your collecting. You can even have metadata for traffic matching specific rules going to separate pflow devices and collectors.

My field notes in Yes, You Too Can Be An Evil Network Overlord - On The Cheap With OpenBSD, pflow And nfsen offers some practical examples and insights, including how we used a pflow setup to track down a noisy machine on a somewhat critical network as well as some pointers to valueable further reading.

LibreSSL, the great deobfuscation

People tell me they think that the reason LibreSSL was created was the Heartbleed bug, but no, actually not, just damn close.

The LibreSSL project was in fact started a few weeks before heartbleed became common knowledge. LibreSSL is the result of a group of OpenBSD developers taking the existing OpenSSL code and starting to fix it.

This time it was not a matter of a bad license. No, this was the result of the number of OpenBSD developers who took a look at the OpenSSL code that had been part of the OpenBSD base system since quite early on, and turned away in disgust and with symptoms of physical pain, reached a critical mass of sorts. I had heard OpenBSD developers complain about the absolute horror of the OpenSSL code for at least ten years. The code quality was just that bad.

What happened next was that a group of hardened OpenBSD developers grabbed the OpenSSL code and started two activities in parallel. One was looking in the OpenSSL request tracker for bugs that had not been addressed. The other was reformatting the OpenSSL code into something resembling the OpenBSD style of readable and maintainable C.

With the code in more readable form, discovering what it did became easier. In addition to a few obvious eye-stinging bugs the LibreSSL developers found a number of oddities, including, but not limited to

• Code was never deleted even when it became irrelevant or obsolete
• OpenSSL did not use the system memory allocation system, but rather opted for their own which never actually deallocated memory, but rather used LIFO recycling, and could easily be made to put private info into logs
• all written in "OpenSSL C", which according to beck@ is a dialect of the "worst common denominator"

It is worth digging out the various articles and presentations made by LibreSSL developers over the years, with specific emphasis on Bob Beck's BSDCan talk on the first 30 days of LibreSSL (available on youtube), which is the original source of the term code flensing.

Since the OpenBSD 5.6 release in 2014, LibreSSL has been the default TLS library in OpenBSD. LibreSSL has been ported elsewhere based on the -portable variant.

For my own part I can only attest to not ever running into a TLS problem that was LibreSSL's fault. It probably still has bugs, but it is a lot more of a healthy choice than its predecessor.

This was my list of life improving OpenBSD events - I'd love to hear yours

As I warned earlier, this has been about my personal list of OpenBSD events that I remember fondly.

I am sure your list is at least a little different. I am sure there are things from the innovations page that I have simply forgotten about.

Each release comes with a detailed list of changes, such as this one for OpenBSD 6.9, and the page has pointers back to the equivalent pages for previous releases.

I would love to hear about your favorite OpenBSD moments.

---

More items for your OpenBSD reading

www.openbsd.org is the official OpenBSD web site. If you want to donate, go to the donations page and find the most appropriate option. Corporate entities may prefer to donate via The OpenBSD Foundation, which is a Canadian non-profit corporation.

undeadly.org is the OpenBSD Journal news site.

bsdly.blogspot.com My rant^H^H^H^Hblog posts

https://flak.tedunangst.com/ Ted Unangst (tedu@) on developments

Michael W Lucas: Absolute OpenBSD, 2nd edition

Peter N. M. Hansteen: The Book of PF, 3rd edition

Henning Brauer: OpenBSD sucks (… least)

---

Addendum 2021-11-06

 It soon became apparent, however, that there were divergent goals for further development.  This led to the creation of FreeBSD.  So we had one community that amicably divided itself into two separate groups."

by Peter N. M. Hansteen (noreply@blogger.com) atNovember 07, 2021 06:32 PM

October 26, 2021

Ole Aamot GNOME Development Blog

GNOME Gingerblue 2.0.1 Recording Software supports XSPF 1.0

GNOME Gingerblue 2.0.1 is Free Recording Software for GNOME.

In the 2.0.1 release I have added support for XSPF 1.0 from Xiph.org.

If you launch the software, a immediate recording is stored into the
$HOME/Music folder along with the playlist in $HOME/Music/GNOME.xspf

You can download the source code from download.gnome.org/sources/gingerblue/2.0/gingerblue-2.0.1.tar.xz for the software and read more about the free playlist standard at www.xspf.org.

Distribution packages are available from
Fedora Core 34
Fedora Core 35
Ubuntu 21.04 LTS

by oleaamot atOctober 26, 2021 12:00 AM

October 24, 2021

Petter Reinholdtsen

Debian still an excellent choice for Lego builders

The Debian Lego team saw a lot of activity the last few weeks. All the packages under the team umbrella has been updated to fix packaging, lintian issues and BTS reports. In addition, a new and inspiring team member appeared on both the debian-lego-team Team mailing list and IRC channel #debian-lego. If you are interested in Lego CAD design and LEGO Minestorms programming, check out the team wiki page to see what Debian can offer the Lego enthusiast.

Patches has been sent upstream, causing new upstream releases, one even the first one in more than ten years, and old upstreams was released with new ones. There are still a lot of work left, and the team welcome more members to help us make sure Debian is the Linux distribution of choice for Lego builders. If you want to contribute, join us in the IRC channel and become part of the team on Salsa.

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

October 24, 2021 05:10 AM

October 10, 2021

Dag-Erling Smørgrav

Automatic Let’s Encrypt certificates in Apache with mod_md

Since 2.4.30, Apache comes with experimental support for ACME certificates (Let’s Encrypt et al.) in the form of mod_md (short for “managed domains”). It’s kind of a pain but it’s still better than what I had before, i.e. a mess of shell and Perl scripts based on Crypt::LE, and if your use case is limited to Apache, it appears to be simpler than Certbot as well. Unfortunately for me, it’s not very well documented and I wasted a considerable amount of time figuring out how to use it. Fortunately for you, I then decided to blog about it so you don’t have to repeat my mistakes.

Edit: the author of mod_md, Stefan Eissing, got in touch and pointed me to his own documentation, which is far superior to the one available from Apache.

My starting point is a freshly installed FreeBSD 13.0 server with Apache 2.4, but this isn’t really OS dependent.

First, you will need mod_ssl (of course) and a session cache, and you will need to tweak the TLS parameters, as the defaults are far from fine.

LoadModule ssl_module libexec/apache24/mod_ssl.so
SSLProtocol +TLSv1.3 +TLSv1.2
SSLCipherSuite TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
SSLHonorCipherOrder off
SSLCompression off

LoadModule socache_dbm_module libexec/apache24/mod_socache_dbm.so
SSLSessionCache dbm:/var/db/httpd_ssl_cache.db

You will also need to load mod_md, of course, and mod_watchdog, which mod_md needs to function.

LoadModule watchdog_module libexec/apache24/mod_watchdog.so
LoadModule md_module libexec/apache24/mod_md.so
MDCertificateAgreement accepted
MDContactEmail acme@example.com

The MDCertificateAgreement directive indicates that you have read and accepted Let’s Encrypt’s subscriber agreement, while MDContactEmail is the email address that you used to sign up to Let’s Encrypt.

You will also need mod_rewrite to redirect HTTP requests to HTTPS and mod_headers for HSTS.

LoadModule rewrite_module libexec/apache24/mod_rewrite.so
LoadModule headers_module libexec/apache24/mod_headers.so

By default, Apache only listens on port 80, so you’ll need an extra Listen directive for port 443.

Listen 443

And as always with Apache, you should probably set ServerName and ServerAdmin to sensible values.

ServerName server.example.com
ServerAdmin www@example.com

Next, set up an HTTP-only virtual host that you can use to check the status of mod_md.

<VirtualHost *:80>
  ServerName localhost
  <Location />
    Require ip 127.0.0.1/8 ::1
  </Location>
  <Location "/md-status">
    SetHandler md-status
  </Location>
</VirtualHost>

(Once Apache is running, you will be able to query it at any time as http://localhost/md-status.)

On to the actual website. First, you need to tell mod_md to manage certificates for it.

MDomain site.example.com

Next, set up a redirect from HTTP to HTTPS for everything except ACME challenge tokens.

<VirtualHost localhost:80>
  ServerName site.example.com
  RewriteEngine on
  RewriteRule "^/(?!.well-known/acme-challenge)(.*)" https://site.example.com/$1 [R=301,L]
  ErrorLog /www/site.example.com/logs/http-error.log
  CustomLog /www/site.example.com/logs/http-access.log combined
</VirtualHost>

And finally, the site itself, including HSTS and strict SNI:

<VirtualHost *:443>
  ServerName site.example.com
  SSLEngine on
  SSLStrictSNIVHostCheck On
  Header always set Strict-Transport-Security "max-age=15552000; includeSubdomains;"
  DocumentRoot /www/site.example.com/data
  IncludeOptional /www/site.example.com/etc/*.conf
  ErrorLog /www/site.example.com/logs/https-error.log
  CustomLog /www/site.example.com/logs/https-access.log combined
</VirtualHost>

Now start Apache and monitor the error log. You should see something like this pretty quickly:

[Sun Oct 10 16:15:27.450401 2021] [md:notice] [pid 12345] AH10059: The Managed Domain site.example.com has been setup and changes will be activated on next (graceful) server restart.

Once you do as it says (apachectl graceful), your site will be up and running and you can head over to the Qualys SSL Server Test and admire your solid A+.

Download the sample configuration and try it out yourself.

by Dag-Erling Smørgrav atOctober 10, 2021 06:19 PM

September 10, 2021

Petter Reinholdtsen

Hvilke partier støttet datalagringsdirektivet 2 og 3?

Relativt stille har Stortinget den siste perioden vedtatt datalagringsdirektivet versjon to og tre, og slik økt overvåkningstrykket på Internett i Norge betraktelig. Det blir mer og mer viktig å bruke Tor og god kryptering for å sikre sin privatsfære på Internett.

Først ut var ny etterretningstjenestelov vedtatt 2020-06-11, der samtlige partier på stortinget (Arbeiderpartiet, Fremskrittspartiet, Høyre, Kristelig Folkeparti, Miljøpartiet De Grønne, Senterpartiet, Sosialistisk Venstreparti, Venstre og et forhenværende Fremskrittspartimedlem) unntatt Rødt støttet loven i sin helhet da den ble vedtatt.

Dernest kom ny ekomlov vedtatt 2021-06-08 med støtte fra Arbeiderpartiet, Fremskrittspartiet, Høyre, Kristelig Folkeparti, Senterpartiet, Venstre og et forhenværende Fremskrittspartimedlem mot stemmene fra Sosialistisk Venstreparti, Miljøpartiet De Grønne og Rødt.

Intet parti som støtter slike inngrep i privatsfæren får min støtte i valg. Heldigvis er det mange parter også utenfor Stortinget som kan ha nytte av den økonomiske støtten som følger av hver stemme de får i stortingsvalget i år, selv om de ikke kommer inn på Stortinget i denne omgang.

Som et annet målepunkt på hvor mye vern av privatsfæren betyr for stortingspartiene kan jeg fortelle at ikke et eneste av partiene på Stortinget har besvart brevet med spørsmål rundt vern av privatsfæren som de fikk i sommer.

Hvis du lurer på hva som er problemet med datalagringsdirektivet, anbefaler jeg å lese artiklene fra Jon Wessel-Aas om temaet.

Som vanlig, hvis du bruker Bitcoin og ønsker å vise din støtte til det jeg driver med, setter jeg pris på om du sender Bitcoin-donasjoner til min adresse 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b. Merk, betaling med bitcoin er ikke anonymt. :)

September 10, 2021 06:35 AM

August 17, 2021

Salve J. Nilsen

Protected: A modest overture for a modest future

There is no excerpt because this is a protected post.

by sjn atAugust 17, 2021 07:18 PM

May 15, 2021

NUUG news

Vet du hva du mister når du bare klikker OK for å komme i gang med å bruke noe?

Retten til privatlivets fred, retten til å reparere og retten til å velge verktøy er sider av samme sak. En ny rettsavgjørelse i Italia kan hjelpe oss å vinne tilbake rettigheter vi ble manipulert til å si fra oss.

Du tenker nok ikke på det så ofte, men om du er en vanlig IT-bruker i et industrialisert land har du sannsynligvis blitt lurt til å si fra deg rettigheter. Dette skjer i et slikt omfang at menneskerettsinteresserte burde være bekymret.

Tenk på når du skal ta i bruk noe du er interessert i, enten det er en datamaskin av noe slag som for eksempel PC, nettbrett eller telefon, eller en nettbasert tjeneste.

La oss først se nærmere på hva som skjer når du får ny datamaskin, nettbrett eller telefon i hus. Noe av det første som skjer etter at du har slått på strømmen for den nye enheten, og helt sikkert før du får mulighet til å bruke dingsen til det du ønsker å gjøre, er at du må godta en juridisk bindende avtale som er utformet av og for de som har produsert utstyret. For å kunne bruke det du har kjøpt, må du godta en avtale som styrer hva du kan bruke enheten til.

I mange tilfeller er det flere slike avtaler som blir presentert, hver med sin egen registrering av om du godtar eller ikke.

Noen av disse avtalene begrenser hva du kan bruke enheten til, mens andre gir leverandøren eller noen som samarbeider med leverandøren lov til å samle inn informasjon om deg og hva du foretar deg med enheten.

Mange av disse ja/nei-spørsmålene gir inntrykk av at du har mulighet til å nekte å godta, men du vil se at du sannsynligvis ikke kommer videre til å ha en gjenstand som er reelt brukbar til tiltenkt bruk før du har godtatt alle disse avtalene.

En av de mest tydelige konsekvensene av COVID 19-krisen er at en større andel av befolkningen ble presset over til nesten helt digital tilværelse, der kommunikasjon både i jobb- og skolesammenheng foregår via digitale enheter og via tjenester som leveres på vilkår av avtaler som er diktert av leverandørene. For noen av oss har tilværelsen vært nær heldigital i en årrekke allerede, men for mange er det en ny situasjon og det går langsomt opp for flere at viktige friheter og rettigheter kan være i ferd med å gå tapt.

Problemstillingen er ikke ny. Mange av oss i IT-miljøer har lenge advart mot at det vi regner som menneskerettigheter eller borgerrettigheter er i ferd med å bli gradvis slipt vekk til fordel for enkelte bedrifter og deres eiere.

Når du slår på en ny datamaskin eller telefon for første gang, blir du sannsynligvis nesten med en gang bedt om å godta en "sluttbrukerlisens" for operativsystemet, altså programvaren som styrer enheten. I sin enkleste form er en lisens et dokument som angir vilkårene for at noen andre enn den som har laget et åndsverk (her programvaren) får tillatelse til å lage eksemplarer av verket. Men i mange tilfeller inneholder lisensdokumentet mer detaljerte og omfattende vilkår. Ofte er lisensavtalen formulert som om du har rett til å avslå å bruke operativsystemet og slette eksemplarer som følger med eller levere tilbake fysiske eksemplarer og få tilbake pengene, men at du kan fortsette å bruke den fysiske maskinen. En del av oss som har kjøpt PCer og annet har vært i stand til å installere et annet system enn det som ble levert med maskinen, og valgt å leve det digitale livet ved hjelp av frie alternativer som for eksempel Linux eller OpenBSD. En del av oss gjør dette for å få mer direkte kontroll over verktøyene vi bruker.

Om vi har forsøkt å få tilbake penger for en ubrukt operativsystemlisens har de fleste av oss aldri klart å få det til. Men det skal vi komme tilbake til.

Om du har klart å installere et fritt alternativ til det operativsystemet som enheten ble levert med, har du slått et slag for retten til å velge verktøy og retten til å reparere og råde over dine egne eiendeler. Men dessverre er ikke dette det eneste punktet i ditt digitale liv der rettighetene dine er i fare.

Uansett om du godtok sluttbrukerlisensen eller ikke, kommer du fort ut for for programvare eller nettbaserte tjenester som presenterer sine egne sluttbrukeravtaler. Det er en stor sjanse for at du bare klikker OK uten å lese vilkårene i avtalen.

Ta gjerne nå en pause for å sjekke hva du faktisk har gått med på. Sannsynligvis finner du at både operativsystemleverandører og sosiale medier-tjenester har fått deg til å gi dem tillatelse til å registrere hva du foretar deg når du bruker systemet eller tjenesten. Ta gjerne tiden til å sjekke alle produkter og tjenester du har registrert deg hos. Det er sannsynlig at ikke bare en, men de aller fleste av de tjenestene og produktene du bruker på en nett-tilkoblet enhet har gitt seg selv retten til å fange inn og lagre data om hva du foretar deg. Hvis du bruker enheten til noe som helst privat eller følsomt, er det verd å se nøye etter hvilke konsekvenser disse avtalene har for din rett til privatliv og beskyttelse av privatsfæren.

På papiret (om vi skal uttrykke oss gammeldags) har vi som bor i EU og EØS-land rett til å få utlevert data som er lagret om oss og eventuelt få rettet feil eller til og med få slettet data i samsvar med EUs personvernforordning (GDPR). Hvis det du fant ut mens du sjekket avtalene mens du tok pause fra å lese denne teksten gjør deg usikker eller bekymret er det god grunn til å ta i bruk retten til innsyn, utlevering, retting eller sletting. Om du ikke får meningsfylt svar, ta kontakt med Datatilsynet eller Forbrukertilsynet, som bør stå klare til å hjelpe.

Men hva så med retten til å reparere eller retten til å velge verktøy? Jo, også på det feltet er det grunn til håp. Etter en omfattende prosess kom nemlig en domstol i Italia frem til at ikke bare hadde en Linux-entusiast rett til å installere Linux på sin nye Lenovo-datamaskin, slik at kunden også hadde rett til å refundert prisen for operativsystemet som ikke ville bli brukt. Og siden Lenovo hadde prøvd å ikke etterleve sine forpliktelser som var angitt i sluttbrukerlisensen som ble presentert for kunden, ble de ilagt en bot på 20 000 Euro.

En slik rettsavgjørelse er ikke direkte presedensskapende for andre europeiske land, og det finnes avgjørelser i andre land som ikke ga kunden medhold i at operativsystem og datamaskin kunne behandles som separate varer. Vi i den norske Unix-brukergruppen (Norwegian Unix User Group - NUUG) deltar nå i et samarbeid som koordineres av Free Software Foundation Europe (FSFE) for å forsvare og styrke din og min rett til privatliv, rett til å reparere og rett til å velge verktøy for å styre vår digitale tilværelse.

Hvis noe av det du nå har lest bekymrer deg, gjør deg forvirret, sint eller bare engasjert for å styrke våre borger- og menneskeretter i den digitale tilværelsen vil vi bli glade for å høre fra deg.

Peter N. M. Hansteen
Styreleder i Norwegian Unix User Group (NUUG)

Den italienske rettsavgjørelsen som gir oss håp er beskrevet på FSFEs nettsted: Refund of pre-installed Windows: Lenovo must pay 20,000 euros in damages

An English version is available as Are you aware what you lose by just clicking OK to get started using something?

May 15, 2021 11:10 AM

May 11, 2021

Nicolai Langfeldt

DLNA servers for our Samsung TV on Linux

We have a Samsung Smart TV, I like it. We also have a cable box, a BluRay player (because sometimes we borrow movies at the library and anyone in the family needs to be able to play them with no help from dad), and a Chromecast.  All three HDMI inputs on the TV are used.

Samsung was pretty big on DLNA, a UPnP based protocol for media playback. Now it's not a feature they tout a lot: People want Netflix. Samsung also had very good codec support from the start which reduces the need for transcoding greatly.  They still have the DLNA client builtin in their TVs and the codec support is even better now.  So this solves the in-house streaming problem very neatly without needing a extra box by the TV to do it. The server in the basement ought to be enough when the TV had DLNA.

I understand that DLNA is a bit low featured in 2021: no on screen movie/episode synopsis, not very slick navigation and on screen controls for everything. But it's enough. DLNA is right featured. The Samsung DLNA client has very nice movie navigation and on screen menus for subtitle selection. And I don't particularly want a TV box though it would certainly play all kinds of video with no issues at all and support subs better and have a nice graphical interface. It seems I'm not in tune with the world on this though: DLNA servers seem to be more rare than before.

Since the start around 2010 I've used tvMobili, a closed source but fully functional DLNA server software that just worked with our TV, including on screen subtitles without transcoding. It also just worked with phone based DLNA software. Since my wife is deaf and not all the kids are that steady in english yet subtitles are a required feature. Subtitles are also nice for late night TV viewing with low volume.

Recently the disk the tvMobili software stored its database on went full and the database went corrupt.  I tidied the disk and nuked the database. Unfortunately that reset the software and it gave me it's out of the box experience: "Please create your account at tvMobili to proceed". tvMobili, the company, gave up in 2013. There was no registration service running any more and I could not get past the registration screen. And also I had no backup of that directory since it only contained stuff I could reconstruct anyway. I thought. (I realize I might try to write the registration service and run it locally, but maybe time to try something new?).

So what to do for in-house streaming now then? This is important to me:

• Subtitles
• Works with Samsung TV
• and phones
• Minimal need for transcoding
• No cloud required. I don't like to be dependent on the cloud to play my own movies
• No external box needed - other than the server
• I will use closed source for a reasonable price ... if it works

The candidates.

• Plex is a commercial product which I've known of for some years.  It was the first software I tried. I though I might be able to use it paying quite reasonable one time fees since I don't want their internet streaming services, just the in-house streaming functions. They have a app for our TV, for phones and a DLNA service.  Also very slick GUI and very nice meta-data. The GUI features a very nice browser based video player which is handy.
• Unfortunately right now subtitle support in the TV app is buggy and has been since about October 2020.
• And the DLNA service less than reliable: the TV rejected files that were never a problem or transcoded by tvMobili. Other files played, with subtitles.
• Plex is probably cloud dependent.

• Kodi: It has a slick GUI that many folks run on a Raspberry Pi (or something like that) attached to their TV. I set up the GUI in a Xvnc server, configured, enabled DLNA and tested it on TV and phone. Kodi also has a metadata module which finds stuff about your movies and series on the internet.
• The DLNA service works perfectly with my phone. Better than tvMobili.
• All the files I tested play on the TV, but there are two issues:
• Directory listings may be truncated and lack titles on the icons
• Subs have to be TEXT in matroska files to work on the TV. SRT files does not work, nor does MOV_TEXT in mp4 files. There is a fairly simple and lossless way to mux the subs into matroska files using ffmpeg so this is manageable.
• Universal Media Server - UMS: It specializes in DLNA playback only and their website features TV manufacturers' icons. A colleague used it in the past with great success with his Samsung TV. It has a web based player too.
• Unfortunately it does not play all videos on our TV
• But when it does subs of all kinds are shown
• Did not test on phone
• Jellyfin: It has a nice web GUI with a very polished out of the box experience.  It supports multiple users that don't share libraries.  It has all the usual support for metadata. It plays all the files I've tested, and supports subs.  On the TV and on the phone.  There is some issue on the TV with going automatically to the next episode. Which I can live with.  There is no TV app - not a problem.  There is no GUI for running on a TV-attached computer - not a problem.
• Emby: The origin of jellyfin. Formerly open source, now closed source, and I found that Jellyfin works very nicely so I did not test.

by nicolai (noreply@blogger.com) atMay 11, 2021 12:54 PM

May 04, 2021

Nicolai Langfeldt

New Linux on old hardware

Since our family relies on the servers in our basement for email, music, movies, and so on it's been bothering me that we have no extra hardware in case something goes wrong.

I've been given two old Dell machines, I want to use one for stuff and one as spares in case something breaks.

Both needed reinstalling: I like Ubuntu and made myself a memory stick with Ubutnus usb-creator software.  The machines never managed to boot of those.  The Ubuntu server ISOs are too large to fit on a CD. Found ubuntus netboot ISOs for 18.04 which fits a CD ten times over.  Again they didn't boot from the memory stick nor off a CD with various error messages or just passing it by and booting off the old OS on the harddrive.

After a while I recalled "unetbootin" - a old, but still updated - tool to make memory sticks from ISO images. It currently lives here: https://unetbootin.github.io/

I had to repartition my memory stick (cfdisk /dev/sdc in my case) and make a msdos filesystem (mkfs.msdos /dev/sdc1) and mount it.

Then unetbootin could make it bootable and indeed my quite old hardware was able to see my memory stick as a hard drive and boot for it with no further issues.

Win of the day.

by nicolai (noreply@blogger.com) atMay 04, 2021 08:29 PM

March 07, 2021

NUUG Foundation

Reisestipend - 2021

NUUG Foundation utlyser reisestipender for 2021. Søknader kan sendes inn til enhver tid.

March 07, 2021 09:46 AM

September 22, 2020

Dag-Erling Smørgrav

wtf, zsh

wtf, zsh

% uname -sr
FreeBSD 12.1-RELEASE-p10
% for sh in sh csh bash zsh ; do printf "%-8s" $sh ; $sh -c 'echo \\x21' ; done 
sh      \x21
csh     \x21
bash    \x21
zsh     !
% cowsay wtf, zsh       
 __________ 
< wtf, zsh >
 ---------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

I mean. Bruh. I know it’s intentional & documented & can be turned off, but every other shell defaults to POSIX semantics…

BTW:

% ln -s =zsh /tmp/sh
% /tmp/sh -c 'echo \x21'
\x21

by Dag-Erling Smørgrav atSeptember 22, 2020 01:11 PM

September 18, 2020

NUUG events video archive

Introduksjon til bygging av Debianpakker

September 18, 2020 05:15 AM

July 10, 2020

Salve J. Nilsen

A FIXIT-dive into an old CPAN module

Let’s have a thought experiment. Assume there is an Open Source-licensed Perl module published on CPAN that you care about, and that hasn’t had any updates in a very long time – what are your options? In this blog post, I’ll take a dive into this problem, and use the Geo::Postcodes::NO module as an example. … Continue reading A FIXIT-dive into an old CPAN module →

by sjn atJuly 10, 2020 04:25 PM

May 19, 2020

NUUG news

NUUG bygger bokskanner - arbeidet er i gang

Det finnes millioner av bøker der vernetiden er utløpt. Noen av dem er norske bøker, og endel av dem finnes ikke tilgjengelig digitalt. For å forsøke å gjøre noe med det siste, har NUUG vedtatt å få bygget en bokskanner. Utformingen er basert på en enkel variant i plast (byggeinstrukser), men vil bli laget i aluminium for lengre levetid.

Oppdraget med å bygge scanneren er gitt til våre venner i Oslo Sveisemek, som er godt igang med arbeidet. Her ser du en skisse over konstruksjonen:

Grunnrammen er montert, men det gjenstår fortsatt en god del:

Tanken er at medlemmer og andre skal kunne låne eller leie bokskanner ved behov, og de av oss som er interessert kan gå igang med å digitalisere bøker med OCR og pågangsmot. Ta kontakt med aktive (at) nuug.no hvis dette er noe for deg, eller stikk innom #nuug.

(Fotograf er Jonny Birkelund)

May 19, 2020 06:00 PM

December 15, 2019

NUUG Foundation

Reisestipend - 2020

NUUG Foundation utlyser reisestipender for 2020. Søknader kan sendes inn til enhver tid.

December 15, 2019 09:46 AM

May 31, 2018

Kevin Brubeck Unhammer

Kan samisk brukes i det offentlige rom?

Hvis vi hadde laget et program som oversatte fra norsk til samisk, ville resultatet ha vært en samisk som er minst like dårlig som den norsken vi er i stand til å lage nå. Norsk og samisk er grammatisk sett svært ulike, og det er vanskelig å få til god samisk på grunnlag av norsk. Et slikt program vil føre til publisering av en hel masse svært dårlig samisk. En situasjon der mesteparten av all samisk publisert på internett kommer fra våre program fortoner seg som et mareritt. Det ville rett og slett ha ødelagt den samiske skriftkulturen.

Sjå kronikken: https://www.nordnorskdebatt.no/samisk-sprak/digitalisering/facebook/kan-samisk-brukes-i-det-offentlige-rom/o/5-124-48030

by unhammer atMay 31, 2018 09:00 AM

October 23, 2017

Espen Braastad

ZFS NAS using CentOS 7 from tmpfs

Following up on the CentOS 7 root filesystem on tmpfs post, here comes a guide on how to run a ZFS enabled CentOS 7 NAS server (with the operating system) from tmpfs.

Hardware

• HP ProLiant MicroServer
• HP ProLiant MicroServer Remote Access Card
• AMD Turion™ II Neo N54L Dual-Core Processor
• 2x 8 GB 1333 MHZ ECC memory modules
• 4x 4 TB SATA hard drives (HGST, Western Digital and Seagate)
• Built-in Broadcom Limited NetXtreme BCM5723 Gigabit Ethernet adapter

Preparing the build environment

The disk image is built in macOS using Packer and VirtualBox. Virtualbox is installed using the appropriate platform package that is downloaded from their website, and Packer is installed using brew:

$ brew install packer

Building the disk image

Three files are needed in order to build the disk image; a Packer template file, an Anaconda kickstart file and a shell script that is used to configure the disk image after installation. The following files can be used as examples:

• template.json (Packer template example file)
• ks.cfg (Anaconda kickstart example file)
• provision.sh (Provision shell script example file)

Create some directories:

$ mkdir ~work/centos-7-zfs/
$ mkdir ~work/centos-7-zfs/http/
$ mkdir ~work/centos-7-zfs/scripts/

Copy the files to these directories:

$ cp template.json ~work/centos-7-zfs/
$ cp ks.cfg ~work/centos-7-zfs/http/
$ cp provision.sh ~work/centos-7-zfs/scripts/

Modify each of the files to fit your environment.

Start the build process using Packer:

$ cd ~work/centos-7-zfs/
$ packer build template.json

This will download the CentOS 7 ISO file, start an HTTP server to serve the kickstart file and start a virtual machine using Virtualbox:

The virtual machine will boot into Anaconda and run through the installation process as specified in the kickstart file:

When the installation process is complete, the disk image will be available in the output-virtualbox-iso folder with the vmdk extension.

The disk image is now ready to be put in initramfs.

Putting the disk image in initramfs

This section is quite similar to the previous blog post CentOS 7 root filesystem on tmpfs but with minor differences. For simplicity reasons it is executed on a host running CentOS 7.

Create the build directories:

$ mkdir /work
$ mkdir /work/newroot
$ mkdir /work/result

Export the files from the disk image to one of the directories we created earlier:

$ export LIBGUESTFS_BACKEND=direct
$ guestfish --ro -a packer-virtualbox-iso-1508790384-disk001.vmdk -i copy-out / /work/newroot/

Modify /etc/fstab:

$ cat > /work/newroot/etc/fstab << EOF
tmpfs       /         tmpfs    defaults,noatime 0 0
none        /dev      devtmpfs defaults         0 0
devpts      /dev/pts  devpts   gid=5,mode=620   0 0
tmpfs       /dev/shm  tmpfs    defaults         0 0
proc        /proc     proc     defaults         0 0
sysfs       /sys      sysfs    defaults         0 0
EOF

Disable selinux:

echo "SELINUX=disabled" > /work/newroot/etc/selinux/config

Disable clearing the screen on login failure to make it possible to read any error messages:

mkdir /work/newroot/etc/systemd/system/getty@.service.d
cat > /work/newroot/etc/systemd/system/getty@.service.d/noclear.conf << EOF
[Service]
TTYVTDisallocate=no
EOF

Now jump to the Initramfs and Result sections in the CentOS 7 root filesystem on tmpfs and follow those steps until the end when the result is a vmlinuz and initramfs file.

ZFS configuration

The first time the NAS server boots on the disk image, the ZFS storage pool and volumes will have to be configured. Refer to the ZFS documentation for information on how to do this, and use the following command only as guidelines.

Create the storage pool:

$ sudo zpool create data mirror sda sdb mirror sdc sdd

Create the volumes:

$ sudo zfs create data/documents
$ sudo zfs create data/games
$ sudo zfs create data/movies
$ sudo zfs create data/music
$ sudo zfs create data/pictures
$ sudo zfs create data/upload

Share some volumes using NFS:

zfs set sharenfs=on data/documents
zfs set sharenfs=on data/games
zfs set sharenfs=on data/music
zfs set sharenfs=on data/pictures

Print the storage pool status:

$ sudo zpool status
  pool: data
 state: ONLINE
  scan: scrub repaired 0B in 20h22m with 0 errors on Sun Oct  1 21:04:14 2017
config:

	NAME        STATE     READ WRITE CKSUM
	data        ONLINE       0     0     0
	  mirror-0  ONLINE       0     0     0
	    sdd     ONLINE       0     0     0
	    sdc     ONLINE       0     0     0
	  mirror-1  ONLINE       0     0     0
	    sda     ONLINE       0     0     0
	    sdb     ONLINE       0     0     0

errors: No known data errors

October 23, 2017 11:20 PM

February 13, 2017

Mimes brønn

En innsynsbrønn full av kunnskap

Mimes brønn er en nettjeneste som hjelper deg med å be om innsyn i offentlig forvaltning i tråd med offentleglova og miljøinformasjonsloven. Tjenesten har et offentlig tilgjengelig arkiv over alle svar som er kommet på innsynsforespørsler, slik at det offentlige kan slippe å svare på de samme innsynshenvendelsene gang på gang. Du finner tjenesten på

https://www.mimesbronn.no/

I følge gammel nordisk mytologi voktes kunnskapens kilde av Mime og ligger under en av røttene til verdenstreet Yggdrasil. Å drikke av vannet i Mimes brønn ga så verdifull kunnskap og visdom at den unge guden Odin var villig til å gi et øye i pant og bli enøyd for å få lov til å drikke av den.

Nettstedet vedlikeholdes av foreningen NUUG og er spesielt godt egnet for politisk interesserte personer, organisasjoner og journalister. Tjenesten er basert på den britiske søstertjenesten WhatDoTheyKnow.com, som allerede har gitt innsyn som har resultert i dokumentarer og utallige presseoppslag. I følge mySociety for noen år siden gikk ca 20 % av innsynshenvendelsene til sentrale myndigheter via WhatDoTheyKnow. Vi i NUUG håper NUUGs tjeneste Mimes brønn kan være like nyttig for innbyggerne i Norge.

I helgen ble tjenesten oppdatert med mye ny funksjonalitet. Den nye utgaven fungerer bedre på små skjermer, og viser nå leveringsstatus for henvendelsene slik at innsender enklere kan sjekke at mottakers epostsystem har bekreftet mottak av innsynshenvendelsen. Tjenesten er satt opp av frivillige i foreningen NUUG på dugnad, og ble lansert sommeren 2015. Siden den gang har 121 brukere sendt inn mer enn 280 henvendelser om alt fra bryllupsutleie av Operaen og forhandlinger om bruk av Norges topp-DNS-domene .bv til journalføring av søknader om bostøtte, og nettstedet er en liten skattekiste av interessant og nyttig informasjon. NUUG har knyttet til seg jurister som kan bistå med å klage på manglende innsyn eller sviktende saksbehandling.

– «NUUGs Mimes brønn var uvurderlig da vi lyktes med å sikre at DNS-toppdomenet .bv fortsatt er på norske hender,» forteller Håkon Wium Lie.

Tjenesten dokumenterer svært sprikende praksis i håndtering av innsynshenvendelser, både når det gjelder responstid og innhold i svarene. De aller fleste håndteres raskt og korrekt, men det er i flere tilfeller gitt innsyn i dokumenter der ansvarlig etat i ettertid ønsker å trekke innsynet tilbake, og det er gitt innsyn der sladdingen har vært utført på en måte som ikke skjuler informasjonen som skal sladdes.

– «Offentlighetsloven er en bærebjelke for vårt demokrati. Den bryr seg ikke med hvem som ber om innsyn, eller hvorfor. Prosjektet Mimes brønn innebærer en materialisering av dette prinsippet, der hvem som helst kan be om innsyn og klage på avslag, og hvor dokumentasjon gjøres offentlig. Dette gjør Mimes Brønn til et av de mest spennende åpenhetsprosjektene jeg har sett i nyere tid.» forteller mannen som fikk åpnet opp eierskapsregisteret til skatteetaten, Vegard Venli.

Vi i foreningen NUUG håper Mimes brønn kan være et nyttig verktøy for å holde vårt demokrati ved like.

by Mimes Brønn atFebruary 13, 2017 02:07 PM

January 06, 2017

Espen Braastad

CentOS 7 root filesystem on tmpfs

Several years ago I wrote a series of posts on how to run EL6 with its root filesystem on tmpfs. This post is a continuation of that series, and explains step by step how to run CentOS 7 with its root filesystem in memory. It should apply to RHEL, Ubuntu, Debian and other Linux distributions as well. The post is a bit terse to focus on the concept, and several of the steps have potential for improvements.

The following is a screen recording from a host running CentOS 7 in tmpfs:

Build environment

A build host is needed to prepare the image to boot from. The build host should run CentOS 7 x86_64, and have the following packages installed:

yum install libvirt libguestfs-tools guestfish

Make sure the libvirt daemon is running:

systemctl start libvirtd

Create some directories that will be used later, however feel free to relocate these to somewhere else:

mkdir -p /work/initramfs/bin
mkdir -p /work/newroot
mkdir -p /work/result

Disk image

For simplicity reasons we’ll fetch our rootfs from a pre-built disk image, but it is possible to build a custom disk image using virt-manager. I expect that most people would like to create their own disk image from scratch, but this is outside the scope of this post.

Use virt-builder to download a pre-built CentOS 7.3 disk image and set the root password:

virt-builder centos-7.3 -o /work/disk.img --root-password password:changeme

Export the files from the disk image to one of the directories we created earlier:

guestfish --ro -a /work/disk.img -i copy-out / /work/newroot/

Clear fstab since it contains mount entries that no longer apply:

echo > /work/newroot/etc/fstab

SELinux will complain about incorrect disk label at boot, so let’s just disable it right away. Production environments should have SELinux enabled.

echo "SELINUX=disabled" > /work/newroot/etc/selinux/config

Disable clearing the screen on login failure to make it possible to read any error messages:

mkdir /work/newroot/etc/systemd/system/getty@.service.d
cat > /work/newroot/etc/systemd/system/getty@.service.d/noclear.conf << EOF
[Service]
TTYVTDisallocate=no
EOF

Initramfs

We’ll create our custom initramfs from scratch. The boot procedure will be, simply put:

1. Fetch kernel and a custom initramfs.
2. Execute kernel.
3. Mount the initramfs as the temporary root filesystem (for the kernel).
4. Execute /init (in the initramfs).
5. Create a tmpfs mount point.
6. Extract our CentOS 7 root filesystem to the tmpfs mount point.
7. Execute switch_root to boot on the CentOS 7 root filesystem.

The initramfs will be based on BusyBox. Download a pre-built binary or compile it from source, put the binary in the initramfs/bin directory. In this post I’ll just download a pre-built binary:

wget -O /work/initramfs/bin/busybox https://www.busybox.net/downloads/binaries/1.26.1-defconfig-multiarch/busybox-x86_64

Make sure that busybox has the execute bit set:

chmod +x /work/initramfs/bin/busybox

Create the file /work/initramfs/init with the following contents:

#!/bin/busybox sh

# Dump to sh if something fails
error() {
	echo "Jumping into the shell..."
	setsid cttyhack sh
}

# Populate /bin with binaries from busybox
/bin/busybox --install /bin

mkdir -p /proc
mount -t proc proc /proc

mkdir -p /sys
mount -t sysfs sysfs /sys

mkdir -p /sys/dev
mkdir -p /var/run
mkdir -p /dev

mkdir -p /dev/pts
mount -t devpts devpts /dev/pts

# Populate /dev
echo /bin/mdev > /proc/sys/kernel/hotplug
mdev -s

mkdir -p /newroot
mount -t tmpfs -o size=1500m tmpfs /newroot || error

echo "Extracting rootfs... "
xz -d -c -f rootfs.tar.xz | tar -x -f - -C /newroot || error

mount --move /sys /newroot/sys
mount --move /proc /newroot/proc
mount --move /dev /newroot/dev

exec switch_root /newroot /sbin/init || error

Make sure it is executable:

chmod +x /work/initramfs/init

Create the root filesystem archive using tar. The following command also uses xz compression to reduce the final size of the archive (from approximately 1 GB to 270 MB):

cd /work/newroot
tar cJf /work/initramfs/rootfs.tar.xz .

Create initramfs.gz using:

cd /work/initramfs
find . -print0 | cpio --null -ov --format=newc | gzip -9 > /work/result/initramfs.gz

Copy the kernel directly from the root filesystem using:

cp /work/newroot/boot/vmlinuz-*x86_64 /work/result/vmlinuz

Result

The /work/result directory now contains two files with file sizes similar to the following:

ls -lh /work/result/
total 277M
-rw-r--r-- 1 root root 272M Jan  6 23:42 initramfs.gz
-rwxr-xr-x 1 root root 5.2M Jan  6 23:42 vmlinuz

These files can be loaded directly in GRUB from disk, or using iPXE over HTTP using a script similar to:

#!ipxe
kernel http://example.com/vmlinuz
initrd http://example.com/initramfs.gz
boot

January 06, 2017 08:34 PM

July 15, 2016

Mimes brønn

Hvem har drukket fra Mimes brønn?

Mimes brønn har nå vært oppe i rundt et år. Derfor vi tenkte det kunne være interessant å få en kortfattet statistikk om hvordan tjenesten er blitt brukt.

I begynnelsen av juli 2016 hadde Mimes brønn 71 registrerte brukere som hadde sendt ut 120 innsynshenvendelser, hvorav 62 (52%) var vellykkede, 19 (16%) delvis vellykket, 14 (12%) avslått, 10 (8%) fikk svar at organet ikke hadde informasjonen, og 12 henvendelser (10%; 6 fra 2016, 6 fra 2015) fortsatt var ubesvarte. Et fåtall (3) av hendvendelsene kunne ikke kategoriseres. Vi ser derfor at rundt to tredjedeler av henvendelsene var vellykkede, helt eller delvis. Det er bra!

Tiden det tar før organet først sender svar varierer mye, fra samme dag (noen henvendelser sendt til Utlendingsnemnda, Statens vegvesen, Økokrim, Mediatilsynet, Datatilsynet, Brønnøysundregistrene), opp til 6 måneder (Ballangen kommune) eller lenger (Stortinget, Olje- og energidepartementet, Justis- og beredskapsdepartementet, UDI – Utlendingsdirektoratet, og SSB har mottatt innsynshenvendelser som fortsatt er ubesvarte). Gjennomsnittstiden her var et par uker (med unntak av de 12 tilfellene der det ikke har kommet noe svar). Det følger av offentlighetsloven § 29 første ledd at henvendelser om innsyn i forvaltningens dokumenter skal besvares «uten ugrunnet opphold», noe som ifølge Sivilombudsmannen i de fleste tilfeller skal fortolkes som «samme dag eller i alle fall i løpet av 1-3 virkedager». Så her er det rom for forbedring.

Klageretten (offentleglova § 32) ble benyttet i 20 av innsynshenvendelsene. I de fleste (15; 75%) av tilfellene førte klagen til at henvendelsen ble vellykket. Gjennomsnittstiden for å få svar på klagen var en måned (med unntak av 2 tillfeller, klager sendt til Statens vegvesen og Ruter AS, der det ikke har kommet noe svar). Det er vel verdt å klage, og helt gratis! Sivilombudsmannen har uttalt at 2-3 uker ligger over det som er akseptabel saksbehandlingstid for klager.

Flest henvendelser var blitt sendt til Utenriksdepartementet (9), tett etterfulgt av Fredrikstad kommune og Brønnøysundregistrene. I alt ble henvendelser sendt til 60 offentlige myndigheter, hvorav 27 ble tilsendt to eller flere. Det står over 3700 myndigheter i databasen til Mimes brønn. De fleste av dem har dermed til gode å motta en innsynshenvendelse via tjenesten.

Når vi ser på hva slags informasjon folk har bedt om, ser vi et bredt spekter av interesser; alt fra kommunens parkeringsplasser, reiseregninger der statens satser for overnatting er oversteget, korrespondanse om asylmottak og forhandlinger om toppdomenet .bv, til dokumenter om Myanmar.

Myndighetene gjør alle mulige slags ting. Noe av det gjøres dÃ¥rlig, noe gjør de bra. Jo mer vi finner ut om hvordan  myndighetene fungerer, jo større mulighet har vi til Ã¥ foreslÃ¥ forbedringer pÃ¥ det som fungerer dÃ¥rlig… og applaudere det som  bra.  Er det noe du vil ha innsyn i, sÃ¥ er det bare Ã¥ klikke pÃ¥ https://www.mimesbronn.no/ og sÃ¥ er du i gang

by Mimes Brønn atJuly 15, 2016 03:56 PM

June 01, 2016

Kevin Brubeck Unhammer

Maskinomsetjing vs NTNU-eksaminator

Twitter-brukaren @IngeborgSteine fekk nyleg ein del merksemd då ho tvitra eit bilete av nynorskutgåva av økonomieksamenen sin ved NTNU:

Dette var min økonomieksamen på "nynorsk". #nynorsk #noregsmållag #kvaialledagar https://t.co/RjCKSU2Fyg—
Ingeborg Steine (@IngeborgSteine) May 30, 2016

Kreative nyvinningar som *kvisleis og alle dialektformene og arkaismane ville vore usannsynlege å få i ei maskinomsett utgåve, så då lurte eg på kor mykje betre/verre det hadde blitt om eksaminatoren rett og slett hadde brukt Apertium i staden? Ingeborg Steine var så hjelpsam at ho la ut bokmålsutgåva, så då får me prøva

Ingen kvisleis og fritt for tær og fyr, men det er heller ikkje perfekt: Visse ord manglar frå ordbøkene og får dermed feil bøying, teller blir tolka som substantiv, ein anna maskin har feil bøying på førsteordet (det mangla ein regel der) og at blir ein stad tolka som adverb (som fører til det forunderlege fragmentet det verta at anteke tilvarande). I tillegg blir språket gjenkjent som tatarisk av nettsida, så det var kanskje litt tung norsk? Men desse feila er ikkje spesielt vanskelege å retta på – utviklingsutgåva av Apertium gir no:

Det er enno eit par småting som kunne vore retta, men det er allereie betre enn dei fleste eksamenane eg fekk utdelt ved UiO …

by unhammer atJune 01, 2016 09:45 AM

October 18, 2015

Anders Nordby

Fighting spam with SpamAssassin, procmail and greylisting

On my private server we use a number of measures to stop and prevent spam from arriving in the users inboxes: - postgrey (greylisting) to delay arrival (hopefully block lists will be up to date in time to stop unwanted mail, also some senders do not retry) - SpamAssasin to block mails by scoring different aspects of the emails. Newer versions of it has URIBL (domain based, for links in the emails) in addtition to the tradional RBL (IP based) block lists. Which works better. I also created my own URIBL block list which you can use, dbl.fupp.net. - Procmail. For user on my server, I recommend this procmail rule: :0 * ^X-Spam-Status: Yes .crapbox/ It will sort emails that has a score indicating it is spam into mailbox "crapbox". - blocking unwanted and dangerous attachments, particularly for Windows users.

by Anders (noreply@blogger.com) atOctober 18, 2015 01:09 PM

April 14, 2015

NUUG events video archive

20111108_lisp

April 14, 2015 11:13 AM

January 06, 2015

thefastestwaytobreakamachine

NSA-proof SSH

One of the biggest takeaways from 31C3 and the most recent Snowden-leaked NSA documents is that a lot of SSH stuff is .. broken.

I’m not surprised, but then again I never am when it comes to this paranoia stuff. However, I do run a ton of SSH in production and know a lot of people that do. Are we all fucked? Well, almost, but not really.

Unfortunately most of what Stribika writes about the “Secure Secure Shell” doesn’t work for old production versions of SSH. The cliff notes for us real-world people, who will realistically be running SSH 5.9p1 for years is hidden in the bettercrypto.org repo.

Edit your /etc/ssh/sshd_config:


Ciphers aes256-ctr,aes192-ctr,aes128-ctr
MACs hmac-sha2-512,hmac-sha2-256,hmac-ripemd160
KexAlgorithms diffie-hellman-group-exchange-sha256


Basically the nice and forward secure aes-*-gcm chacha20-poly1305 ciphers, the curve25519-sha256 Kex algorithm and Encrypt-Then-MAC message authentication modes are not available to those of us stuck in the early 2000s. That’s right, provably NSA-proof stuff not supported. Upgrading at this point makes sense.

Still, we can harden SSH, so go into /etc/ssh/moduli and delete all the moduli that have 5th column < 2048, and disable ECDSA host keys:

cd /etc/ssh
mkdir -p broken
mv moduli ssh_host_dsa_key* ssh_host_ecdsa_key* ssh_host_key* broken
awk '{ if ($5 > 2048){ print } }' broken/moduli > moduli
# create broken links to force SSH not to regenerate broken keys
ln -s ssh_host_ecdsa_key ssh_host_ecdsa_key
ln -s ssh_host_dsa_key ssh_host_dsa_key
ln -s ssh_host_key ssh_host_key

Your clients, which hopefully have more recent versions of SSH, could have the following settings in /etc/ssh/ssh_config or .ssh/config:

Host all-old-servers

    Ciphers aes256-gcm@openssh.com,aes128-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
    MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-ripemd160-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512,hmac-ripemd160
    KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256

Note: Sadly, the -ctr ciphers do not provide forward security and hmac-ripemd160 isn’t the strongest MAC. But if you disable these, there are plenty of places you won’t be able to connect to. Upgrade your servers to get rid of these poor auth methods!

Handily, I have made a little script to do all this and more, which you can find in my Gone distribution.

There, done.

Updated Jan 6th to highlight the problems of not upgrading SSH.
Updated Jan 22nd to note CTR mode isn’t any worse.
Go learn about COMSEC if you didn’t get trolled by the title.

by kacper atJanuary 06, 2015 04:33 PM

December 08, 2014

thefastestwaytobreakamachine

sound sound

Intermission..

Recently I been doing some video editing.. less editing than tweaking my system tho.
If you want your jack output to speak with Kdenlive, a most excellent video editing suite,
and output audio in a nice way without choppyness and popping, which I promise you is not nice,
you’ll want to pipe it through pulseaudio because the alsa to jack stuff doesn’t do well with phonom, at least not on this convoluted setup.

Remember, to get that setup to work, ALSA pipes to jack with the pcm.jack { type jack .. thing, and remove the alsa to pulseaudio stupidity at /usr/share/alsa/alsa.conf.d/50-pulseaudio.conf

So, once that’s in place, it won’t play even though Pulse found your Jack because your clients are defaulting out on some ALSA device… this is when you change /etc/pulse/client.conf and set default-sink = jack_out.

by kacper atDecember 08, 2014 12:18 AM

October 31, 2011

Anders Nordby

Taile wtmp-logg i 64-bit Linux med Perl?

Jeg liker å la ting skje hendelsesbasert, og har i den forbindelse lagd et script for å rsynce innhold etter opplasting med FTP. Jeg tailer da wtmp-loggen med Perl, og starter sync når brukeren er eller har blitt logget ut (kort idle timeout). Å taile wtmp i FreeBSD var noe jeg for lenge siden fant et fungerende eksempel på nettet:

$typedef = 'A8 A16 A16 L'; $sizeof = length pack($typedef, () ); while ( read(WTMP, $buffer, $sizeof) == $sizeof ) { ($line, $user, $host, $time) = unpack($typedef, $buffer); # Gjør hva du vil med disse verdiene her }

FreeBSD bruker altså bare verdiene line (ut_line), user (ut_name), host (ut_host) og time (ut_time), jfr. utmp.h. Linux (x64, hvem bryr seg om 32-bit?) derimot, lagrer en hel del mer i wtmp-loggen, og etter en del Googling, prøving/feiling og kikking i bits/utmp.h kom jeg frem til:

$typedef = "s x2 i A32 A4 A32 A256 s2 l i2 i4 A20"; $sizeof = length pack($typedef, () ); while ( read(WTMP, $buffer, $sizeof) == $sizeof ) { ($type, $pid, $line, $id, $user, $host, $term, $exit, $session, $sec, $usec, $addr, $unused) = unpack($typedef, $buffer); # Gjør hva du vil med disse verdiene her }

Som bare funker, flott altså. Da ser jeg i sanntid brukere som logger på og av, og kan ta handlinger basert på dette.

by Anders (noreply@blogger.com) atOctober 31, 2011 07:37 PM