#+TITLE: BSD #+AUTHOR: Michał Sapka #+URL: https://michal.sapka.me/bsd/ #+STARTUP: show2levels indent logdone #+HUGO_BASE_DIR: ~/ghq/vcs.sapka.me/michal-sapka-me/ #+HUGO_WEIGHT: auto #+HUGO_SECTION: bsd * DONE Daemonic BSD Site :@bsd: CLOSED: [2024-03-06 Wed 14:45] :PROPERTIES: :EXPORT_FILE_NAME: home :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract A site dedicated to BSD family of systems :EXPORT_HUGO_PAIRED_SHORTCODES: recent-updates menu img-r :END: BSDs are a family of operating systems derivative from Berkley Standard Distribution. While not a popular as GNU/Linux, they are the defect descendants of the Unix Operating System. This site is dedicated to two of those systems: FreeBSD, and OpenBSD. Those are the systems I daily run on my personal computers and servers. And, while they come with significant problems, it's very easy to fall in love them. ** History of BSD Short history of how came to be. - [[/bsd/history/][BSD history]] ** FreeBSD #+attr_shortcode: "freebsd-beastie.png" #+begin_img-r Beastie, the BSD mascott #+end_img-r FreeBSD is the oldest living BSD system. It's full of features which I found essential to my happy computer usage. It's hard to live without ZFS or Jail system. Here are my guides to those features: #+attr_shortcode: "bsd-freebsd" #+begin_menu FreeBSD #+end_menu *** FreeBSD on a Thinkpad X1 Extreme Gen2 I run FreeBSD on my personal laptop. This comes with significant problems: battery life is bad and WiFi is a hit miss. You can find some of my wins bellow: #+attr_shortcode: "bsd-thinkpad" #+begin_menu FreeBSD #+end_menu ** OpenBSD #+attr_shortcode: "openbsd.png" #+begin_img-r OpenBSD Logo #+end_img-r The sites you are browsing is running on OpenBSD. The idea behind this idea is to make it security-first. And this is what the system achieved: the most secure OS I know of. It comes with a lot of interesting (and strange) things, so it's also an interesting experiense. Here you can find some of my guides: #+attr_shortcode: "bsd-openbsd" #+begin_menu FreeBSD #+end_menu ** Varia Sometimes I write non-technical articles on technical subjects: #+attr_shortcode: "bsd-varia" #+begin_menu FreeBSD #+end_menu * OpenBSD :@bsd: :PROPERTIES: :EXPORT_HUGO_MENU: :menu bsd :parent "openbsd" :END: ** DONE OpenBSD webstack: Relayd, Httpd and Acme-client CLOSED: [2023-07-19 Mon 19:08] :PROPERTIES: :EXPORT_FILE_NAME: open-bsd-web-stack :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract How to setup the web server stack work? :EXPORT_HUGO_MENU_OVERRIDE: :identifier "obsdweb" :name "Webstack: Relayd, Httpd and Acme-Client" :END: OpenBSD comes with three great tools out of the box: - httpd(8) - an HTTP daemon - relayd(8) - a relay daemon - acme-client(1) - a client for Automatic Certificate Management Environment (ACME) With those free things, we can serve static webpages over TLS. While you most likely already use [[https://www.nginx.com/][NGINX]] or [[https://httpd.apache.org/][Apache]][fn:win], those solutions are complex. They work amazingly in enterprise environments where you have people with doctorates in NGINX configuration, but most real-world examples don't need that complexity. A static blog most likely doesn't. Let's set it up. Due to security concerns, OpenBSD comes with doas(1) instead of sudo(1). Copy `/etc/examples/doas.conf` file to `/etc/doas.conf`. For all intends, and purposes, from now on doas(1) will work the same as sudo(1). When the system boots for the very first time, ports 80 and 443 are closed, and only the SSH port is open. This alone was a nice surprise for me. But it gets better: since all utilities are part of the OSes, they work together perfectly. Assuming your domain is already pointing at the correct IPs, let's start listening for unencrypted HTTP traffic. I will use "michal.sapka.me" as the domain in all examples. First, Open =/etc/httpd.conf= in your favorite editor and add #+begin_src shell server "michal.sapka.me" { listen on * port 80 root "/htdocs/michal-sapka-me" } #+end_src Then create a simple HTML file under =/var/www/htdocs/michal-sapka-me/index.html=. Httpd(8) works chrooted to /var/www/, so it threats this directory as root. This makes the "root" option shorter to write, but it also means that the process doesn't have access to anything outside of /var/www/. Even if an attacker can break in via the daemon, he will be locked in the www folder, so there is no risk to the rest of the system. As I said, OpenBSD is secure by default[fn:nginx-sec]. All we need to do now it to enable the daemon via the handy rcctl(8) tool. #+begin_src shell $ doas rcctl enable httpd #+end_src and to start it #+begin_src shell $ doas rcctl start httpd #+end_src And boom. Opening http://michal.sapka.me shows on our site both on IPv4 and IPv6. One thing to note here is the limitation of up to HTTP 1.1. HTTP 2 is not yet supported. Let's add TLS, so we have this cute lock icon. For this, we will request a certificate from [[https://letsencrypt.org/][Let's Encrypt]] using acme-client(1). If you used certbot, this will look familiar - just tidier. First, let's add config to =/etc/acme-client.conf= #+begin_src shell -n authority letsencrypt { api url "https://acme-v02.api.letsencrypt.org/directory" account key "/etc/acme/letsencrypt-privkey.pem" } authority letsencrypt-staging { api url "https://acme-staging.api.letsencrypt.org/directory" account key "/etc/acme/letsencrypt-staging-privkey.pem" } domain michal.sapka.me { domain key "/etc/ssl/private/michal.sapka.me.key" domain full chain certificate "/etc/ssl/michal.sapka.me.crt" sign with letsencrypt } #+end_src Lines 1-9 tell our acme-client(1) how to talk with Let's Encrypt, while lines 11-15 allow us to request a certificate for our domain. OpenBSD comes preconfigured for Let's Encrypt, so we just enable provided settings. Nice! Next, we need to allow Let's Encrypt challenges. Acme-client(1) will manage all required files, and Let's Encrypt can read them via httpd(8). Again, like cogs in a well-oiled machine. By default, acme-client(1) will write to =/var/www/acme=, so we need to redirect =/.well-known/acme-challenge/*= there. Let's change our =httpd.conf=: #+begin_src shell server "michal.sapka.me" { listen on * port 80 root "/htdocs/michal-sapka-me" location "/.well-known/acme-challenge/*" { root "/acme" request strip 2 } } #+end_src We can now either restart httpd(8) or reload it. Let's for the latter. #+begin_src shell $ doas rcctl reload httpd #+end_src Now we can request the certificates #+begin_src shell $ doas rcctl reload httpd $ doas acme-client -v michal.sapka.me #+end_src OpenBSDs supplied tools don't print unnecessary information to the user, so we add the =-v= to see what's happening. Assuming everything went fine, let's start serving the page with TLS! For this, we will use relayd(8). We could use only httpd(8), but moving it one layer up is easier. Relayd(8) also gives us nice options for changing headers or moving some locations to a different process, like we will do with Plaroxy soon. This also shows us the big difference between this simple solution and NGINX: while NGINX shovels everything into one process and config, OpenBSD splits it into narrow focus areas. Let's open =/etc/relayd.conf= and add: #+begin_src shell -n table { 127.0.0.1 } http protocol "https" { tls keypair "michal.sapka.me" match request quick header "Host" value "michal.sapka.me" forward to } relay "https" { listen on 0.0.0.0 port 443 tls protocol https forward to port 8080 } relay "https6" { listen on :: port 443 tls protocol https forward to port 8080 } #+end_src Now, I won't go into much detail here, but what happens here is: 1. We create two relays, one for ipv4 and one for ipv6. One relay can listen on a single port for given IP. Each relay uses protocol "https" to modify and steer the request to a given process. 2. Both relays set up forwarding to httpd (IP taken from the table on the head of the file) on port 8080. 3. https protocol adds a TLS key pair for the session. We've got the files from Let's Encrypt in the step above. 4. We then test each request, and if the host matches "michal.sapka.me" it will be forwarded to httpd(8). You can also see that relayd(8) can listen on a given IP or all IPs (:: in case of IPv6) But our httpd(8) listens only on port 80! Let's fix that by changing the `httpd.conf` file: #+begin_src shell server "michal.sapka.me" { listen on * port 8080 #+end_src We also need to redirect HTTP to HTTPS. Since we use Relayd(8) only for HTTPS, this will be done in httpd(8). Let's add a second server to our `httpd.conf`: #+begin_src shell server "michal.sapka.me" { listen on * port 80 location * { block return 301 "https://$HTTP_HOST$REQUEST_URI" } } #+end_src Now, when the user enters the site, the flow will look like: 1. httpd(8) will respond to :80 requests and return a 301 redirect to HTTPS 2. relayd(8) will catch the request to :443 and forward it on port :8080 to httpd(8) 3. httpd(8) will serve our site and pass the response to relayd(8) again 4. relayd(8) can modify headers before returning the response to the client. Talking about modifying headers, let's apply some extra security! We can expand our https protocol with the following: #+begin_src shell # Return HTTP/HTML error pages to the client return error match request header set "X-Forwarded-For" value "$REMOTE_ADDR" match request header set "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT" match response header remove "Server" match response header append "Strict-Transport-Security" value "max-age=31536000; includeSubDomains" match response header append "X-Frame-Options" value "SAMEORIGIN" match response header append "X-XSS-Protection" value "1; mode=block" match response header append "X-Content-Type-Options" value "nosniff" match response header append "Referrer-Policy" value "strict-origin" match response header append "Content-Security-Policy" value "default-src https:; style-src 'self' \ 'unsafe-inline'; font-src 'self' data:; script-src 'self' 'unsafe-inline' 'unsafe-eval'" match response header append "Permissions-Policy" value "accelerometer=(), camera=(), \ geolocation=(), gyroscope=(), magnetometer=(), microphone=(), payment=(), usb=()" # set recommended tcp options tcp { nodelay, sack, socket buffer 65536, backlog 100 } # set up certs tls { no tlsv1.0, ciphers "HIGH:!aNULL:!SSLv3:!DSS:!ECDSA:!RSA:-ECDH:ECDHE:+SHA384:+SHA256" } #+end_src I won't discuss details here as each header has a dedicated MDM webdoc. Most of the headers here are considered a standard. Besides adding headers, we configure TLS here, disabling weak ciphers and old TLS versions and adding some standard config. Lastly, we can automate refreshing the certificate via cron(8): #+begin_src shell 0~59 0~23 * * 1 acme-client michal.sapka.me && rcctl reload relayd #+end_src It looks almost like a normal cron. The "0~59" and "0~29" parts are unique to OpenBSD: Cron(8) will evenly split all tasks between specified time boxes so that no two jobs run simultaneously. We now have created a fully working web server without any 3rd party packages. All OpenBSD provided, all secure, all simple, all cool as ice. To further your knowledge, you can challenge the assumption that BSD has the best doc and read man pages for =httpd.conf(5)=, =relayd.conf(5)=, and =acme-client.conf(5)=. I also can't recommend enough "Httpd and Relayd Mastery" by Michael W. Lucas[fn:mwl2] [fn:nginx-sec] The ports collection of OpenBSD contains a fork of NGINX with a similar security treatment. [fn:mwl2] yeah, the one from the top of this article. He's a household name and a staple of the BSD community. I'm primarily a software engineer, and all this sysadmin thing I am doing is a side quest for me. His books make it so much easier. I've already read four of his books, and I will read more as they are amazing. Even a dense person like yours truly comes out smarter after the lecture. While I'm not a [Full Michael](https://www.tiltedwindmillpress.com/product/full-michael-2023-06/) kind of person, it seems my library will soon have a very strong representation of his. [fn:win] because there is no fourth way. Please repeat after me: there is no webserver in Windows. ** DONE OpenBSD: Blocking bad bots using Relayd CLOSED: [2023-12-11 Mon 19:08] :PROPERTIES: :EXPORT_FILE_NAME: blocking-bad-bots-openbsd :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract How do I fight bad crawlers? :EXPORT_HUGO_MENU_OVERRIDE: :parent "obsdweb" :name "Blocking bad bots using Relayd" :END: The bane of existence for most of small pages: web crawlers. They create most traffic this site sees and makes my [[https://michal.sapka.me/site/info/#site-stats][site stats]] overly optimistic. We can go with [[https://en.wikipedia.org/wiki/Robots_Exclusion_Protocol][robots.txt]], but what if it's not enough? I can tell a valuable bot to not index some part of my site, but: a) some bots ignore it b) what if I don't want some bots to even have the chance to ask? Get that SEO scanning and LLM training out of here! *** Blocking crawlers The rest of this guide assumes webstack: Relayd and Httpd. Relayd is great and since it works on higher level than pf, we can read headers. Luckily, those crawlers send usable "User-Agents" which we can block. First, let's see who uses my site the most. Assuming you use "forwarded"[fn:log-style] style for logs, we can do: #+begin_src shell awk -F '"' '{print $6}' | sort | uniq -c | sort# #+end_src Then we need to manually select agents we want to block. It won't be easy, as the strings are long and contain a lot of unnecessary information - which includes plain lies. You need to define which part of the full User-Agent is common and can be used for blocking. Then we can create block rules in a Relayd protocol. Relayd doesn't use regexp, and instead allows using case-sensitive Lua globs. Stars will match everything. #+begin_src shell block request method "GET" header "User-Agent" value "**" #+end_src Remember that config assumes last-one-wins, so the block rules should be the last matching. I just put those end the end of my config. You can create a `block quick...` rule if you want - it will short-circuit the entire protocol. Therefore, my "https" protocol now has a series of blocks: #+begin_src shell http protocol "https" { # most of the procol omitted block request method "GET" header "User-Agent" value "*Bytespider*" block request method "GET" header "User-Agent" value "*ahrefs*" block request method "GET" header "User-Agent" value "*censys*" block request method "GET" header "User-Agent" value "*commoncrawl*" block request method "GET" header "User-Agent" value "*dataforseo*" block request method "GET" header "User-Agent" value "*mj12*" block request method "GET" header "User-Agent" value "*semrush*" block request method "GET" header "User-Agent" value "*webmeup*" block request method "GET" header "User-Agent" value "*zoominfo*" } #+end_src (usage of globs was proposed to me on [OpenBSD mailing list](https://marc.info/?l=openbsd-misc&m=170206886109953&w=2) [fn:log-style]: vide https://man.openbsd.org/httpd.conf.5#style ** DONE OpenBSD: Forwarding requests from Relayd to a custom webserver CLOSED: [2023-07-19 Mon 19:30] :PROPERTIES: :EXPORT_FILE_NAME: relayd-custom-webserver :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract How to forward request to webserver? :EXPORT_HUGO_MENU_OVERRIDE: :parent "obsdweb" :name "Forwarding requests from Relayd to a custom webserver" :END: One thing that OpenBSD doesn't provide (yet?) is an HTTP proxy. I use [[https://plausible.io/][[Plausible]][fn:nope] for basic visitor analytics [fn:privacy] here, and one of the cool things you can do is to break all adblockers via serving Plausible from my own domain[fn:adblock] After two evenings of failed attempts, I reminded myself that I am a programmer, and I wrote one myself. You can find it on my [no longer available]. It was a great learning exercise and a chance to touch Golang[fn:ruby] for the first time. Assuming you have it running (it works on my machine!), let's adjust our relayd(8). Plaprox listens on port 9090, and we want to relay all requests to =/js/script.js= there. Let's add it to our relays in =relayd.conf=: #+begin_src shell -n table { 127.0.0.1 } http protocol "https" { # all our previous content omitted match request quick path "/js/script.js" forward to match request quick path "/api/event" forward to } relay "https" { listen on 0.0.0.0 port 443 tls protocol https forward to port 8080 forward to port 9090 } relay "https6" { listen on :: port 443 tls protocol https forward to port 8080 forward to port 9090 } #+end_src You can also move the port number to a table. Remember that in Relayd(8) last one wins. We already have a match for the domain and added another matcher for the path. The request will be forwarded to the last marching matcher - so we put our new matchers at the end of the protocol definition. *** Updates 2023-07-28: remove wrong information abot PF. 2023-07-30: fix invalid cron format 2023-12-12: extracted to a dedicated article [fn:privacy] Yes, I want to know what people are reading! For details, refer to my [[https://michal.sapka.me/about/#privacy-policy][two sence long privacy policy]]. [fn:nope] [[https://michal.sapka.me/site/updates/2023/removed-plausible/][this is no longer the case]] [fn:adblock] yes, it's a dick move. But my reasoning was simple: Plausible gathers so little information that the harm is almost nonexistent, and I really want to know what people are reading. [fn:ruby] I am a Ruby developer by trade and heart, but I will try anything that is not an IDE-driven language. LSP for Java/Scala is still a joke, and I refuse to pollute my system with Intellij. [[https://go.dev/][Go][, on the other hand, is a modern language designed for humans. I am not good at it, but I am infinitetly[fn:infinit] better than a week ago. [fn:infinit] Any positive number would be infinite progress compared to zero, or as an old wise man once said: "to have a nickel and to not a nickel is already two nickles". ** DONE OpenBSD: Live from OpenBSD in Amsterdam CLOSED: [2023-07-19 Mon 22:47] :PROPERTIES: :EXPORT_FILE_NAME: moved-to-openbsd :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract A short info on OpenBSD Amsterdam :EXPORT_HUGO_MENU_OVERRIDE: :name "Moved to OpenBSD" :EXPORT_HUGO_PAIRED_SHORTCODES: img-r :END: This site, in its infancy, was running Debian on Linode. Then I moved [fn:fbsd] to [[https://freebsd.org][FreeBSD]] on Vultr. Today marks a day of another migration: hello from [[https://www.openbsd.org/][OpenBSD]] running on [[https://openbsd.amsterdam/][OpenBSD Amsterdam]].[fn:bloggers] [fn:fbsd] [[https://michal.sapka.me/2023/early-freebsd-thoughts/][Early FreeBSD Thoughts]] [fn:bloggers] for technical folks, tinkering with their sites is just as fun as making them. I still have to create a "Yet Another Blog System", but discovering BSD was a great award in itself. *** OpenBSD #+attr_shortcode: "openbsd.png" #+begin_img-r OpenBSD Logo #+end_img-r OpenBSD is one of the three most popular BSD distributions. While [[https://www.netbsd.org/][NetBSD]] focuses on running on obscure hardware[fn:netbsd], and [[https://freebsd.org][FreeBSD]] has ZFS as its killer feature, OpenBSD is all about security[fn:security]. I was very happy with FreeBSD, but at the same time, I was never fully confident in my ability to configure it securely. Not that my server hosts anything of real value[fn:mwl-mail], but I still wouldn't like a machine I administer to become a cog in some botnet. Between learning forensics and a new OS, the latter seems nicer. OpenBSD's official project goal[fn:goals] states that even though they aim to provide the most secure OS, each developer has their own goals and can freely pursue them as long as the project adheres to these goals. It's a very different approach to what we see anywhere else. There is no 10-year roadmap and constant consultations. Instead, we have a hacker-oriented[fn:hackathon] culture. This resulted in multiple projects having their inception in OpenBSD, like [[https://www.openssh.com/][OpenSSH]] or [[https://www.libressl.org/][LibreSSL]]. OpenBSD ships with a secure by-default mindset. All non-essential services are disabled, and those running are using sensible configurations. For example, I had huge problems configuring a firewall on FreeBSD, especially for IPv6[fn:ipv6]. On OpenBSD, it was much simpler. OpenBSD being a BSD, provides a complete system - system and user space are developed together. No GNU tools are needed, as everything comes together. At the same time, BSDs come with a lot of surprising things out of the box. FreeBSD wowed me with Jails[fn:jail]. All in all, a lot of things I've learned on FreeBSD are easily transplantable to OpenBSD. They say that all BSDs are separate OSes, a stark difference from distributions of GNU/Linux. I fail to see it, as so much works the same. The package manager of FreeBSD may be more modern, and the separation between system space and user space[fn:hier] is not so evident here, but so many things work the same. I can not pretend to be a pro-BSDer, but I fail to see evidence of them diverging so narrowly to call them completely different OSes. But then again, maybe it's just my poor judgment and love for POSIX. And still no SystemD(1) in sight. I don't have enough willpower to learn forensics or Rust, not even to mention an OS-level complex PID1 process. ## OpenBSD Amsterdam #+attr_shortcode: "openbsd-amsterdam.png" #+begin_img-r OpenBSD Amsterdam logo #+end_img-r I had a similar exodus of server providers. First, it was Linode, then Vultr. Linode became useless when I wanted to try BSD. Vultr was great as it provided images of FreeBSD and OpenBSD for its VMs. But why stop halfway? Vultr doesn't use BSD as the base system. While it may not be a big deal, I've recently learned of[[https://openbsd.amsterdam/][ OpenBSD Amsterdam]][fn:aws]. OpenBSD Amsterdam is a small company based in (to the surprise of everyone reading this) Amsterdam. What's even better is that they serve OpenBSD VMS from OpenBSD hosts via vmm(4) and vmd(8) - a small virtualization driver baked into OpenBSD. Cool. What's even cooler is that they give a significant part of their earnings to the [[https://www.openbsdfoundation.org/][OpenBSD Fundation]]. I could not resist, and a day after learning about them, I had already paid for a full year. *** Updates 2023-12-12: moved info about web stack to a [[/bsd/open-bsd-web-stack][dedicated article]]. [fn:mwl-mail] at least until "[[https://www.tiltedwindmillpress.com/product/ryoms-esponsor/][Run Your Own Mail Server]]" finally lands in my digital hands [fn:netbsd] There is a semi-widely known story about running NetBSD on a [[https://www.embeddedts.com/blog/netbsd-toaster-powered-by-the-ts-7200-arm9-sbc/][toaster]]. It may not support a modern WiFi card, but if the device is old, you can run NetBSD on it. [fn:security] At least officially. In reality, I'm test-driving it on my laptop and have much fewer problems than with FreeBSD[fn:tphistory]. [fn:tphistory] You may want to check my writing about this epic fight - [FreeBSD on Thinkpad X1 Extreme G2](https://michal.sapka.me/2023/freebsd-on-thinkpad-x1-extreme-g2/). [fn:goals]: [[https://www.openbsd.org/goals.html][OpenBSD Project Goals]] [fn:hackathon]: enough said that OpenBSD coined the term "Hackathon" before corporations stole it - like the internet. [fn:jail]: Jails are FreeBSD containerization mechanisms based solely on chroot(8). Ever since I learned how simple it can be, I started vocalizing my disgust for Docker. [fn:ipv6]: [[https://michal.sapka.me/2023/fixing-ipv6-and-securing-the-domain/][Fixing IPv6 and securing the domain]] [fn:hier]: vide hier(7)of [[https://man.openbsd.org/hier][OpenBSD]] and of [[https://man.freebsd.org/cgi/man.cgi?hier(7)][FreeBSD]] [fn:aws]: notice the lack of Amazon Web Services. Screw them. They have almost all of the interwebs in their server farm, but they will not have this blog! * FreeBSD :@bsd: :PROPERTIES: :EXPORT_HUGO_MENU: :menu bsd :parent "freebsd" :END: ** DONE FreeBSD: examples of Jail usage CLOSED: [2023-12- Mon 21:29] :PROPERTIES: :EXPORT_FILE_NAME: jails-usage-examples :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract How can we use FreeBSD Jails? :EXPORT_HUGO_MENU_OVERRIDE: :name "Some examples of using Jails" :END: Creating Jails in FreeBSD is very easy. It takes few minutes to follow the [[https://docs.freebsd.org/en/books/handbook/jails/][official handbook]] and voilà! We've a dedicated container. But what can we do with it? A very common use pattern on FreeBSD desktop is to use Jails to encapsulate dependencies, so they don't pollute the main OS. Don't want that pesky Pyton? Straight to Jail. Java? Straight to jail! *** Simple jail - Ltex-Ls I am not a native speaker, so my English skills are far from perfect. My terrible typing skills don't add anything good into the mix. Therefore, I am in need of a good grammar and spell checker. There is Grammarly, but it's closed source and cloud based. We don't want that. Luckily an alternative exist - [[https://languagetool.org/][LanguageTool]]. It is quite good and can run locally! I use it via an LSP - [[https://valentjn.github.io/ltex/ltex-ls/installation.html][Ltex-Ls]]. Technically it's aimed at Latex, but it can work with any filetype. The problem with ltex-ls is that is runs on JVM. I really don't need that on my system. Let's lock it in a jail and allow our NeoVim[^lsp] to use it. [^lsp]: I can, of course, run in any other editor which supports the LSP standard, be it Emacs or some bad one. First, let's create a Jail and call it `ltex`. For now we can allow it access to the internet, as we will download some files. #+begin_src shell ltex { ip4 = inherit; interface = em0; } #+end_src We will remove network access after we are done with the setup. We don't want any of our writings to leak to the web. Then, let's log into the jail #+begin_src shell doas jexec ltex /bin/sh #+end_src and add the dreaded java #+begin_src shell pkg install openjdk-jre #+end_src then, let's fetch our latex-ls #+begin_src shell cd /root wget https://github.com/valentjn/ltex-ls/releases/download/16.0.0/ltex-ls-16.0.0.tar.gz tar -xvf ltex-ls-16.0.0.tar.gz #+end_src for ease of use, let's remove the version number from the directory #+begin_src shell mv ltex-ls-16.0.0/ ltex #+end_src And our jail is ready, so let's leave it (either `exit` or the good, old Ctrl+d). We can now open our nvim config file. This is not a guide about this part, so let's just assume you have LSP and LSP-Config intalled. Our config will look like this: #+begin_src lua lspconfig.ltex.setup{ on_attach = on_attach, cmd = { "doas", "jexec", "ltex", "/root/ltex/bin/ltex-ls" }, -- rest of config omitted } #+end_src Notice, that we now run the command as root inside the Jail. It would make sense to allow passwordless-doas to our user due to `doas`. I will update this guide if I figure out if we can commit this security nightmare here. But let's go a step further. Ltex-Ls allows to use machine learning based grammar check based on ngram data. We can add it to our jail. Let's log back in #+begin_src shell doas jexec ltex /bin/sh #+end_src Next we need to fetch the ngram data (you can find it on [[https://dev.languagetool.org/finding-errors-using-n-gram-data.html][LanguageTool website]]. We need to have a `ngrams` folder which contains `en` (as the language shortcut). The ngrams should be inside the `en`. I propose you move the files to `/var/ngrams/en` inside the Jail. We can now tell `NeoVim` to inform the `ltex-ls` runtime to use the ngrams. #+begin_src lua -- our old config lspconfig.ltex.setup{ on_attach = on_attach, cmd = { "doas", "jexec", "ltex", "/root/ltex/bin/ltex-ls" }, -- ngram settings added settings = { ltex = { additionalRules = { languageModel = '/var/ngrams/', }, }, }, -- rest of config still omitted #+end_src Note that we instructed the LSP to use =/var/ngrams= folder. For program running inside a Jail, the Jail is full system. Even though we can access the ngram data on the host OS under =/jail/containers/ltex/var/ngrams= (depending on your config), for Jail, the =/jail/containers/ltex/= prefix doesn't exist as it points to root for the jail. We can now remove networking from the jail, so our Jail config just defines the existence of the jail #+begin_src shell ltex { } #+end_src And boom. We've got machine language grammar checking which is completely offline and does not pollute our primary system. Our tinfoil friends will be proud of us. [this article will be expanded with more examples in the near future] * Thinkpad :@bsd: :PROPERTIES: :EXPORT_HUGO_SECTION: bsd/thinkpad :EXPORT_HUGO_MENU: :menu bsd-thinkpad :END: ** DONE FreeBSD on Thinkpad X1 Extreme G2 CLOSED: [2024-04-10 Wed 22:42] :PROPERTIES: :EXPORT_FILE_NAME: _index :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract My my laptop on FreeBSD :EXPORT_HUGO_PAIRED_SHORTCODES: img-c img-r menu :EXPORT_HUGO_MENU: :menu bsd :parent freebsd :name "Thinkpad X1 Extreme G2 support" :END: This is my personal machine, where I run FreeBSD-release daily. *** Current status of components: | Name | Comment | Status | |--------------------+--------------------------------------------------------+----------------------------| | Audio | | works | | Intel GPU | switching doesn't work. You either use Intel or Nvidia | works | | Nvidia GPU | " | | | Bluetooth | never tried | unsupported | | Camera | | works | | Ethernet | | works | | Fingerprint sensor | | not tested | | Fn Keys | | works | | HDMI | | works | | Hibernate | works is done to address | unsupported | | SD card reader | | not tested | | Suspend | | works | | Touchpad | | works | | Trackpoint | | works | | USB-C | | works | | USB | | works | | WiFi | a lot of problems | technically works[fn:wifi] | [fn:wifi] I have replaced my WiFi card *** Articles Some articles I've written about trying to get it to a usable state: #+attr_shortcode: "bsd-thinkpad" #+begin_menu History of BSD #+end_menu ** DONE FreeBSD: Review of Thinkpad Extreme G2 CLOSED: [2023-02-25 Mon 22:30] :PROPERTIES: :EXPORT_FILE_NAME: freebsd-on-thinkpad-x1-extreme-g2 :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract it works, but there are drawbacks :EXPORT_HUGO_MENU_OVERRIDE: :name "Impressions, Instalation and problems" :EXPORT_HUGO_PAIRED_SHORTCODES: img-c img-r :END: My wife got a new computer, so I can easily break my laptop whenever I want - so it's time for FreeBSD! *All this applies to FreeBSD 13.1 at the time of publishing. I'll add links to any additions and errata in the future* *** Installation The installation process is great. It's more involved than something like Fedora, and some concepts were foreign to me. [[https://docs.freebsd.org/en/books/handbook/bsdinstall/][Handbook's chapter on installation]] guided me through every step, so there were no problems. Within 15 mins of booting from the USB Drive, I had a working hardened system running on an encrypted ZFS drive with wireless networking and essential services configured. #+attr_shortcode: "freebsd-setup-fs.png" "https://docs.freebsd.org/en/books/handbook/bsdinstall/" #+begin_img-c Partitioning #+end_img-c Many things worked out of the box, but not all of them. *** Hardware Setting X-Org was a breeze. Nvidia drivers are [[https://docs.freebsd.org/en/books/handbook/x11/#x-configuration-nvidia][available and ready to go]] no additional configuration is necessary. (update: I was wrong, but it is fixed now) Sound, of all things, work out of the box. Unfortunately, it doesn't auto-switch to headphone output, but there is [[https://freebsdfoundation.org/freebsd-project/resourcesold/audio-on-freebsd/][a known way to do this]] via device hints. The integrated camera also works after running =webcamd -d ugen0.2 -i 0 -v 0=. Tested via =pwcview=. My laptop uses AX200 wireless card, which is [[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244261][not yet fully supported by the system]]. It is recognized and works, but only up to WiFi 3 (g). I'd be ok with WiFi 4(n), but the driver is not ready, and WiFi 5 (AC) [[https://wiki.freebsd.org/WiFi/80211ac][is not supported by the OS]] at all. Funny enough, it [[https://man.openbsd.org/man4/iwx.4][seems to be supported by OpenBD]]. I have yet to learn how different BSDs intertwine and different. FreeBSD is supposed to be more user-friendly, but it seems not to be the case here. Work on [[https://wiki.freebsd.org/WiFi/Iwlwifi][fully supporting]] the card is already planned, but I have no idea when I can expect results. From what I've learned, the team can't reuse code from Linux due to licensing incompatibilities between [[https://www.gnu.org/licenses/gpl-3.0.html][GPL]] and [[https://docs.freebsd.org/en/articles/license-guide/][BSD license]] [update: there are more problems]. #+attr_shortcode: "freebsd-beastie.png" #+begin_img-r FreeBSD Beastie #+end_img-r This is one of the few instances when I am rethinking my life choices, and I would love to be able to help with C code. Also, Bluetooth on this card is not supported, and there is no work done to address it - but luckily, I am already de-wirelessing my life. USB devices are detected automatically and mostly work. However, my monitor (Dell P2723QE) has an integrated 1000Base-T ethernet connection, but on FreeBSD, only 100Base-T worked. Another problem is Suspend/Resume. I can easily [[https://wiki.freebsd.org/SuspendResume][suspend]] the device, but after resuming it, the screen is still black. This seems to be a known problem across different OSes for this laptop and has some [[https://www.thinkwiki.org/wiki/Problem_with_display_remaining_black_after_resume#Solution_for_ThinkPads_with_Intel_Extreme_Graphics_2][known warkarounds]] for Linux, but I have no idea how to apply them to BSD. [update: there is a fix working] The biggest problem here is battery drain. I have =power= enabled, but =acpinfo= reports about over 1% per minute. I was getting about the same drain on Arch Linux, but Manajaro acted much better. I have not tested the fingerprint reader as I've never used it. **** Conclusion My ThinkPad is far from being a brick under FreeBSD. It is, however, severely hindered. I plan to fix the memory drain and allow for a resume after suspension. This will make it a proper laptop again, as there are always USB dongles when faster WiFi is needed. Dongle town, however, is not what I want in the long term, and I'll need to follow the progress of the driver implementation closely. It seems that for a desktop computer everything would work. I'm falling in love with the system and its simplicity and logic. So even if I fail at fixing the above, I'll try to stick with it. And even if I get fed up with the state of hardware support, I'll keep FreeBSD as a secondary system. For personal servers, however, I see no way of abandoning BSD. *** Updates - 2023-02-27: [[/2023/fixing-resume-on-thinkpad-x1-extreme-g2-on-freebsd][Fixing resume due to graphic drivers]] - 2023-02-04: [[/2023/freebsd-configuring-nvidia-and-xorg-on-thinkpad-x1-extreme-g2][Configuring NVIDIA]]. - 2023-03-15: [[/2023/freebsd-on-modern-intel-wifi-cards-and-resume][Problems with WiFi after resume]] ** DONE FreeBSD: Fixing ThinkPad X1 Wifi CLOSED: [2023-08-03 Mon 21:40] :PROPERTIES: :EXPORT_FILE_NAME: fixing-thinkpad-x1-wifi-on-freebsd :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract replacing a wificard fixes everything :EXPORT_HUGO_MENU_OVERRIDE: :name "Fixing WiFi for good" :EXPORT_HUGO_PAIRED_SHORTCODES: img-c :END: As much as I like FreeBSD, my laptop has mostly sat dormant for the last few weeks. It rocked an AX200, an excellent WiFi adapter unless you want to use it in FreeBSD. There were three reasons for this, with one primary cause: 1. WiFi speeds up to WiFi 2, 2. inability of the system to resume after suspend 3. occasional kernel panics Long story short[fn:wifiart], the firmware is yet to be properly reverse-engineered, and the card is still unsupported[fn:lies]. The team can't simply copy the Linux driver due to BSD/GPL license incompabilities[fn:openbsd], so the work needs to continue. Luckily, ThinkPads are still good laptops, and the card was not soldered. So, there was a way: buy a better-supported card and just replace it. Unfortunately, Lenovo is not a good company. You can't simply buy any random card matching the port and be sure it will work. The BIOS has a whitelist of supported hardware, and if it detects anything outside of this list, the machine won't boot. Lenovo's support proved itself useless. I tried to contact them and get the list of whitelisted WiFi adapters, but at first, they had no idea what am I talking about, and when we finally got on the same page, they started to ignore me. After a few nags met with silence, I just gave up and ordered a used [[https://www.intel.com/content/www/us/en/products/sku/99445/intel-wirelessac-9260/specifications.html][Intel AC 9260]]. Have I mentioned that ThinkPads are still good devices? Replacing the WiFi adapter was sparkly[fn:spark] but easy. Just pop the two antenna connectors, unscrew a single screw, remove the card, and do the same in reverse for the new one. Try to do that with a MacBook![fn:battery] #+attr_shortcode: "intel-ac9260.jpg" #+begin_img-c Sitting and working nicely #+end_img-c Then, with a single reinstall[^reinstall] of the system, everything started working. I'm still limited to WiFi 2, but it works over 5GHz. It's a small problem because my system can finally suspend and resume. I no longer need to power off/power on all the time because it's no longer necessary. I no longer need to be annoyed by the booting speed[fn:systemd] because it will no longer be a constant sight for me. I also have a (not backed by any analysis) feeling that the laptop runs colder. With this, I am now a two BSD[fn:golang] guy: [[/2023/moved-to-openbsd][OpenBSD]] on the server and FreeBSD on the computer. Why not go fully into one? Mostly, BSDs are cool, and it's nice to get to know each other. But also each of them has its strengths and weaknesses. OpenBSD is secure, has httpd/relayd and modern PF[fn:pf] but a smaller number of ported software, no ZFS, and finding answers on the information highway is more difficult. For a server, those are non-issues, as I have no intention of installing random crap there. But for my computer, I want to experiment more. I will break the system so ZFS will be a great addition. And having more applications ready to =pkg install= will make it this much nicer. [fn:spark] don't be a moron like me and disable the internal battery in BIOS before randomly poking the motherboard with a metal screwdriver. [fn:battery] or with battery. I'm replacing mine in a few days. If I went with Apple, I would need to go to a service station as my ungluing skills are nonexistent. [fn:wifiart] Vide [FreeBSD on modern Intel WiFi cards and resume](/2023/freebsd-on-modern-intel-wifi-cards-and-resume/) [fn:lies] technically [it is](https://wiki.freebsd.org/WiFi/Iwlwifi), but no real use case is feasable. [fn:openbsd] The OpenBSD team had no such problems, and the drivers are downloaded during installation and work out of the box. [fn:reinstall] I'm a simple bare metal guy and was toying with OpenBSD. I don't know if a reinstall would be required if I had a working FreeBSD. [fn:systemd] Which is one of the few good things about [systemd](https://michal.sapka.me/2023/systemd-is-fast/) [fn:golang] I could have learned to Go, but I chose a totally unmarketable skill for a programmer. I think it makes it even cooler. [fn:pf] I am currently reading /[[https://nostarch.com/pf3][[The Book of PF]]/ so I can have any benefit. Great book. Would recommend. ** DONE FreeBSD: Switching Between Speakers and Headphones CLOSED: [2023-03-16 Mon 23:02] :PROPERTIES: :EXPORT_FILE_NAME: switching-between-speakers-and-headphones-on-freebsd :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract How to change the audio sevices? :EXPORT_HUGO_MENU_OVERRIDE: :name "Switching beteen headphnes and speakers" :END: I want my laptop to switch to wired headphones when I attach them. FreeBSD has its own [[https://wiki.freebsd.org/Sound][Sound System]], so it's a great learning experience. I have yet to automate it (it is possible, but an attempt to do so forced me to do a complete rollback of the system state), but for now, this is working. First, check which audio outputs your device supports: #+begin_src shell $ cat /dev/sndstat #+end_src In the case of my ThinkPad, this returns #+begin_src shell Installed devices: pcm0: (play) pcm1: (play) pcm2: (play) pcm3: (play/rec) default pcm4: (play) No devices installed from userspace. #+end_src The ones I care about are: pcm3 - the speakers pcm4 - the headphone jack I can now easily switch between them: #+begin_src shell # enable speakers $ sysctl hw.snd.default_unit=3 # enable headphones $ sysctl hw.snd.default_unit=4 #+end_src (replace the value with the correct id from `sndstat` file.) This, however, comes with a huge caveat. Some apps (khem khem, Firefox) not native to FreeBSD come configured with PulseAudio instead of FreeBSD's Sound System. This creates a level of indirection, and changing system output may not work instantly. In the case of Firefox, I need to reload the tab. Some apps, as I've heard, require a restart. ** DONE FreeBSD: Fixing Resume on ThinkPad X1 Extreme G2 due to integrated graphic card CLOSED: [2023-03-16 Mon 23:02] :PROPERTIES: :EXPORT_FILE_NAME: fixing-resume-on-thinkpad-x1-extreme-g2-on-freebsd :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract you need to load the driver :EXPORT_HUGO_MENU_OVERRIDE: :name "Fixing resume due to Intel driver" :END: *This applies to FreeBSD 13.1* Recently I [[/2023/freebsd-on-thinkpad-x1-extreme-g2][posted]] about my problems with FreeBSD. One of them was resume. After installing FreeBSD, I was able to put my laptop to sleep via #+begin_src shell acpiconf -s 3 #+end_src And this worked fine. However, I was not able to resume it back from sleep. After pressing the power button laptop woke, but the screen was still black. I could =reboot=, and it would work, so only the screen was the problem. After asking about this on [[https://forums.freebsd.org/threads/resume-on-thinkpad-x1-extreme-g2-ends-in-black-screen.88162/][FreeBSD Forums]], [[https://forums.freebsd.org/members/bsduck.61635/][bsdduck]] and [[https://forums.freebsd.org/members/smithi.71028/][smithi]] pointed me to dedicated drivers for the integrated Intel GPU. And it worked like a charm. All I had to do was: #+begin_src shell pkg install drm-kmod sysrc -f /etc/rc.conf kld_list+=i915kms reboot #+end_src (via [[https://wiki.freebsd.org/Graphics#Intel_Integrated_Graphics_.28aka_HD_Graphics.29][FreeBSD Wiki]]). Now the computer can sleep and resume without any problems. At least when using [[https://man.freebsd.org/cgi/man.cgi?acpiconf(8)][sleep mode 3]]. The 4th doesn't work for me at all. ** DONE FreeBSD: configuring Nvidia and Xorg CLOSED: [2023-03-16 Mon 23:02] :PROPERTIES: :EXPORT_FILE_NAME: freebsd-configuring-nvidia-and-xorg-on-thinkpad-x1-extreme-g2 :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract A tutorial on making Nvidia work :EXPORT_HUGO_MENU_OVERRIDE: :name "Setting up Nvidia" :END: First, the bad news: I could not make FreeBSD work with Hybrid Graphics, so I use only the discrete one. To ensure this, open BIOS and: 1. Configuration 2. Display 3. Graphics Device 4. select `Discrete Graphics` Then, log in as root and install the drivers: #+begin_src shell pkg install nvidia-driver nvidia-xconfig #+end_src The next step is to enable the drivers. #+begin_src shell sysrc kld_list+=nvidia sysrc kld_list+=nvidia-modeset #+end_src Some people advise adding Linux (=sysrc kld_list+=linux=) to kld_list, but I got my GPU working without that. After that, either load the drivers manually or give the computer an old, good reboot. Login as root again and use the NVIDIA configurator to get Xorg configured. #+begin_src shell nvidia-xconfig #+end_src Then try starting your desktop environment, windows manager, or startx. You may be done, but I got an error about =Screen not found=. Tell Xorg where the NVIDIA GPU is if you have the same problem. Try probing the system for GPUs #+begin_src shell pciconf -l | grep vga #+end_src You will see one on more elements on the list. The critical part is in the first column, for example: #+begin_src shell vgapci0@pci0:1:0:0 #+end_src Our GPU is available under BUS 1:0:0 (we skip the first 0). You may need to try different elements from the list. #+begin_quote For PCI/AGP cards, the bus−id string has the form PCI:bus:device:function (e.g., “PCI:1:0:0” might be appropriate for an AGP card). [[https://www.x.org/releases/X11R7.7/doc/man/man5/xorg.conf.5.xhtml#heading10X][Xorg documentation]] #+end_quote Open =/etc/X11/xorg.conf=, look for =Section "Device"= and add: #+begin_src shell BusID "PCI:1:0:0" #+end_src In my case, everything worked fine after that. Notes: I learned the BUS trick from [[https://nudesystems.com/how-to-fix-no-screen-found-xorg-error-on-freebsd/][Nude Systems]]. * Unix history :@bsd: :PROPERTIES: :EXPORT_HUGO_SECTION: bsd/history :EXPORT_HUGO_CUSTOM_FRONT_MATTER: :hasNavSection bsd-history :END: ** DONE History of BSD and Unix CLOSED: [2024-04-08 Mon 09:37] :PROPERTIES: :EXPORT_FILE_NAME: _index :EXPORT_HUGO_CUSTOM_FRONT_MATTER+: abstract the long and turbolent history :EXPORT_HUGO_PAIRED_SHORTCODES: menu img-r :EXPORT_HUGO_MENU: :menu bsd :END: Does computer history have any real appliance? Not really. Does it explain certain, potentially baffling things? Certainly. But does it make great beer talk? Absolutely! Especially when discussing systems with such rich history as BSD. I will be gentle here, I promise. No /Hardcore History/ in sight. #+attr_shortcode: "bsd-history" #+begin_menu History of BSD #+end_menu ** DONE History of BSD part I: Multics CLOSED: [2024-03-09 Sat 21:03] :PROPERTIES: :EXPORT_FILE_NAME: 01_multics :EXPORT_HUGO_CUSTOM_FRONT_MATTER+: abstract BSD history starts with Multics :EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :shortname Part I: Multics :EXPORT_HUGO_MENU: :menu bsd-history :name "Multics" :EXPORT_HUGO_PAIRED_SHORTCODES: img-c :END: *** Origins of time-sharing Let's start our journey back when dinosaurs roamed the earth, engineers wore ties, and Barbie was first gaining popularity - the 60s. Nowadays, we have grown accustom to companies selling devices at lower profit margins - with most profit coming from software subscriptions. But 70 years ago it was the complete opposite. Companies bought expensive computers, huge machines, and what was running on them was of lesser value. And I really mean /expensive/. An IBM System/360 Model 20, presented in 1964, could have been bought starting from USD 62,710 (USD 622,626 adjusted for inflation) or rented for USD 1280 (USD 12,708 adjusted)[fn:IBM360]. Sellers earned a lot from the devices. But, naturally, companies making all those investments wanted a nice return. This led to the creation of /time-sharing/. #+attr_shortcode: "ibm-360.jpg" "https://www.nbcnews.com/tech/gadgets/5-reasons-love-mad-mens-new-star-ibm-360-n101716" #+begin_img-c IBM 360 in an official photoshoot. #+end_img-c This concept seems natural now: multiple processes were able to share computer resources, so multiple applications could run at the same time. Applications could even run for different users. Ergo, time-sharing allowed for multi-user multitask processing. This is in stark contrast to batch-processing, where only a single program would be able to compute at any particular time. An example of such processing would be EDSAC, the first electronic computer[fn:root]. I won't go into detail of time-sharing, but you can read more in "Time sharing in large computers", C. Strachey, 1959. But what is important here is how it was used. All computation happened on a single, large server. End users would use /computer terminals/ which were /multiplexed/ into that server, called a /mainframe/. What is multiplexing you ask? Simply said, it's a way to combine different signals into a shared medium. It was used extensively for land-line telephony, where all signals were transferred over shared wires. #+attr_shortcode: "edsac.jpg" "https://www.datacenterdynamics.com/en/analysis/rebuilding-edsac-the-first-real-computer/" #+begin_img-c EDSAC, the "first computer". #+end_img-c *** Multics Let's jump to 1969. /Multics (MULTiplexed Information and Computing)/ was an early time-sharing operating system developed by /MIT/, /General Electrics/ and /Bell Labs/. It pioneered many of innovations which are still widely used in the computing systems of today: hierarchical file system, redirection as interprocess communication, or the existence of a shell[fn:wiki], as well as memory pages, memory protection, or the ability for a single machine to use multiple CPUs and memory[fn:allen]. It was however also huge, both in terms of memory usage (the resident kernel could occupy a huge part of memory living not enough for applications) and code size (it consisted of about 1,5k source modules)[fn:wiki]. /Multics/ was experimental and therefore ambitious, complex - designed by trial and error. It was delivered late, early on had performance problems, and in 1969 /Bell Labs/ withrew from the project[fn:earlylin]. #+attr_shortcode: "multics-login.png" "https://en.wikipedia.org/wiki/Multics#/media/File:Multics-Login.png" #+begin_img-c Multics login screen. #+end_img-c #+attr_shortcode: "thompson-ritchie.jpg" "https://computerhistory.org/blog/discovering-dennis-ritchies-lost-dissertation/" #+begin_img-c Ken Thompson and Denis Ritchie. #+end_img-c Ken Thompson, Dennis Ritchie, Douglas McIlroy, and Joe Ossanna, frustrated with their experiences with /Multics/ Operating System are starting to work on their own alternative. Their work will become one of the most beloved computer products of all time - /UNIX/. The team armed with previous experience set up to create simple, manageable OS that would still fulfill all the requirements /Multics/ was to fulfill. Currently, however, they have problems convincing /Bell Labs/ management to get them a new computer. This has not stopped them from designing the system on black blackboards and paper. [fn:IBM360] [[https://ibms360.co.uk/?p=902][IBM 360 Model 20 Rescue and Restoration: Machine Overview]] [fn:root] [[http://blog.wovenmemories.net/2023/10/30/First.Operating.System_Part.1.html][First Operating System -- Part One]] [fn:wiki] [[https://en.wikipedia.org/wiki/Multics][Multics --- {W}ikipedia{,} The Free Encyclopedia]] [fn:allen] [[https://www.youtube.com/watch?v=UYb6WqWBTE0][Mark Allen - Before Unix: An Early History of Timesharing Systems]] [fn:earlylin] [[https://www.youtube.com/watch?v=ECCr_KFl41E][The early days of Unix at Bell Labs - Brian Kernighan (LCA 2022 Online)]] ** DONE History of BSD part II: Unix CLOSED: [2024-03-16 Sat 21:00] :PROPERTIES: :EXPORT_FILE_NAME: 02_unix :EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :abstract How Unix came to be? :EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :shortname Part II: Unix :EXPORT_HUGO_MENU: :menu bsd-history :name "Unix" :EXPORT_HUGO_PAIRED_SHORTCODES: img-c :END: *** Bell Labs Let's pause here and discuss a few things. /Bell Labs/ was a research institute founded by no other, but the famous Alexander Graham Bell in 1876. It is known as one of the most innovative places ever conceived. It was the birthplace of radio astronomy, the transistor, the laser, the photovoltaic cell, the charge-coupled device (CCD), information theory, and the programming languages B, C, C++, S, SNOBOL, AWK, AMPL, and the UNIX operating system. The work completed there was awarded 10 times with a Nobel Prize[fn:wikibell]. But it wasn't all pretty. AT&T (owner of the lab) was /the/ telephone company in the USA. It had the /de facto/ monopoly for most of the 20th century. This led to a consent decree with US Government under which AT&T was forbidden to branch out to other markets. This agreement happened in 1956 and will be very important for our story[fn:hiddenunix]. Another cool tidbit: back then computer screens as we know them now were yet to be invented. There were ways for computer to present information on a screen, but it was not feasable to present information back to the user in real time. The monitor was a separate machine and an /assembly/ code to light up individual pixels would need to be sent every time to it. If the user wanted to see what the edited document looked like, the only way to achieve that was via a printer. Try to print a page every few lines - let alone every character. Even the bravest of the brave would not escape the anger of the finance department. *** Getting a computer After this short intermission, let's return to /UNIX/. The popular history of /UNIX/ goes as follows: Ken Thompson finds a PDP-7 microcomputer, starts playing with it and suddenly realizes that he has created a new operating system. What he actually wanted to do, was to play a damn game. A completely reasonable reason to revolutionize the industry if you ask me. Ken was a huge fan of /Space Travel/, but it was only available on mainframes. And since those costed an arm and a leg, end users were paying for time they used. A single game of /Space Travel/ could cost Ken USD 50-75[fn:uvlist]. Luckily, he was able to find a discarded PDP-7 from another department[fn:hiddenunix]. #+attr_shortcode: "space-travel.png" "https://www.uvlist.net/game-164857-Space+Travel" #+begin_img-c Space Travel #+end_img-c What the PDP-7 was, was a refrigerator size 18bit monstrosity[fn:hiddenunix]. It was at the time a 5-year-old and obsolete[fn:earlylin] leftover from an ended research into audio-psychology. No one cared what would happen to this particular machine. Have I mentioned just how stinky rich the /Bell Lab/ was? So, Ken started rewriting the game for that PDP-7 as /GECOS Space Travel/. Porting the game proved challenging and difficult to debug, so they developed a system for PDP-7 to make developing the game easier. This was how /PDP UNIX/ was born[fn:lcm]. A guy wanted to play a god-damn game and now bankrupt in the process[fn:hiddenunix] Labs management was very happy with this turn of events as /UNIX/ was something to show after the /Multics/ fiasco. They allowed Ken to get himself a /PDP-11/ and port /UNIX/ to it. It was a cheaper machine compared to /PDP-7/, but it was also 16bit - which was a nice change. Officially, the new system would allow for document preparation for patent applications[fn:earlylin]. The biggest benefit however was the popularity of this machine - over the next decades it could be seen everywhere across enterprise America. You can buy a working one from /eBay/ today. It won't be easy nor cheap, but definitely possible. #+attr_shortcode: "pdp-7.jpeg" "https://en.wikipedia.org/wiki/PDP-7#/media/File:Pdp7-oslo-2005.jpeg" #+begin_img-c PDP-7 #+end_img-c #+attr_shortcode: "thompson-ritchie-pdp11.jpg" "https://www.bell-labs.com/institute/blog/invention-unix/" #+begin_img-c Ken Thompson and Denis Ritchie working on the legendary PDP-11. #+end_img-c *** Unix 1 So here we are, November 1971, and /UNIX 1/ for /PDP-11/ is released[fn:50] A year later Ken notes: #+begin_quote [...]the number of UNIX installations has grown to 10, with more expected. None of these has exactly the same complement of hardware or software. Therefore, at any particular installation, it is quite possible that this manual will give inappropriate information. [...] Also, not all installations have the latest versions of all the software.'' -- Unix Programmers Manual, Second Edition. Thompson Ken, Ritchie Denis, 1972 #+end_quote Note, that this is 10 mainframes across /Bell Labs/, so user count was significantly higher. It's also worth to talk about release cycle. UNIX was in constant development, and each installation would use current snapshot at the moment of installation. This is why Ken notes that each part of /UNIX/ could differ between installations. What is also important is that due to licensing of /UNIX/ (or rather lack of thereof), each installation would be accompanied by full source code. Administrators were able to modify the system. Crazy times. Still, in June 1972 the 2nd Edition of /UNIX/ was released and every 7–8 months a new release was out. In November 1973, a 4th Edition was released, and it was the first one released to Universities. It made quite the buzz, and first user groups started forming under the /USENIX/ name[fn:hiddenunix]. It's worth to mention that this version was rewritten in C, as previous versions which were created in Assembly This made it the first portable /UNIX/, as it could be recompiled to any compared, as long as it also had a C compiler[fn:earlylin]. The system was presented by Ken and Denis during the /4th Symposium on Operating System Principles/ the same year, and a year later Ken and Dennis publish the first paper on /UNIX/ - "The UNIX Time-Sharing System" in the /Communications/ journal of ACM. #+attr_shortcode: "unix-tech-journal.jpg" "https://leancrew.com/all-this/2011/10/dennis-ritchie-unix-and-clarity/" #+begin_img-c Bell System Technical Journal cover. #+end_img-c The word was out and /UNIX/ became known outside of /Bell Labs/. In the next episode of History of BSD: the scariest creature of them all. Lawyers. [fn:wikibell] [[https://en.wikipedia.org/wiki/Bell_Labs][Bell Labs --- {W}ikipedia{,} The Free Encyclopedia]] [fn:hiddenunix] [[https://www.youtube.com/watch?v=xuzeagzqwrs][The hidden early history of unix the forgotten history of early unix]] [fn:earlylin] [[https://www.youtube.com/watch?v=ECCr_KFl41E][The early days of Unix at Bell Labs - Brian Kernighan (LCA 2022 Online)]] [fn:uvlist] [[https://www.uvlist.net/game-164857-Space+Travel][UVL]] [fn:50] The 50th anniversary of this event was very big for a small group of fans. [fn:lcm] [[https://livingcomputers.org/Blog/Restoring-UNIX-v0-on-a-PDP-7-A-look-behind-the-sce.aspx][Living Computer Museum]] has a real PDP-7 running /PDP Unix/ if want to chek it. * Varia :@bsd: :PROPERTIES: :EXPORT_HUGO_MENU: :menu bsd :parent "varia" :END: ** DONE FreeBSD: Early thoughts CLOSED: [2023-02-15 Mon 21:12] :PROPERTIES: :EXPORT_FILE_NAME: early-freebsd-thoughts :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract I've been using FreeBSD on my server for the last few weeks and I like it! :EXPORT_HUGO_MENU_OVERRIDE: :name "FreeBSD: early thoughts" :EXPORT_HUGO_PAIRED_SHORTCODES: img-r img-c :END: I'm leaning more and more towards joining the [[https://www.freebsd.org/][FreeBSD]] crowd. The community is small and welcoming, and I'm driven towards more minor groups. But I was surprised to find out hoh welcoming it was. People seem to be actually happy to help a noob - something the Linux crowd forgot how to do. #+attr_shortcode: "freebsd-beastie.png" #+begin_img-r FreeBSD Beastie #+end_img-r Another aspect is the documentation. People say it's excellent, and I consider it to be selling short. I'm reading [[https://docs.freebsd.org/en/books/handbook/][The Official Handbook]] It starts with the assumption that the reader has close to 0 knowledge but never treats him as a moron. And chapter by chapter explains how and why things work this way. It may not be for everyone, as you are expected to want to learn - but it is invaluable if you are in the target group. It's worth reading even if you don't want to move to BSD, as a lot applies to other NIXs, like Linux. And, of course, the system itself. I've been using unix-inspired OSs exclusively for over a decade (and quite often before that). FreeBSD is so close, that from day 0, I am able to navigate it. And what I see is a very well-thought system without many pitfalls Linux fell into. Just two examples that strike me the most. In Linux, the root partition is a mess. System and userland are intertwined, and I wonder if anyone understands where things should go. Just look at how many explanations of the structure there are! Should this particular config be in /var/ or maybe in /etc/? AFAIK there are no generic guidelines, just tribal knowledge. If FreeBSD, there's a [[https://docs.freebsd.org/en/books/handbook/basics/#dirstructure][dedicated chapter]] in the documentation! There's also a strict rule where userland should live - in /usr. Everything you install goes to /usr - the executables, the configs, etc. Finally, a new user can experiment without fearing breaking the system! The other one is the `rc` subsystem. The Linux world has a neverending war between Systemd and, well, everything else. Here? The system itself dictates how to manage the cattle - elegantly and logically. #+attr_shortcode: "freebsd13-bootloader.png" #+begin_img-c How the OS greats us. #+end_img-c FreeBSD comes with two package managers: pkg and ports. Pkg is a standard replacement for brew/apt/pacman or whatever else is there. What is nice is that the user can configure to use packages updated quarterly or the latest. Want to have a stable infrastructure? Go with quarterly - bug fixes will be included in between updates. Want modern thingies? Go with the latest. My biggest issue with Ubuntu and its derivatives is how far behind the packages in apt are, as they are tied to the yearly distro update circle. You can mitigate this by using personal repositories, but those are a nuance to set up. FreeBSD comes prepared for servers and workstations at the same time. And then there are ports for the demanding crowd. Since BSD is semi-compatible with Linux, you can compile most of its software. But there are some differences, so it requires some manual configuration or looking for dependencies. Or rather, it would, as FreeBSD has you covered. Ports is a single repository with makefiles for different projects tailored for the system. You can either compile anything with default settings or adjust the parameters easily. Want Firefox without JS support? Why not! I have yet to use ports, as they seem excessive for my humble VPS, but I love the idea. So, you have the best features from Ubuntu (stable versions), Arch (cutting edge), and from Gentoo (compile from source code) right at your disposal. I am **this** close to installing FreeBSD on my personal computer. My work-issued Macbook is [[https://en.wikipedia.org/wiki/Darwin_(operating_system)#Kernel][already running a BSD derivative]]... for better or worse. ** DONE FreeBSD on the Desktop won't improvide unless people are using it CLOSED: [2023-03-29 Mon 22:09] :PROPERTIES: :EXPORT_FILE_NAME: desktop-freebsd-wont-improve-unless-people-are-using-it :EXPORT_HUGO_CUSTOM_FRONT_MATTER: abstract I have hits from /r/bsd, but almost none of those people are using BSD :EXPORT_HUGO_MENU_OVERRIDE: :name "Desktop FreeBSD won't improve unless people are using it" :EXPORT_HUGO_PAIRED_SHORTCODES: img-r img-c :END: Shamelessly, I posted my previous post, [[https://d-s.sh/2023/freebsd-on-thinkpad-x1-extreme-g2/][FreeBSD on a Thinkpad Extreme G2]] on [[https://www.reddit.com/r/BSD/comments/124v5cm/freebsd_on_a_thinkpad_x1_extreme_g2/][/r/bsd Reddit]]. The result, some 24 hours later, is 100 visitors. Out of that 100, 57 are using a desktop. Out of that 57, only 2 used FreeBSD—2%. No other BSDs are recorded. People who are into BSD don't use BSD. This seems to be a reason for lacking hardware support. If no one uses FreeBSD, no one will encounter those problems. If no one encounters them, no one will fix them. *** Update 2023-04-14 The article, got quite the round around the internets, gathering some interests from [[https://old.reddit.com/r/freebsd/comments/126fvkz/desktop_freebsd_wont_improve_unless_people_are/][Reddit]], [Hacker News](https://news.ycombinator.com/item?id=35378367), Twitter, Discover BSD, or [[https://vermaden.wordpress.com/2023/04/03/valuable-news-2023-04-03/][Vermaden]]. With all that interest come quite a few questions and comments. The following is an attempt to summarize it all. **** People who use FreeBSD don't care about FreeBSD hardware This makes perfect sense. If your FreeBSD installation on X220 works flawlessly, you may not care about anything more modern. But there will come a time when you will need to replace the hardware. #+attr_shortcode: "freebsd-beastie.png" #+begin_img-r FreeBSD Beastie #+end_img-r This comment, however, came as a proof that the sample from my blog is invalid. This may be the case, but I don't buy it. All traffic on the aforementioned post came from Reddit's BSD forum. It's the one place where you could expect that people using BSD would hang. It may also be that it's quite a random sample - it's small, and people who have yet to become into BSD but are BSD-curious opened my blog post. I am in no place to debunk or confirm this. I, however, know that many people presenting at FreeBSD conferences do it using Macs or Windows. So even if the numbers are dubious, the overall feeling remains sorrowful. To add to the above: there are also stats for the commented opinion piece. Two hundred forty-four people opened it from /r/freebsd. Of that, 24 people were using FreeBSD, and just 2 were using OpenBSD. **** Your statistics may be invalid as people mask their browser agent. This also may be the case. Why, then, is the referer not spoofed? It's a much more invasive data point than the underlying OS. But I'm a simple Firefox user, never used Librewolf. **** FreeBSD is a server OS Yeah, this is the sentiment I've read before jumping aboard. My problem with this idea is that each and every FOSS OS is a value in itself. The current poster boy, Linux, also had huge problems getting to work on various machines. In my opinion, it's limiting OS to a single use case is a completely valid point - your use case for FreeBSD is on a server, and this is where it currently shines (or not, depending on your experience). Some folks despise allocating any FreeBSD dev time to the desktop as there are many server issues. But again, I don't see it this way. Limiting FreeBSD to the server only is short-lighted. Unless you are already powering your servers with BSD, there will always be a question: "Why not Linux. It's what everyone else is doing". And Linux got into its current position not by being a great server machine but rather by attracting the interest of some very skillful people. And it did it by allowing more and more people to free themselves from Windows on their machines. I see FreeBSD problems as having two primary causes: the [[https://en.wikipedia.org/wiki/Unix_wars][Unix wars]] of the past and limited resources now. If FreeBSD were easier to use on a wide range of end-user machines (which tend to be laptops), the easier it would for people to want to develop it. BSDs are now a far second choice. Why would someone invest time? They may fall in love with the OS, but unless they try it, it will never happen. **** I like our small userbase I'm as elitist as the other person. [[https://dwm.suckless.org/][DWM]] stated that #+begin_quote "This keeps its userbase small and elitist. No novices asking stupid questions. #+end_quote I can't find this quote anymore, but the sentiment seems similar. However, there are two aspects here. FreeBSD comes with no graphical interface by default. This makes it much closer to minimalist distros than Ubuntu. This still allows anyone to feel like a hacker. The second, however, is that some problems are unsolvable by end-user. Writing drivers is EXTREMELY difficult, and, as I've recently learned (thanks, Jeff!), this is especially true when it comes to WiFi drivers, as there is no open implementation. This means that any progress requires a trial-and-error process based on reverse engineering. No one without deep knowledge of low-level programming will be able to make any progress, and even those few will need people with real hardware for testing. **** Hardware support is years behind Linux Yes, and this is what I was referring to. **** Why would anyone use BSD on a desktop? It's a great system, just needs a lot of work on hardware support :-) **** Your post is worthless, and only the comments are interesting It's more than I anticipated. That post was small and written without any deeper research. But the discussion around it makes me believe that I hit something real. * WIP ** TODO OpenBSD: XMPP (Jabber) server /intro/ *** Installing prosody #+BEGIN_SRC shell pkg_install prosodyctl #+END_SRC - daemon *** Configuring DNS prosodyctl check dns *** Getting certificates - acme config (ignore makefile) - importing certificates #+BEGIN_SRC shell prosodyctl --root cert import HOSTNAME /path/to/certificates #+END_SRC *fullchain* #+BEGIN_SRC shell prosodyctl check certs #+END_SRC *** Final tests #+BEGIN_SRC shell prosodyctl check connectivity #+END_SRC *** Clients - emacs - terminal - android - ios *** Modern XMPP ** TODO BSD and Linux