diff options
author | mms <michal@sapka.me> | 2023-12-12 21:34:42 +0100 |
---|---|---|
committer | mms <michal@sapka.me> | 2023-12-12 21:34:42 +0100 |
commit | 72b44f302d41df3caceed23cb925dc382b864996 (patch) | |
tree | a25867874cea69e5482ad00274693445980f417c | |
parent | 867d235681b1ad5201a63c04b8d9dbad365ae9c5 (diff) |
tidy up: fixes and bsd extaction
-rw-r--r-- | content/blog/2023/tar-is-for-tape-drive.md | 8 | ||||
-rw-r--r-- | content/bsd/blocking-bad-bots-openbsd.md | 5 | ||||
-rw-r--r-- | content/bsd/home.md | 5 | ||||
-rw-r--r-- | content/bsd/moved-to-openbsd.md | 259 | ||||
-rw-r--r-- | content/bsd/open-bsd-web-stack.md | 210 | ||||
-rw-r--r-- | content/bsd/relayd-custom-service.md | 57 | ||||
-rw-r--r-- | content/bsd/relayd-custom-webserver.md | 58 | ||||
-rw-r--r-- | static/style.css | 2 |
8 files changed, 343 insertions, 261 deletions
diff --git a/content/blog/2023/tar-is-for-tape-drive.md b/content/blog/2023/tar-is-for-tape-drive.md index e8c332c..ad6044b 100644 --- a/content/blog/2023/tar-is-for-tape-drive.md +++ b/content/blog/2023/tar-is-for-tape-drive.md @@ -29,3 +29,11 @@ This explains two things I never got about tar: Tar is one of the things where computer archeology makes things clearer. Nowadays, we often forget that computers have a history and a lot of things we take for granted need to be invented. Many quirks can be understood if we explore the past a bit. +**Errata** + +[Humm](https://bsd.network/@humm) pointed to me that the above is not technically true. + +1. Looking at low level, what makes tar slower than ZIP is lack of index. Zip files have an index which allows finding and reading the expected file. In tar, we need to seek and read file header to determine if this part of the archive contains the file we need. And this forces us to read all files before, so if it's our file let's extract it; otherwise look at the next header. +2. It is fully possible to compress a file with a streaming archiver, like gzip and read the headers as we are streaming the archive. + +Thanks! diff --git a/content/bsd/blocking-bad-bots-openbsd.md b/content/bsd/blocking-bad-bots-openbsd.md index ef3f1ed..a52c1b3 100644 --- a/content/bsd/blocking-bad-bots-openbsd.md +++ b/content/bsd/blocking-bad-bots-openbsd.md @@ -21,13 +21,14 @@ Get that SEO scanning and LLM training out of here! The rest of this guide assumes webstack: Relayd and Httpd. Relayd is great and since it works on higher level than pf, we can read headers. Luckily, those crawlers send usable "User-Agents" which we can block. -First, let's see who uses my site the most. Assuming you use "forwarded" style for logs, we can do: +First, let's see who uses my site the most. Assuming you use "forwarded"[^log-style] style for logs, we can do: +[^log-style]: vide https://man.openbsd.org/httpd.conf.5#style {{<highlight shell>}} awk -F '"' '{print $6}' <path to log file> | sort | uniq -c | sort {{</highlight>}} -Then we need to manually select agents we want to block. It won't be easy, as the strings are long and contain a lot of unnecessary information - which includes plain lies. You need to define which part of the full Uer-Agent is common and can be used for blocking. +Then we need to manually select agents we want to block. It won't be easy, as the strings are long and contain a lot of unnecessary information - which includes plain lies. You need to define which part of the full User-Agent is common and can be used for blocking. Then we can create block rules in a Relayd protocol. Relayd doesn't use regexp, and instead allows using case-sensitive Lua globs. Stars will match everything. diff --git a/content/bsd/home.md b/content/bsd/home.md index 2366758..917fe48 100644 --- a/content/bsd/home.md +++ b/content/bsd/home.md @@ -30,6 +30,7 @@ Since at least a year, I've been a BSD type of a guy. My personal laptop is runn - [Switching between speakers and headphones](/bsd/thinkpad/switching-between-speakers-and-headphones-on-freebsd) - [Fixing WiFi by replacing card](/bsd/thinkpad/fixing-thinkpad-x1-wifi-on-freebsd) - OpenBSD server - - [OpenBSD Amsterdam](/bsd/moved-to-openbsd/) - - [Webstack - Httpd(8), Relayd(8)](/bsd/moved-to-openbsd/#httpd8-and-relayd8) + - [Webstack - Httpd(8), Relayd(8)](/bsd/open-bsd-web-stack/) - [Blocking bad bots and crawlers](/bsd/blocking-bad-bots-openbsd/) + - [Forwarding request to a webserver](/bsd/relayd-custom-webserver) + - [OpenBSD Amsterdam](/bsd/moved-to-openbsd/) diff --git a/content/bsd/moved-to-openbsd.md b/content/bsd/moved-to-openbsd.md index b8a7d3b..66ec760 100644 --- a/content/bsd/moved-to-openbsd.md +++ b/content/bsd/moved-to-openbsd.md @@ -1,7 +1,7 @@ --- title: "Live from OpenBSD in Amsterdam" category: bsd -abstract: +abstract: A short info on OpenBSD Amsterdam date: 2023-07-19T12:27:54+02:00 year: draft: false @@ -16,8 +16,6 @@ tags: - plausible - plaprox --- - - This site, in its infancy, was running Debian on Linode. Then I moved [^fbsd] to [FreeBSD](https://freebsd.org) on Vultr. Today marks a day of another migration: hello from [OpenBSD](https://www.openbsd.org/) running on [OpenBSD Amsterdam](https://openbsd.amsterdam/).[^bloggers] [^fbsd]: [Early FreeBSD Thoughts](https://michal.sapka.me/2023/early-freebsd-thoughts/) [^bloggers]: for technical folks, tinkering with their sites is just as fun as making them. I still have to create a "Yet Another Blog System", but discovering BSD was a great award in itself. @@ -58,260 +56,9 @@ What's even cooler is that they give a significant part of their earnings to the I could not resist, and a day after learning about them, I had already paid for a full year. -## Httpd(8) and Relayd(8) - -So here we are: OpenBSD VM. What now? Let's configure a webserver! - -OpenBSD comes with three great tools out of the box: - -- httpd(8) - an HTTP daemon -- relayd(8) - a relay daemon -- acme-client(1) - a client for Automatic Certificate Management Environment (ACME) - -With those free things, we can serve static webpages over TLS. While you most likely already use [NGINX](https://www.nginx.com/) or o[Apache](https://httpd.apache.org/)[^win], those solutions are complex. They work amazingly in enterprise environments where you have people with doctorates in NGINX configuration, but most real-world examples don't need that complexity. A static blog most likely doesn't. -[^win]: because there is no fourth way. Please repeat after me: there is no webserver in Windows. - -Let's set it up. - -Due to security concerns, OpenBSD comes with doas(1) instead of sudo(1). Copy `/etc/examples/doas.conf` file to `/etc/doas.conf`. For all intends, and purposes, from now on doas(1) will work the same as sudo(1). - -When the system boots for the very first time, ports 80 and 443 are closed, and only the SSH port is open. This alone was a nice surprise for me. But it gets better: since all utilities are part of the OSes, they work together perfectly. - -Assuming your domain is already pointing at the correct IPs, let's start listening for unencrypted HTTP traffic. I will use "michal.sapka.me" as the domain in all examples. - -First, Open `/etc/httpd.conf` in your favorite editor and add - -{{<highlight shell "linenos=inline">}} -server "michal.sapka.me" { - listen on * port 80 - root "/htdocs/michal-sapka-me" -} -{{</highlight>}} - -Then create a simple HTML file under `/var/www/htdocs/michal-sapka-me/index.html`. - -Httpd(8) works chrooted to /var/www/, so it threats this directory as root. This makes the "root" option shorter to write, but it also means that the process doesn't have access to anything outside of /var/www/. Even if an attacker can break in via the daemon, he will be locked in the www folder, so there is no risk to the rest of the system. As I said, OpenBSD is secure by default[^nginx-sec]. -[^nginx-sec]: The ports collection of OpenBSD contains a fork of NGINX with a similar security treatment. - -All we need to do now it to enable the daemon via the handy rcctl(8) tool. - -{{<highlight shell>}} -$ doas rcctl enable httpd -{{</highlight>}} - -and to start it - -{{<highlight shell>}} -$ doas rcctl start httpd -{{</highlight>}} - -And boom. Opening http://michal.sapka.me shows on our site both on IPv4 and IPv6. One thing to note here is the limitation of up to HTTP 1.1. HTTP 2 is not supported. - -Let's add TLS, so we have this cute lock icon. For this, we will request a certificate from [Let's Encrypt](https://letsencrypt.org/) using acme-client(1). If you used certbot, this will look familiar - just tidier. - -First, let's add config to `/etc/acme-client.conf` - -{{<highlight shell "linenos=inline">}} -authority letsencrypt { - api url "https://acme-v02.api.letsencrypt.org/directory" - account key "/etc/acme/letsencrypt-privkey.pem" -} - -authority letsencrypt-staging { - api url "https://acme-staging.api.letsencrypt.org/directory" - account key "/etc/acme/letsencrypt-staging-privkey.pem" -} - -domain michal.sapka.me { - domain key "/etc/ssl/private/michal.sapka.me.key" - domain full chain certificate "/etc/ssl/michal.sapka.me.crt" - sign with letsencrypt -} -{{</highlight>}} - -Lines 1-9 tell our acme-client(1) how to talk with Let's Encrypt, while lines 11-15 allow us to request a certificate for our domain. OpenBSD comes preconfigured for Let's Encrypt, so we just enable provided settings. - -Nice! Next, we need to allow Let's Encrypt challenges. Acme-client(1) will manage all required files, and Let's Encrypt can read them via httpd(8). Again, like cogs in a well-oiled machine. By default, acme-client(1) will write to `/var/www/acme`, so we need to redirect `/.well-known/acme-challenge/*` there. Let's change our `httpd.conf`: - -{{<highlight shell "linenos=inline">}} -server "michal.sapka.me" { - listen on * port 80 - root "/htdocs/michal-sapka-me" - - location "/.well-known/acme-challenge/*" { - root "/acme" - request strip 2 - } -} -{{</highlight>}} - -We can now either restart httpd(8) or reload it. Let's for the latter. - -{{<highlight shell>}} -$ doas rcctl reload httpd -{{</highlight>}} - -Now we can request the certificates - -{{<highlight shell>}} -$ doas acme-client -v michal.sapka.me -{{</highlight>}} - -OpenBSDs supplied tools don't print unnecessary information to the user, so we add the `-v` to see what's happening. Assuming everything went fine, let's start serving the page with TLS! - -For this, we will use relayd(8). We could use only httpd(8), but moving it one layer up is easier. Relayd(8) also gives us nice options for changing headers or moving some locations to a different process, like we will do with Plaroxy soon. This also shows us the big difference between this simple solution and NGINX: while NGINX shovels everything into one process and config, OpenBSD splits it into narrow focus areas. - -Let's open `/etc/relayd.conf` and add: - -{{<highlight shell "linenos=inline">}} -table <httpd> { 127.0.0.1 } - -http protocol "https" { - tls keypair "michal.sapka.me" - - match request quick header "Host" value "michal.sapka.me" forward to <httpd> -} - -relay "https" { - listen on 0.0.0.0 port 443 tls - protocol https - forward to <httpd> port 8080 - -} -relay "https6" { - listen on :: port 443 tls - protocol https - forward to <httpd> port 8080 -} -{{</highlight>}} - -Now, I won't go into much detail here, but what happens here is: -1. We create two relays, one for ipv4 and one for ipv6. One relay can listen on a single port for given IP. Each relay uses protocol "https" to modify and steer the request to a given process. -2. Both relays set up forwarding to httpd (IP taken from the table on the head of the file) on port 8080. -3. https protocol adds a TLS key pair for the session. We've got the files from Let's Encrypt in the step above. -4. We then test each request, and if the host matches "michal.sapka.me" it will be forwarded to httpd(8). - -You can also see that relayd(8) can listen on a given IP or all IPs (:: in case of IPv6) - -But our httpd(8) listens only on port 80! Let's fix that by changing the `httpd.conf` file: - -{{<highlight shell "linenos=inline">}} -server "michal.sapka.me" { - listen on * port 8080 -{{</highlight>}} - -We also need to redirect HTTP to HTTPS. Since we use Relayd(8) only for HTTPS, this will be done in httpd(8). Let's add a second server to our `httpd.conf`: - -{{<highlight shell "linenos=inline">}} -server "michal.sapka.me" { - listen on * port 80 - location * { - block return 301 "https://$HTTP_HOST$REQUEST_URI" - } -} -{{</highlight>}} - -Now, when the user enters the site, the flow will look like: - -1. httpd(8) will respond to :80 requests and return a 301 redirect to HTTPS -2. relayd(8) will catch the request to :443 and forward it on port :8080 to httpd(8) -3. httpd(8) will serve our site and pass the response to relayd(8) again -4. relayd(8) can modify headers before returning the response to the client. - -Talking about modifying headers, let's apply some extra security! We can expand our https protocol with the following: - -{{<highlight shell "linenos=inline">}} - # Return HTTP/HTML error pages to the client - return error - match request header set "X-Forwarded-For" value "$REMOTE_ADDR" - match request header set "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT" - match response header remove "Server" - match response header append "Strict-Transport-Security" value "max-age=31536000; includeSubDomains" - match response header append "X-Frame-Options" value "SAMEORIGIN" - match response header append "X-XSS-Protection" value "1; mode=block" - match response header append "X-Content-Type-Options" value "nosniff" - match response header append "Referrer-Policy" value "strict-origin" - match response header append "Content-Security-Policy" value "default-src https:; style-src 'self' \ - 'unsafe-inline'; font-src 'self' data:; script-src 'self' 'unsafe-inline' 'unsafe-eval'" - match response header append "Permissions-Policy" value "accelerometer=(), camera=(), \ - geolocation=(), gyroscope=(), magnetometer=(), microphone=(), payment=(), usb=()" - - # set recommended tcp options - tcp { nodelay, sack, socket buffer 65536, backlog 100 } - - # set up certs - tls { no tlsv1.0, ciphers "HIGH:!aNULL:!SSLv3:!DSS:!ECDSA:!RSA:-ECDH:ECDHE:+SHA384:+SHA256" } -{{</highlight>}} - -I won't discuss details here as each header has a dedicated MDM webdoc. Most of the headers here are considered a standard. - -Besides adding headers, we configure TLS here, disabling weak ciphers and old TLS versions and adding some standard config. - -Lastly, we can automate refreshing the certificate via cron(8): - -{{<highlight shell>}} -0~59 0~23 * * 1 acme-client michal.sapka.me && rcctl reload relayd -{{</highlight>}} - -It looks almost like a normal cron. The "0~59" and "0~29" parts are unique to OpenBSD: Cron(8) will evenly split all tasks between specified time boxes so that no two jobs run simultaneously. - -We now have created a fully working web server without any 3rd party packages. All OpenBSD provided, all secure, all simple, all cool as ice. - -To further your knowledge, you can challenge the assumption that BSD has the best doc and read man pages for `httpd.conf(5)`, `relayd.conf(5)`, and `acme-client.conf(5)`. - -I also can't recommend enough "Httpd and Relayd Mastery" by Michael W. Lucas[^mwl2] -[^mwl2]: yeah, the one from the top of this article. He's a household name and a staple of the BSD community. I'm primarily a software engineer, and all this sysadmin thing I am doing is a side quest for me. His books make it so much easier. I've already read four of his books, and I will read more as they are amazing. Even a dense person like yours truly comes out smarter after the lecture. While I'm not a [Full Michael](https://www.tiltedwindmillpress.com/product/full-michael-2023-06/) kind of person, it seems my library will soon have a very strong representation of his. - -## Plaprox - -One thing that OpenBSD doesn't provide (yet?) is an HTTP proxy. I use [Plausible](https://plausible.io/) for basic visitor analytics [^privacy] here, and one of the cool things you can do is to break all adblockers via serving Plausible from my own domain[^adblock] -[^privacy]: Yes, I want to know what people are reading! For details, refer to my [two sence long privacy policy](https://michal.sapka.me/about/#privacy-policy). -[^adblock]: yes, it's a dick move. But my reasoning was simple: Plausible gathers so little information that the harm is almost nonexistent, and I really want to know what people are reading. - -After two evenings of failed attempts, I reminded myself that I am a programmer, and I wrote one myself. You can find it on my [VCS page](https://vcs.sapka.me/plaprox/). It was a great learning exercise and a chance to touch Golang[^ruby] for the first time. -[^ruby]: I am a Ruby developer by trade and heart, but I will try anything that is not an IDE-driven language. LSP for Java/Scala is still a joke, and I refuse to pollute my system with Intellij. [Go](https://go.dev/), on the other hand, is a modern language designed for humans. I am not good at it, but I am infinitetly[^infinit] better than a week ago. -[^infinit]: Any positive number would be infinite progress compared to zero, or as an old wise man once said: "to have a nickel and to not a nickel is already two nickles". - -Assuming you have it running (it works on my machine!), let's adjust our relayd(8). Plaprox listens on port 9090, and we want to relay all requests to `/js/script.js` there. - -Let's add it to our relays in `relayd.conf`: - -{{<highlight shell "linenos=inline">}} -table <plausibleproxyd> { 127.0.0.1 } - -http protocol "https" { - - # all our previous content omitted - - match request quick path "/js/script.js" forward to <plausibleproxyd> - match request quick path "/api/event" forward to <plausibleproxyd> -} - - -relay "https" { - listen on 0.0.0.0 port 443 tls - protocol https - forward to <httpd> port 8080 - forward to <plausibleproxyd> port 9090 -} -relay "https6" { - listen on :: port 443 tls - protocol https - forward to <httpd> port 8080 - forward to <plausibleproxyd> port 9090 -} -{{</highlight>}} - -You can also move the port number to a table. - -Remember that in Relayd(8) last one wins. We already have a match for the domain and added another matcher for the path. The request will be forwarded to the last marching matcher - so we put our new matchers at the end of the protocol definition. - -## Next +## Updates -What are my next steps? It looks like OpenBSD much better supports the hardware of my laptop than FreeBSD, so I'll try to migrate it. +2023-12-12: moved info about web stack to a [dedicated article](/bsd/open-bsd-web-stack/) -## Updates -2023-07-28: remove wrong information abot PF. -2023-07-30: fix invalid cron format diff --git a/content/bsd/open-bsd-web-stack.md b/content/bsd/open-bsd-web-stack.md new file mode 100644 index 0000000..7b66252 --- /dev/null +++ b/content/bsd/open-bsd-web-stack.md @@ -0,0 +1,210 @@ +--- +title: "OpenBSD webstack: Relayd, Httpd and Acme-client" +category: +- bsd +abstract: Short guide on how to setup webserver on OpenBSD +date: 2023-07-19T12:27:54+02:00 +year: +draft: false +tags: +--- +OpenBSD comes with three great tools out of the box: + +- httpd(8) - an HTTP daemon +- relayd(8) - a relay daemon +- acme-client(1) - a client for Automatic Certificate Management Environment (ACME) + +With those free things, we can serve static webpages over TLS. While you most likely already use [NGINX](https://www.nginx.com/) or o[Apache](https://httpd.apache.org/)[^win], those solutions are complex. They work amazingly in enterprise environments where you have people with doctorates in NGINX configuration, but most real-world examples don't need that complexity. A static blog most likely doesn't. +[^win]: because there is no fourth way. Please repeat after me: there is no webserver in Windows. + +Let's set it up. + +Due to security concerns, OpenBSD comes with doas(1) instead of sudo(1). Copy `/etc/examples/doas.conf` file to `/etc/doas.conf`. For all intends, and purposes, from now on doas(1) will work the same as sudo(1). + +When the system boots for the very first time, ports 80 and 443 are closed, and only the SSH port is open. This alone was a nice surprise for me. But it gets better: since all utilities are part of the OSes, they work together perfectly. + +Assuming your domain is already pointing at the correct IPs, let's start listening for unencrypted HTTP traffic. I will use "michal.sapka.me" as the domain in all examples. + +First, Open `/etc/httpd.conf` in your favorite editor and add + +{{<highlight shell "linenos=inline">}} +server "michal.sapka.me" { + listen on * port 80 + root "/htdocs/michal-sapka-me" +} +{{</highlight>}} + +Then create a simple HTML file under `/var/www/htdocs/michal-sapka-me/index.html`. + +Httpd(8) works chrooted to /var/www/, so it threats this directory as root. This makes the "root" option shorter to write, but it also means that the process doesn't have access to anything outside of /var/www/. Even if an attacker can break in via the daemon, he will be locked in the www folder, so there is no risk to the rest of the system. As I said, OpenBSD is secure by default[^nginx-sec]. +[^nginx-sec]: The ports collection of OpenBSD contains a fork of NGINX with a similar security treatment. + +All we need to do now it to enable the daemon via the handy rcctl(8) tool. + +{{<highlight shell>}} +$ doas rcctl enable httpd +{{</highlight>}} + +and to start it + +{{<highlight shell>}} +$ doas rcctl start httpd +{{</highlight>}} + +And boom. Opening http://michal.sapka.me shows on our site both on IPv4 and IPv6. One thing to note here is the limitation of up to HTTP 1.1. HTTP 2 is not supported. + +Let's add TLS, so we have this cute lock icon. For this, we will request a certificate from [Let's Encrypt](https://letsencrypt.org/) using acme-client(1). If you used certbot, this will look familiar - just tidier. + +First, let's add config to `/etc/acme-client.conf` + +{{<highlight shell "linenos=inline">}} +authority letsencrypt { + api url "https://acme-v02.api.letsencrypt.org/directory" + account key "/etc/acme/letsencrypt-privkey.pem" +} + +authority letsencrypt-staging { + api url "https://acme-staging.api.letsencrypt.org/directory" + account key "/etc/acme/letsencrypt-staging-privkey.pem" +} + +domain michal.sapka.me { + domain key "/etc/ssl/private/michal.sapka.me.key" + domain full chain certificate "/etc/ssl/michal.sapka.me.crt" + sign with letsencrypt +} +{{</highlight>}} + +Lines 1-9 tell our acme-client(1) how to talk with Let's Encrypt, while lines 11-15 allow us to request a certificate for our domain. OpenBSD comes preconfigured for Let's Encrypt, so we just enable provided settings. + +Nice! Next, we need to allow Let's Encrypt challenges. Acme-client(1) will manage all required files, and Let's Encrypt can read them via httpd(8). Again, like cogs in a well-oiled machine. By default, acme-client(1) will write to `/var/www/acme`, so we need to redirect `/.well-known/acme-challenge/*` there. Let's change our `httpd.conf`: + +{{<highlight shell "linenos=inline">}} +server "michal.sapka.me" { + listen on * port 80 + root "/htdocs/michal-sapka-me" + + location "/.well-known/acme-challenge/*" { + root "/acme" + request strip 2 + } +} +{{</highlight>}} + +We can now either restart httpd(8) or reload it. Let's for the latter. + +{{<highlight shell>}} +$ doas rcctl reload httpd +{{</highlight>}} + +Now we can request the certificates + +{{<highlight shell>}} +$ doas acme-client -v michal.sapka.me +{{</highlight>}} + +OpenBSDs supplied tools don't print unnecessary information to the user, so we add the `-v` to see what's happening. Assuming everything went fine, let's start serving the page with TLS! + +For this, we will use relayd(8). We could use only httpd(8), but moving it one layer up is easier. Relayd(8) also gives us nice options for changing headers or moving some locations to a different process, like we will do with Plaroxy soon. This also shows us the big difference between this simple solution and NGINX: while NGINX shovels everything into one process and config, OpenBSD splits it into narrow focus areas. + +Let's open `/etc/relayd.conf` and add: + +{{<highlight shell "linenos=inline">}} +table <httpd> { 127.0.0.1 } + +http protocol "https" { + tls keypair "michal.sapka.me" + + match request quick header "Host" value "michal.sapka.me" forward to <httpd> +} + +relay "https" { + listen on 0.0.0.0 port 443 tls + protocol https + forward to <httpd> port 8080 + +} +relay "https6" { + listen on :: port 443 tls + protocol https + forward to <httpd> port 8080 +} +{{</highlight>}} + +Now, I won't go into much detail here, but what happens here is: +1. We create two relays, one for ipv4 and one for ipv6. One relay can listen on a single port for given IP. Each relay uses protocol "https" to modify and steer the request to a given process. +2. Both relays set up forwarding to httpd (IP taken from the table on the head of the file) on port 8080. +3. https protocol adds a TLS key pair for the session. We've got the files from Let's Encrypt in the step above. +4. We then test each request, and if the host matches "michal.sapka.me" it will be forwarded to httpd(8). + +You can also see that relayd(8) can listen on a given IP or all IPs (:: in case of IPv6) + +But our httpd(8) listens only on port 80! Let's fix that by changing the `httpd.conf` file: + +{{<highlight shell "linenos=inline">}} +server "michal.sapka.me" { + listen on * port 8080 +{{</highlight>}} + +We also need to redirect HTTP to HTTPS. Since we use Relayd(8) only for HTTPS, this will be done in httpd(8). Let's add a second server to our `httpd.conf`: + +{{<highlight shell "linenos=inline">}} +server "michal.sapka.me" { + listen on * port 80 + location * { + block return 301 "https://$HTTP_HOST$REQUEST_URI" + } +} +{{</highlight>}} + +Now, when the user enters the site, the flow will look like: + +1. httpd(8) will respond to :80 requests and return a 301 redirect to HTTPS +2. relayd(8) will catch the request to :443 and forward it on port :8080 to httpd(8) +3. httpd(8) will serve our site and pass the response to relayd(8) again +4. relayd(8) can modify headers before returning the response to the client. + +Talking about modifying headers, let's apply some extra security! We can expand our https protocol with the following: + +{{<highlight shell "linenos=inline">}} + # Return HTTP/HTML error pages to the client + return error + match request header set "X-Forwarded-For" value "$REMOTE_ADDR" + match request header set "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT" + match response header remove "Server" + match response header append "Strict-Transport-Security" value "max-age=31536000; includeSubDomains" + match response header append "X-Frame-Options" value "SAMEORIGIN" + match response header append "X-XSS-Protection" value "1; mode=block" + match response header append "X-Content-Type-Options" value "nosniff" + match response header append "Referrer-Policy" value "strict-origin" + match response header append "Content-Security-Policy" value "default-src https:; style-src 'self' \ + 'unsafe-inline'; font-src 'self' data:; script-src 'self' 'unsafe-inline' 'unsafe-eval'" + match response header append "Permissions-Policy" value "accelerometer=(), camera=(), \ + geolocation=(), gyroscope=(), magnetometer=(), microphone=(), payment=(), usb=()" + + # set recommended tcp options + tcp { nodelay, sack, socket buffer 65536, backlog 100 } + + # set up certs + tls { no tlsv1.0, ciphers "HIGH:!aNULL:!SSLv3:!DSS:!ECDSA:!RSA:-ECDH:ECDHE:+SHA384:+SHA256" } +{{</highlight>}} + +I won't discuss details here as each header has a dedicated MDM webdoc. Most of the headers here are considered a standard. + +Besides adding headers, we configure TLS here, disabling weak ciphers and old TLS versions and adding some standard config. + +Lastly, we can automate refreshing the certificate via cron(8): + +{{<highlight shell>}} +0~59 0~23 * * 1 acme-client michal.sapka.me && rcctl reload relayd +{{</highlight>}} + +It looks almost like a normal cron. The "0~59" and "0~29" parts are unique to OpenBSD: Cron(8) will evenly split all tasks between specified time boxes so that no two jobs run simultaneously. + +We now have created a fully working web server without any 3rd party packages. All OpenBSD provided, all secure, all simple, all cool as ice. + +To further your knowledge, you can challenge the assumption that BSD has the best doc and read man pages for `httpd.conf(5)`, `relayd.conf(5)`, and `acme-client.conf(5)`. + +I also can't recommend enough "Httpd and Relayd Mastery" by Michael W. Lucas[^mwl2] +[^mwl2]: yeah, the one from the top of this article. He's a household name and a staple of the BSD community. I'm primarily a software engineer, and all this sysadmin thing I am doing is a side quest for me. His books make it so much easier. I've already read four of his books, and I will read more as they are amazing. Even a dense person like yours truly comes out smarter after the lecture. While I'm not a [Full Michael](https://www.tiltedwindmillpress.com/product/full-michael-2023-06/) kind of person, it seems my library will soon have a very strong representation of his. + diff --git a/content/bsd/relayd-custom-service.md b/content/bsd/relayd-custom-service.md new file mode 100644 index 0000000..0dbac80 --- /dev/null +++ b/content/bsd/relayd-custom-service.md @@ -0,0 +1,57 @@ +--- +title: "forwarding requests from Relayd to custom webserver" +category: +- bsd +abstract: How to forward requests to a webserver? +date: 2023-07-19T12:27:54+02:00 +year: +draft: false +tags: +--- +## Plaprox + +One thing that OpenBSD doesn't provide (yet?) is an HTTP proxy. I use [Plausible](https://plausible.io/) for basic visitor analytics [^privacy] here, and one of the cool things you can do is to break all adblockers via serving Plausible from my own domain[^adblock] +[^privacy]: Yes, I want to know what people are reading! For details, refer to my [two sence long privacy policy](https://michal.sapka.me/about/#privacy-policy). +[^adblock]: yes, it's a dick move. But my reasoning was simple: Plausible gathers so little information that the harm is almost nonexistent, and I really want to know what people are reading. + +After two evenings of failed attempts, I reminded myself that I am a programmer, and I wrote one myself. You can find it on my [VCS page](https://vcs.sapka.me/plaprox/). It was a great learning exercise and a chance to touch Golang[^ruby] for the first time. +[^ruby]: I am a Ruby developer by trade and heart, but I will try anything that is not an IDE-driven language. LSP for Java/Scala is still a joke, and I refuse to pollute my system with Intellij. [Go](https://go.dev/), on the other hand, is a modern language designed for humans. I am not good at it, but I am infinitetly[^infinit] better than a week ago. +[^infinit]: Any positive number would be infinite progress compared to zero, or as an old wise man once said: "to have a nickel and to not a nickel is already two nickles". + +Assuming you have it running (it works on my machine!), let's adjust our relayd(8). Plaprox listens on port 9090, and we want to relay all requests to `/js/script.js` there. + +Let's add it to our relays in `relayd.conf`: + +{{<highlight shell "linenos=inline">}} +table <plausibleproxyd> { 127.0.0.1 } + +http protocol "https" { + + # all our previous content omitted + + match request quick path "/js/script.js" forward to <plausibleproxyd> + match request quick path "/api/event" forward to <plausibleproxyd> +} + + +relay "https" { + listen on 0.0.0.0 port 443 tls + protocol https + forward to <httpd> port 8080 + forward to <plausibleproxyd> port 9090 +} +relay "https6" { + listen on :: port 443 tls + protocol https + forward to <httpd> port 8080 + forward to <plausibleproxyd> port 9090 +} +{{</highlight>}} + +You can also move the port number to a table. + +Remember that in Relayd(8) last one wins. We already have a match for the domain and added another matcher for the path. The request will be forwarded to the last marching matcher - so we put our new matchers at the end of the protocol definition. + +# Updates + +2023-12-12: extracted to a dedicated article diff --git a/content/bsd/relayd-custom-webserver.md b/content/bsd/relayd-custom-webserver.md new file mode 100644 index 0000000..a361fc2 --- /dev/null +++ b/content/bsd/relayd-custom-webserver.md @@ -0,0 +1,58 @@ +--- +title: "forwarding requests from Relayd to custom webserver" +category: +- bsd +abstract: How to forward requests to a webserver? +date: 2023-07-19T12:27:54+02:00 +year: +draft: false +tags: +--- +One thing that OpenBSD doesn't provide (yet?) is an HTTP proxy. I use [Plausible](https://plausible.io/)[^nope] for basic visitor analytics [^privacy] here, and one of the cool things you can do is to break all adblockers via serving Plausible from my own domain[^adblock] +[^privacy]: Yes, I want to know what people are reading! For details, refer to my [two sence long privacy policy](https://michal.sapka.me/about/#privacy-policy). +[^nope]: [this is no longer the case](https://michal.sapka.me/site/updates/2023/removed-plausible/) +[^adblock]: yes, it's a dick move. But my reasoning was simple: Plausible gathers so little information that the harm is almost nonexistent, and I really want to know what people are reading. + +After two evenings of failed attempts, I reminded myself that I am a programmer, and I wrote one myself. You can find it on my [VCS page](https://vcs.sapka.me/plaprox/). It was a great learning exercise and a chance to touch Golang[^ruby] for the first time. +[^ruby]: I am a Ruby developer by trade and heart, but I will try anything that is not an IDE-driven language. LSP for Java/Scala is still a joke, and I refuse to pollute my system with Intellij. [Go](https://go.dev/), on the other hand, is a modern language designed for humans. I am not good at it, but I am infinitetly[^infinit] better than a week ago. +[^infinit]: Any positive number would be infinite progress compared to zero, or as an old wise man once said: "to have a nickel and to not a nickel is already two nickles". + +Assuming you have it running (it works on my machine!), let's adjust our relayd(8). Plaprox listens on port 9090, and we want to relay all requests to `/js/script.js` there. + +Let's add it to our relays in `relayd.conf`: + +{{<highlight shell "linenos=inline">}} +table <plausibleproxyd> { 127.0.0.1 } + +http protocol "https" { + + # all our previous content omitted + + match request quick path "/js/script.js" forward to <plausibleproxyd> + match request quick path "/api/event" forward to <plausibleproxyd> +} + + +relay "https" { + listen on 0.0.0.0 port 443 tls + protocol https + forward to <httpd> port 8080 + forward to <plausibleproxyd> port 9090 +} +relay "https6" { + listen on :: port 443 tls + protocol https + forward to <httpd> port 8080 + forward to <plausibleproxyd> port 9090 +} +{{</highlight>}} + +You can also move the port number to a table. + +Remember that in Relayd(8) last one wins. We already have a match for the domain and added another matcher for the path. The request will be forwarded to the last marching matcher - so we put our new matchers at the end of the protocol definition. + +## Updates + +2023-07-28: remove wrong information abot PF. +2023-07-30: fix invalid cron format +2023-12-12: extracted to a dedicated article diff --git a/static/style.css b/static/style.css index d33617c..ea5f47a 100644 --- a/static/style.css +++ b/static/style.css @@ -170,7 +170,7 @@ h1 { text-decoration: none; } sup { - font-size: 0.3em; + font-size: 0.6em; } /* generic_header_end */ |