diff options
Diffstat (limited to 'content')
| -rw-r--r-- | content/bsd/blocking-bad-bots-openbsd.md | 74 | ||||
| -rw-r--r-- | content/bsd/desktop-freebsd-wont-improve-unless-people-are-using-it.md | 120 | ||||
| -rw-r--r-- | content/bsd/history/01_multics.md | 79 | ||||
| -rw-r--r-- | content/bsd/home.md | 16 |
4 files changed, 95 insertions, 194 deletions
diff --git a/content/bsd/blocking-bad-bots-openbsd.md b/content/bsd/blocking-bad-bots-openbsd.md deleted file mode 100644 index 51ed5ec0..00000000 --- a/content/bsd/blocking-bad-bots-openbsd.md +++ /dev/null @@ -1,74 +0,0 @@ -+++ -title = "OpenBSD: Blocking bad bots using Relayd" -author = ["Michał Sapka"] -date = 2023-12-11T19:08:00+01:00 -categories = ["bsd"] -draft = false -weight = 2002 -abstract = "How do I fight bad crawlers?" -[menu] - [menu.bsd-openbsd] - weight = 2002 - identifier = "openbsd-blocking-bad-bots-using-relayd" - parent = "obsdweb" - name = "Blocking bad bots using Relayd" -+++ - -The bane of existence for most of small pages: web crawlers. -They create most traffic this site sees and makes my [site stats](https://michal.sapka.me/site/info/#site-stats) overly optimistic. -We can go with [robots.txt](https://en.wikipedia.org/wiki/Robots_Exclusion_Protocol), but what if it's not enough? -I can tell a valuable bot to not index some part of my site, but: -a) some bots ignore it -b) what if I don't want some bots to even have the chance to ask? - -Get that SEO scanning and LLM training out of here! - - -## Blocking crawlers {#blocking-crawlers} - -The rest of this guide assumes webstack: Relayd and Httpd. -Relayd is great and since it works on higher level than pf, we can read headers. -Luckily, those crawlers send usable "User-Agents" which we can block. - -First, let's see who uses my site the most. Assuming you use "forwarded"[^fn:1] style for logs, we can do: - -```shell -awk -F '"' '{print $6}' <path to log file> | sort | uniq -c | sort# -``` - -Then we need to manually select agents we want to block. -It won't be easy, as the strings are long and contain a lot of unnecessary information - which includes plain lies. -You need to define which part of the full User-Agent is common and can be used for blocking. - -Then we can create block rules in a Relayd protocol. -Relayd doesn't use regexp, and instead allows using case-sensitive Lua globs. -Stars will match everything. - -```shell -block request method "GET" header "User-Agent" value "*<common part>*" -``` - -Remember that config assumes last-one-wins, so the block rules should be the last matching. -I just put those end the end of my config. -You can create a \`block quick...\` rule if you want - it will short-circuit the entire protocol. - -Therefore, my "https" protocol now has a series of blocks: - -```shell -http protocol "https" { - # most of the procol omitted - block request method "GET" header "User-Agent" value "*Bytespider*" - block request method "GET" header "User-Agent" value "*ahrefs*" - block request method "GET" header "User-Agent" value "*censys*" - block request method "GET" header "User-Agent" value "*commoncrawl*" - block request method "GET" header "User-Agent" value "*dataforseo*" - block request method "GET" header "User-Agent" value "*mj12*" - block request method "GET" header "User-Agent" value "*semrush*" - block request method "GET" header "User-Agent" value "*webmeup*" - block request method "GET" header "User-Agent" value "*zoominfo*" -} -``` - -(usage of globs was proposed to me on [OpenBSD mailing list](<https://marc.info/?l=openbsd-misc&m=170206886109953&w=2>) - -[^fn:1]: : vide <https://man.openbsd.org/httpd.conf.5#style> diff --git a/content/bsd/desktop-freebsd-wont-improve-unless-people-are-using-it.md b/content/bsd/desktop-freebsd-wont-improve-unless-people-are-using-it.md deleted file mode 100644 index 723c82c4..00000000 --- a/content/bsd/desktop-freebsd-wont-improve-unless-people-are-using-it.md +++ /dev/null @@ -1,120 +0,0 @@ -+++ -title = "FreeBSD on the Desktop won't improvide unless people are using it" -author = ["Michał Sapka"] -date = 2023-03-29T22:09:00+02:00 -categories = ["bsd"] -draft = false -weight = 2002 -abstract = "I have hits from /r/bsd, but almost none of those people are using BSD" -[menu] - [menu.bsd-varia] - weight = 2002 - identifier = "freebsd-on-the-desktop-won-t-improvide-unless-people-are-using-it" - name = "Desktop FreeBSD won't improve unless people are using it" -+++ - -Shamelessly, I posted my previous post, [FreeBSD on a Thinkpad Extreme G2](https://d-s.sh/2023/freebsd-on-thinkpad-x1-extreme-g2/) on [/r/bsd Reddit](https://www.reddit.com/r/BSD/comments/124v5cm/freebsd_on_a_thinkpad_x1_extreme_g2/). - -The result, some 24 hours later, is 100 visitors. -Out of that 100, 57 are using a desktop. -Out of that 57, only 2 used FreeBSD—2%. No other BSDs are recorded. - -People who are into BSD don't use BSD. -This seems to be a reason for lacking hardware support. -If no one uses FreeBSD, no one will encounter those problems. If no one encounters them, no one will fix them. - - -## Update 2023-04-14 {#update-2023-04-14} - -The article, got quite the round around the internets, gathering some interests from [Reddit](https://old.reddit.com/r/freebsd/comments/126fvkz/desktop_freebsd_wont_improve_unless_people_are/), [Hacker News](<https://news.ycombinator.com/item?id=35378367>), Twitter, Discover BSD, or [Vermaden](https://vermaden.wordpress.com/2023/04/03/valuable-news-2023-04-03/). -With all that interest come quite a few questions and comments. -The following is an attempt to summarize it all. - - -### People who use FreeBSD don't care about FreeBSD hardware {#people-who-use-freebsd-don-t-care-about-freebsd-hardware} - -This makes perfect sense. -If your FreeBSD installation on X220 works flawlessly, you may not care about anything more modern. -But there will come a time when you will need to replace the hardware. - -{{< img-r "freebsd-beastie.png" >}} -FreeBSD Beastie -{{< /img-r >}} - -This comment, however, came as a proof that the sample from my blog is invalid. -This may be the case, but I don't buy it. -All traffic on the aforementioned post came from Reddit's BSD forum. -It's the one place where you could expect that people using BSD would hang. -It may also be that it's quite a random sample - it's small, and people who have yet to become into BSD but are BSD-curious opened my blog post. -I am in no place to debunk or confirm this. -I, however, know that many people presenting at FreeBSD conferences do it using Macs or Windows. -So even if the numbers are dubious, the overall feeling remains sorrowful. - -To add to the above: there are also stats for the commented opinion piece. -Two hundred forty-four people opened it from /r/freebsd. -Of that, 24 people were using FreeBSD, and just 2 were using OpenBSD. - - -### Your statistics may be invalid as people mask their browser agent. {#your-statistics-may-be-invalid-as-people-mask-their-browser-agent-dot} - -This also may be the case. -Why, then, is the referer not spoofed? -It's a much more invasive data point than the underlying OS. -But I'm a simple Firefox user, never used Librewolf. - - -### FreeBSD is a server OS {#freebsd-is-a-server-os} - -Yeah, this is the sentiment I've read before jumping aboard. -My problem with this idea is that each and every FOSS OS is a value in itself. -The current poster boy, -Linux, also had huge problems getting to work on various machines. -In my opinion, it's limiting OS to a single use case is a completely valid point - your use case for FreeBSD is on a server, and this is where it currently shines (or not, depending on your experience). -Some folks despise allocating any FreeBSD dev time to the desktop as there are many server issues. - -But again, I don't see it this way. -Limiting FreeBSD to the server only is short-lighted. -Unless you are already powering your servers with BSD, there will always be a question: "Why not Linux. It's what everyone else is doing". -And Linux got into its current position not by being a great server machine but rather by attracting the interest of some very skillful people. -And it did it by allowing more and more people to free themselves from Windows on their machines. - -I see FreeBSD problems as having two primary causes: the [Unix wars](https://en.wikipedia.org/wiki/Unix_wars) of the past and limited resources now. -If FreeBSD were easier to use on a wide range of end-user machines (which tend to be laptops), the easier it would for people to want to develop it. -BSDs are now a far second choice. -Why would someone invest time? -They may fall in love with the OS, but unless they try it, it will never happen. - - -### I like our small userbase {#i-like-our-small-userbase} - -I'm as elitist as the other person. [DWM](https://dwm.suckless.org/) stated that - -> "This keeps its userbase small and elitist. -> No novices asking stupid questions. - -I can't find this quote anymore, but the sentiment seems similar. -However, there are two aspects here. - -FreeBSD comes with no graphical interface by default. -This makes it much closer to minimalist distros than Ubuntu. -This still allows anyone to feel like a hacker. - -The second, however, is that some problems are unsolvable by end-user. -Writing drivers is EXTREMELY difficult, and, as I've recently learned (thanks, Jeff!), this is especially true when it comes to WiFi drivers, as there is no open implementation. -This means that any progress requires a trial-and-error process based on reverse engineering. -No one without deep knowledge of low-level programming will be able to make any progress, and even those few will need people with real hardware for testing. - - -### Hardware support is years behind Linux {#hardware-support-is-years-behind-linux} - -Yes, and this is what I was referring to. - - -### Why would anyone use BSD on a desktop? {#why-would-anyone-use-bsd-on-a-desktop} - -It's a great system, just needs a lot of work on hardware support :-) - - -### Your post is worthless, and only the comments are interesting {#your-post-is-worthless-and-only-the-comments-are-interesting} - -It's more than I anticipated. That post was small and written without any deeper research. But the discussion around it makes me believe that I hit something real. diff --git a/content/bsd/history/01_multics.md b/content/bsd/history/01_multics.md new file mode 100644 index 00000000..b9265529 --- /dev/null +++ b/content/bsd/history/01_multics.md @@ -0,0 +1,79 @@ ++++ +title = "History of BSD part I: Multics" +author = ["Michał Sapka"] +date = 2024-03-09T21:03:00+01:00 +categories = ["bsd"] +draft = false +weight = 2001 +abstract = "BSD history starts with Multics" +[menu] + [menu.bsd-history] + weight = 2001 + identifier = "history-of-bsd-part-i-multics" + name = "Multics" ++++ + +## Origins of time-sharing {#origins-of-time-sharing} + +Let's start our journey back when dinosaurs roamed the earth, engineers wore ties, and Barbie was first gaining popularity - the 60s. +Nowadays, we are accused to companies selling devices with a loss, as all the money comes from software subscriptions. +But 70 years ago it was the complete opposite. +Companies bought expensive computers, huge machines, and what was running on them was of lesser value. +And I really mean _expensive_. +An IBM System/360 Model 20, presented in 1964, could have been bought starting from USD 62,710 (USD 622,626 adjusted for inflation) or rented for USD 1280 (USD 12,708 adjusted)[^fn:1]. +Sellers earned a lot from the devices. +But, naturally, companies making all those investments wanted a nice return. +This led to creation of _time-sharing_. + +{{< img-c "ibm-360.jpg" "https://www.nbcnews.com/tech/gadgets/5-reasons-love-mad-mens-new-star-ibm-360-n101716" >}} +IBM 360 in an official photoshoot. +{{< /img-c >}} + +This concept seems natural now: multiple processes were able to share computer resources, so multiple applications could run at the same time. +Applications could even run for different users. +Ergo, time-sharing allowed for multi-user multitasking processing. +This is in stark contrast to batch-processing, where only a single program would be able to compute at any particular time. +An example of such processing would be EDSAC, the first electronic computer[^fn:2]. +I won't go into detail of time-sharing, but you can read more in "Time sharing in large computers", C. Strachey, 1959. +But what is important here is how it was used. +All computation happened on a single, large server. +End users would use _computer terminals_ which were _multiplexed_ into that server, called _mainframe_. +What is multiplexing you ask? +Simply said, it's a way to different combine signals into a shared medium. +It was used extensively for land-line telephony, where all signals were transferred over shared wires. + +{{< img-c "edsac.jpg" "https://www.datacenterdynamics.com/en/analysis/rebuilding-edsac-the-first-real-computer/" >}} +EDSAC, the "first computer". +{{< /img-c >}} + + +## Multics {#multics} + +Let's jump to 1969. + +_Multics (MULTiplexed Information and Computing)_ was an early time-sharing operating system developed by _MIT_, _General Electrics_ and _Bell Labs_. +It pioneered many of innovations which are still widely used in computing systems of today: hierarchical file system, redirection as interprocess communication, or the existence of a shell[^fn:3], as well as memory pages, memory protection, or the ability for a single machine to use multiple CPUs and memory[^fn:4]. +It was however also huge, both in terns of memory usage (the resident kernel could occupy a huge part of memory living not enough for applications) and code size (it consisted of about 1,5k source modules)[^fn:3]. +_Multics_ was experimental and therefore ambitious, complex, and designed by trial and error. +It was delivered late, early on had performance problems, and in 1969 _Bell Labs_ withrew from the project[^fn:5]. + +{{< img-c "multics-login.png" "https://en.wikipedia.org/wiki/Multics#/media/File:Multics-Login.png" >}} +Multics login screen. +{{< /img-c >}} + +{{< img-c "thompson-ritchie.jpg" "https://computerhistory.org/blog/discovering-dennis-ritchies-lost-dissertation/" >}} +Ken Thompson and Denis Ritchie. +{{< /img-c >}} + +Ken Thompson, Dennis Ritchie, Douglas McIlroy, and Joe Ossanna, frustrated with their experiences with _Multics_ Operating System are starting to work on their own alternative. +Their work will become one of the most beloved computer products of all time - _UNIX_. +The team armed with previous experience set up to create simple, manageable OS that would still fulfill all the requirements _Multics_ was to fulfill. + +Currently, however, they have problems convincing _Bell Labs_ management to get them a new computer. +This has not stopped them from designing the system on black blackboards and paper. + +[^fn:1]: [IBM 360 Model 20 Rescue and Restoration: Machine Overview](https://ibms360.co.uk/?p=902) +[^fn:2]: [First Operating System -- Part One](http://blog.wovenmemories.net/2023/10/30/First.Operating.System_Part.1.html) +[^fn:3]: [Multics --- {W}ikipedia{,} The Free Encyclopedia](https://en.wikipedia.org/wiki/Multics) +[^fn:4]: [Mark Allen - Before Unix: An Early History of Timesharing Systems](https://www.youtube.com/watch?v=UYb6WqWBTE0) +[^fn:5]: [The early days of Unix at Bell Labs - Brian Kernighan (LCA 2022 Online)](https://www.youtube.com/watch?v=ECCr_KFl41E) diff --git a/content/bsd/home.md b/content/bsd/home.md index 85afab1b..58557a60 100644 --- a/content/bsd/home.md +++ b/content/bsd/home.md @@ -15,6 +15,22 @@ Those are the systems I daily run on my personal computers and servers. And, while they come with significant problems, it's very easy to fall in love them. +## History of BSD {#history-of-bsd} + +Does computer history have any real appliance? +Not really. +Does it explain certain, potentially baffling things? Certainly. +But does it make great beer talk? +Absolutely! +Especially when discussing systems with such rich history as BSD. +I will be gentle here, I promise. +No _Hardcore History_ in sight. + +{{< menu "bsd-history" >}} +History of BSD +{{< /menu >}} + + ## FreeBSD {#freebsd} {{< img-r "freebsd-beastie.png" >}} |