{"id":11099,"date":"2024-01-06T01:13:05","date_gmt":"2024-01-06T00:13:05","guid":{"rendered":"https:\/\/monodes.com\/predaelli\/?p=11099"},"modified":"2024-01-06T01:14:25","modified_gmt":"2024-01-06T00:14:25","slug":"how-standard-ebooks-serves-millions-of-requests-per-month-with-a-2gb-vps-or-a-paean-to-the-classic-web-alex-cabal","status":"publish","type":"post","link":"https:\/\/monodes.com\/predaelli\/2024\/01\/06\/how-standard-ebooks-serves-millions-of-requests-per-month-with-a-2gb-vps-or-a-paean-to-the-classic-web-alex-cabal\/","title":{"rendered":"How Standard Ebooks serves millions of requests per month with a 2GB VPS; or, a paean to the classic web – Alex Cabal"},"content":{"rendered":"\n

Source: How Standard Ebooks serves millions of requests per month with a 2GB VPS; or, a paean to the classic web – Alex Cabal<\/a><\/em><\/p>\n\n\n\n\n\n\n\n

\n

Standard Ebooks<\/a> is a project that takes transcriptions of public domain literature, like the kind typically available at Project Gutenberg<\/a>, and creates beautiful, modern ebooks out of them using a detailed style manual<\/a>. Volunteers from all over the world work to produce these ebooks, and then we release them free of cost and free of copyright restrictions.<\/p>\n\n\n\n

Books that are in the US public domain are typically books that were published at least 95 years ago. (At the time of writing, this means published before 1927.) As you can imagine, we spend a lot of our time working on really, really old books. So why not create a website out of really, really old technology?<\/p>\n\n\n\n

In 2021 the Standard Ebooks server served about 1.4 million ebook downloads over 15.5 million page views. That comes out to a little over 1.2 million page views per month, and that excludes requests for our OPDS<\/a> and RSS feeds<\/em>, which also number in the millions per month. It\u2019s been on the front page of Hacker News at least<\/a> three<\/a> times<\/a> in the past several years without breaking a sweat. And we did all this with a tiny VPS serving a website that has no Javascript\u2014or even a database!<\/p>\n\n\n\n

The amount of traffic we get isn\u2019t that<\/em> crazy. But it\u2019s a demonstration of how inexpensive, classic-web technology can serve a surprising amount of traffic, without relying on any of the technological crutches that so many of us reflexively reach for nowadays when starting new projects.<\/p>\n\n\n\n

What the SE server does<\/h2>\n\n\n\n

SE runs its entire operations using one single-core VPS with 2GB of RAM. It\u2019s responsible for:<\/p>\n\n\n\n

    \n
  1. Hosting all of the Git repositories for all of our ebooks, serving as the source of truth for the ebooks we produce. (While we also host all of our ebooks on GitHub, GitHub is actually a mirror<\/em>, not the origin, for our ebooks. We think keeping local control is important, and self-hosting a GitHub-like solution to allow the public to view ebook sources and submit corrections, instead of relying on GitHub, is on our long-term roadmap.)<\/li>\n\n\n\n
  2. Running the ebook build process when an ebook is released or updated.<\/li>\n\n\n\n
  3. Serving the SE website, including the OPDS and RSS feeds.<\/li>\n<\/ol>\n\n\n\n

    The Standard Ebooks website<\/a> runs on a classic LAP stack. What\u2019s that, you say? LAP? That\u2019s right: LAMP without the M. There isn\u2019t even a traditional database behind the SE website.<\/em><\/p>\n\n\n\n

    Why PHP?<\/h2>\n\n\n\n

    I\u2019ve been writing PHP for almost twenty years now. It\u2019s a language and paradigm I\u2019m familiar with. As one gets older, one feels less and less excitement for the new tech hotness\u2014after all, when one must drive a nail, there\u2019s only so many different kinds of hammers to get excited about before one stops caring and just reaches for the go-to. So when I want to start a new website, I reach for PHP.<\/p>\n\n\n\n

    Fortunately, PHP is by one measure the server-side language of choice for 78% of the web<\/a>, so hitching your wagon to such a popular language means it\u2019ll be easy to find someone to maintain it for at least the next few decades.<\/p>\n\n\n\n

    Modern PHP isn\u2019t half bad, and it has at least two major benefit over some of its competitors:<\/p>\n\n\n\n

      \n
    1. Each request is a totally independent request that rebuilds the world. There\u2019s no shared state (unless you want there to be).<\/li>\n\n\n\n
    2. It\u2019s an interpreted language, so there\u2019s no code to recompile and no servers to restart when deploying. You make a change in your script, save your file, refresh your browser, and there it is. That makes development and debugging really fast<\/em> and deployment really easy<\/em>.<\/li>\n<\/ol>\n\n\n\n

      What many people also don\u2019t know is that PHP is also a fairly good templating language out of the box! While it\u2019s easy to assume that any PHP web project at the bare minimum requires one of the big<\/a> frameworks<\/a>, in truth it\u2019s easy to use basic PHP as its own simple templating system. For many basic classic-web websites, like SE\u2019s, that\u2019s often enough to get the job done; and the big benefit is that you\u2019re not stuck with having to learn and maintain<\/em> a huge bells-and-whistles 3rd-party framework in perpetuity.<\/p>\n\n\n\n

      I think people really underestimate the burden of maintaining<\/em> a 3rd-party framework even after development<\/em> of the website is complete. If your website is alive for even just five short years, how many times will you have to update the framework, even if only for security updates, and keep up with API updates to make sure there are no breaking changes? Once is too much for what\u2019s essentially scaffolding that I never want to see again.<\/p>\n\n\n\n

      How the SE server handles everything without falling over<\/h2>\n\n\n\n

      Serving a web request<\/h3>\n\n\n\n

      Requests start at Apache, which usually starts by rewriting the URL using mod_rewrite<\/code>\u2014for example, rewriting an ebook path into a PHP script with a query string. This is kind of equivalent to the router in a server-side framework, except that it\u2019s done before PHP even enters the picture.<\/p>\n\n\n\n

      Once Apache decides what PHP script to run, it forwards it to PHP-FPM. PHP-FPM<\/a> is the process manager that replaced the ailing and feeble mod_php<\/code> many years ago, and is one of the major reasons why modern PHP websites can be so fast.<\/p>\n\n\n\n

      Once a request reaches PHP, we start creating output for the browser using a very, very small file-based templating system<\/a>. It uses raw PHP instead of a custom templating syntax. As I mentioned earlier, PHP is already a surprisingly good templating language in its own right. Why reinvent the wheel?<\/p>\n\n\n\n

      Dynamic content without a database<\/h3>\n\n\n\n

      At this point in the request, we may have to look up some data before we can present it to the browser. This is where one might query a database\u2014but we don\u2019t have a database. So how do we do it?<\/p>\n\n\n\n

      You may recall that the SE server also hosts all of the Git repositories for all our ebooks. These ebooks are rich in metadata, including title, author, descriptions, word count, series information, and so on. Because our corpus of ebooks is relatively small (about 630 ebooks at the time of writing), we\u2019re able to fit all of them in memory<\/em>: specifically, using PHP\u2019s built-in APCu object cache<\/a>.<\/p>\n\n\n\n

      This cache survives across PHP processes, so we only have to initialize it once when PHP-FPM first starts, and we only have to update it when an ebook is updated. If the PHP process crashes or the server restarts, the website rebuilds the cache from scratch whenever it starts up again. Building this cache does take a few seconds, but since crashes and restarts are rare it\u2019s not a big deal.<\/p>\n\n\n\n

      You may be thinking that this is a pretty silly approach to a dynamic website. Why not just use a database?<\/p>\n\n\n\n

      I don\u2019t entirely disagree. But it\u2019s that way because SE started out as\u2014and in many ways still is\u2014a small hobby project, and small hobby projects want to spend time on the meat (ebooks) and not the potatoes (a website).<\/p>\n\n\n\n

      To build a website, we would have already<\/em> had to parse the XML in our ebooks, if only to insert it into a database for later lookup. If we\u2019re already creating ebook objects in PHP, why not just skip the database, the ORM layer, the glue code, the data model, and all that ancillary stuff, and simply cache the ebook objects themselves, to save development time on this hobby?<\/p>\n\n\n\n

      It turned out that for our small data set and very basic data processing requirements, this approach is perfectly performant. Of course, we\u2019ll eventually<\/em> reach a point where we\u2019ll need a real database, but we\u2019ll probably reach that point first because we want our website to do more complex things, not because the size of our ebook corpus calls for one.<\/p>\n\n\n\n

      And in the meantime, we\u2019re still serving millions of page views just fine.<\/p>\n\n\n\n

      Javascript not necessary<\/h3>\n\n\n\n

      If we haven\u2019t shocked you with our deeply-questionable design decisions yet, this one might shock the younger developers out there most of all: our website doesn\u2019t use any Javascript. Zero. (Well, with the exception of a link to a manifest.json<\/code> file, because there\u2019s no way around that.)<\/p>\n\n\n\n

      In terms of content, the SE website is very much a classic-web website. There\u2019s a homepage, a search page with a paginated list of items, a detail view of those items, some very basic forms here and there, and a few ancillary, mostly static pages. And it just so happens that plain HTML with some clever CSS has just about all of the functionality needed by that kind of website:<\/p>\n\n\n\n