Making BlogGen

---

Table of Contents

1. What is BlogGen?
2. BlogGen - Concept
3. BlogGen - Setup
4. How the Magic Happens
5. Discussion, Retrospective
6. Conclusion

This blog post is a bit of a retrospective, and a way of condensing some older blog posts that I had written during the initial development of BlogGen. Those posts were a bit more stream-of-consciousness, and didn’t provide as concise of a summary of this project as I would have hoped. So, this post is meant to condense those thoughts and provide an overview of BlogGen.

What is BlogGen?

BlogGen started out as a summer coding project. I had set up my own personal websites in the prior months, and had been wanting to make a personal blog for some time so that I could write about things I was interested in.

I had taken a quick survey of blogging solutions, such as Jekyll, Hugo, and of course WordPress, but none of them seemed appealing to me. I wanted a solution that was designed to be self-hosted, with minimal configuration, and had a simple but effective security model. Though some of the tools I found could be configured to meet these goals, I ultimately settled on trying to bulid my own solution. Why rely on these large software projects maintained by experienced engineers when I could instead make my own, _____ version? (fill in the blank after you read the code)

BlogGen - Concept

As detailed in the (somewhat hard to follow) idea doc, the goal was to make a blogging framework comprised of 3 main parts:

1. A CLI tool, used to generate post templates and convert them from markdown into HTML
2. A Server, which accepted connections from the CLI tool, did some basic authentication, and stored uploaded posts
3. A Frontend, which dynamically displayed blog posts after being processed by the server.

My vision for the project was to have the server and frontend be a one-time setup task, and from then on, the user would only ever need to use the CLI tool to make blog posts.

Additionally, I wanted the project to be environment agnostic with a simple, but effective security model. As a result, every server-side component (Nginx, Next.js, and the custom Server code) runs in its own Docker container, and all you need to have is SSH access on the underlying machine. All client-server communications work on top of SFTP, which lies on top of SSH; that way, only people with SSH access to the underlying server can add blog posts to the site. This is a reasonable assumption to make, and requires essentially no extra configuration for most scenarios (if you don’t have SSH access to your webserver you have bigger problems).

BlogGen - Setup

The server really only has one configuration file needed, which is just a simple JSON file. This file is parsed, and provides some basic information for server operations (listening port, log output, etc.). Outside of some minor nginx configuration depending on your environment, all you need to focus on is really just the styling of the frontend to your liking.

After these steps are completed, you simply run everything with Docker Compose, and your entire server side is done!

The client is also simple to configure; you just specify the server and the port when you’re issuing commands, and you can even set up an environment variable to avoid the hassle.

How the Magic Happens

With the tool configured, the workflow becomes very simple:

1. Start a blog post with the CLI tool: bloggen post init <post-name>
2. Go into the new directory and start writing in <post-name>.md
3. Add linked assets (photos, GIFs, etc.) directly to the assets folder if you want, or they’ll be automatically copied in at upload time
4. Once you’re finished, type bloggen post upload in the post directory

In a few seconds, the post should appear on the website! How does this happen? Well, to take a brief look behind the curtain:

1. The client initializes an SFTP session with the server (with the client’s SSH keypair)
2. The post’s markdown is converted into HTML, assets are copied locally, and all these files are sent over our SFTP session.
3. The server places these files in a directory linked to the frontend
4. The Next.js frontend soon discovers the new files and displays them statically using SSG.

It might not be as glamorous internally as one would expect, but the end result is shockingly simple; just a few commands to have a blog post appear on the website, and no WordPress/Wix bill. Dreams really do come true. The fact that everything is Dockerized also makes the server-side nice to use. Each component is modularized, easy to update in-place, and handles its own dependencies. Bundling everything up into Docker containers makes the application surprisingly robust; containers only share the data that they need to function, and only listen on external ports where absolutely necessary.

Discussion, Retrospective

The design of BlogGen’s backend isn’t meant to be groundbreaking, nor is it meant to be on the cutting edge in terms of performance. The use case is very simple and not very restrictive; a personal blogging platform doesn’t need to have incredible scaling or be implemented as a distributed system. The overall goal was ease of use, and I believe that I met this goal quite well in implementation.

That isn’t to say that the system is perfect, though. Far from it. One of the most difficult problems was that of content distribution; if you’re an engineer looking at the last section, you’re probably wondering: “Why is this guy using SFTP? Does anyone even use that anymore? I’m pretty sure that died out before AOL.” Indeed, the first thought most engineers would have when tasked with building a blogging platform is to be storing files in some sort of relational database. This was my prevailing thought process initially too (look guys I promise I’m not a hack), but the same problem continued to appear: access control. If I set up blog posts in a relational database, then used REST or GraphQL for access, how could I stop unauthorized entities from issuing malicious requests?

The answer was always the same: secondary authentication. The user needed to have a password, or a new keypair, or some secret token. Maybe the user could have an interactive log-on through some REST API call, present their authentication token/password, then be given an access token with an expiration date? Then they could make calls (upload blog posts) with that token? No matter how I split it, the authentication process was either too convoluted (given that blog post uploads happened relatively infrequently), or required the client to do extra work. Namely, remembering a new password or storing a new security code. Since ease of use was my fundamental goal, I settled on SFTP because it just meant that the client used their SSH key. Since everything happened over SSH, data was encrypted in transit and the client’s identity was authenticated by the server.

The issue with that decision was ultimately reflected in performance and fault tolerance. The files are simply dropped into the right place over SFTP, but that doesn’t allow any redundancy or scaling. It’s also generally inefficient; the files aren’t compressed or stored in a binary format, as they would be in a relational database. They’re simple UNIX files that exist somewhere in the directory tree. So a user on the machine may delete them by accident, or move them, and they would simply disappear from the site. Again, not ideal.

Another major problem is the styling of the frontend. When I was building the project, as most developers tend to do (…right?), I ignored the frontend until the very end. This meant that the way files were actually going to be represented as blog posts was eventually a major issue. The main concern here was how to keep all the structure of React and all the styling of Tailwind, when the blog posts themselves were converted into plain HTML files. It wasn’t exactly pretty; I ended up writing a lot of manual CSS rules for styling pages that would probably make a modern frontend dev cringe. Blog posts were then essentially plugged into pages as plain HTML, and the more general CSS rules would take over from the lack of Tailwind to make things look ok.

The design of the frontend was thus one of my largest shortcomings of the project. If a user wants to change the way the site looks for their blog, they have to go through various React components to change Tailwind classes, as well as through some more basic CSS. This doesn’t exactly hinder the “ease of use” ideology behind the project (because you don’t have to change the styling of the site for it to function), but it does make it harder to customize blogs within the framework. This issue might’ve been much easier to address if the backend used a relational database. Then, rather than plain HTML, posts would only be stored as text, and styling could be applied as the text was injected into the page. But, as stated before, this requires some sort of secondary authentication method that makes the application less ‘plug-n-play.’ So, ultimately, I made the decision to keep the frontend design as it was, and prioritize ease of use over ease of customization.

Conclusion

I learned a lot from this project. It was an incredibly fun journey to take it from idea to execution, and it’s also pretty neat that I’m able to use the tool as I’m writing this blog post.

There’s still a lot that could be done for the project; it is by no means a complete software (is there such a thing?). I don’t expect to see anyone using it for production-grade, large-scale blogs any time soon, but it would be immensely exciting to see it deployed by someone other than me. If you’re reading this and you’re charitable enough to run BlogGen on your machine, shoot me an email with your experience/deployment. I would be absolutely thrilled to see someone using the service, even if they give me an earful of criticisms along the way.