MyJekyllBlog

MyJekyllBlog is an all-in-one multi-user CMS and hosting platform for Jekyll blogs.

Administrators can log into the web panel and configure hosted domains that users can host blogs under (i.e. hosted-blog.com).

Users can log into the web panel and configure a blog on their own domain or a subdomain of the hosted domains (i.e. mycookingblog.hosted-blog.com). Once they have a blog, they can use the CMS features to create posts, upload media, and otherwise manage their blog.

Whenever a user updates their blog, the blog is rebuilt and deployed the to webservers. Each update is a git commit, and a history panel allows users to restore their blog to any past state.

MyJekyllBlog comes with a complete set of ansible roles to automate the installation.

Table of Contents

1. MyJekyllBlog
   1. Table Of Contents
   2. Meet The Servers
      1. Panel
      2. Build
      3. Store
      4. Certbot
      5. WebServer
   3. Overview
2. Installation Guide
   1. Designing The Network
   2. Running The Installation With Ansible
   3. Configure The Panel
   4. Configure The Store
3. Operations Guide
   1. How to perform a system backup
   2. How to perform a system restore
4. Development Guide
   1. Enable development mode
   2. Working on the web panel
   3. Working on the database model
   4. Working on the ssl certificates
   5. Working on the build and deploy process
   6. Working on the nginx web servers

Meet The Servers

This table shows a brief overview of the server types and their relationships. Additional descriptions for each server type follows the table.

Server	Description	Services	Talks To
Panel	Runs customer-facing web interface	mjb.panel, nginx	build, store
Build	Runs site builders, deploys blogs	mjb.worker	store, webservers
Store	Source of truth - Database, Gitea	postgresql, gitea	panel, build, certbot
Certbot	Handles getting/updating SSL certs	mjb.certbot	store, webservers
WebServer	Hosts customer blogs on the internet	nginx	certbot, build

Panel

The Panel server hosts the web application that customers can use to provision blogs, publish articles, upload media and otherwise manage their blogs. Administrators can use it to check users/blogs on the system, run maintenance tasks, and configure some aspects of the system.

Build

The build server processes Jekyll git repositories into static websites and deploys the fully built website to the webservers for hosting.

Store

The store server hosts two database with postgresql. One database supports MyJekyllBlog::DB and another supports Minion. The Panel, Build, and Certbot servers all need access to these databases.

The store server also hosts an installation of Gitea so that each Jekyll blog may have its own central git repository. The panel server will checkout and commit/push to this server. The build server will checkout the repository from this server for building.

Certbot

The CertBot server handles obtaining SSL certificates from Let's Encrypt and pushing them to the webservers.

When HTTP challenges are used, /.well-known/ is proxied from ALL webserver nodes to the certbot node and --standalone is used from the certbot node to obtain an SSL certificate.

When DNS challenges are used, wildcard certificates may be obtained (and is recommended for hosts expecting many sub-domains to be made).

The /etc/letsencrypt directory is synced with webserver nodes through rsync whenever new certificates are obtained. An administrator can update and sync SSL certificates from the admin panel.

WebServer

WebServers run nginx and host static content for Jekyll blogs. When a blog is provisioned, an SSL certificate will be requested for the site and an nginx configuration file will be created.

The build servers will sync the blog content with webservers each time the blog is updated through the Panel.

Overview

This diagram shows a view of a user updating a post and somebody from the Internet viewing a post.

flowchart TB
    subgraph one[Panel]
        a2[MyJekyllBlog::Web]
        a3[Nginx]
        a3 -- Hypnotoad PSGI --> a2
    end

    subgraph two[Build]
        b2[MyJekyllBlog Worker<br><br>- Pulls blog repo<br>- Builds blog with jekyll<br>- Deploys to each WebServer]
    end

    subgraph four[Store]
        d1[PostgreSQL]
        d2[Gitea]
        d2 <-- Second: Minion fetches repo --> b2
        d1 <-- First: Minion Picks Up Job --> b2
        a2 -- Queue blog sync job --> d1
        a2 -- Update file in repo --> d2

    end

    subgraph three[WebServer]
        c1[nginx]
    end

    b2 -- rsync to each WebServer Node --> c1
    q[Internet User] <-- View Blog Post -->c1
    z[MyJekyllBlog User] <-- Submit Post Update -->a3

Installation Guide

This guide will follow my process of installing the software to run on mds-stage.com and serve blogs on mds-stage-blog.com. If you follow along, you could have your own version up and running in a couple of hours.

Designing The Network

I need at least one of each server type. I will, however, use two WebServers and have one on the west coast and another on the east coast. I will need six servers, and I will name them panel, store, build, certbot, web-west, west-east.

I want the database server running on store not to be exposed to the Internet at large, and I will use the private networking feature of my VPS provider to get private IPs for panel, store, build, and certbot. This means I also need to have all of those machines running in the same datacenter.

Each machine should be running Debian 11.

Machine	Public IP	Private IP	Domain
panel	45.79.31.186	192.168.213.90	panel.mjb-stage.com
build	45.33.25.211	192.168.188.226	build.mjb-stage.com
store	69.164.204.212	192.168.216.75	store.mjb-stage.com
certbot	96.126.122.198	192.168.163.105	certbot.mjb-stage.com
web-west	173.255.249.43	N/A	web-west.mjb-stage.com
web-east	173.255.225.48	N/A	web-east.mjb-stage.com

Once I have these machines provisioned I lay out the information about them in the table above. I will need this information to begin writing the configuration file.

For each of these machines, I update DNS records so that the domain maps to the public IP address for each machine.

Additionally, I add two DNS records for *.mds-stage-blog.com, one A record with 173.255.249.43 and another A record with 173.255.225.48. This maps all sub-domains of mds-stage-blog.com to the web servers so that they may serve the blogs to people on the Internet. People on the Internet will go to one or the other server.

Before proceeding from this section, review the section checklist to ensure you have completed all item.

X	Section Checklist Items
[ ]	Provision machine for panel, build, store, certbot
[ ]	Provision one or more machines for webservers
[ ]	Create table with your machine information
[ ]	Add DNS records for each machine
[ ]	Add wildcard DNS record for each WebServer

Running The Installation With Ansible

The installation process has been codified into a number of ansible roles that are included in this repository. I will need ansible installed so that I have access to the ansible-playbook command, and I will need to have SSH credentials to each of these machines so that ansible may install and configure the network.

From the root directory of a clone of this repository, I do the following.

cd devops/ansible/
mkdir -p env/stage

# Copy and edit the inventory file for your network.
cp env/example/inventory.yml env/stage/inventory.yml
vim env/stage/inventory.yml

# Copy and edit the secrets for your network
cp env/example/vault.yml env/stage/vault.yml
vim env/stage/vault.yml

# Create a vault password to encrypt your secrets with
perl -e'print join("", map { ('A'..'Z','a'..'z',0..9)[int rand 62] } ( 0 .. 128 )), "\n"' > .vault_password

# Encrypt your secrets with the vault password
ansible-vault encrypt --vault-password-file .vault_password env/stage/vault.yml

I named the configuration file env/stage/inventory.yml, since this will be a staging environment. I placed this in its own directory because some environment specific files will be stored in the inventory directory, and keeping seperate directories will prevent file clobbering. One should pay special attention to go through this example config file and update it with details of their network. I updated the vault file with new passwords and then encrypted it. Once this is complete, the installation should be smooth sailing with ansible. I use the following command to get everything installed.

ansible-playbook -i env/stage/inventory.yml --vault-password-file .vault_password -e @env/stage/vault.yml site.yml

This command took about two and a half hours to complete, it should largely setup the whole platform across all of the machines.

Before proceeding from this section, review the section checklist to ensure you have completed all item.

X	Section Checklist Items
[ ]	Checked repo out to a machine with ansible and ssh access
[ ]	Network specific ansible inventory file was created
[ ]	Ansible runs through the entire playbook with no errors

Configure The Panel

I need to create an account on the panel, https://panel.mjb-stage.com/register with the email address manager@mjb-stage.com.

Once I create the account, I need to promote it to an admin. From a shell on panel.mjb-stage.com server, I run the following.

cd mjb/Web
./script/mjb flip_admin manager@mjb-stage.com

Now that I have an admin account, I can access the Servers tab at https://panel.mjb-stage.com/admin/servers

The tab configures web servers that the system will deploy blogs to. Each of the webservers that were configures by Ansible should go here, so I enter web-west.mjb-stage.com and then web-west.mjb-stage.com. The servers tab now lists these two servers.

Next I need to go to the Domains tab and add mjb-stage-blog.com.

There is a drop down for SSL Challenge. When selecting HTTP, each time a blog is added, certbot will be used to complete an HTTP challenge for the domain. When selecting DNS-Linode, a Wildcard SSL certificate will be obtained and then blogs will not need their own certificates. DNS-Linode requires a Linode account and API credentials.

X	Section Checklist Items
[ ]	Created admin account, can login and view Admin Panel
[ ]	Added Web Servers to Admin Panel -> Servers
[ ]	Added Hosted Domains to Admin Panel -> Domains

Configure The Store

During the installation process, an SSH keypair was created. The public key must be added to the Gitea user that was setup. This must be done through the Gitea web panel.

1. Login to Gitea on the store server, using the credentials for gitea user/pass from the inventory file.
2. Click the user drop down in the upper right
3. Click Settings from the drop down menu
4. Click "SSH / GPG Keys"
5. Click "Add Key" under "Manage SSH Keys"
6. Type a title
7. Paste the contents of env/staging/files/ssh/id_rsa.pub
8. Click to add the key

Once this is done, you'll need to create the mjb organization.

1. Click the + Plus button drop down
2. Click "New Organization"
3. Name the organization "mjb"
4. Click "Create Organization"

Now we need to add a Jekyll blog as a template.

Get a shell into the build server and create a new Jekyll blog.

# Create the default blog
alias jekyll="podman run -ti --rm -v .:/srv/jekyll -e JEKYLL_ROOTLESS=1 docker.io/jekyll/jekyll jekyll"
jekyll new jekyll-default

# Push this default to the panel_config.jekyll_init_repo repository
cd jekyll-default
git init
git remote add origin git@store.mjb-stage.com:manager/jekyll-default.git
git add * .gitignore
git commit -m "Initial Commit"
git push origin master

Now that the panel_config.jekyll_init_repo repository exists, we should be ready to rock and roll; give provisioning a blog a go.

X	Section Checklist Items
[ ]	Confirmed login to the Gitea install on store server
[ ]	Added SSH key to the gitea user account
[ ]	Added mjb organization for blog repos to be added under
[ ]	Pushed the jekyll repo to the jekyll_init_repo address

Step 3: Confirm It All Works!

1. Create a user account
2. Create a blog
3. Create a post
4. Delete a post

Operations Guide

How to perform a system backup

How to perform a system restore

Development Guide

Enable development mode

Working on the web panel

Working on the database models

Working on the ssl certificates

Working on the build and deploy process

Working on the nginx web servers

Development Guide

MJB::Web Panel Development

As root you will need to stop the MJB::Web app from running in production.

systemctl stop mjb.panel

As the manager user you can run the application in development mode.

cd mjb/Web
morbo ./script/mjb --listen http://127.0.0.1:8080

Now it will automatically reload when you make changes to the libraries and templates. Additionally, it will show stack traces during crashes and debug information in your terminal.

Jekyll

You can run Jekyll by getting into a build server and running the following:

alias jekyll="podman run -ti --rm -v .:/srv/jekyll -e JEKYLL_ROOTLESS=1 docker.io/jekyll/jekyll jekyll"

Once you've done that, jekyll command will work.

Operations Guide