Avery Tan, Security Professional

Example Talk

Sat, 01 Jun 2030 13:00:00 +0000

Click on the Slides button above to view the built-in slides feature.

Slides can be added in a few ways:

Create slides using Hugo Blox Builder’s feature and link using slides parameter in the front matter of the talk file
Upload an existing slide deck to static/ and link using url_slides parameter in the front matter of the talk file
Embed your slides (e.g. Google Slides) or presentation video on this page using .

Further event details, including such as image galleries, can be added to the body of this page.

Max Redundancy with 3-2-1!

Mon, 01 Dec 2025 00:00:00 +0000

Introduction

Backups are important and an often overlooked aspect in the minds of the day-to-day average joes. I definitely was reminded the importance of having a backup strategy during a failure of a routine kernel upgrade on my main Linux workhorse a few years ago. In this particular absolut tragedy, I count myself incredibly lucky at being able to leverage to successfully recover most of my data.

Backups are even more doubly important in production enterprise systems holding client and/or otherwise business-critical data. This of course, should be an obvious and needless-to-mention axiom, and a lack of a backup strategy or adequate disaster recovery response plan would be utterly devastating to any meaningfully sized organization, resulting in all sorts of legal and/or compliance risk, not to mention potential financial losses.

I personally might not be a organization beholden to the risks of non-compliance to such industry and regulatory frameworks and the repurcussions that come with it, but I do have what I consider to be critically important personal data whose loss would be a true tragedy on a personal level in the form of family photos, historical personal budget/financial data, media server configuration setting files, personally curated recipes collection, and more in the form of a variety of self-hosted applications running on individual docker containers on my home server.

The 3-2-1 Strategy

The 3-2-1 backup strategy is one of the most widely recommended approaches for protecting data. It’s simple, flexible, and works across personal and enterprise contexts.

This strategy comprises of having 3 copies of your data:

the primary “prod” data; and
2 backup copies, 1 of which is stored offsite.

Having a copy of the data located offsite provides geographic separation, which reduces the risk of data loss via fire, theft, floods, or other natural or man-made disasters!

Offsite can mean cloud storage, a safety deposit box, or a drive you rotate to another location.

Our Implementation

In our specific case, we will implement a version of the 3-2-1 strategy using 3 SSD harddisks:

SSD #1 - our primary drive installed in a Hard Disk enclosure connected to our home application server
SSD #2 - our backup drive attached directly to our home application server via a PCI-to-usb adapter
SSD #3 - our offsite backup drive attached to a separate backup server located in geographically separated area from our main application server

The relevant directory structure on the primary application server is the following:

/home/s1na/
├── media-stack
│   └── starr
│ ├── docker-compose.yml
│ ├── pinchflat
│ │ └── config/
│ ├── jellyseer
│ │ └── config/
│ ├── jellyfin
│ │ ├── cache/
│ │ └── config/
│   └── other-media-server-related-docker-containers...
│ └── config/
/mnt/f15/
 ├── ntfy
 │ ├── docker-compose.yml
 │ ├── config
 │ │ └── server.yml
 │ └── cache/
 └── docker-containers
 ├── firefly
 │ ├── docker-compose.yml
 │ └── volumes/
 ├── immich
 │ ├── docker-compose.yml
 │ └── volumes/
 ├── calibre
 │ ├── docker-compose.yml
 │ └── volumes/
 ├── mealie
 │ ├── docker-compose.yml
 │ └── volumes/
 └── other-hosted-applications...
 ├── docker-compose.yml
 └── volumes/

In implementing our 3-2-1 backup strategy, we are interested primarily in the docker applications located in /mnt/f15/docker-containers but only the config data in our *arr media stack folders. For the media server, the actual media data lives at a separate location and the sheer volume of media data (movies + tv shows) and the relatively low importance make including full backups of the media server data impractical.

We will also be using ntfy in order to trigger email notifications on success or failure. The shutdown of all application containers within the docker-containers directory before a backup copy is taken is . Hence we run an instance of ntfy outside of /mnt/f15/docker-containers to ensure that ntfy is always running to report the status of the backup process, both success and failure.

nfty

Docker compose file for ntfy

services:
 ntfy:
 image: binwiederhier/ntfy
 container_name: ntfy
 command:
 - serve
 volumes:
 - ./cache:/var/cache/ntfy
 - ./config:/etc/ntfy
 ports:
 - 10700:80
 healthcheck: # optional: remember to adapt the host:port to your environment
 test: ["CMD-SHELL", "wget -q --tries=1 http://localhost:80/v1/health -O - | grep -Eo '\"healthy\"\\s*:\\s*true' || exit 1"]
 interval: 60s
 timeout: 10s
 retries: 3
 start_period: 40s
 restart: unless-stopped

To set up access to Gmail smtp server, we need a server.yml file within the mapped config volume:

# this is the local ip of the application server
# ntfy is running out of port 10700
base-url: http://192.168.1.76:10700
smtp-sender-addr: "smtp.gmail.com:587"
smtp-sender-user: "sender.email.addr@gmail.com"
smtp-sender-pass: "app-password"
smtp-sender-from: "sender.email.addr@gmail.com"

As of writing, Google now requires the use of something called an .

With the ntfy container spun up and running, email notifications can then be triggered via a HTTP POST request:

curl \
 -H "Email: recipient.email.addr@gmail.com" \
 -H "Tags: +1,tada" \
 -d "Everything went fine with the backup." \
 http://localhost:10700/docker-backup

ntfy email notification

Borg & Rsync

After extensive consultation with the likes of ChatGPT and Claude, Borg will be the primary backup tool we use to implement our 3-2-1 strategy. Borg has considerable strengths when compared to Rsync, primarily which is its ability to perform versioned and deduplicated incremental backups.

Prior to implementing Borg for this strategy, Rsync was used in a previous iteration of this strategy. Rsync would periodically make a incremental backup copy which overrides the existing backup copy.

Versioned backups are critically important for the reason that if the current prod data somehow became corrupted and unreadable, making a copy of this corrupted unreadable data and then overwriting the existing readable backup copy would then result in an absolut tragic loss of data!

# initiating a new empty borg backup dir
borg init --encryption=none /mnt/docker/back/up/dir
# executing the backup with some nifty informative flags
borg create --stats --progress --compression zstd,9 /mnt/docker/back/up/dir::docker-backup-$(date +%F) /source/dir/to/back/up
# list all backup copies
borg list /mnt/docker/back/up/dir
# prune to remove old outdated backup copies
borg prune -v --keep-daily=14 --keep-weekly=4 --keep-monthly=6 --keep-yearly=2 /mnt/docker/back/up/dir

Another strength of Borg is its ability to perform encryption at rest via the –encryption mode flag. In our specific implementation, I made the calculated trade-off opting not to encrypt data at rest since none of my servers or services are publicly internet-facing.

Communication channels between the prod application server and the remote offsite backup server is done via , giving us a reliable connection while at the same time not having either servers exposing naked services directly to the scary dangerous internet.

The Backup Scripts

We will create 2 separate backup scripts:

one for backing up the media server configuration files (in case tragedy ever befalls our *arr stack, we can quickly spin up a fully configured plug-n-play replacement); and
one for a full backing up of the more ‘critical’ self-hosted applicationa and their respective data.

My *arr stack media server is a basic implementation as described in the .

The *arr stack media server

Containers running as part of the media server are specified in the single docker-compose.yml file located at the home/s1na/media-stack/starr directory.

The following bash script:

Checks that both the source and destination directories exist.
If those checks pass, it then stops the containers specified in home/s1na/media-stack/starr/docker-compose.yml.
The script then calls Borg to perform the incremental backup and prunes old outdated copies.
The containers are then spun up.
Email notifications are executed via ntfy if any intermediary step fails or if the backup is successful

#!/bin/bash

# Define variables
BACKUP_DIR_LOCAL="/mnt/bae/media-server-config-backup-dir"
DOCKER_CONTAINERS_DIR='/home/s1na/media-stack'
LOG_FILE="/var/log/media-server-config-backup.log"


# Function to send email via ntfy
send_email() {
 local subject=$1
 local message=$2
 curl \
 -H "Email: recipient.email.addr@gmail.com" \
 -H "Tags: cd" \
 -d "$subject $message" \
 http://192.168.1.76:10700/media-server-config-backup
}


# Log function
log() {
 echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | sudo tee -a "$LOG_FILE" > /dev/null
}

# Start script
log "Backup process started."

# Check if backup directory exists
if [ ! -d "$BACKUP_DIR_LOCAL" ]; then
 log "Backup directory $BACKUP_DIR_LOCAL does not exist."
 send_email "Media Server Config Backup Failed" "Backup directory $BACKUP_DIR_LOCAL does not exist."
 exit 1
fi

# Check if backup directory exists
if [ ! -d "$DOCKER_CONTAINERS_DIR" ]; then
 log "Docker-containers directory $DOCKER_CONTAINERS_DIR does not exist."
 send_email "Media Server Config Backup Failed" "Docker-containers directory $DOCKER_CONTAINERS_DIR does not exist."
 exit 1
fi


# Stop all containers
log "Stopping all containers..."
for app_dir in "$DOCKER_CONTAINERS_DIR"/*; do
 if [ -d "$app_dir" ] && [ -f "$app_dir/docker-compose.yml" ]; then
 log "Stopping containers for app in: $app_dir"

 # Change to the app directory
 cd "$app_dir" || {
 log "Failed to change directory to $app_dir";
 send_email "Media Server Config Backup Failed" "Failed to change dir while trying to stop containers into $app_dir during the backup process"
 exit 1;
 }

 if sudo docker compose down; then
 log "Containers for app $app_dir stopped successfully."
 else
 log "Failed to stop containers for app $app_dir."
 send_email "Media Server Config Backup Failed" "Failed to stop containers in $app_dir."
 exit 1
 fi
 fi
done

# Borg incremental backup
log "Starting borg backup..."
if sudo borg create --stats --progress --compression zstd,9 $BACKUP_DIR_LOCAL::media-server-config-backup-$(date +%F) $DOCKER_CONTAINERS_DIR; then
 log "Borg backup completed successfully."
else
 log "Borg backup failed."
 send_email "Media Server Config Backup Failed" "Borg backup failed."
 exit 1
fi

# Borg Clean up
if sudo borg prune -v --keep-daily=14 --keep-weekly=4 --keep-monthly=12 --keep-yearly=5 $BACKUP_DIR_LOCAL; then
 log "Borg cleanup completed successfully."
else
 log "Bor cleanup failed."
 send_email "Docker Cleanup Failed" "Borg Cleanup failed."
 exit 1
fi

# Restart containers
log "Restarting containers..."
for app_dir in "$DOCKER_CONTAINERS_DIR"/*; do
 if [ -d "$app_dir" ] && [ -f "$app_dir/docker-compose.yml" ]; then
 log "Restarting containers for app in: $app_dir"

 # Change to the app directory
 cd "$app_dir" || {
 log "Failed to change directory to $app_dir";
 send_email "Media Server Config Backup Failed" "Failed to change dir into $app_dir during the backup process"
 exit 1;
 }

 if sudo docker compose up -d; then
 log "Containers for app $app_dir restarted successfully."
 else
 log "Failed to restart containers for app $app_dir."
 send_email "Media Server Config Backup Failed" "Failed to restart containers in $app_dir."
 exit 1
 fi
 fi
done

log "Backup process completed successfully."
send_email "Media Server Config Backup Successful" "Backup process completed successfully. See log for details: $LOG_FILE"
exit 0

This script is periodically run via cron:

# runs every first of the month at 6:00 AM MST.
0 6 1 * * /mnt/f15/desktop/backup-scripts/media_server_config_backup_script.sh

The self-hosted applications

The containers in the /mnt/f15/docker-containers directory form the critical assets where we want to implement full backup of everything – configuration settings as well as full application data and files. The goal is for speedy and full recovery in the event of a full disk failure of the primary harddisk.

Each self-hosted application lives in its own directory within the docker-containers directory. Typically, each of these application directories contains a docker-compose.yml file and a volumes subdirectory where all application configuration files and data files live.

# Typical directory structure
/mnt/f15/docker-containers
 └── some-application
 ├── docker-compose.yml
 └── volumes
 ├── config
 │ └── config-files
 ├── data
 │ └── data-files
 └── cache-dir-or-other-app-specific-volume-dirs
 └── cache-files-or-other-app-specific-files

The backup bash script targeting these applications is similar to the one for the media config backup, with a small difference in how containers are stopped and started. The following takes place:

Check for existence of both source and destination directories
Stops all containers by iteratively entering each subdir of /mnt/f15/docker-containers/docker-containers
Borg performs the incremental backup and prunes old outdated copies.
The containers are then spun back up.
Email notifications are executed via ntfy if any intermediary step fails or if the backup is successful.

#!/bin/bash

# Define variables
BACKUP_DIR_LOCAL="/mnt/bae/borg-backup-dir"
DOCKER_CONTAINERS_DIR='/mnt/f15/docker-containers'
LOG_FILE="/var/log/docker-backup.log"


# Function to send email via ntfy
send_email() {
 local subject=$1
 local message=$2
 curl \
 -H "Email: recipient.email.addr@gmail.com" \
 -H "Tags: cd" \
 -d "$subject $message" \
 http://192.168.1.76:10700/docker-backup
}


# Log function
log() {
 echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | sudo tee -a "$LOG_FILE" > /dev/null
}

# Start script
log "Backup process started."

# Check if backup directory exists
if [ ! -d "$BACKUP_DIR_LOCAL" ]; then
 log "Backup directory $BACKUP_DIR_LOCAL does not exist."
 send_email "Docker Backup Failed" "Backup directory $BACKUP_DIR_LOCAL does not exist."
 exit 1
fi

# Check if backup directory exists
if [ ! -d "$DOCKER_CONTAINERS_DIR" ]; then
 log "Docker-containers directory $DOCKER_CONTAINERS_DIR does not exist."
 send_email "Docker Backup Failed" "Docker-containers directory $DOCKER_CONTAINERS_DIR does not exist."
 exit 1
fi


# Stop all containers
log "Stopping all containers..."
for app_dir in "$DOCKER_CONTAINERS_DIR"/*; do
 if [ -d "$app_dir" ] && [ -f "$app_dir/docker-compose.yml" ]; then
 log "Stopping containers for app in: $app_dir"

 # Change to the app directory
 cd "$app_dir" || {
 log "Failed to change directory to $app_dir";
 send_email "Docker Backup Failed" "Failed to change dir while trying to stop containers into $app_dir during the backup process"
 exit 1;
 }

 if sudo docker compose down; then
 log "Containers for app $app_dir stopped successfully."
 else
 log "Failed to stop containers for app $app_dir."
 send_email "Docker Backup Failed" "Failed to stop containers in $app_dir."
 exit 1
 fi
 fi
done

# Borg incremental backup
log "Starting borg backup..."
if sudo borg create --stats --progress --compression zstd,9 $BACKUP_DIR_LOCAL::docker-backup-$(date +%F) $DOCKER_CONTAINERS_DIR; then
 log "Borg backup completed successfully."
else
 log "Borg backup failed."
 send_email "Docker Backup Failed" "Borg backup failed."
 exit 1
fi

# Borg Clean up
if sudo borg prune -v --keep-daily=14 --keep-weekly=4 --keep-monthly=12 --keep-yearly=5 $BACKUP_DIR_LOCAL; then
 log "Borg cleanup completed successfully."
else
 log "Bor cleanup failed."
 send_email "Docker Cleanup Failed" "Borg Cleanup failed."
 exit 1
fi

# Restart containers
log "Restarting containers..."
for app_dir in "$DOCKER_CONTAINERS_DIR"/*; do
 if [ -d "$app_dir" ] && [ -f "$app_dir/docker-compose.yml" ]; then
 log "Restarting containers for app in: $app_dir"

 # Change to the app directory
 cd "$app_dir" || {
 log "Failed to change directory to $app_dir";
 send_email "Docker Backup Failed" "Failed to change dir into $app_dir during the backup process"
 exit 1;
 }

 if sudo docker compose up -d; then
 log "Containers for app $app_dir restarted successfully."
 else
 log "Failed to restart containers for app $app_dir."
 send_email "Docker Backup Failed" "Failed to restart containers in $app_dir."
 exit 1
 fi
 fi
done

log "Backup process completed successfully."
send_email "Docker Backup Successful" "Backup process completed successfully. See log for details: $LOG_FILE"
exit 0

The crontab entry for this:

# runs every other day at 4:00 AM MST
0 4 */2 * * /mnt/f15/desktop/backup-scripts/docker_backup_script.sh

The resultant local backups

A recent ntfy notification email

s1na@big-dumpling:~$ borg list /mnt/bae/docker-backup-dir/
docker-backup-2025-06-19 Wed, 2025-06-18 21:52:46 [4f175980a4ca28ad26ec8a14e224b6aa5c4c5a0485027e5b6dfee33c81de88dd]
docker-backup-2025-07-13 Sat, 2025-07-12 23:28:56 [492cb6d014c6bac827cd21f0f10b5feba652342039af8319328361da8e79aab8]
docker-backup-2025-07-21 Sun, 2025-07-20 22:01:00 [23fb593765cf59f8f7c99edf9d98a51609c3826df2c37fceba117a17ec20427e]
docker-backup-2025-07-27 Sat, 2025-07-26 22:00:59 [aec28e0224ebcb6a8ae8ba0d9fffb8f84a9fb64d67e9281c5c0097fa40e93496]
docker-backup-2025-08-01 Thu, 2025-07-31 22:01:00 [e32cef441520f19e4b62752fa1c1bd9792baa4f311063db3dc301075b79bef9d]
docker-backup-2025-08-03 Sat, 2025-08-02 22:00:58 [449d8d699abc67f88d7dac8efc4a7a254dd22d4ae75375479b40c88366bd5726]
docker-backup-2025-08-09 Fri, 2025-08-08 22:01:03 [93311a0e5eacc7564e3c0471ecdec5da051949153b018bb2ddceef7db796b5af]
docker-backup-2025-08-11 Sun, 2025-08-10 22:01:03 [2f25c96fc8a919062015f24b150e9289417ab73366feb4ae841bb94d14b9f59d]
docker-backup-2025-08-13 Wed, 2025-08-13 04:01:04 [f8318d0f435d0ecc3f018c42a981696abdea287761d0c059d7e6b4227b581872]
docker-backup-2025-08-15 Fri, 2025-08-15 04:01:01 [7c41a0f970d653bd6bcd3799ed62d0b6ede0f1a3f241283b94418acf0847193c]
docker-backup-2025-08-19 Tue, 2025-08-19 04:01:04 [abd740749118c55068e7dcb87c12ed105986e8691ea076205c8d78ca77bf41d7]
docker-backup-2025-08-21 Thu, 2025-08-21 04:01:00 [da9ddb1a6cc6f72e5cc46b328b321472d710230c7a3f5d64979e51bec53cb247]
docker-backup-2025-08-23 Sat, 2025-08-23 04:00:50 [fb0c86b220a22bb9d7343ddf97a55f4b35a4e32893b2325e37086552195a309d]
docker-backup-2025-08-25 Mon, 2025-08-25 04:00:51 [9f3e9f2c3ddd8fe514608995791254eea6ea820bad38f17c250f2a320fa82a5c]
docker-backup-2025-08-27 Wed, 2025-08-27 04:00:50 [5727271708eefe625c9d1d12cff43c433df07231777af274b4dcda6ab1ce6ce1]
docker-backup-2025-08-29 Fri, 2025-08-29 04:00:55 [1a5d6b36da5a02c29750dccdcbb9cb74de2899e992c0cd90809c9677fd5d9149]
docker-backup-2025-08-31 Sun, 2025-08-31 04:00:51 [2cf17793a4ae580007d504518209d45fd8b352b7b41150b8230da5b991671c08]
docker-backup-2025-09-01 Mon, 2025-09-01 04:00:55 [87c4d2ad57f27cdded1735baa346e01efbe1cf7f5a588205ea62dded8cff7892]
docker-backup-2025-09-03 Wed, 2025-09-03 04:00:51 [71234a94c8836af7c2baaa36956051d15f7c8d34c5474f304252ec2bf9cfc2c3]
docker-backup-2025-09-05 Fri, 2025-09-05 04:00:52 [6db1ecd6da8ac1173366266a41867453ef19d57f84c30611b7cc98af4e77c725]

s1na@big-dumpling:~$ borg info /mnt/bae/borg-backup-dir/
Repository ID: 2290625c5b22dc05897fe230602d386c376780016a6f393cf9578c9a0b73008f
Location: /mnt/bae/docker-backup-dir
Encrypted: No
Cache: /root/.cache/borg/2290625c5b22dc05897fe230602d386c376780016a6f393cf9578c9a0b73008f
Security dir: /root/.config/borg/security/2290625c5b22dc05897fe230602d386c376780016a6f393cf9578c9a0b73008f
------------------------------------------------------------------------------
 Original size Compressed size Deduplicated size
All archives: 2.63 TB 2.59 TB 126.98 GB

 Unique chunks Total chunks
Chunk index: 101445 1931093


s1na@big-dumpling:~$ borg list /mnt/bae/media-server-config-backup-dir/
media-server-config-backup-2025-08-02 Sat, 2025-08-02 05:31:20 [a058d03472f47d91f51f5d9728ec598a40a8b52d19accc844c6a783cec4aab3e]
media-server-config-backup-2025-09-01 Mon, 2025-09-01 06:00:13 [4ba86566575f271f7391cca663d4d7f56efcc5e9b9c8b0598d752f84ef384d39]

s1na@big-dumpling:~$ borg info /mnt/bae/media-server-config-backup-dir/
Repository ID: f57b38ff467ef220a76fbd056de15e0c906822bbfb0410740b2b4f555692652c
Location: /mnt/bae/media-server-config-backup-dir
Encrypted: No
Cache: /root/.cache/borg/f57b38ff467ef220a76fbd056de15e0c906822bbfb0410740b2b4f555692652c
Security dir: /root/.config/borg/security/f57b38ff467ef220a76fbd056de15e0c906822bbfb0410740b2b4f555692652c
------------------------------------------------------------------------------
 Original size Compressed size Deduplicated size
All archives: 27.94 GB 25.81 GB 19.61 GB

 Unique chunks Total chunks
Chunk index: 43801 70102

Sending a backup copy offsite

Unlike for making the versioned incremental local backups where we employ borg, sending a copy of these local borg backup directories to a geographically separate remote server will be done via rsync. We assume that rsync will never run a backup job to the remote server at the same that borg is performing its own incremental backup, and will schedule our cron jobs to ensure this.

We will also need to perform passwordless ssh login to our remote server. The following generates a new private/public key pair and then copies the public key to our remote server. Our remote server is aptly named lil-dumpling.

ssh-keygen -t ed25519 -C "rsync backup key" -f ~/.ssh/rsync_backup_key
ssh-copy-id -i ~./ssh/rsync_backup_key.pub s1na@lil-dumpling

Add a ssh profile to our ~/.ssh/config file:

Host lil-dumpling
 HostName 10.77.88.1
 User s1na
 IdentityFile ~/.ssh/rsync_backup_key

Testing ssh-ing our remote server

The following script performs checks similarly to the previous 2 backup scripts. It checks that both locations exist, but also attempts to verify that there is no other borg processes currently executing. The script also checks the health of the output borg repos on the remote server after the job is complete.

Additionally, the rsync call also uses the -c flag to more securely and accurately detect changed files by calculating checksums to determine change. While slower, it ensures absolute data integrity which is a critical priority of this backup strategy.

#!/bin/bash

# Define variables
LOCAL_DIR="/mnt/bae"
REMOTE_DIR="/mnt/bak"
PARTIAL_DIR=".rsync-partial"
LOG_FILE="/var/log/rsync-to-lil-dumpling.log"
REMOTE_USER="s1na"
REMOTE_HOST="lil-dumpling"

# Function to send email via ntfy
send_email() {
 local subject=$1
 local message=$2
 curl \
 -H "Email: recipient.email.addr@gmail.com" \
 -H "Tags: cd" \
 -d "$subject $message" \
 http://192.168.1.76:10700/rsync-to-lil-dumpling
}


# Log function
log() {
 echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | sudo tee -a "$LOG_FILE" > /dev/null
}

# Start script
log "Backup process started."

# Check if backup directory exists
if [ ! -d "$LOCAL_DIR" ]; then
 log "Backup directory $LOCAL_DIR does not exist."
 send_email "Rsync Backup Failed" "Backup directory $LOCAL_DIR does not exist."
 exit 1
fi


# Check if backup directory exists
if ! ssh -o BatchMode=yes -o ConnectTimeout=10 "$REMOTE_USER@$REMOTE_HOST" "[ -d \"$REMOTE_DIR\" ]"; then
 log "Remote directory $REMOTE_DIR does not exist on $REMOTE_HOST."
 send_email "Rsync Backup Failed" "Remote directory $REMOTE_DIR does not exist on $REMOTE_HOST."
 exit 1
fi


# Check if partial directory exists
if ! ssh -o BatchMode=yes -o ConnectTimeout=10 "$REMOTE_USER@$REMOTE_HOST" "[ -d \"$REMOTE_DIR/$PARTIAL_DIR\" ]"; then
 log "Remote partial directory $REMOTE_DIR/$PARTIAL_DIR does not exist on $REMOTE_HOST."
 send_email "Rsync Backup Failed" "Remote partial directory $REMOTE_DIR/$PARTIAL_DIR does not exist on $REMOTE_HOST."
 exit 1
fi


# Check if any borg process is running
if ps aux | grep "[b]org" > /dev/null; then
 log "A Borg process is currently running. Aborting backup."
 send_email "Rsync Backup Failed" "Detected running borg process"
 exit 1
fi


# Rsync incremental backup
log "Starting rsync backup..."
if sudo rsync -avcz --partial --partial-dir=$PARTIAL_DIR --progress -e "ssh -o BatchMode=yes -o ConnectTimeout=10" $LOCAL_DIR $REMOTE_USER@$REMOTE_HOST:$REMOTE_DIR; then
 log "Rsync backup completed successfully."
else
 log "Rsync backup failed."
 send_email "Rsync backup to lil-dumpling Failed" "Rsync backup failed."
 exit 1
fi


# Verifying integrity of rsynced borg dirs
log "Checking Borg integrity on $REMOTE_HOST..."
# Get list of subdirectories on remote
borg_dirs=$(ssh "$REMOTE_USER@$REMOTE_HOST" "find '$REMOTE_DIR' -mindepth 1 -maxdepth 1 -type d")

# Loop over each Borg repo dir
for borg_dir in $borg_dirs; do
 log "Checking: $borg_dir"
 # Run borg check remotely
 ssh "$REMOTE_USER@$REMOTE_HOST" "borg check --verify-data '$borg_dir'"
 RC=$?

 if [ $RC -eq 0 ]; then
 log "Repo $borg_dir valid"
 elif [ $RC -eq 1 ]; then
 log "Repo $borg_dir valid with warnings"
 send_email "Borg health checks" "Borg repo $borg_dir valid but with warnings???"
 else
 log "Repo $borg_dir unhealthy"
 send_email "Borg health check failed" "Borg repo $borg_dir unhealthy"
 exit 1
 fi
done


log "Rsync Backup process completed successfully."
send_email "Rsync Backup Successful" "Rsync Backup process completed successfully. See log for details: $LOG_FILE"
exit 0

# runs every Tuesday and Saturday of the month at 6:00 AM MST.
0 6 * * 2,5 /mnt/f15/desktop/backup-scripts/rsync_to_lil_dumpling.sh

Testing of backups

Backups require periodic testing to ensure they actually work. Nothing is worse than finding out your backups are corrupted, malformed, improperly copied, or otherwise unuseable when push comes to shove and you actually suffer a tragic data loss incident.

Simulating such an incident in our implementation is even simpler thanks to the use of docker containers which make spinning up and down applications and cleaning up applications incredibly easy and elegant.

Testing the resiliency of our backup strategy is as simple as spinning down the self-hosted applications on our main prod application server, unmounting the primary harddisk, extracting the borg backup repos on the local secondary harddisk, and spinning up the containers using docker compose and then verifying the integrity and availability of application data. The following details these steps:

# spins down all running docker containers
docker stop $(docker ps -q)
# unmounts the primary prod harddisk
sudo umount /mnt/f15
# extracts a backup copy
mkdir /mnt/bae/docker-containers
borg extract --progress --verbose /mnt/bae/borg-backup-dir::docker-backup-2025-06-19 /mnt/bae/docker-containers
# spins up the container from the extracted backup. A bash script could be 
# written for this step to iteratively spin up all backed up apps
cd /mnt/bae/docker-containers/some-app
docker compose up -d
#clean up
cd /mnt/bae/docker-containers/some-app
docker compose down
rm -rf /mnt/bae/docker-containers

Alternatively, similarly testing the remote offsite backups is also possible by simply extracting the borg backups on the remote server locally and spinning them up and comparing with the prod application counterparts without actually having to spin down the prod applications themselves:

# On the remote offsite server
mkdir /mnt/remote/docker-containers
borg extract --progress --verbose /mnt/bak/borg-backup-dir::docker-backup-2025-06-19 /mnt/remote/docker-containers
# spin up the container from the extracted backup. A bash script could be 
# written for this step to iteratively spin up all backed up apps
cd /mnt/remote/docker-containers/some-app
docker compose up -d
#clean up
cd /mnt/remote/docker-containers/some-app
docker compose down
rm -rf /mnt/remote/docker-containers

Defcon 33 Afterthoughts

Thu, 04 Sep 2025 00:00:00 +0000

As August rolled in, Las Vegas once again became the global gathering place for hackers, researchers, and curious minds. Defcon 33 kicked off just as Black Hat wrapped up, bringing together thousands of attendees for a whirlwind of talks, contests, and parties.

This year marked my fourth Defcon — my first being Defcon 27 in 2019.

Humble Beginnings

I can’t quite remember how I first stumbled onto Defcon, but I do know it traces back to Jack Rhysider’s Darknet Diaries podcast sometime in 2019. At the time, I was a fresh graduate returning once again to central British Columbia for what would be my final season of tree planting — my third season of reforestation work.

A view of the planting ‘block’ with my bags and shovel in foreground

I distinctly remember listening to Darknet Diaries episodes during long drives out to the planting blocks. Tales of espionage and sabotage, physical pentesters infiltrating banks and corporate offices filled these long days — and Defcon kept being mentioned. I had taken CMPUT 333: Security in a Networked World during my undergrad so I had some very crude rudimentary knowledge of hashes, ciphers, and cryptography, but nothing close to preparing me for what I would see at my first Defcon.

That first Defcon experience was pretty unforgettable. I shared a room at the LINQ with two other attendees. These were the times before corporate sponsorship would cover and reimburse my travel expenses. The time where Walmart baloney sandwiches in ziplock bags made the bulk of my Defcon diet. Circumstances were a lot more no-frills for me during that first Defcon.

Nonetheless it was an eye-opening affair, one that I do credit being the significant event that marked the historical boundary signaling the start of the ‘cybersecurity era’ for me.

When three or more hack for satan badges are in close proximity, lights flicker in a seance of satanic goodness

In between wandering from village to village and being bombarded with things that looked and sounded like magic, I ended up one evening in some suite at Bally’s for the CONadian party allong with my two attendee roomates. In between drinks at their open bar, a key conversation that night yielded me the start of my personal ‘security roadmap’.

My original cyber roadmap

While I never did end up following this original roadmap I set myself all those years ago, I would say Defcon 27 definitely provided me the insight and a trajectory to penetrate the world of hacking. As a result, I’ve always felt a sense of gratitude and high reverance for what Defcon means to me, that lifechanging historical boundary that marked my entrypoint into the world of infosec, and so it only makes sense that I return for Defcon 33!

DEFCON 33

Defcon 33 was held again this year all “under one roof” at the Las Vegas Convention Centre from August 7 to August 10 2025. I lined up for merch about an hour before things opened and was incredibly pleasantly surprised I was able to essential be one of the first 100 people in line! What a surprise and subversion of expectations, especially seeing how I was in line for 4+ hours for merch the last time I was at Defcon.

Unsurprisingly, AI was centre-stage this year. The AIxCC contest continued to expand in size and real estate. This was a contest sponsored by DARPA going into its second year where contestants deploy AI systems that can detect, exploit, and patch vulnerabilities in systems mimicking industrial infrastructure. The activity at other villages were still ever-present, but AI and LLM tooling and discussions dominated the narrative.

OpenAI execs on the AIxCC stage

A key theme was the use of AI tools, particularly LLMs, and how they democratizing offensive security in ways that outpace their current defensive applications. Hackers demonstrated , and the AIxCC contest itself was a strong showcase of this particular trend. I don’t believe this particular emerging trend is a surprise to really anyone that actually uses LLMs in their day-to-day, even for minor troubleshooting or debugging/coding purposes. An obvious consequence of this democratization is that it lowers the barrier of entry for offensive security, both for aspiring whitehacks as well as for cybercriminals, and this trend is already readily showing up in !

Tricking an LLM into helping with offensive security activites

I also managed to snag a signed copy of ‘Red Teaming AI’ which ended up selling out on the first day of the convention! Definitely the loot highlight of the con for me.

Prized loot

Another highlight for me was the Deepfake demo held at the AI Village. This demo was running the opensource DeepFaceLab and via a personal consumer-grade laptop sporting a gtx 4090. For the cost of a mid range gaming laptop, anyone can create deep fakes. Deep fakes for all!!

Hardware running the deepfake demo

Look Mom, I’m Keanu Reeves!

Policy discussions were also dominated by AI, particularly its broad and disruptive impact across domains like application security, privacy, and national security. While the tools evolve rapidly, policymakers are still catching up; tale as old as time.

Other notable highlights were keynotes by the likes of Microsoft, Anthropic, and OpenAI. And stickers, lots of stickers.

sticker monster mode

Closing Thoughts

Defcon creates this aura and feeling of being enveloped and surrounded in the bleeding edge frontier of cybersecurity. The long-running legend now ingrained in Vegas and Defcon myth of having the ‘most hostile wifi network on earth’ persists and I’ve had a few baristas and ‘civies’, seeing my attendee badge and then remarking, ‘so, I guess I shouldn’t connect to any wifi networks for the rest of the week right?’.

Myths and traditions aside, I had a blast at hacker summer camp this year, and am forever grateful that this event persists year after year, ever evolving with the times, forming a place where the curious and the brightest meet and discuss the latest and greatest.

Advantage Blackjack: Analysis with Simulation and Reinforcement Learning Methods

Mon, 21 Jul 2025 00:00:00 +0000

A figure from your page’s folder or your assets/media/ media library

caption

A work in progress!

Code generated by Claude Opus

Projects

Sun, 19 May 2024 00:00:00 +0000

🎉 Easily create your own simple yet highly customizable blog

Fri, 27 Oct 2023 00:00:00 +0000

Welcome 👋

Table of Contents

Overview

The Hugo Blox website builder for Hugo, along with its starter templates, is designed for professional creators, educators, and teams/organizations - although it can be used to create any kind of site
The template can be modified and customised to suit your needs. It’s a good platform for anyone looking to take control of their data and online identity whilst having the convenience to start off with a no-code solution (write in Markdown and customize with YAML parameters) and having flexibility to later add even deeper personalization with HTML and CSS
You can work with all your favourite tools and apps with hundreds of plugins and integrations to speed up your workflows, interact with your readers, and much more

Get Started

👉
📚
💬 or
🐦 Twitter: #MadeWithHugoBlox
💡
⬆️ Updating Hugo Blox? View the and

Crowd-funded open-source software

To help us develop this template and software sustainably under the MIT license, we ask all individuals and businesses that use it to help support its ongoing maintenance and development via sponsorship.

As a token of appreciation for sponsoring, you can unlock awesome rewards and extra features 🦄✨

Ecosystem

: Automatically import publications from BibTeX

Inspiration

are building with this template.

Features

Page builder - Create anything with no-code and
Edit any type of content - Blog posts, publications, talks, slides, projects, and more!
Create content in , , or
Plugin System - Fully customizable
Display Code and Math - Code syntax highlighting and LaTeX math supported
Integrations - , , Maps, Contact Forms, and more!
Beautiful Site - Simple and refreshing one-page design
Industry-Leading SEO - Help get your website found on search engines and social media
Media Galleries - Display your images and videos with captions in a customizable gallery
Mobile Friendly - Look amazing on every screen with a mobile friendly version of your site
Multi-language - 35+ language packs including English, 中文, and Português
Multi-user - Each author gets their own profile page
Privacy Pack - Assists with GDPR
Stand Out - Bring your site to life with animation, parallax backgrounds, and scroll effects
One-Click Deployment - No servers. No databases. Only files.

Themes

Hugo Blox and its templates come with automatic day (light) and night (dark) mode built-in. Visitors can choose their preferred mode by clicking the sun/moon icon in the header.

for your site. Themes are fully customizable.

License

Released under the license.

🧠 Sharpen your thinking with a second brain

Thu, 26 Oct 2023 00:00:00 +0000

Create a personal knowledge base and share your knowledge with your peers.

Hugo Blox web framework empowers you with one of the most flexible note-taking capabilities out there.

Create a powerful knowledge base that works on top of a local folder of plain text Markdown files.

Use it as your second brain, either publicly sharing your knowledge with your peers via your website, or via a private GitHub repository and password-protected site just for yourself.

Mindmaps

Hugo Blox supports a Markdown extension for mindmaps.

With this open format, can even edit your mindmaps in other popular tools such as Obsidian.

Simply insert a Markdown code block labelled as markmap and optionally set the height of the mindmap as shown in the example below.

Mindmaps can be created by simply writing the items as a Markdown list within the markmap code block, indenting each item to create as many sub-levels as you need:


```markmap {height="200px"}
- Hugo Modules
- Hugo Blox
- blox-plugins-netlify
- blox-plugins-netlify-cms
- blox-plugins-reveal
```

renders as

- Hugo Modules
- Hugo Blox
- blox-plugins-netlify
- blox-plugins-netlify-cms
- blox-plugins-reveal

Anh here’s a more advanced mindmap with formatting, code blocks, and math:


```markmap
- Mindmaps
- Links
- [Hugo Blox Docs](https://docs.hugoblox.com/)
- [Discord Community](https://discord.gg/z8wNYzb)
- [GitHub](https://github.com/HugoBlox/hugo-blox-builder)
- Features
- Markdown formatting
- **inline** ~~text~~ *styles*
- multiline
text
- `inline code`
-
```js
console.log('hello');
console.log('code block');
```
- Math: $x = {-b \pm \sqrt{b^2-4ac} \over 2a}$
```

renders as

- Mindmaps
- Links
- [Hugo Blox Docs](https://docs.hugoblox.com/)
- [Discord Community](https://discord.gg/z8wNYzb)
- [GitHub](https://github.com/HugoBlox/hugo-blox-builder)
- Features
- Markdown formatting
- **inline** ~~text~~ *styles*
- multiline
text
- `inline code`
-
```js
console.log('hello');
console.log('code block');
```
- Math: $x = {-b \pm \sqrt{b^2-4ac} \over 2a}$

Highlighting

Highlight important text with mark:

<mark>Highlighted text</mark>

Callouts

Use (aka asides, hints, or alerts) to draw attention to notes, tips, and warnings.

By wrapping a paragraph in {{% callout note %}} ... {{% /callout %}}, it will render as an aside.

{{% callout note %}}
A Markdown aside is useful for displaying notices, hints, or definitions to your readers.
{{% /callout %}}

renders as

A Markdown aside is useful for displaying notices, hints, or definitions to your readers.

Or use the warning callout type so your readers don’t miss critical details:

A Markdown aside is useful for displaying notices, hints, or definitions to your readers.

Communicate your results effectively with the best data visualizations

Wed, 25 Oct 2023 00:00:00 +0000

Hugo Blox is designed to give technical content creators a seamless experience. You can focus on the content and Hugo Blox handles the rest.

Use popular tools such as Plotly, Mermaid, and data frames.

Charts

Hugo Blox supports the popular format for interactive data visualizations. With Plotly, you can design almost any kind of visualization you can imagine!

Save your Plotly JSON in your page folder, for example line-chart.json, and then add the {{< chart data="line-chart" >}} shortcode where you would like the chart to appear.

Demo:

You might also find the useful.

Diagrams

Hugo Blox supports the Mermaid Markdown extension for diagrams.

An example flowchart:

```mermaid
graph TD
A[Hard] -->|Text| B(Round)
B --> C{Decision}
C -->|One| D[Result 1]
C -->|Two| E[Result 2]
```

renders as

graph TD A[Hard] -->|Text| B(Round) B --> C{Decision} C -->|One| D[Result 1] C -->|Two| E[Result 2]

An example sequence diagram:

```mermaid
sequenceDiagram
Alice->>John: Hello John, how are you?
loop Healthcheck
John->>John: Fight against hypochondria
end
Note right of John: Rational thoughts!
John-->>Alice: Great!
John->>Bob: How about you?
Bob-->>John: Jolly good!
```

renders as

sequenceDiagram Alice->>John: Hello John, how are you? loop Healthcheck John->>John: Fight against hypochondria end Note right of John: Rational thoughts! John-->>Alice: Great! John->>Bob: How about you? Bob-->>John: Jolly good!

An example class diagram:

```mermaid
classDiagram
Class01 <|-- AveryLongClass : Cool
Class03 *-- Class04
Class05 o-- Class06
Class07 .. Class08
Class09 --> C2 : Where am i?
Class09 --* C3
Class09 --|> Class07
Class07 : equals()
Class07 : Object[] elementData
Class01 : size()
Class01 : int chimp
Class01 : int gorilla
Class08 <--> C2: Cool label
```

renders as

classDiagram Class01 <|-- AveryLongClass : Cool Class03 *-- Class04 Class05 o-- Class06 Class07 .. Class08 Class09 --> C2 : Where am i? Class09 --* C3 Class09 --|> Class07 Class07 : equals() Class07 : Object[] elementData Class01 : size() Class01 : int chimp Class01 : int gorilla Class08 <--> C2: Cool label

An example state diagram:

```mermaid
stateDiagram
[*] --> Still
Still --> [*]
Still --> Moving
Moving --> Still
Moving --> Crash
Crash --> [*]
```

renders as

stateDiagram [*] --> Still Still --> [*] Still --> Moving Moving --> Still Moving --> Crash Crash --> [*]

Data Frames

Save your spreadsheet as a CSV file in your page’s folder and then render it by adding the Table shortcode to your page:

{{< table path="results.csv" header="true" caption="Table 1: My results" >}}

renders as

Table 1: My results
customer_id	score
1	0
2	0.5
3	1

👩🏼‍🏫 Teach academic courses

Tue, 24 Oct 2023 00:00:00 +0000

is designed to give technical content creators a seamless experience. You can focus on the content and the Hugo Blox Builder which this template is built upon handles the rest.

Embed videos, podcasts, code, LaTeX math, and even test students!

On this page, you’ll find some examples of the types of technical content that can be rendered with Hugo Blox.

Video

Teach your course by sharing videos with your students. Choose from one of the following approaches:

Youtube:

{{< youtube D2vj0WcvH5c >}}

Bilibili:

{{< bilibili BV1WV4y1r7DF >}}

Video file

Videos may be added to a page by either placing them in your assets/media/ media library or in your , and then embedding them with the video shortcode:

{{< video src="my_video.mp4" controls="yes" >}}

Podcast

You can add a podcast or music to a page by placing the MP3 file in the page’s folder or the media library folder and then embedding the audio on your page with the audio shortcode:

{{< audio src="ambient-piano.mp3" >}}

Try it out:

Test students

Provide a simple yet fun self-assessment by revealing the solutions to challenges with the spoiler shortcode:

{{< spoiler text="👉 Click to view the solution" >}}
You found me!
{{< /spoiler >}}

renders as

👉 Click to view the solution

You found me 🎉

Math

Hugo Blox Builder supports a Markdown extension for $\LaTeX$ math. Enable math by setting the math: true option in your page’s front matter, or enable math for your entire site by toggling math in your config/_default/params.yaml file:

features:
 math:
 enable: true

To render inline or block math, wrap your LaTeX math with $...$ or $$...$$, respectively.

Example math block:

$$
\gamma_{n} = \frac{ \left | \left (\mathbf x_{n} - \mathbf x_{n-1} \right )^T \left [\nabla F (\mathbf x_{n}) - \nabla F (\mathbf x_{n-1}) \right ] \right |}{\left \|\nabla F(\mathbf{x}_{n}) - \nabla F(\mathbf{x}_{n-1}) \right \|^2}
$$

renders as

$$\gamma_{n} = \frac{ \left | \left (\mathbf x_{n} - \mathbf x_{n-1} \right )^T \left [\nabla F (\mathbf x_{n}) - \nabla F (\mathbf x_{n-1}) \right ] \right |}{\left \|\nabla F(\mathbf{x}_{n}) - \nabla F(\mathbf{x}_{n-1}) \right \|^2}$$

Example inline math $\nabla F(\mathbf{x}_{n})$ renders as $\nabla F(\mathbf{x}_{n})$.

Example multi-line math using the math linebreak (\\):

$$f(k;p_{0}^{*}) = \begin{cases}p_{0}^{*} & \text{if }k=1, \\
1-p_{0}^{*} & \text{if }k=0.\end{cases}$$

renders as

$$ f(k;p_{0}^{*}) = \begin{cases}p_{0}^{*} & \text{if }k=1, \\ 1-p_{0}^{*} & \text{if }k=0.\end{cases} $$

Code

Hugo Blox Builder utilises Hugo’s Markdown extension for highlighting code syntax. The code theme can be selected in the config/_default/params.yaml file.

```python
import pandas as pd
data = pd.read_csv("data.csv")
data.head()
```

renders as

import pandas as pd
data = pd.read_csv("data.csv")
data.head()

Inline Images

{{< icon name="python" >}} Python

renders as

Python

Experience

Tue, 24 Oct 2023 00:00:00 +0000

Experience

Tue, 24 Oct 2023 00:00:00 +0000

Learn JavaScript

Tue, 24 Oct 2023 00:00:00 +0000

is designed to give technical content creators a seamless experience. You can focus on the content and the Hugo Blox Builder which this template is built upon handles the rest.

Embed videos, podcasts, code, LaTeX math, and even test students!

On this page, you’ll find some examples of the types of technical content that can be rendered with Hugo Blox.

Video

Teach your course by sharing videos with your students. Choose from one of the following approaches:

Youtube:

{{< youtube w7Ft2ymGmfc >}}

Bilibili:

{{< bilibili id="BV1WV4y1r7DF" >}}

Video file

Videos may be added to a page by either placing them in your assets/media/ media library or in your , and then embedding them with the video shortcode:

{{< video src="my_video.mp4" controls="yes" >}}

Podcast

You can add a podcast or music to a page by placing the MP3 file in the page’s folder or the media library folder and then embedding the audio on your page with the audio shortcode:

{{< audio src="ambient-piano.mp3" >}}

Try it out:

Test students

Provide a simple yet fun self-assessment by revealing the solutions to challenges with the spoiler shortcode:

{{< spoiler text="👉 Click to view the solution" >}}
You found me!
{{< /spoiler >}}

renders as

👉 Click to view the solution

You found me 🎉

Math

Hugo Blox Builder supports a Markdown extension for $\LaTeX$ math. You can enable this feature by toggling the math option in your config/_default/params.yaml file.

To render inline or block math, wrap your LaTeX math with {{< math >}}$...${{< /math >}} or {{< math >}}$$...$${{< /math >}}, respectively.

We wrap the LaTeX math in the Hugo Blox math shortcode to prevent Hugo rendering our math as Markdown.

Example math block:

{{< math >}}
$$
\gamma_{n} = \frac{ \left | \left (\mathbf x_{n} - \mathbf x_{n-1} \right )^T \left [\nabla F (\mathbf x_{n}) - \nabla F (\mathbf x_{n-1}) \right ] \right |}{\left \|\nabla F(\mathbf{x}_{n}) - \nabla F(\mathbf{x}_{n-1}) \right \|^2}
$$
{{< /math >}}

renders as

Example inline math {{< math >}}$\nabla F(\mathbf{x}_{n})${{< /math >}} renders as $\nabla F(\mathbf{x}_{n})$ .

Example multi-line math using the math linebreak (\\):

{{< math >}}
$$f(k;p_{0}^{*}) = \begin{cases}p_{0}^{*} & \text{if }k=1, \\
1-p_{0}^{*} & \text{if }k=0.\end{cases}$$
{{< /math >}}

renders as

$$ f(k;p_{0}^{*}) = \begin{cases}p_{0}^{*} & \text{if }k=1, \\ 1-p_{0}^{*} & \text{if }k=0.\end{cases} $$

Code

Hugo Blox Builder utilises Hugo’s Markdown extension for highlighting code syntax. The code theme can be selected in the config/_default/params.yaml file.

```python
import pandas as pd
data = pd.read_csv("data.csv")
data.head()
```

renders as

import pandas as pd
data = pd.read_csv("data.csv")
data.head()

Inline Images

{{< icon name="python" >}} Python

renders as

Python

Learn Python

Tue, 24 Oct 2023 00:00:00 +0000

is designed to give technical content creators a seamless experience. You can focus on the content and the Hugo Blox Builder which this template is built upon handles the rest.

Embed videos, podcasts, code, LaTeX math, and even test students!

On this page, you’ll find some examples of the types of technical content that can be rendered with Hugo Blox.

Video

Teach your course by sharing videos with your students. Choose from one of the following approaches:

Youtube:

{{< youtube w7Ft2ymGmfc >}}

Bilibili:

{{< bilibili id="BV1WV4y1r7DF" >}}

Video file

Videos may be added to a page by either placing them in your assets/media/ media library or in your , and then embedding them with the video shortcode:

{{< video src="my_video.mp4" controls="yes" >}}

Podcast

You can add a podcast or music to a page by placing the MP3 file in the page’s folder or the media library folder and then embedding the audio on your page with the audio shortcode:

{{< audio src="ambient-piano.mp3" >}}

Try it out:

Test students

Provide a simple yet fun self-assessment by revealing the solutions to challenges with the spoiler shortcode:

{{< spoiler text="👉 Click to view the solution" >}}
You found me!
{{< /spoiler >}}

renders as

👉 Click to view the solution

You found me 🎉

Math

Hugo Blox Builder supports a Markdown extension for $\LaTeX$ math. You can enable this feature by toggling the math option in your config/_default/params.yaml file.

To render inline or block math, wrap your LaTeX math with {{< math >}}$...${{< /math >}} or {{< math >}}$$...$${{< /math >}}, respectively.

We wrap the LaTeX math in the Hugo Blox math shortcode to prevent Hugo rendering our math as Markdown.

Example math block:

{{< math >}}
$$
\gamma_{n} = \frac{ \left | \left (\mathbf x_{n} - \mathbf x_{n-1} \right )^T \left [\nabla F (\mathbf x_{n}) - \nabla F (\mathbf x_{n-1}) \right ] \right |}{\left \|\nabla F(\mathbf{x}_{n}) - \nabla F(\mathbf{x}_{n-1}) \right \|^2}
$$
{{< /math >}}

renders as

Example inline math {{< math >}}$\nabla F(\mathbf{x}_{n})${{< /math >}} renders as $\nabla F(\mathbf{x}_{n})$ .

Example multi-line math using the math linebreak (\\):

{{< math >}}
$$f(k;p_{0}^{*}) = \begin{cases}p_{0}^{*} & \text{if }k=1, \\
1-p_{0}^{*} & \text{if }k=0.\end{cases}$$
{{< /math >}}

renders as

$$ f(k;p_{0}^{*}) = \begin{cases}p_{0}^{*} & \text{if }k=1, \\ 1-p_{0}^{*} & \text{if }k=0.\end{cases} $$

Code

Hugo Blox Builder utilises Hugo’s Markdown extension for highlighting code syntax. The code theme can be selected in the config/_default/params.yaml file.

```python
import pandas as pd
data = pd.read_csv("data.csv")
data.head()
```

renders as

import pandas as pd
data = pd.read_csv("data.csv")
data.head()

Inline Images

{{< icon name="python" >}} Python

renders as

Python

✅ Manage your projects

Mon, 23 Oct 2023 00:00:00 +0000

Easily manage your projects - create ideation mind maps, Gantt charts, todo lists, and more!

Ideation

Hugo Blox supports a Markdown extension for mindmaps.

Simply insert a Markdown code block labelled as markmap and optionally set the height of the mindmap as shown in the example below.

Mindmaps can be created by simply writing the items as a Markdown list within the markmap code block, indenting each item to create as many sub-levels as you need:


```markmap {height="200px"}
- Hugo Modules
- Hugo Blox
- blox-plugins-netlify
- blox-plugins-netlify-cms
- blox-plugins-reveal
```

renders as

- Hugo Modules
- Hugo Blox
- blox-plugins-netlify
- blox-plugins-netlify-cms
- blox-plugins-reveal

Diagrams

Hugo Blox supports the Mermaid Markdown extension for diagrams.

An example Gantt diagram:

```mermaid
gantt
section Section
Completed :done, des1, 2014-01-06,2014-01-08
Active :active, des2, 2014-01-07, 3d
Parallel 1 : des3, after des1, 1d
Parallel 2 : des4, after des1, 1d
Parallel 3 : des5, after des3, 1d
Parallel 4 : des6, after des4, 1d
```

renders as

gantt section Section Completed :done, des1, 2014-01-06,2014-01-08 Active :active, des2, 2014-01-07, 3d Parallel 1 : des3, after des1, 1d Parallel 2 : des4, after des1, 1d Parallel 3 : des5, after des3, 1d Parallel 4 : des6, after des4, 1d

Todo lists

You can even write your todo lists in Markdown too:

- [x] Write math example
 - [x] Write diagram example
- [ ] Do something else

renders as

Write math example
- Write diagram example
Do something else

Spawning an Parrot Attack Machine on the Cloud

Tue, 17 Aug 2021 00:00:00 +0000

It is sometimes convenient to have a second attack machine somewhere else not connected to the same network. Your local machine at home may experience slow network connection to the internet, or we might also inadvertently blacklist our home ip address by sending too many requests to a target or account. In cases like these, it is handy to have a second attack machine that can get around these minor inconveniences. We also bump into some minor errors and other problems while setting this up that we will need to resolve and troubleshoot as we are setting up and configuring our machine.

In this post, I’m going to spin up a Debian droplet on Digital Ocean, one of the many cloud service providers offering PaaS packages. Digital Ocean allows us to essentially ‘rent’ a computer system from a selection of regions around the world. One of the key differences between Cloud Providers and their earlier ’legacy’ Hosting Providers, is that Cloud Providers are more flexible as it is possible to ‘rent’ out their machines on a per hour or a per use basis as opposed to the traditional Hosting Providers which usually only rent out their machines on a per month basis. This has great advantages to the economics of businesses and industry where they might only have certain ‘peak’ usage times and thus are able to only pay for the time that they are using the resource. I’ve elected to use Digital Ocean as my Cloud Provider, but any similar Cloud Providers such as AWS, Azure, or Alibaba Cloud would work as well.

We’ll be installing an instance of ParrotSec, a Debian-based Linux Operating System distribution geared towards security professionals and comes with a sleuth of tools tailored towards pentesting, forensics, and other security activities. In a way, it is the slightly lesser known cousin of the widely popular Kali Linux security distribution.

Signing into my Digital Ocean dashboard, we create a new droplet.

Create a new Digital Ocean droplet

We will then select the configuration that our spawned droplet will have.

We will spawn a Debian 10 machine with a general purpose dedicated CPU

Our machine will be located in Singapore

We will add our public ssh key for public key authentication

After creating out droplet, we are now able to ssh into the machine as root at the ip address provided by Digital ocean

ssh-ing into our newly spawned machine

We inspect the parrot documentation page for the repository locations. These are links and locations that affect what software packages are available for download and what versions, and contain details of who packages the software. In this case, we are interested in the software packaged by Parrot.

Finding the Parrot software repositories

We need to add these entries to our sources.list file so our machine knows which repositories to look at for software updates and installation.

The sources.list file on our Debian machine

Now that we have the needed Parrot entries in our sources.list file, we should perform an ‘apt update’ to download package information from our newly added Parrot sources and see which software we need to install.

However, when I ran the ‘apt update’, I got a GPG no public key found error

No public GPG key found

GPG is a public key crytographic algorithm that is used, in this case, as a digital signature to verify that the repository we are downloading packages from really are authored and belong to the Parrot team. (See digital signatures)

Doing some Google-fu, I found a solution here and here to add the required public keys and verify the authenticity of the Parrot sources.

Adding the missing GPG public key

Now that we have added Parrot’s public GPG key, we rerun our ‘apt update’, then search for the correct Parrot package we want to install.

apt update (Note that we have 274 packages that can be upgraded. Best practise is to update these packages before installing Parrot.)

Searching for Parrot

Scrolling down, we find the Parrot we are looking for.

The right parrot

We then enter ‘apt install parrot-meta-all -y’ to begin installation

Another stumbling block we find is during the installation, we are returning a number of 404 errors and a failed installation.

404 errors

If we ran a whois lookup on that offending IP address, we see that it is registered to the Singaporean branch of OVH , a French webhosting and cloud provider.

OVH Singapore

My guess of what is happening is that our Parrot entries we added to our sources.list file is being resolved to a server hosting the Parrot packages in Singapore that, as of this writting is down for whatever reason (Software packages are usually hosted on a number of servers around the world for redundancy as well as performance).

With this in mind, we go back to the Parrot documentation and look for the list of known mirrors (locations of other servers hosting the Parrot packages)

The other mirrors

Seeing that our machine is spawned in Singapore, let’s select a mirror that is fairly close to that region of the globe. We copy and paste the entry for Taiwan’s NCHC’s Free Software Lab into our sources.list file.

Adding the Taiwanese mirror to our sources.list file

With this done, let’s do another ‘apt update’ to download package data from our newly added sources.

Updating our package information from our newly added sources

Then let’s try reinstall the parrot-meta-all package again. This time, the installation completes with no failures. We then update the distribution, and remove any unneeded packages before restarting the machine.

Upgrading the distribution

System restart

Give the system a few minutes to reboot, and when we ssh into it again, we see that Parrot OS is installed and we now have access to all our favourite hacker tools!

Parrot is online

Creating an Active Directory Lab Environment

Wed, 11 Aug 2021 00:00:00 +0000

When I bought, set up, and upgraded my desktop PC, the goal was to have a pretty powerful machine that I can use to power a home lab that could handle running multiple virtual machine instances with ease. Since I’ve recently upgraded my RAM up to 24GB and had some free time, I thought I would get on that and write a post about it.

We will be setting up a simple Active Directory lab. The network would consist of probably 3-4 machines with one Domain Controller. We’ll set it up, and then we can use this AD network to practise AD hardening concepts, as well as simulate attacks, enumeration and other AD-related security tasks.

It is typically used in a corporate environment and allows centralized authentication and authorization of all users and computers and allows access to these shared resources and the setting of a common shared settings in an office.

A Windows Active Directory deployment consists of the following elements • Objects which represent a single entity (a printer, a computer, a user, a group (of users) ) and its attributes. An object has a unique name. • Domains are a collection of objects that are all stored on the same Active Directory database. Domains have their own policies (password policy, user accounts, machines, update schedule, etc…) • Trees are a collection of domains • A Forests is a collection of trees

Domains are managed by a special Windows server known as the Domain Controller. It is the centralized server that other resources rely on for authentication into the network and access to the directory service. Due to the centralized nature of Active Directory, most corporate networks have more than one Domain Controller to provide failover redundancy in case of failure of the primary Domain Controller. Larger networks also have more Domain Controllers to scale up performance. The Domain Controller then can be thought of as the central brain of an Active Directory network. It is the machine that handles authentication and authorizes machines/users onto the network and for resource access.

The machines: I downloaded a bunch of Windows Operating Systems from Microsoft’s ‘Windows Evaluation Centre’. We’ll set up an Active Directory environment using the following systems: • Windows Server 2019 • 2 Windows 10 Enterprise Systems • Windows 7 Professional

I’m using virtualbox as my hypervisor of choice to set up these virtual machines. This post will not go in-depth into the steps to install a Windows Virtual Machine (VM). There are many guides online such as this.

Our 4 VMs are installed and online

The Windows Server 2019 VM will be our sole domain controller on this AD network. We rename the PC to ‘CYDONIA-DC’, and install Active Directory Domain Services through the Server Manager on this server.

CYDONIA-DC; our soon-to-be Domain Controller. Click on ‘Add Roles and Features’ under the ‘Manage’ tab

Click next

Have ‘Role-based or feature-based installation’ selected and hit Next

Have ‘Active Directory Domain Services’ selected and click Next

Keep the rest of the options as defualts and confirm the installation to install Active Directory Domain Services.

Installation can take a few minutes. Once this is done installing, we need to promote this server to a Domain Controller. We’ll need to create a new forest and give our root domain a name.

Select ‘Promote this server to a domain controller’

We’ll add a new forest and set our domain name to ‘SANCTUARY.local’

We’ll also need to create a password

The NetBIOS domain name is ‘SANCTUARY’

We’ll keep most of the rest of the settings as default.

Once the server finishes installation and restarts, we’ll see that our login screen now shows that we are logging into the SANCTUARY domain as the domain user Administrator.

Logging in as ‘Administrator’ on the ‘SANCTUARY’ domain

The next step is to configure Users, Groups, and Policies objects. We’ll add a bunch of new users.

Select ‘Active Directory Users and Computers’ under the ‘Tools’ tab

Enter the user details and hit ‘Next’

Create a password for our user and hit ‘Next’ and then ‘Finish’

We should then see our newly created users

We’ll also set up a file share on our Domain Controller.

We will also set up a SQL service account. We have to set up something called a Service Principal Name (SPN) which is used by clients to identify a service instance, in this case, our SQL Service instance, and allow authentication to associate a service instance with a service logon account. This will be important for some of the attacks we will be conducting in the future on our Active Directory (AD) lab.

We’ll create a new User and name it SQL Service

Setting up SPN via the command line

Confirming SPN was set via the command line

Our Domain Controller is now up and running. We will return in the future to modify and edit this server but for now, we can move on to setting up the host machines.

We will boot up our host machines, rename them, and create a new share in our C: drive

We then need to change our Network Adapter settings to perform DNS through our Domain Controller.

We can note from running ‘ipconfig’ on a command line on our Domain Controller that its IP address is 192.168.33.111. This is the IP address we need to set as the DNS server on each of our host machines.

Getting the IP address of our Domain Controller

In the ‘Network Connections’ section of the control panel, right click on our Ethernet connection and select ‘Properties’

Add the IP address of the Domain Controller as the Preferred DNS server of our host machines.

Finally, we will connect our host machine to the domain.

Select ‘Join this device to a local Active Directory domain’

Type in our domain name we want to connect to

Authenticate into the domain as one of our domain users

Our host machine will then prompt us to restart, and when we do, we can note that we are now prompted to log into our domain

Upon reboot, we can now log in as e.tonra on the SANCTUARY domain

We can then perform the same steps to set up the other hosts and connect them to the domain.

Our fully operational AD lab with 4 VMs

I booted up another 5th VM running an instance of Parrot Security and conducted a quick nmap scan on the network

# Nmap 7.91 scan initiated Tue Aug 10 23:54:34 2021 as: nmap -n -sT -sC -oA nmap-generic 192.168.33.0/24
Nmap scan report for 192.168.33.1
Host is up (0.0067s latency).
Not shown: 996 closed ports
PORT STATE SERVICE
22/tcp open ssh
| ssh-hostkey:
| 1024 52:9d:3b:c8:e6:69:02:00:db:52:8f:3d:b2:4b:af:94 (DSA)
|_ 1039 51:59:7f:80:47:b3:18:f0:73:b8:e5:18:01:3c:ea:3d (RSA)
23/tcp open telnet
80/tcp open http
|_http-title: Site doesn't have a title (text/html; charset=utf-8).
1900/tcp open upnp
MAC Address: 7C:8B:CA:C3:D0:94 (Tp-link Technologies)

Nmap scan report for 192.168.33.111
Host is up (0.0022s latency).
Not shown: 989 filtered ports
PORT STATE SERVICE
53/tcp open domain
88/tcp open kerberos-sec
135/tcp open msrpc
139/tcp open netbios-ssn
389/tcp open ldap
445/tcp open microsoft-ds
464/tcp open kpasswd5
593/tcp open http-rpc-epmap
636/tcp open ldapssl
3268/tcp open globalcatLDAP
3269/tcp open globalcatLDAPssl
MAC Address: 08:00:27:A6:C1:72 (Oracle VirtualBox virtual NIC)

Host script results:
|_nbstat: NetBIOS name: CYDONIA-DC, NetBIOS user: <unknown>, NetBIOS MAC: 08:00:27:a6:c1:72 (Oracle VirtualBox virtual NIC)
| smb2-security-mode:
| 2.02:
|_ Message signing enabled and required
| smb2-time:
| date: 2021-08-11T03:55:23
|_ start_date: N/A

Nmap scan report for 192.168.33.112
Host is up (0.0014s latency).
Not shown: 999 filtered ports
PORT STATE SERVICE
135/tcp open msrpc
MAC Address: 08:00:27:B5:A5:BA (Oracle VirtualBox virtual NIC)

Nmap scan report for 192.168.33.113
Host is up (0.0015s latency).
Not shown: 999 filtered ports
PORT STATE SERVICE
135/tcp open msrpc
MAC Address: 08:00:27:9F:F0:4B (Oracle VirtualBox virtual NIC)

Nmap scan report for 192.168.33.115
Host is up (0.0016s latency).
Not shown: 999 filtered ports
PORT STATE SERVICE
135/tcp open msrpc
MAC Address: 08:00:27:D0:AC:7D (Oracle VirtualBox virtual NIC)

Nmap scan report for 192.168.33.137
Host is up (0.051s latency).
Not shown: 999 closed ports
PORT STATE SERVICE
902/tcp open iss-realsecure
MAC Address: 44:33:4C:49:F7:BF (Shenzhen Bilian electronic)

Nmap scan report for 192.168.33.118
Host is up (0.048s latency).
All 1000 scanned ports on 192.168.33.118 are closed

# Nmap done at Tue Aug 10 23:56:03 2021 -- 256 IP addresses (7 hosts up) scanned in 89.78 seconds

We can see from this initial scan that the Domain Controller has a number of services running on it. We can thus determine that the network layout of our lab looks like the following.

Network Diagram of the network

With this AD homelab set up, we now have an entry point to experiment with Windows Active Directory and test out some Active Directory-specific attacks and exploits as well as practise hardening the network and other blue team activities!

Bulk Scanning Stuff with VirusTotal

Sat, 17 Jul 2021 00:00:00 +0000

I have a number of files that I have accumulated of unknown and ambiguous origin. I’d like to scan these files for any potential malicious contents.

Virustotal is a website created by Spanish security company Hispasec Sistemas which was acquired by Google. Virustotal aggregates many antivirus products and online scan engines to check for viruses. It allows users to upload files up to 650MB and send suspected malicious URLs for scanning.

virustotal webapp

Looking at all the files I am interested in scanning, we have a total of 287 files that require scanning. It would be beyond tedious to submit each file manually one by one, wait for virustotal to scan it, give back results and then uploading the next file for scanning.

Using some bash magic to find all the files!

Luckily, virustotal also has an api and command line interface tool that we can use to automate this.

The VirusTotal command line tool repo

Inspecting the README file on the repository

Install instructions

First we need to download the repository

Cloning the repo

then make

Building the tool

The documentation mentioned that we need to sign up to get a free apikey in order to use the command line tool. So let’s sign up and grab the api key. After signing up, I initialized the command line tool with my api key.

Initializing vt with our api key

VirusTotal processes files in 2 steps. There is an upload step where files are uploaded to VirusTotal and a unique ID is given back to the user. The user can then use this unique ID to query results of the VirusTotal scan. The command line tool has commands for each of these steps.

We will need a list of all file names and their location on my computer and send this to a file so we have a list of all our files to be scanned.

find '/home/s1na/Documents/some/location' -type f > /home/s1na/Documents/war-room/file-list.txt

Inspecting this new file, we get a list of all our files we want to scan. We can then read off this file line by line and send a post request to the virustotal api endpoint to get our id.

a list of our 287 files to be scanned

The VirusTotal command line tool has a scan file function that will return us an id that we can then use with the vt analysis command. We can even send in our list of files to it!

vt scan file command

Running this and tee-ing the results into a new file. tee outputs results to the screen and also into a seperate file

cat ~/Documents/war-room/files-list.txt | go run main.go scan file - | tee ~/Documents/war-room/engineering-personal/list-file-id

Inspecting the new file, we see that our results have given us our list of IDs!

We can then do some bash processing to take only the IDs and compile them into a separate list to run analysis on them. The structure of the file is a space separated entries of file names and the virustotal ID.

getting only the IDs

Let’s place these into a new file and running a quick checksum to make sure we have all lines of our code.

collecting the IDs into a new file

let’s pull up the analysis command on the tool

vt analysis command

we see that we can pass this command a list just as we did for the file scan command. We’ll throw the results into a file aptly named results

begin analyzing

Once this completes, we’ll need a way to sift through the results in a meaningful way to look for any files that may have been flagged as potentially malicious.

Upon closer inspection of the results, it is not returned in JSON format. Probing around the tool, I wasn’t able to find a way to make it return the results as a JSON object. So we’ll have to to do some quick hacking in order to process these results. Note also that this screenshot shows the results for one file entry and we have 287 such entries. The first entry is 532 lines long. I’ve also collapsed the results field for brevity.

a look at the results

The results field just shows the details of the antivirus scanner used

the results field

The total length of the results file is about ~150 000 lines long! Imagine manually scrolling through the file line by line looking for any malicious or suspicious flags!

Looking at these results, we are interested in the id field, and the entries within the ‘stats’ field. Particularly, we are interested in any malicious or suspicious entries found.

We’ll use python to perform our analysis on the results. We will use python to read every line from our results file, and when it finds an id field, it will add that id and the 8 ‘stats’ we are interested in as an entry into a Python Dictionary.

#!/usr/bin/python3

f = open('results', 'r') #open the file names 'results' as read-only
results_dict = dict() #where we will store our final data

curr_file_id = '' #what current file we are processing

for line in f: #go through each line in our results file
 splitted_line = line.split() #split the line by any whitespace
 if (splitted_line[0] == '-'): #if first element is a -, then this line will give us an id 
 curr_file_id = splitted_line[2] #the 3rd element is the id
 results_dict[curr_file_id] = dict() #make an entry in our results_dict

 #if the first element is any of the 8 interesting fields, record the stats
 if (splitted_line[0] == 'confirmed-timeout:'):
 results_dict[curr_file_id]['confirmed-timeout'] = splitted_line[1]
 if (splitted_line[0] == 'failure:'):
 results_dict[curr_file_id]['failure'] = splitted_line[1]
 if (splitted_line[0] == 'harmless:'):
 results_dict[curr_file_id]['harmless'] = splitted_line[1]
 if (splitted_line[0] == 'malicious:'):
 results_dict[curr_file_id]['malicious'] = splitted_line[1]
 if (splitted_line[0] == 'suspicious:'):
 results_dict[curr_file_id]['suspicious'] = splitted_line[1]
 if (splitted_line[0] == 'timeout:'):
 results_dict[curr_file_id]['timeout'] = splitted_line[1]
 if (splitted_line[0] == 'type-unsupported:'):
 results_dict[curr_file_id]['type-unsupported'] = splitted_line[1]
 if (splitted_line[0] == 'undetected:'):
 results_dict[curr_file_id]['undetected'] = splitted_line[1]

this will load only the data we are interested in into a python dictionary called results_dict and allow us to manipulate and ‘query’ this dictionary for any files that may have been flagged as malicious or suspicious.

we will go through each entry of our results_dict one by one, and check for any malicious or suspicious entries, if we find any, we will add these to an alert_list. Finally, we’ll print out the alert list

alert_list = list() #a list that will contain any malicious messages

for entry in results_dict: #go through each entry in our processed data
 alerts = ''
 #if we have any malicious or suspicious flags, add an error message to alerts
 if int(results_dict[entry]['suspicious']) > 0:
 alerts = alerts + entry + ' has ' + results_dict[entry]['suspicious'] + ' suspicious flags! '
 if int(results_dict[entry]['malicious']) > 0:
 alerts = alerts + entry + ' has ' + results_dict[entry]['malicious'] + ' malicious flags! '
 #if alerts has any errors in them, add them to our alert_list
 if alerts:
 alert_list.append(alerts)


#print every entry present in alert_list
for i in alert_list:
 print(i)

Running the program and boom, we find 2 potentially malicious files. The internet is a scary place.

2 files detected as malicious

This is not a full-proof solution as it could still be very possible that some malicious files have not been submitted or detected by the scanners used by virustotal. Nevertheless we have somewhat increased our confidence, at least a little bit, that besides these 2 files, there is a lower probability that the other files are malicious in nature.

Automating HTTP form submissions using Python and cronjobs

Wed, 24 Feb 2021 00:00:00 +0000

Nobody likes doing tedious and repetitive tasks. I received an email notifying me that I had to submit a daily ‘health declaration check’ as a form on a website. Since I lead a boring and routine lifestyle, I thought I would automate this task. It would be a good excuse to refresh my knowledge on Python’s Requests module as well as get more hands on experience with cron jobs.

The health declaration webform

Inspecting the web form using Firefox developer tools

Using the built-in Firefox dev tools, we see that the web form consist of a pretty standard HTML form tag sent via a HTTP POST request.

Web traffic travels in requests and responses. HTTP Requests are sent out by hosts (your browser or computer) for specific web resources, which are then answered by HTTP Responses that provide the web resource requested. Requests come in multiple flavours but the most common requests are GET requests and POST requests. GET requests are HTTP requests usually sent to ‘get’ or retrieve information from the web while POST requests are are used to send or ‘post’ data to the internet (this could be adding a blog post, making a social media comment, or in our case, submitting a web form)

Inspecting the POST request with Burp Suite

POST Requests are separated into 2 sections called the Head, which contain Headers, and the Body. For this particular request, the Header takes up the first 15 lines of the request and the body of our request begins on line 17.

We can see where each of the fields in the form correspond to the fields in line 17 of the request. We can thus write some code to modify the contents submittied in line 17 depending on day of the week and where I will be on that day.

Now that we have some idea of what a valid HTTP request and corresponding response looks like, we can use Python and its request module to craft and send our own custom HTTP POST request to help submit this form for us automatically.

#!/usr/bin/python3

import requests
import datetime


URL = "https://some-endpoint/somewhere/admin-ajax.php"

headers = {
 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0',
 'accept': 'application/json, text/javascript, */*; q=0.01',
 'accept-language': 'en-US,en;q=0.5',
 'accept-encoding': 'gzip, deflate',
 'content-type': 'application/x-www-form-urlencoded; charset=UTF-8',
 'x-requested-with': 'XMLHttpRequest',
 'content-length':'661',
 'origin': 'https://some.origin.com',
 'DNT': '1',
 'Connection': 'close',
 'cookie': 'TawkConnectionTime=0',
 'sec-gpc': '1'
}

usual_places = {
 "home": "Home",
 0: "Era",
 1: "Past",
 2: "Cashflow",
 3: "Dreams-of-flight",
 4: "Stallion-Cooper-Coffee",
 5: "Ministop"
}

url_encoded_comma = '%2C'


day = datetime.datetime.today().day
if (day < 10):
 day = '0'+str(day)
else:
 day = str(day)

month = datetime.datetime.today().month
if (month < 10):
 month = '0'+str(month)
else:
 month = str(month)

home = usual_places['home']
places = usual_places[4]+url_encoded_comma+'+'+usual_places[5]


data = "action=formcraft3_form_submit&field1%5B%5D=T%E1%BB%9C+KHAI+Y+T%E1%BA%BE+H%C3%80NG+NG%C3%80Y%2F+DAILY+HEALTH+DECLARATION+FORM+2021&field19=&field3%5B%5D=Avery+Tan&field20=+&field14="+day+"%2F"+month+"%2F2021&field6%5B%5D=B%C3%ACnh+th%C6%B0%E1%BB%9Dng%2FNormal&field25%5B%5D=&field9%5B%5D="+places+"&field21="+home+"&field30%5B%5D=Kh%C3%A1c%2FOthers&field32%5B%5D=&field33%5B%5D=&field34%5B%5D=&field35%5B%5D=&field36%5B%5D=I+use+Grab&field18%5B%5D=I+agree&field28=&website=&id=51&location=https%3A%2F%2F%2F%2F&emails=&hidden=field25, field32, field33, field34, field35&redirect=&type=all&triggerIntegration=undefined"


r = requests.post(url = URL, headers = headers, data = data)


# for logging
time_now = str(datetime.datetime.now())

print(time_now + '. Request sent. Response is ' + r.text+ '\n\n')
f = open('/home/kali/Documents/health-automated/health-logs','a+')
f.write(time_now + '. Request sent. Response is ' + r.text+ '\n\n')
f.close()

I map the headers to a Python dictionary and also use another dictionary to record my places of interests. I perform some logic to ensure that the date is submitted in a format that the form expects it to be. Finally, I add these variables into my data string that will form the body of our POST request before finally sending out the request.

Now that we have our simple Python script, we now have to schedule a cronjob to automatically run the script every day.

Cron jobs are a unix time-based job scheduler where Users can set up and schedule jobs to run periodically at fixed times, dates, or intervals.

In order for a scheduled cronjob to be run at the set time, the computer needs to be running and connected to the internet during the scheduled job for it to be executed. Instead of running the cronjob from my laptop and having to ensure it is on and connected at the scheduled time of execution, I am going to use the deth* (pronounced deathstar), my raspberry pi 4 server which I have hooked into the back of a tv in my room. It’s a kali machine which I use for lightweight, in-depth, thorough scans where leaving my laptop running for 8hours is no option.

I use Dataplicity, which offers a VPN-like service to connect remotely into your Raspberry Pi without having to set up a portfoward or deal with any dynamic DNS services. Using Dataplicity, I can remotely access deth* wherever I have internet.

deth*, my raspberry pi 4 server

Cronjobs can be scheduled on a per-user basis via something called crontab. To access the user-specific crontab, we do so via:

crontab -e

crontab

The syntax of a cronjob command is 5 optional parameters followed by the command you want to run at the specified time. I’m running my cronjob at the start of the 50th minute of the 2nd hour of the day, or at 02 50 UTC time, which is about 9.50am local time.

I also redirect output from the python file to logs so that I can check periodically to make sure the cronjob is running as expected.

With this done, I can rest easy knowing that my forms are being submitted in a automated non-intervention-requiring manner!

Creating a swapfile for Linux machines running a btrfs filesystem

Thu, 11 Feb 2021 00:00:00 +0000

I have an i5 Huawei Matebook with 4GB RAM. This isn’t the worst specs for most things I need to do day to day (save for something like maybe intense password cracking). However, I also run virtualbox on my computer to host an instance of Parrot OS that I use for other purposes and I have noticed performance issues which at times would force me to do a hard shutdown of the whole computer.

Swap is a space on a disk that is used when a computer’s physical RAM is full. It’s a way to use harddisk space in order to simulate extra RAM on a computer. When a computer’s RAM is full, idle programs are copied over to the swapfile to free up RAM memory.

When I initially set up my native Parrot OS on my computer, I was given the option to create separate partitions for a swap disk. The alternative setting was to settle for a swap file (I have since learnt that using a swap file is more desirable compared to a swap disk). Being the Linux noob that I was (and still probably am), I did not want to mess around too much with partitions on the hard disk.

Only recently have I found out, by running the command swapon –show that a swapfile was not automatically created during the initial installation of Linux. A further obstacle presented itself when I was attempting to provision a swapfile in that the harddisk on my computer was using the . At the point of enabling the swapfile, I was getting constant Invalid Argument errors.

A bit of further research and I found out the required extra steps to configuring a swapfile. The steps needed to be taken to create a swapfile on a btrfs harddisk is:

touch /swapfile

We’ll create the swapfile at the root directory.

chattr +C /swapfile

This is the magic command that fixed the Invalid argument error I was getting when trying to enable my swapfile previously. This command is asking for COW (a btrfs-specific resource management technique) to be disabled.

sudo fallocate -l 4G /swapfile

In my case, I’m creating a swapfile of size 4GB

chmod 0600 /swapfile

Change the file permissions to be read and write only by root.

mkswap /swapfile

set up this directory as a swap area

swapon /swapfile

enable the swapfile

swapon --show

check that your swapfile is enabled and how much of it is being used at the moment

Do not forget to set a swappiness value. Swappiness can be set between values of 0 to 100 and determine how often or how aggressively the computer system will make use of the swapfile. A lower value means the kernel avoids swapping as much as possible while a high value will allow the kernel to more liberally make use of the swapfile. By default swappiness is set to 60. There are circles of engineers discouraging the use of frequent read and write operations into SSD harddrives due to degradation concerns. While I disagree in terms of the degree of impact these read and write operations during swapping will actually cause, and have a high level of confidence in our modern harddrive technologies, I set my swappiness to a level of 10. (No need to use the space if you actually don’t need it right)

To do this, add

vm.swappiness = 10

to /etc/sysctl.conf.

This can also be temporarily set by using

sudo sh -c 'echo 10 > /proc/sys/vm/swappiness'

to temporarily change the swappiness value. The changes will not persist after reboot though.

After setting up a swapfile, I have noticed increased performance and reliability. I have not had to perform a hard shutdown of my PC since. It’s never run as smoothly and has rid me of the frustration of having to reboot the VM, losing all work and worse yet when I have to reboot not only the VM but the entire system.

Using a swapfile, I turned my machine with 4GB of RAM into a machine with 8GB of RAM, and I didn’t have to spend a single dollar on RAM sticks.

An example preprint / working paper

Sun, 07 Apr 2019 00:00:00 +0000

This work is driven by the results in my on LLMs.

Create your slides in Markdown - click the Slides button to check out the example.

Add the publication’s full text or supplementary notes here. You can use rich formatting such as including .

An example journal article

Tue, 01 Sep 2015 00:00:00 +0000

Click the Cite button above to demo the feature to enable visitors to import publication metadata into their reference management software.

Create your slides in Markdown - click the Slides button to check out the example.

Add the publication’s full text or supplementary notes here. You can use rich formatting such as including .

An example conference paper

Mon, 01 Jul 2013 00:00:00 +0000

Click the Cite button above to demo the feature to enable visitors to import publication metadata into their reference management software.

Create your slides in Markdown - click the Slides button to check out the example.

Add the publication’s full text or supplementary notes here. You can use rich formatting such as including .

Blog

Mon, 01 Jan 0001 00:00:00 +0000

Avery Tan, Security Professional

Example Talk

Max Redundancy with 3-2-1!

Introduction

The 3-2-1 Strategy

Our Implementation

nfty

Borg & Rsync

The Backup Scripts

The *arr stack media server

The self-hosted applications

The resultant local backups

Sending a backup copy offsite

Testing of backups

Defcon 33 Afterthoughts

Humble Beginnings

DEFCON 33

Closing Thoughts

Advantage Blackjack: Analysis with Simulation and Reinforcement Learning Methods

A figure from your page’s folder or your assets/media/ media library

Projects

🎉 Easily create your own simple yet highly customizable blog

Overview

Get Started

Crowd-funded open-source software

Ecosystem

Inspiration

Features

Themes

License

🧠 Sharpen your thinking with a second brain

Mindmaps

Highlighting

Callouts

Did you find this page helpful? Consider sharing it 🙌

Communicate your results effectively with the best data visualizations

Charts

Diagrams

Data Frames

Did you find this page helpful? Consider sharing it 🙌

👩🏼‍🏫 Teach academic courses

Video

Podcast

Test students

Math

Code

Inline Images

Did you find this page helpful? Consider sharing it 🙌

Experience

Experience

Learn JavaScript

Video

Podcast

Test students

Math

Code

Inline Images

Did you find this page helpful? Consider sharing it 🙌

Learn Python

Video

Podcast

Test students

Math

Code

Inline Images

Did you find this page helpful? Consider sharing it 🙌

✅ Manage your projects

Ideation

Diagrams

Todo lists

Did you find this page helpful? Consider sharing it 🙌

Spawning an Parrot Attack Machine on the Cloud

Creating an Active Directory Lab Environment

Bulk Scanning Stuff with VirusTotal

Automating HTTP form submissions using Python and cronjobs

Creating a swapfile for Linux machines running a btrfs filesystem

An example preprint / working paper

An example journal article

An example conference paper

Blog