2021-02-10 08:17:57 +00:00

3.8 MiB

Raw Blame History

theme	_class	backgroundColor	backgroundImage	marp
gaia	lead		url('https://0.s3.envato.com/files/138380962/Preview%20Image%20White%20Low%20Poly%20Background2.jpg')	true

Personal Infrastructure

Marcus Noble

🔗 https://marcusnoble.co.uk 🐦 @Marcus_Noble_ ✨ AverageMarcus

A little about me and the past

I work professionally as a "DevOps" engineer at Elsevier with a focus on Kubernetes and containers
Previously a JavaScript developer, now a Go developer
I've been running my own servers for as long back as I can recall
I like to tinker with "Smart Home" stuff

1. Personal network setup

2. Smart home

3. Replacing Google Analytics

Personal network setup

Where things are at today...

87 uniquely identifiable devices connected to my home network
A Zigbee network with 32 devices
A single-node Kubernetes cluster running on a dedicated desktop machine hosting 14 applications
An autoscaling 1-3 node Kubernetes cluster hosted on Scaleway running ~30 applications
1 Raspberry Pi running Home Assistant to control all smart devices in my home
1 Raspberry Pi running OctoPi to control my 3D printer

Previously

Raspberry Pi 4 running k3s Kubernetes cluster
- Only ARM arch support - required me to manually rebuild some images
- I had stability and performance issues
Dokku running on a VPS server hosted on bHost (now Mythic Beasts)

Local Network setup

FTTP - Zen Internet ↯ ISP Modem ↯ Powerline ↯ Router - TP-Link Archer C7 ↯ Mesh WiFi beacons - TP-Link Deco M4 (x3) ↯ 87 devices

Local Network setup

5 devices connected via ethernet

Everything else via WiFi

All devices have static IPs and assigned hostnames (based on the periodic table of elements)

Kubernetes - local

An old desktop machine repurposed as a single-node Kubernetes cluster

OS: Ubuntu Kubernetes Version: Microk8s v1.20.1 Specs: CPU - 4 core | Memory - 12Gb | HDD - Lots

Kubernetes - local

Primary Use: Hosting sensitive or local-specific applications that I want to keep off the internet. E.g.

custom built CCTV monitoring application covering the 8 cameras I currently have set up
automated download tools so I can kick off downloads remotely while I'm out
DNS-based ad blocker (AdGuard)

Kubernetes - cloud

Hosted on Scaleway's managed Kubernetes platform, Kapsule. 1-3 DEV1_M instances, autoscaled based on load.

All defined in code using Terraform and stored in git.

Kubernetes Version: v1.20 Specs: CPU - 3 vCPUs | Memory - 4Gb | HDD - 40Gb + 14 PVCs of varying size

Kubernetes - cloud

Primary Use: Hosting internet facing services and providing a secure, authenticated tunnel to my local cluster E.g.

My personal blog plus some related sites
Self-hosted alternatives to cloud services: Gitea, Nextcloud, Harbor, Photoprism, etc.
Tekton - handle all my CI/CD needs, triggered by webhooks from Gitea
"Utility" applications to make my life easier:
- Convert SVG to DXF - used when creating 3D print designs from SVG
- website-to-remarkable - lets me send a webpage as a PDF to my reMarkable tablet for later reading

Time for a quick break

☕️

Questions so far?

Smart Home

Home Assistant centrally controlling all devices
Combination of Zigbee and WiFi devices
Amazon Echo in every room
Most rooms have smart lighting: ceiling bulbs, floor lamps, LED strips
Motion and light sensors used to trigger turning lights on
Outdoor lighting with LED strips + motion/light sensors
Cheap Android tablets acting as dashboards

Typical Day

🌙 Wake up at 5am 🥱
Take my phone off charge 📱 and my office lights automatically turn on along with my monitor 💡🖥
🚶🏽‍♂️ Walk downstairs and as I pass through the landing and the hallway the lights automatically turn on at 20% brightness 💡
Enter the kitchen to get a drink and again the light turns on at 20% brightness 🥛💡
Go through to the living room and tap on the tablet by the door to turn on the sofa lamp 🛋
Open the back door to let the dog out 🐶 the LED strip along the garden fence lights up as the door opens

Typical Day

Time to get some work done 👨🏻‍💻
Check my calendar for meetings - morning free so decide to set my 3D printer going
Tap on the tablet mounted in front of my monitor to turn on my 3D printer and start it pre-heating 🔥
Access my OctoPrint webpage and select the model to print 🖨
Some time later access my 📹 CCTV webpage to check what the dog is up to downstairs 🐕
Head down to the kitchen for a snack 🍪 - this time the lights come on at 100% brightness as does the lamp in the kitchen 💡

Typical Day

Lunchtime - head down for food 🧆 this time the lights don't come on as there is already enough natural light
While eating my 3D printer finishes printing and shortly after turns itself off 🔌
Back to work... 👨🏻‍💻

Typical Day

Evening - take the dog for a walk 🦮 outdoor lights come on as the front door is opened, turning off automatically a few minutes later
When returning back the lights again turn on automatically as I approach to illuminate the keyhole
Dinner time 🍔 put food in the over then ask Echo to set a timer
Timer sounds - take food out and plate up. 🍛 Tap a button on the tablet that triggers an announcement on all the Echos in the house letting the family know food is ready 📣
Daughter's bedtime 🛏 lights on her desk flash briefly letting her know it's bedtime and the lights under her bed turn on to a dim glow

Typical Day

Bedtime 🛏
While getting ready for bed I check my phone to make sure all lights are turned off through the house and if not toggle them off 🎚
When in bed - "Echo, turn the lights off" 🌑
😴

Security

Where possible devices have been flashed with custom firmware that makes them run in a local-only mode. This is done on all cameras, all compatible WiFi plugs and most of the LED strips.

Unnecessary external network traffic is blocked by AdGuard.

Firewall on router blocks all incoming ports.

Amazon Echo's ... 🤷🏼‍♂️

Secure external access is provides via a tunnel between my local cluster and my remote cluster. All endpoints authenticated and monitored. Local cluster connects to internet via a VPN.

Misc

Automation handles brightness of lights based on time of day and amount of light already in the room
Automation controls the volume of the Echos throughout the house based on time of day
Alerts sent to phones when outside doors, garden gate or garage door are left open for too long
Most lights set to automatically turn off after a period of inactivity
4 Tablets mounted throughout the house as dashboards

Time for a quick break

☕️

Questions so far?

Replacing Google Analytics

Why?

Never used 99% of the functionality that Google Analytics provides
Slowly de-google-ing myself so I should do the same for my website visitors
Client-side analytics are flakey at best
All I really wanted to know was:
- What pages are popular when?
- How are bots trying to do naughty things?
- Anything throwing error codes?

The solution!

Traefik + Loki + Grafana

Traefik - Handles all incoming HTTP requests and logs out to an access log in JSON format.
Loki - Collects all logs from the application in my cluster and stores them ready for querying.
Grafana - Queries the Loki data and displays it as a dashboard

Example Access Log

{
    "BackendAddr": "100.64.128.16:8000", "BackendName": "http-https_marcusnoble.co.uk/",
    "BackendURL": { "Scheme": "http", "Opaque": "", "User": null, "Host": "100.64.128.16:8000", "Path": "", "RawPath": "", "ForceQuery": false, "RawQuery": "", "Fragment": "" },
    "ClientAddr": "100.64.0.1:10948", "ClientHost": "100.64.0.1", "ClientPort": "10948", "ClientUsername": "-",
    "DownstreamContentSize": 1245, "DownstreamStatus": 200,"DownstreamStatusLine": "200 OK",
    "Duration": 25958433,
    "FrontendName": "http-https_marcusnoble.co.uk/",
    "OriginContentSize": 1245, "OriginDuration": 25716428, "OriginStatus": 200, "OriginStatusLine": "200 OK",
    "Overhead": 242005,
    "RequestAddr": "marcusnoble.co.uk", "RequestContentSize": 0, "RequestCount": 50101, "RequestHost": "marcusnoble.co.uk",
    "RequestLine": "GET /css/main.css?202102052202 HTTP/1.1",
    "RequestMethod": "GET", "RequestPath": "/css/main.css?202102052202", "RequestPort": "-", "RequestProtocol": "HTTP/1.1",
    "RetryAttempts": 0,
    "StartLocal": "2021-02-09T10:04:23.441099709Z", "StartUTC": "2021-02-09T10:04:23.441099709Z",
    "downstream_Accept-Ranges": "bytes", "downstream_Cache-Control": "public, max-age=0", "downstream_Content-Encoding": "gzip", "downstream_Content-Type": "text/css; charset=UTF-8",
    "downstream_Date": "Tue, 09 Feb 2021 10:04:23 GMT", "downstream_Etag": "W/\"13dc-1777437f9e8\"", "downstream_Last-Modified": "Fri, 05 Feb 2021 22:02:41 GMT",
    "downstream_Vary": "Accept-Encoding", "downstream_X-Powered-By": "Express",
    "level": "info",
    "msg": "",
    "origin_Accept-Ranges": "bytes", "origin_Cache-Control": "public, max-age=0", "origin_Content-Encoding": "gzip", "origin_Content-Type": "text/css; charset=UTF-8",
    "origin_Date": "Tue, 09 Feb 2021 10:04:23 GMT", "origin_Etag": "W/\"13dc-1777437f9e8\"", "origin_Last-Modified": "Fri, 05 Feb 2021 22:02:41 GMT", "origin_Vary": "Accept-Encoding", "origin_X-Powered-By": "Express",
    "request_Accept": "text/css,*/*;q=0.1", "request_Accept-Encoding": "gzip,deflate,br", "request_Accept-Language": "en-US", "request_Connection": "keep-alive", "request_From": "googlebot(at)googlebot.com",
    "request_Referer": "https://marcusnoble.co.uk/",
    "request_User-Agent": "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36",
    "time": "2021-02-09T10:04:23Z"
}

Fin!

Question?

3.8 MiB Raw Blame History Unescape Escape

Personal Infrastructure

A little about me and the past

1. Personal network setup

2. Smart home

3. Replacing Google Analytics

Personal network setup

Where things are at today...

Previously

Local Network setup

Local Network setup

Kubernetes - local

Kubernetes - local

Kubernetes - cloud

Kubernetes - cloud

Time for a quick break

Smart Home

Typical Day

Typical Day

Typical Day

Typical Day

Typical Day

Security

Misc

Time for a quick break

Replacing Google Analytics

Why?

The solution!

Example Access Log

Fin!

3.8 MiB

Raw Blame History