• Weeknotes - Latency, SQLite statically, printing with netcat

    A couple of weeks ago my POE switch at the mountain site kept finding itself in quite the unresponsive state. This is a photo of me cruising into the microwave shed with my small, inadequate replacement switch and POE injector. The autopsy is ongoing...

    Bookmarked

    Latency sneaks up on you
    https://brooker.co.za/blog/2021/08/05/utilization.html

    • How measuring latency != measuring efficiency.

    Hosting SQLite databases on Github Pages
    https://phiresky.github.io/blog/2021/hosting-sqlite-databases-on-github-pages/

    • Compile SQLite to WebAssembly, wrap it in JS, and fetch only what you need.

    Bye CUPS: Printing with netcat
    https://retrohacker.substack.com/p/bye-cups-printing-with-netcat

    • nc 192.168.1.223 9100 < doc.pdf (it's real!)
  • Playing tag with DFS

    If you've ever worked in the fixed wireless space then dynamic frequency selection, or DFS, might sound familiar. DFS is a channel allocation scheme for the 5.8 GHz frequency band intended to prevent interference with other C-band usages such as aircraft radar, weather radar, and satellite communications.

    By requiring device certification, DFS channels remain relatively noise-free when compared the the rest of the 5.8 GHz band. This can permit greater RF performance without needing access to FCC-licensed bands.

    Caveats!

    Devices certified to use DFS must passively listen to the requested channel to ensure no radar is present before transmitting. The fun begins when the device detects radar whilst operating on a DFS channel. When this happens, the device must immediately stop transmitting on the selected channel and move to a non-DFS center frequency. Performance will likely degrade as the noise present in the rest of the 5.8 GHz band returns. This is often called a "radar strike."

    I have a Ubiquiti LTU Rocket access point at about 2000' AGL in Santa Barbara County and rely on DFS for adequate performance in this noisy environment. When a radar strike occurs and the radio moves to a non-DFS channel, it will not automatically revert to the previously selected DFS channel. Often the quickest way to get back to the DFS channel is to—believe it or not—reboot the radio.

    A heavy-handed solution

    dfs-bot is a brute-force, cron-friendly container to steer a wandering radio back to the trail.

    I use Ubiquiti's UISP as a largely hands-free network management system which exposes a generous API. dfs-bot uses this API to both determine if the active center frequency has deviated from the target DFS channel, and to reboot the radio to coax it back into its happy place. The alpine-based container uses cron to run the following bash every four hours or so.

    Core logic:

    #!/usr/bin/env bash
    set -e

    # Check if necessary environment variables are set
    if [[ -z $UISP_DOMAIN || -z $DEVICE_ID || -z $TARGET_FREQ ]]; then
    echo "One or more variables are undefined."
    echo "The following must be set:"
    echo "- UISP_DOMAIN"
    echo "- DEVICE_ID"
    echo "- TARGET_FREQ"
    exit 1
    fi

    # Check if UISP API token exists
    if [[ -z $UISP_API_TOKEN && ! -f "/run/secrets/uisp_api_token" ]]; then
    echo "The UISP API token cannot be found."
    echo "The following must be set:"
    echo "- UISP_API_TOKEN (will also check /run/secrets/uisp_api_token)"
    exit 1
    elif [[ -z $UISP_API_TOKEN ]]; then
    export UISP_API_TOKEN=$(cat /run/secrets/uisp_api_token)
    fi

    # Define API routes
    status_route="${UISP_DOMAIN}/nms/api/v2.1/devices/airmaxes/${DEVICE_ID}"
    restart_route="${UISP_DOMAIN}/nms/api/v2.1/devices/${DEVICE_ID}/restart"

    # Variable to store current device center frequency
    device_freq=$(curl -s -X GET $status_route -H "accept: application/json" -H "x-auth-token: ${UISP_API_TOKEN}" | jq '.airmax.frequencyCenter')

    # Let's see if any DFS events have occured
    if [[ $device_freq != $TARGET_FREQ ]]; then
    echo "Frequency off target [${device_freq}]. Rebooting access point..."
    curl -s -X POST $restart_route -H "accept: application/json" -H "x-auth-token: ${UISP_API_TOKEN}"
    else
    echo "Frequency on target [${TARGET_FREQ}]. No reboot needed."
    fi

    Usage

    Clone the repository:

    git clone git@github.com:raylas/dfs-bot.git

    Configure cron job in cronjobs:

    0 */4 * * * /dfs_bot.sh

    Build image:

    docker build dfs-bot:latest .

    Run container:

    docker run \
    -e UISP_DOMAIN=<uisp_domain> \
    -e UISP_API_TOKEN=<api_token> \
    -e DEVICE_ID=<device_id> \
    -e TARGET_FREQ=<target_frequency> \
    dfs-bot:latest

    Docker Compose:

    docker-compose up -d
  • Weeknotes - Cat hair, pulleys, Kafka, Docker Swarm

    That string and pulley tuning mechanism is likely the most interesting piece of technology engineering I've seen lately.

    This old Pioneer SX-780 receiver is something my partner had when we initially moved in together. It's quite literally a corner piece to our living room and brings both our turntable and our wireless audio solution to life.

    I recently had the top case removed to remove about a year's worth of cat hair from the inside (sorry audiophiles) and couldn't help but appreciate the refreshing simplicity of the internals. From the chunky diodes and capacitors to the colored wires jumpered from one side of the early PCB to the other, I knew I could make repairs on my own if it came to that.

    The original manual!

    Some technical highlights:

    • Receives AM/FM
    • A + B speaker outputs
    • 4 RCA inputs: AUX, Tape 1, Tape 2, Phono
    • 2 RCA outputs: Tape 1, Tape 2
    • Loudness switch
    • 15hz low frequency filter
    • Headphone output

    Bookmarked

    Gently down the stream
    https://gentlydownthe.stream

    • River otters explain Apache Kafka, of course.

    RFC 2100
    https://www.rfc-editor.org/rfc/rfc2100.html

    • "This RFC is a commentary on the difficulty of deciding upon an acceptably distinctive hostname for one's computer, a problem which grows in direct proportion to the logarithmically increasing size of the Internet."

    What I've learned about data recently
    https://seldo.com/posts/what-i-ve-learned-about-data-recently

    • Having recently joined a company focused on "big data" it's nice to see discussion about how engineers, not just scientists, are crucial to a functioning data pipeline.

    The Apple M1, ARM/x86 Linux Virtualization, and BOINC
    https://www.sevarg.net/2021/01/09/arm-mac-mini-and-boinc/

    • Still loving my M1 laptop 6 months in

    Docker Swarm Rocks
    https://dockerswarm.rocks

    • Nomad and Kubernetes are solid orchestrators, but you're telling me I could deploy application stacks in a distributed cluster with the same Docker Compose YAML file?
  • Weeknotes - Wildfires, telecom documents, eBPF

    Flagstaff, Arizona from ~40,000 feet with wildfires visible to the north.

    Bookmarked

    The Telecom Archive
    http://www.telecomarchive.com

    • 22,385 documents on all things Bell Systems

    The modern web on a slow connection
    https://danluu.com/web-bloat/

    • What does it mean to have a performant website?

    An incomplete list of skills senior engineers need, beyond coding
    https://skamille.medium.com/an-incomplete-list-of-skills-senior-engineers-need-beyond-coding-8ed4a521b29f

    • Undoubtedly useful for non-senior engineers as well

    How Netflix uses eBPF flow logs at scale for network insight
    https://netflixtechblog.com/how-netflix-uses-ebpf-flow-logs-at-scale-for-network-insight-e3ea997dca96

    • I'll never not share a post about network observability.
  • Weeknotes - SDR 1G cellular, cross-platform Docker on M1 Macs

    I'm working remote from the east coast for the next week.

    Main observations:

    • It's hyper-stimulating to be in a city for the first time in over a year
    • Thanks to the vaccine people are out in droves enjoying summer
    • I think I'm okay with the first two points

    Bookmarked

    Gaining real control of an Amazon Echo
    https://andrerh.gitlab.io/echoroot/

    • This may finally push me to pull my old, unused Echo out of storage.

    Running a 1G analog phone from 1997
    https://medium.com/geekculture/howto-running-the-1g-analog-phone-from-1997-3caec77a9df9

    • Another legacy celluluar/radio device revived thanks to a little SDR.

    The simple power of docker-compose in an ARM environment
    https://til.simonwillison.net/macos/running-docker-on-remote-m1

    • The excuses for staying on x86 keep dwindling...

    An easy environment to start working with data in git
    https://octo.github.com/projects/flat-data

    • Why not waste some time and automate the scraping/ingestion of data into git with GitHub Actions?
  • Weeknotes - Static endgame, burrito cannons, ONIE fun

    Hexo was my static site generator of choice for many years, until this last weekend. I wasn't pleased with my CI/CD environment, nor was it clean to only have the blog/ section of the site statically generated. Additionally, it felt like time to move on from the DigitalOcean App Platform—where this site was hosted until now—to something with a robust CDN like Netlify.

    I can't say enough positive things about my latest, three-faceted setup:

    • Eleventy - flexibile and unopinionated static site generation
    • Netlify - hosting and build infrastructure emphasizing the JAMstack architecture
    • Netlify CMS - a git-backed CMS that can be used with any static site generator

    Bookmarked

    The Alameda-Weehawken Burrito Tunnel
    https://idlewords.com/2007/04/the_alameda_weehawken_burrito_tunnel.htm

    • An old, unfortunately fiction, but always delightful read by Maciej Cegłowski.

    You can still use a pager in 2021
    https://debugger.medium.com/howto-using-a-pager-in-the-21st-century-6a57454ecde8

    • A pager is something I yearned for as a young kid before the era of ubiquitous cellular devices. Why not bring one back to life with a little bit of reverse engineering and SDR?

    Replacing Elasticsearch with Rust and SQLite
    https://nickb.dev/blog/replacing-elasticsearch-with-rust-and-sqlite

    • Even though Elasticsearch is a do-it-all storage engine it can be worthwhile to replace it with something more single-purpose and performant!

    Teaching a cheap ethernet switch new tricks
    https://blog.benjojo.co.uk/post/dell-switch-hacking

    • Having lived in the world of bare metal networking hardware, it's awesome to see ONIE-capable devices becoming accessible to more enthusiasts.
  • Weeknotes - Vault Agent, WireGuard, more WireGuard

    Not a ton of side project work the last week or so. Though, I've been slowly incorporating Go into my toolbox at work. Think SSO OAuth API calls wrapped in a friendly, portable CLI tool.

    I did however update query-bot to use Slack's "App" API authentication flow. Thanks to this PR finally merging. Expect a proper release soon-ish!

    Dynamic WireGuard configurations with Vault Agent Templates

    Also in the background I've been toiling away on a project to bring together a WireGuard-gated Nomad, Consul, and Vault deployment model that is packed full of things like mTLS, ACLs, and dynamic, Vault-derived secrets.

    Last week I implemented a Vault Agent Template for my WireGuard configuration and it's been great not having to bake my WireGuard peer configurations into my machine images.

    A sample of the template:

    [Interface]
    Address = 10.0.0.1/24,fd10::1/64
    ListenPort = 51800
    PostUp = /etc/wireguard/postup.sh
    PostDown = /etc/wireguard/postdown.sh
    {{- with secret "kv/node/wireguard" }}
    PrivateKey = {{ .Data.data.private_key }}
    {{ end }}

    {{ range secrets "kv/metadata/node/wireguard-peers/" -}}
    {{ with secret (printf "kv/node/wireguard-peers/%s" .) -}}
    [Peer]
    # {{ .Data.data.description }}
    PublicKey = {{ .Data.data.public_key }}
    AllowedIPs = {{ .Data.data.allowed_ips }}

    {{ end }}
    {{- end }}

    Vault Agent configuration:

    # WireGuard templates
    template {
    source = "/etc/wireguard/wg0.conf.tpl"
    destination = "/etc/wireguard/wg0.conf"
    command = "systemctl restart wg-quick@wg0.service"
    }

    I know that I can use wg syncconf to hot-reload the configuration, but I have yet to implement that due to some blocking factors.

    Bookmarked

    WireGuard Endpoint Discovery and NAT Traversal using DNS-SD https://www.jordanwhited.com/posts/wireguard-endpoint-discovery-nat-traversal/

    • This type of deep-dive is my happy place. WireGuard + DNS + NAT trickery further makes the case for WireGuard as a minimal, secure, and performant tunnel technology.

    Notes on building debugging puzzles
    https://jvns.ca/blog/2021/04/16/notes-on-debugging-puzzles/

    • Julia Evans never stops introducing novel methods of knowledge sharing, and this post about her foray into interactive, debugging puzzles is right in line with her usual high-quality content. Plus, it's about DNS!
  • Weeknotes - query-bot, HCL2, Docker without Docker

    So I've ended up writing 246 lines of Go in my quest to build a Slack bot...introducing query-bot.

    As I put it simply in the README:

    query-bot is an unintelligent Slack bot for writing HTTP query responses to a given channel when user messages match pre-defined commands.

    It's basically a Slack bot for curl.

    It's pretty bare-bones currently, and I have a pending feature branch with non-legacy authentication and some key refactoring. Though I have to say that I love working with Go. Error handling is dead simple, Viper is a killer configuration module, and my container image is only 14MB!

    Packer templates in HCL2

    Let's leave JSON templates behind:

    $ packer hcl2_upgrade template.json

    Note: v1.7.1 required if you want the above command to also upgrade variables to HCL2 local variables

    Hashicorp's introduction to Packer and HCL2.

    Bookmarked

    Docker without Docker
    https://fly.io/blog/docker-without-docker/

    • fly.io is a platform that I have been wanting to try out (multi-region containers with BGP Anycast + Firecracker micro-VMs!) and this deep dive on their blog about their container to micro-VM deployment pipeline speaks volumes about their investment in building a cool product.

    How to make an awesome Python package in 2021
    https://antonz.org/python-packaging/

    • When you want to make that one-off Python utility a little more official...