• Weeknotes - query-bot, HCL2, Docker without Docker

    So I’ve ended up writing 246 lines of Go in my quest to build a Slack bot…introducing query-bot.

    As I put it simply in the README:

    query-bot is an unintelligent Slack bot for writing HTTP query responses to a given channel when user messages match pre-defined commands.

    It’s basically a Slack bot for curl.

    It’s pretty bare-bones currently, and I have a pending feature branch with non-legacy authentication and some key refactoring. Though I have to say that I love working with Go. Error handling is dead simple, Viper is a killer configuration module, and my container image is only 14MB!

    Packer templates in HCL2

    Let’s leave JSON templates behind:

    packer hcl2_upgrade template.json

    Note: v1.7.1 required if you want the above command to also upgrade variables to HCL2 local variables

    Hashicorp’s introduction to Packer and HCL2.

    Bookmarked

    Docker without Docker
    https://fly.io/blog/docker-without-docker/

    • fly.io is a platform that I have been wanting to try out (multi-region containers with BGP Anycast + Firecracker micro-VMs!) and this deep dive on their blog about their container to micro-VM deployment pipeline speaks volumes about their investment in building a cool product.

    How to make an awesome Python package in 2021
    https://antonz.org/python-packaging/

    • When you want to make that one-off Python utility a little more official…
  • Weeknotes - ACLs, e-ink, PGP git commits

    This week I’ve continued work on my Nomad/Consul architecture and began using the allure of Slack bots to learn a bit of Go.

    Also, I’m trying to let myself post a bit more than a few bookmarks per week—hence the renaming of these weekly posts from Goodies –> Weeknotes (à la Simon Willison’s blog). He’s quite industrious!

    Consul ACLs

    It’s dead simple to automate the deployment a dual-stack Nomad/Consul cluster, but ensuring that proper and sane ACL bootstrapping is part of the process is less than straightforward. HashiCorp documentation is great though often contradicting! Here’s a good primer.

    Bookmarked

    Creating a digital newspaper from a 32” e-ink display
    https://gregraiz.com/i-made-an-eink-newspaper/

    • Absolutely the type of project that’s been on my list for quite some time. If only e-ink displays weren’t so expensive.

    What does a PGP signature on a git commit prove?
    https://people.kernel.org/monsieuricon/what-does-a-pgp-signature-on-a-git-commit-prove

    • An informative piece on how cryptographic signatures on git commits can help guarantee the validity of a repository’s history.
  • Weeknotes - DNS tool in Go, not your normal binary

    Bookmarked

    A fun web tool (built with VueJS and Golang on Netlify functions) to make DNS queries
    https://jvns.ca/blog/2021/02/24/a-little-tool-to-make-dns-queries/

    Another deep dive into personal data warehouse infrastructure
    https://beepb00p.xyz/myinfra.html

    A run-everywhere web server binary that is also the zip archive containing page source.
    Completely nuts.
    https://justine.lol/redbean/index.html

    Sendy - Affordable, self-hosted newsletter platform using Amazon SES
    https://sendy.co/

    Apple Platform Security - Apple Support
    https://support.apple.com/guide/security/welcome/web

  • Weeknotes - S3 sync, mission control, personal data warehouses

    Bookmarked

    Papercraft Models by Rocky Bergen
    http://rockybergen.com/papercraft

    Replacing Dropbox in favor of DigitalOcean spaces (or other S3 storage)
    https://mitjafelicijan.com/replacing-dropbox-in-favor-of-digitalocean-spaces.html

    Open MCT - Open Source Mission Control Software by NASA
    https://nasa.github.io/openmct/

    The Big Little Guide to Message Queues
    https://sudhir.io/the-big-little-guide-to-message-queues/

    Personal Data Warehouses: Reclaiming Your Data by Simon Willison
    https://simonwillison.net/2020/Nov/14/personal-data-warehouses/

  • Weeknotes - 2 million containers, thermometer firmware

    This is the first post of a maybe-once-a-week series of things that I found on the internet that may be worth a look.

    Bookmarked

    Hashicorp - The 2 Million Container Challenge
    https://www.hashicorp.com/c2m

    fasterthanli.me - A half-hour to learn Rust
    https://fasterthanli.me/articles/a-half-hour-to-learn-rust

    Julia Evans - Docker Compose: a nice way to set up a dev environment
    https://jvns.ca/blog/2021/01/04/docker-compose-is-nice/

    Custom firmware for the Xiaomi Thermometer LYWSD03MMC
    https://github.com/pvvx/ATC_MiThermometer

  • Make a DaemonSet schedule a pod on the master node(s)

    Who hasn’t been building a small Pi-based Kubernetes cluster at home?

    Well, on mine I wanted to deploy a DaemonSet with node_exporter for my external Prometheus service to scrape metrics of each node in the cluster, so I wrote up a DaemonSet manifest and applied it to my cluster. On first glance everything seemed happy, but quickly I noticed that the master node was not running a node-exporter pod!

    A quick Stack Overflow search led me to this answer and informed me that since Kubernetes 1.6 you have to add a toleration to the Pod spec section of your DaemonSet manifest:

    tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule

    I added this, and my master node was now running a node-exporter pod—just as I wished.

  • Using Vim to record a maintenance event timeline

    Vim can do lots of things

    After-hours maintenance is something my team and I encounter frequently. Whether it’s customer turn-ups, circuit migrations, or firmware upgrades—we get used to the 10pm to early morning working hours!

    Often, we like to record a rough timeline of events during the maintenance period in order to have timestamped context for re-tracing our steps and/or investigating persistent issues before/during/after the maintenance period. Historically I’ve manually inserted timestamps as we never keep too granular of a timeline. However, this last go-around I decided to dig a little into Vim and look into how to automate the insertion of timestamps whenever I start a new line in the editor.

    Insert mode maps

    Vim/vi is nearly limitless in its customization. One of the configurable properties is key mappings. Key mappings allow the creation of a shortcuts for the purpose of repeating a sequence of keys or commands.

    In my use case, I want to map the automatic input of the formatted, current time using Vim’s internal strftime() function whenever we drop down to a new line in the editor. The mapping below is what I came up with:

    :imap <buffer> <CR> <CR><C-R>=strftime("%Y-%m-%d %H:%M:%S :: ")<CR>
    • imap tells Vim that this is an insert mode map
    • <buffer> tells Vim to only consider the mapping in the buffer in which it is defined
    • The first <CR> is the left-hand-side of the map command, which defines the keys used to activate the shortcut. <CR> and <Enter> are synomymous
    • <C-R> tells vim to insert the following expression at the cursor
    • =strftime("%Y-%m-%d %H:%M:%S :: ") is our strftime() function to grab the current date and time in the preferred format
    • The final <CR> executes the expression

    If the above is entered as a Vim command, a timestamp will be inserted at the beginning of the line every time Enter is pressed while in insert mode.

    Make it a Vim filetype

    Now that we have the functionality we want, the next step is to automate the key mapping when it’s suitable.

    First, enable the filetype plugin in your .vimrc file:

    filetype plugin on

    Create the following directories:

    • ~/.vim/ftdetect
    • ~/.vim/ftplugin

    In my example, I’m going to define a new filetype called toe (timeline of events) with the extension of .toe.

    Let’s define our filetype detection file:

    ~/.vim/ftdetect/toe.vim

    autocmd BufNewFile,BufRead *.toe set filetype=toe

    This sets the Vim filetype to toe when the filename ends in .toe.

    Lastly, we define the command(s) we want active when the filetype is toe:

    ~/.vim/ftplugin/toe.vim

    imap <buffer> <CR> <CR><C-R>=strftime("%Y-%m-%d %H:%M:%S :: ")<CR>

    Now, whenever you manually set the filetype to toe or open a file that ends in .toe, Vim will know to insert a timestamp at the beginning of each new line.

    Originally, I wanted to make a quick CLI tool in Go to replicate this behavior, but I’m glad that I went down the route of learning a bit more about Vim/vi.

  • Updating my profile README with GitHub Actions

    Woah cool

    Having heard of GitHub’s new profile README feature—and seeing many fun and clever takes on its functionality—I decided to take a shot at simple README showing something I like, backed up by a simple GitHub Action that keeps the content updated throughout the day. Having used self-hosted CI/CD tools like Drone to automatically deploy configurations and code in the past, this seemed like an easy introduction to GitHub’s own take on CI/CD.

    For the content I chose a framegrab from my webcam at Gibraltar Peak here in Santa Barbara. Some friends and I rent some tower space on this peak for a small community internet project and have placed a cheap IP camera facing west to catch the daily sunset—and peek above the clouds when the marine layer makes an appearance.

    Create a jpeg from the RTSP stream

    On a server at the peak I have ffmpeg pulling in the RTSP feed and grabbing a single frame, resizing it, and saving it to a publicly-accessibly URL:

    #!/bin/bash
    /usr/local/bin/ffmpeg -y -i "rtsp://<camera_ip_and_port>/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif" -vframes 1 -s 1280x720 /<web_root>/m/frame.jpg

    An entry in the crontab runs the above script every minute.

    README.md

    This is near as simple as a README.md can get:

    ![](https://raw.githubusercontent.com/raylas/raylas/master/frame.jpg)
    `Updated-every-five-minutes frame from my webcam on Gibraltar Peak in Santa Barbara, California.`

    Note that I’ve linked to the raw repository URL of frame.jpg.

    The GitHub action

    Coming from using Drone and its YAML syntax, defining my new action felt pretty familiar.

    Using the supported schedule trigger, the action below performs the following every five minutes:

    • Checks out the repository using GitHub’s first-party checkout action
    • Grabs the latest framegrab from our web server
    • If git diff detects a change, the new frame.jpg is committed and pushed to the master branch
    name: frame-update
    
    on:
      push:
      workflow_dispatch:
      schedule:
        - cron: '*/5 * * * *'
    
    jobs:
      update:
        runs-on: ubuntu-latest
        steps:
          - name: Checkout
            uses: actions/[email protected]
    
          - name: Grab latest stream frame
            run: |-
              wget https://internetmountain.org/m/frame.jpg -O frame.jpg
    
          - name: Commit and push
            run: |-
              git diff
              git config --global user.email "[email protected]"
              git config --global user.name "README-bot"
              git diff --quiet || (git add frame.jpg && git commit -m "Update stream frame")
              git push

    Nice and happy

    I came away from this little tangent of a project amazed by how extensible and capable GitHub actions are. Considering that public repositories get unlimited minutes of actions, I will be spending many spare brain cycles on new ideas of how to make Actions do something fun and useful, even if slightly unnecessary…

    My GitHub profile repository is here:

    https://github.com/raylas/raylas