Portainer upgrade and Microk8s

Woke up today to some microk8 cluster unhappiness. Some auto upgrades caused my leo-ntp-monitoring app to stop. I’ve been lazy and using Portainer as a UI vs remembering all the kubectl command line shizzle. When I tried to pull up Portainer in the browser, it was giving me some message about “Portainer has been upgraded, please restart Portainer to reset the security context” or something like that.

Well shit. I don’t even have a cup of coffee in me and I gotta start using my brain. Here’s the shell script to restart Portainer.

microk8s kubectl get pods -n portainer -o yaml | microk8s kubectl replace --force -f -

Raspberry Pi 4 Microk8s Kubernetes Clustering Gotchas

3 node raspberry pi 4 cluster

I’ve been running a few docker workloads on various stand-alone raspberry pi 4 hosts. This has worked well, but I decided to up my game a bit and setup a Kubernetes cluster. Why? Kubernetes is the container orchestration technology that is taking over the cloud and figured it would be a good learning opportunity to figure out how all the bits play together.

For my workloads, I need a 64 bit OS and I am using raspberry pi 4 8GB boards with a power-over-ethernet (POE) hat. I am using Ubuntu Server 64 bit and I am using Microk8s for the Kubernetes runtime. The tutorials are straight forward and I am not going to rehash that, but instead call out the gotchas to look out for.

CoreDNS

For my infrastructure stuff, I use DHCP reservations with long leases and make an internal DNS entries. This is a lot easier to centrally manage that doing static address assignments. I knew I was going to need k8 DNS support, so I did the following….

microk8s enable dns

And then when I moved my docker hosted container into a pod it failed. After a little troubleshooting to make sure there wasn’t any network layer issues, and validating that I could resolve external DNS names, I knew the problem was CoreDNS wasn’t pointed at my internal DNS servers. There are a couple ways to fix this…

# pass the dns server ips when enabling coredns 
microk8s enable dns:dns1, dns2

# or you can edit the configuration after-the-fact
microk8s kubectl -n kube-system edit configmap/coredns

Private Registry

I wanted to run a private registry to start with. Why? ISP connections can fail and it is also a fast way for me to experiment. Microk8s is the container orchestration layer, and it is using Docker for the container runtime. Docker by default will attempt to use HTTPs when connecting to the registry, which breaks with the private registry. You will see an error such as “…http: server gave HTTP response to HTTPS client.”

I am running a 3 node cluster, and I setup the registry storage on node-01. So we have to make some configuration edits…

# edit the docker /etc/docker/daemon.json file and add the ip address or FQDN to the registry host. I did this on each node of the cluster
{
  "insecure-registries" : ["xx.xx.xx.xx:32000"]
}

# restart docker
sudo systemclt restart docker

# now edit the container template and use the same 
# ip address/FQDN. I did this step on each node in 
# the cluster to make sure everything was consistent.
# The point of a cluster is to let the cluster consider 
# all resources when picking a host, so each node needs 
# to be able to pull the docker images if there is a 
# redeployment, scaling event, etc.
sudo nano /var/snap/microk8s/current/args/containerd-template.toml

 [plugins."io.containerd.grpc.v1.cri".registry.mirrors."xx.xx.xx.xx:32000"]
          endpoint = ["http://xx.xx.xx.xx:32000"]

# after making the edit/saving, restart the microk8s node
microk8s stop
microk8s start

I ported the leo ntp time server monitoring to run in the microk8s cluster. It has worked flawlessly until it croaked. The entire cluster was jacked up. I was using channel=1.20/stable which was the latest release at that time. I have since rebuilt the cluster to use channel=1.21/stable and everything has been bullet proof.

OpenHPSDR PowerSDR mRX PortAudio Errors and Fix

OpenHPSDR PowerSDR mRX PortAudio Error -9999
OpenHPSDR PowerSDR mRX PortAudio Error -9999 Unanticipated host error

You’re about ready to get your radio geek on and you get a PortAudio Error -9999 Unanticipated host error, followed by a PortAudio Error -9988 Invalid stream pointer.  WTF since everything was working fine on Windows XP/Windows 7/Windows 10.  Well, the mighty fine folks at Microsoft decided to tighten up security with applications (“apps” if you’re cool) permissions to access the microphone.

Windows 10 Allow Access to Microphone

The fix is straight-forward:  On Windows 10, you go to settings->privacy->microphone and enable it.  If you are privacy concerned like me, you can then go through the “Choose which apps can access your microphone” and turn them all off.

For you new software engineers – this is a great example of a shitty error message.  If the message said “Access denied while trying to open your microphone or input device.  Please make sure this device is enabled and the access permissions are correct” that would make it much easier to diagnose the issue.

Machine Learning to Eat Free

This hack is brilliant:
In today’s digital age, a large Instagram audience is considered a valuable currency. I had also heard through the grapevine that I could monetize a large following — or in my desired case — use it to have my meals paid for. So I did just that.

I created an Instagram page that showcased pictures of New York City’s skylines, iconic spots, elegant skyscrapers — you name it. The page has amassed a following of over 25,000 users in the NYC area and it’s still rapidly growing.

BMW S1000R Accessory Light Flasher – First Prototype

I built this about a year ago.  I prototyped a circuit that used an ATTiny85 microcontroller to drive a p-channel (high side) mosfet.  The idea was to use the microcontroller to strobe the Clearwater Darla LED accessory lights.

The circuit worked as expected, but that click-click-click-click noise is bad. I thought the functionality of the led “instant-on” was via a 12v signal to the dc-dc circuit in the led light, but it is actually a mechanical relay. Strobing anything mechanical is no bueno.

I  completely changed my strategy after this test.  A little experimenting and I discovered that the accessory lights are controlled by a PWM signal which controls the light intensity (low to full power).

Microsoft Azure Sphere

IoT might be an over-hyped trend, but for ~$8 retail I can buy a EPS8266 NodeMCU board that has built-in WIFI.  Moore’s Law and accessibility will continue to drive costs down, which means eventually all manufacturers will experiment with IoT products.  Why?  Because the data that can be captured is extremely valuable and can be monetized.  One downside is these SoCs do not have any hardware protections where you can stash secrets, such as a code signing key.

For example, when I created the garage-o-matic to monitor and open/close my garage doors, I started looking at how to secure the firmware beyond a simple password.  Ultimately, I would have had to build some additional custom hardware if I wanted to stay in the hobbyist IoT space.

I like the general idea of including some sort of hardware security system, which the Azure Sphere chip is calling “Pluton.”  But what is really telling is Microsoft supporting this ecosystem with a Linux distribution.  I’ve mentioned this before that Microsoft is becoming a cloud-first company, and this really drives that home.  Build whatever you want, using the tooling you want, get even faster time-to-market if you use Visual Studio/Azure boilerplate, and run it on Microsoft’s Azure cloud.

https://azure.microsoft.com/en-us/blog/introducing-microsoft-azure-sphere-secure-and-power-the-intelligent-edge/

Wizbangadry

I met up with one of my peeps for lunch the other week.  We’re chatting about stuff, then we started talking about coding.  We have a nice rivalry – I’m very much about the “art and craft” of software engineering, and he’s all about using the latest/greatest to build stuff.  I call B.S. on his shinny new, and he calls B.S. on my old and crusty.

Me: “Dependency Injection used to be your jammy jam. You told me that my code sucked because I called a constructor directly.  So what’s your newest hotness?” Continue reading “Wizbangadry”

Progressive Web Apps and The Microsoft Store

Welcoming Progressive Web Apps to Microsoft Edge and Windows 10

Microsoft announced that Progressive Web Apps (PWA) will be added to the Microsoft Store (the “Store”).  This means just like a native app (or Universal App in Microsoft Store parlance), you can build a PWA app and have that added to the Store.  From a developer perspective, this is great.  A PWA app in theory should be much more cross-platform than a native app.  But what I find more interesting is thinking about the “why’s” a company would do something.

The big tech companies have been battling for years.  When you are building your business, trying to navigate the cesspool of technologies is a challenge.  You have to be careful of betting on a technology that could get dropped when it isn’t a strategic fit anymore.  Remember a thing called Silverlight?  As developers, we know its possible to have standards and the Web has been that shinning light.  But Apple, Google, and Microsoft all have different objectives.  Unfortunately, rather than evaluating a technology on its technical merits, it’s actually more important to evaluate it on the viability of its long term success. Continue reading “Progressive Web Apps and The Microsoft Store”

Garage Parking Assist – Part One

Intro

Ever since I outfitted my truck for the Zombie Apocalypse, parking it in my garage is something of a pain.  The front and rear bumpers have lengthened the wheelbase so I only have a few inches of margin – if I drive in too far I’ll hit the front wall, and if I don’t drive in far enough the garage door will hit the spare tire and refuse to close.

For years I put up with this until the hassle overcame my laziness factor.  I could have hung a tennis ball from the ceiling, but I’ve got a pretty cool garage “man cave” and a dangling tennis ball isn’t cool.  Plus, I’ve got some geek cred I need to maintain, I can build it myself, and more importantly maybe this can teach/inspire someone else along the way.

I’ve organized this into a two part series.  Part one will give you some background info, instructions on bread-boarding (prototyping) the circuit, and getting the software flashed to the circuit.  Additionally, I’ll point out some of the more important details in the software and give reasons why this was done.

While you might see some of the electronic equipment on my bench, don’t let that stop you.  All you need is a multimeter, and you don’t really need that unless stuff goes whacko.  Start with a simple breadboard and hookup wire.  If you decide to make this more permanent, I’ve included links to “perma-proto” boards.  These look just like the breadboard you are using, but you can solder stuff to them. Continue reading “Garage Parking Assist – Part One”