Building
Zero
ZeroWatch
weather-dashboard
Fauna
TileMapTool
Planned Community

rpjios

An Accidental Data Platform
(and some other stuff)

Abstract

What started as a simple hobby project has proven flexible & stable enough to continually enable the addition of many bits & bobs, accidentally creating an end-to-end platform of sorts. The aim here is to capture all those bits in one place & discuss them more thoroughly.
The primary "data bus" is redis pub/sub while implementation languages vary depending on the platform & requirements. Both Raspberry Pi and Arduino devices feature significantly, and in the fully-deployed system under discussion rely on the Bosch BME280 for T/H/P sensing & the Sensirion SPS30 for particulate matter (air quality) sensing.

The Bits & Bobs

(arranged roughly in order of where each appears in the pipeline, from sources to sinks; maturity varies widely)

RPJiOS

A sensor management & data collection/emission suite for Raspberry Pi devices written in Python 2.7 that acts as the primary data source for the platform. The namesake of the system and this site, it is a portmanteau of my initials (RPJ), "RPI" and "OS".

spheremon

Azure Sphere-based pipeline monitoring devices that inspired the creation of & consume YARL (below).

YARL

Yet Another Redis Library, this one in POSIX-compliant C.

ZeroWatch

Arduino live data display devices that inspired the creation of & consume Arduino-Redis (below).

Arduino-Redis

A Redis client library for Arduino devices written in C++11.

rhp

A rudimentary Redis-to-HTTP proxy, written in Go, that enables the Weather Dashboard (below).

weather-dashboard

A live dashboard of weather & air quality data built with vanilla Javascript/HTML/CSS (and a bit of help from Plotly). Features user-configurable plots with both historic and live data.

Air-quality index dashboard

Enabled by ThinkSpeak and the powerful MatLab integration within, this dashboard features a live air-quality index value based on data from the pipeline, as defined by the US EPA. The MatLab source is available here.

iFAQ

(infrequently asked questions... ok, one person asked once)
"Is Redis really necessary?"
Nope! The key component is the pub/sub pattern, which can be implemented in an incredibly wide variety of ways. I use Redis because I appreciate the flexibility it provides.
"Why not use off-the-shelf solution X for component Y?"
Just as a machinist or woodworker would use all available tools on the job but may well prefer to use simpler, more-direct tools when exercising their craft for enjoyment, I write software both as a profession as well as a hobby & the same motivation applies here: I simply enjoy building this way. That's all!
"What are the specifics of your actual deployment?"
Full diagram coming soon, but in short:
  • zed
    &
    zero
    are two Raspberry Pi Zero W's running RPJIOS that are the primary data sources
    • zero
      is equipped with a BME280 and DHT22
    • zed
      has the SPS30 and another BME280
  • newcharlie
    is an "Atomic Pi" (x86 SBC) that runs the Redis server & redis-to-http proxy
  • charlie
    is an older Ubuntu server that runs all the downsamplers, sqlite-sink & ThinkSpeak forwarders
  • spheremon
    is an AzureSphere device that monitors the system and alerts (via LEDs) when devices are lost
  • ezero
    is an Wemos D1-mini clone (an ESP-8266) running ZeroWatch that displays live data on color-coded 7-segment displays
  • pkd
    is a Raspberry Pi 4 that runs the weather dashboard on a large vertical Acer monitor
  • pearl
    is a Raspberry Pi 2b that acts as the private network entry & exit point and "VPN" provider

And now... something entirely different

("games" & gamedev-related things)

fauna

A game (of sorts) that became an unintended but welcome respite during the COVID-19 quarantine(s). Not yet compelete & and unlikely to ever be, it has been a thouroughly enjoyable endeavour nonetheless. Full source available here.

2d sparse bitmaps library

Created for Fauna, this library for node.js is published in the hope that may help other developers who face similar challenges.

TileMapTool

Created to assit in development of Planned Community, TileMapTool is a single-file, no-dependency, offline-capable tool for exploring, annotating & exporting tilemaps (a.k.a spritesheets). Source is available here.

Planned Community

An attempt to distill the city simulation game down to its essence, in a single-file browser-based package. Still being actively developed. Source is available here.