Today was kind of scattershot and weird. I got a lot of work done, but it was interspersed with naps, reading sessions and all sorts of stuff. As a result, my “work notes” today are just a few unrelated notes. On Focus I’ve finally gotten a copy of Johan Hari’s “Stolen Focus”, and it’s really good. In particular, it is touching on a bunch of different topical areas that I’ve been thinking about myself, but weaves them together in a nice and coherent way.

I wrote this up as a proxy for actually teaching a friend a how Git works the other day. I figure it might as well exist on the internet. A lot of git tutorials focus on the “how”, but I find that too few focus on the “why”. This tries to suggest the how while grounding the why. You definitely want to read something on the “how” after this though.

Booster shot today. Worried it might screw up my next day. Round 2 took me out for a bit. We’ll see. Not a lot of code today, but lots of planning work and meetings. Slowly converging on infrastructure and nailing a bunch of details. Whee. I’m going to say that I won’t bore you with the details, mostly because I can’t be bothered to write them out. But honestly, the details were fun, and you’re missing out.

Sturgeon was right, he just spelled CRUD wrong. 90% of everything is in fact CRUD – Creating, Reading, Updating, Deleting. Except, nowadays, we have schemas to at least partially automate the highly efficient creation of an infinity of crap. My Tech Stack is Janky The problem arises when you have six operational domains: Data warehouse (Original data) Batch processing data store (Largely Parquet files derived from original data) The programs that do the batch processing (mostly Jai, some Fortran, C and Python) need to have some internal representation of the data it is processing Operational data store (PostgreSQL tables derived from original data or batch outputs) Django ORM (Python schema reflecting the ODS) React frontend schemas (Typescript schema reflecting the output from Django REST API) The closer we get to the user, the more the data will have been transformed into something directly useful.

I am equally fascinated by the world’s fascination with zombies, as I am unfascinated with zombies. They appear to represent (in the modern manifestation) some kind of deep fear of the uncontrollable masses. LeCorbusier asked rhetorically, “Is there anything more pitiful than an undisciplined crowd?” The answer is yes. For instance, such a crowd trapped in mediated self-loathing, by way of a zombie trope. Somebody suggested a zombie/vampire feedback loop that follows a right-wing/left-wing sentiment shift: when people are fearful of their prosperity due to competition from other workers, people shift right wing and zombie movies get made.

One of the many ironies of Electron-based apps is that nowadays I rely on so many of them, I wish I could arrange them into tabs. Good morning! Another day, another box of stolen pens, amirite? No, but seriously. I found some energy lurking in the shadowy depths of my tordid existence last night and did half of the conversion to Typescript that I was bitching about yesterday. It’s amazing what you can get done when you just stop complaining for half a minute.

I’ve been doing these “work logs” publicly for a week now. The process is easy – when I sit down to start a workday, I start a new Markdown file. During the day I document what I’m doing, broadly, noting things like breakthroughs, frustrations, insights, useful links that come up. At the end of the day, instead of just forgetting the document exists, I commit it to my blog repo. Perhaps the public nature of this will come back to bite me at some point, but for now this is having a positive effect on me.

Short day today; have to go carry heavy objects for people. Just a few things. MuPDF Cloned MuPDF to try and determine how hard it would be to use as a library for something. Saw it had a makefile. Typed “make”, expecting the usual shenanigans, but no. It just freaking compiled. And quick, too. Like: No CMake shit, no Configure script, none of that madness. Just a well crafted Makefile.

Let us return briefly to the entire GRIB saga. Yesterday I complained about some weird overlaps in the imported data. Upon further inspection, it turns out the dates are being interpreted wrong. The way GRIB files store dates is that the beginning of the message indicates the start date and the templates indicate time step size. Data is then tagged as X time steps from origin, more or less. Except, how the time step is interpreted depends on the template itself, and apparently wgrib2 is reading that wrong for my data.

I’m making some progress. Naturally, I was being a bit dense by the end of the day yesterday. Of course Apache Arrow comes with Debian repositories and good instructions for installation. I mean, it’s not quite as good as just working™, but close enough. Solved. I also read up on the GLib-bindings mechanism that they provide, which makes a lot of things easier. The Ruby bindings base off it, for instance, and probably the Python bindings too.