Meteorological data fun (day 3)

I’m making some progress. Naturally, I was being a bit dense by the end of the day yesterday. Of course Apache Arrow comes with Debian repositories and good instructions for installation. I mean, it’s not quite as good as just working™, but close enough. Solved. I also read up on the GLib-bindings mechanism that they provide, which makes a lot of things easier. The Ruby bindings base off it, for instance, and probably the Python bindings too.

GRIB Woes, Part 2

Yesterday I spent a few hours making a decoder for the GRIB file format, which was a mostly pleasant experience, although the deeper I got the more I realized that there was something very wrong with the state of affairs. The data I’m trying to work with is the ECMWF’s Seasonal-to-Subseasonal daily averaged data, which is a pretty typical data set as meteorology and climate science goes. It’s very well curated, very precisely defined, and deployed to the world in a really difficult to use way.

The GRIB file format

I love a good file format. GRIB, however, appears to be… I don’t even know. It’s a file format for meteorological data, and stands for GRIdded Binary. There are two main versions, and they appear to be binary incompatible. Unsurprisingly for meteorological formats, a lot of the reference code is in Fortran. Sigh. GDAL has support for the GRIB format, but unsurprisingly, since GDAL is designed more around managing geographical imagery data, it makes it a bit complicated to just read the data out of a GRIB file.

jai-subprocess

Made jai-subprocess today. It was starting to be annoying that it was hard to interactively “chat” with subprocesses from Jai. The default system() and execXX() functions in POSIX are only good enough for very basic things. The situation that system() becomes insufficient for is when you want to capture the return status, the STDOUT and STDERR output, and be able to send input while it’s running. Currently the module is still missing the ability to write to STDIN on the child process, but I’ve run out of time for today.

Closing tabs

Much like most of the people in the western world, I’ve had my attention span stolen and decimated by an abundance of distractions, including social media. Being addicted to the dopamine-induced thrill of relevance is counterproductive, stressful, and depressing. For that reason, I’m excited about Johann Hari’s new book, Stolen Focus: Why You Can’t Pay Attention–And How to Think Deeply Again. An excerpt was published in the guardian the other day, and it immediately struck a cord.

Books of 12021

Lots of good books this year. In rough order of completion: Braiding Sweetgrass, Robin Wall Kimmerer Cadillac Desert, Marc Reisner Babel 17, Samuel R. Delany The Ministry for the Future, Kim Stanley Robinson Moon Rush, Leonard David Why Planes Crash, David Soucie What Do You Care What Other People Think?, Richard Feynman The Wild Shore, Kim Stanley Robinson Growing a Revolution, David R. Montgomery Game Programming Patterns, Robert Nystrom The Gold Coast, Kim Stanley Robinson The Sheep Look Up, John Brunner The Book of Shaders, Patricio Gonzalez Vivo and Jen Lowe Pacific Edge, Kim Stanley Robinson Everyday Croatian for Beginners Restoration Agriculture, Mark Shepard Project Hail Mary, Andy Weir Agent running in the field, John LeCarre The Years of Lyndon Johnson, vol 1: The Path to Power, by Robert Caro Designing Data Intensive Applications, Martin Kleppmann The Science of Extreme Weather, Eric R.

Goals 12021

For the last four years, I have maintained a list of goals, updated once a year, but reviewed actively over the course of the year. There are typically between 25-28 goals per year, split into various categories: skills, body, wellbeing, personal infrastructure, finances, work and writing. Additionally, there is one category that starts out empty, named “unexpected wins of great magnitude”, where bonus points can be earned. In setting these goals, which I do on the last two days of the year, the objective is to think what it is that I want out of life, and what is practically attainable.

What else is needed in Jai?

Yesterday I wrote some stuff about Scientific Computing in Jai. After that, I chatted a bit with Felix, who is doing the LAPACK port for Jai, and we discussed more broadly what was needed. My list wasn’t bad as a baseline, but we agreed that a good graphing library was going to be needed. More on that later. Anyway, last night, I used the bindings generator to do a first pass of GraphBLAS bindings, which are now available.

Restarting?

It’s winter solstice today. A time for renewal. There’s a lot to renew. I’ve not had a functioning website for a number of years. A place for my (public) thoughts and an assemblage of my output. A quasi-personal space. I got a bit busy. Oops. It happens. So, here’s an attempt at renewing, replenishing, etc. Let’s see where it goes. For now: I’ve merged in the old content from my previous iterations of website, along with various articles from recent years (mostly in Icelandic).

Scientific Computing with Jai

I spend a fair bit of my time being angry at Fortran code while wishing that it could be replaced with significantly less Jai code. Jai, if you don’t know, is a language under active development that I’ve been beta testing for the past two years. It is an amazingly powerful language, blowing pretty much everything else out of the water, but since it’s quite young there aren’t a lot of libraries and such available for it yet, meaning that a lot of things end up being quite a lot of manual effort in practice – for better or worse.