What else is needed in Jai?

Yesterday I wrote some stuff about Scientific Computing in Jai. After that, I chatted a bit with Felix, who is doing the LAPACK port for Jai, and we discussed more broadly what was needed. My list wasn’t bad as a baseline, but we agreed that a good graphing library was going to be needed. More on that later.

Anyway, last night, I used the bindings generator to do a first pass of GraphBLAS bindings, which are now available. It was way easier than I expected. The bindings generator is excellent. Mostly, I just copied and simplified the generator used for the ImGUI bindings.

While this might mean that we can get those other libraries (GMP, GLPK, GSL etc) done easily, there are two thoughts that are relevant:

1. Just generating bindings is shitty, because it doesn’t take advantage of the features that make Jai powerful.
2. There are a lot of non-scientific tooling and libraries that will also be needed.

I also recognize that while I’m happy to do this work now, I’m not certain that I want to end up being a bindings package maintainer forever, so this work probably needs to be automated and standardized in some way.

Shortcomings of generation

As to the first point, there will always be a bunch of manual adaptation that will make sense. Take for example this definition in GraphBLAS:

1

GxB_Desc_get :: (desc: GrB_Descriptor, field: GrB_Desc_Field, __args: ..Any) -> GrB_Info #foreign libgraphblas;

(note: this is an optional extension as implemented in the SuiteSparse implementation, but it’s a good example anyway.)

The C declaration for the same is (from GraphBLAS.h):

1
2
3
4
5
6
7

GB_PUBLIC
GrB_Info GxB_Desc_get           // get a parameter from a descriptor
(
    GrB_Descriptor desc,        // descriptor to query; NULL means defaults
    GrB_Desc_Field field,       // parameter to query
    ...                         // value of the parameter
) ;

A more elegant way to do this in Jai would be something like:

1

GxB_Desc_get :: (desc: GrB_Descriptor, field: GrB_Desc_Field) -> []Any, GrB_Info;

But to get to there, we will need some manual work (or cleverness in the generator).

Despite this, I think the correct approach for now is to start getting things bound and start to see how people use them, and liberally accept pull requests that include things that make the nicer. This of course is based on the hope that people will do that kind of thing in good and consistent ways, which I really hope not to be let down on.

Other things that need bindings

A long time ago, I made a list for myself of libraries I could imagine wanting bindings for. A lot of the libraries I have made bindings for have been stuff that came up in my work (or play) that happened to be correct predictions off the list. Other things other people have made, or simply are included with the compiler.

• Cryptography
  • libgcrypt
  • OpenSSL
  • GnuTLS
• Databases (and queues)
  • PostgreSQL
  • MySQL
  • Redis
  • SQLite
  • UnQLite
  • rabbitmq
• Compression
  • zip
  • zlib
  • zlib-ng
  • lz4 (1.9.1 and 1.8.3 in std)
• Graphics
  • libtiff
  • libgeotiff
  • GL (in std)
  • SDL (in std)
  • freetype (in std)
• GUI / UI
  • GTK+
  • QT
  • ncurses
• Numerical
  • BLAS
  • GraphBLAS
  • GMP
  • GLPK
  • GSL
  • Cuba
• Serialization and file formats
  • protobuf
  • tpl
  • libavro
  • CSV
  • JSON
  • INI
  • XML
  • YAML
• Networking
  • Curl (in std)
• Utilities
  • Magic
  • GDAL
  • libfuse

In addition, I identified a few other types of things I’d want to be able to do:

• Proper good datetime library
• Markdown parser & HTML converter
• PDF builder
• Graphing library
• Statistics package

I’ve checked off the ones that are available (regardless of who did them), even if they’re in the sorry state listed above. I couldn’t be bothered to add links for everything, sorry.

I’m of course biased towards the kinds of things I’m doing, but if you think I’m missing something or something is redundant, let me know.