Towards Futhark 1.0
The release policy for obscure academic software is a strange and ad-hoc somewhat2 thing. The very first release of Futhark was issued to mark the conclusion of my PhD studies. Since then, we have issued new releases whenever we felt the stars were right, mostly because various package managers tend to include only released software. Sometimes we make a release corresponding to the facilities described in some paper, and sometimes because we are about to use the compiler for teaching, but usually it is just because it has been a while. This has led to 131 releases over the years, with the most recent being 0.27.1.
It is my impression that many programmers are afraid of releasing a 1.0 version of their software. I will claim that I have no such fear, while also recognising that Futhark has been 0.x software for nine years. The reason is that I always intended that the 1.0 release of Futhark would signify feature completeness not in the sense that Futhark would never again change, nor that it actually had all desirable features, but simply that it is able to compile all type-correct Futhark programs to GPU code. Until recently, this was not the case for programs that made use of nonuniform irregular parallelism, but this was recently addressed. This means that the once far-off hope of a 1.0 release will hopefully become reality this year.
Of course, I am not immune to the symbolism of a 1.0 release, so apart from feature completeness, I also want 1.0 to represent some kind of commitment to stability. While we certainly intend to keep improving the language and the compiler, I would like users of Futhark 1.0 to be able to upgrade to later versions while having a high amount of confidence that their usage continues to work. This means that if we want to change the language in major backwards-incompatible ways, now is the time to do it. Further, if there are any tools in the Futhark distribution that we do not want to maintain in the long term, now is the time to remove or change them. There are two such cases to deal with somehow before I feel comfortable issuing a 1.0 release.
The first is a somewhat subtle issue related to alias tracking; so subtle that it merits a dedicated post in the future. The root issue is that Futhark currently does not track aliases for top-level global variables with an abstract type. This is fundamentally just an old bug and it is easy to fix, but we have not fixed it because it causes certain programs to break with aliasing violations, due to a rule (which I will not justify here) that a function may not return an alias to a top level variable. This breaks only programs that use the module system to build parameterised modules, but such programs do exist. The bug is a real soundness issue in the type system, and it must be fixed, but I want to see if we can make alias tracking a bit smarter to reduce the breakage. Futhark as a language has been quite stable for a while (I went into a bit of detail here), but this will break some programs, and I’d rather do it before 1.0.
The second issue is not about the Futhark language at all. Many of Futhark’s built-in tools contain an interpreter for a small language called FutharkScript. It is used to provide a convenient facility for invoking compiled Futhark code without having to go through the server protocol or C API. While FutharkScript is syntactically a trivially small subset of Futhark, it is its own language, and supports various impure things like reading and writing files. Still, I’d rather not maintain two languages, when one could do. Instead, I would like to replace FutharkScript with real Futhark (a really good language, you know), executed through the interpreter, with facilities for invoking compiled functions. This is being worked on by Marcus Jensen, who also implemented automatic differentiation in the interpreter. Apart from the ability to transparently mix compiled and interpreted code, we also need some kind of facility for “magic functions” that perform I/O operations, and obviously only work in the interpreter. The core mechanisms for allowing the interpreter to perform “foreign calls” and transform between the interpreter value representation and “foreign data” has all been built and works quite well, so we just need to do the integration work of tearing out FutharkScript and replacing it with Futhark.
While Futhark is perhaps not strictly speaking a “hobbyist” language, in that most people who work on it are employed as academic researchers, its usability as a programming language goes beyond what is needed for its use as a research vehicle. Its release policy and documentation is much more similar to hobby projects than to either industrial languages or to academic thesis-ware. I do wonder how other hobbyists manage their releases, and when they decide on when to make that supposedly momentous “1.0”.