Source file: literate-video.fut

Generating videos with literate Futhark

A video is just a collection of images called frames. However, for memory reasons it is usually best not to generate all the frames simultaneously. Therefore the :video command is a bit more intricate than :img. To generate a video, we must define a frame function that takes as input a state and returns a frame and a new state value.

First we define a function for converting a light intensity into an u32 encoding an ARGB colour.

def grey (light: f32) : u32 =
  let x = u32.f32(255 * f32.min 1 (f32.max 0 light))
  in (x << 16) | (x << 8) | x

Then we define a frame function where some of the parameters will be partially applied. The state parameter t represents the current time, and is increased by the time delta td for every frame.

entry frame (width: i64) (height: i64) (td: f32) (t: f32): ([height][width]u32, f32) =
  (replicate height (replicate width (grey(0.5+f32.cos(t*10)/2))),
   t + td)

> :video (frame 100i64 100i64 0.02f32, 0f32, 314i64);
fps: 24
format: gif

That is not terribly exciting. Let’s try something more interesting - specifically, a simple ray marcher that renders a blobby sphere. First, we’ll need to do some vector calculations, so we’ll import code from another example.

module vectors = import "3d-vectors"

module vec3 = vectors.mk_vspace_3d f32
type vec3 = vec3.vector

Next we’ll define a function for converting a point on a sphere into latitude and longtitude (UV mapping):

def uv (p: vec3) : (f32,f32) =
  let d = vec3.normalise p
  in (0.5 + f32.atan2 d.x d.z / (2*f32.pi),
      0.5 + f32.asin d.y / f32.pi)

Now we can define a function for determining the radius of a blobby sphere at a given point.

def radius_at (t: f32) (p: vec3) : f32 =
  let (u,v) = uv p
  in (1+f32.sin(u*20*f32.pi+t)*f32.sin(t))/2 +
     (1+f32.cos(v*20*f32.pi+t)*f32.sin(t))/2

The signed distance function is now trivial.

def sdf (t: f32) (p: vec3) : f32 =
  vec3.length p - radius_at t p

To trace the sphere, we perform ray marching into the scene, with up to 128 steps. If we make it to 128, we assume a miss. The logic is a bit convoluted due to lack of recursive functions.

type hit = #hit vec3 | #miss

def trace t (orig: vec3) (dir: vec3) : hit =
  let not_done (i, _) = i < 128
  let march (i, pos) =
    let d = sdf t pos
    in if d < 0
       then (1337, pos)
       else (i + 1, pos vec3.+ ((f32.max (d*0.1) 0.01) `vec3.scale` dir))
  in iterate_while not_done march (0,orig)
     |> \(i, hit) -> if i == 1337 then #hit hit else #miss

Finally, we’ll need a way to compute a surface normal for lighting. This can be done with a single invocation of reverse-mode automatic differentiation.

def grad f x = vjp f x 1f32

def distance_field_normal t pos =
  vec3.normalise (grad (sdf t) pos)

This concludes the actual geometry code. Now we just have to construct camera rays.

def camera_ray width height i j =
  let fov = f32.pi/3
  let x = (f32.i64 i + 0.5) - f32.i64 width/2
  let y = -(f32.i64 j + 0.5) + f32.i64 height/2
  let z = -(f32.i64 height)/(2*f32.tan(fov/2))
  in vec3.normalise {x,y,z}

The actual frame function is quite straightforward - for each pixel generate a ray and see if it collides with the blobby sphere. We use the surface normal to reflect lighting from a light sourse at (10,10,10).

entry blob (width: i64) (height: i64) (td: f32) (t: f32): ([height][width]u32, f32) =
  let f j i =
    let dir = camera_ray width height i j
    in match trace t {x=0, y=0, z=3} dir
       case #miss ->
         0xFFFFFF
       case #hit hit ->
         let light_dir = vec3.normalise ({x=10, y=10, z=10} vec3.- hit)
         let light_intensity = light_dir `vec3.dot` distance_field_normal t hit
         in grey light_intensity
  in (tabulate_2d height width f, t + td)

Finally we can view our sphere it in all its moderately pixelated glory.

> :video (blob 640i64 480i64 0.0314f32, 0f32, 100i64);
fps: 24
format: gif

Generating videos with literate Futhark

See also