Information about the locks available for accumulators.

Which target are we ultimately generating code for? While most of the kernels code is the same, there are some cases where we generate special code based on the ultimate low-level API we are targeting.

Reduce the w first elements of the given arrays, leaving the result in element zero of each. The others are clobbered.

Run code where threads within a warp communicate directly through shared memory. On OpenCL we rely on lockstep execution make all memory accesses volatile; on CUDA and HIP we explicitly synchronise the warp afterwards with WarpSync. Like a barrier, this must be reached by all threads of the warp.

Scan the w first elements of the given arrays, in place.

An array-typed element is kept twice: at index i, where the result is expected, and again at index i + arrs_full_size. It is the latter copy that a thread reads as its x operand, because the operator writes its result through its x parameter, which may well be the former. The elements from index arrs_full_size + blockScanChunkSize on are then used to save the elements of the first chunk, which are overwritten with the per-chunk carry-ins. Only the w first elements need be within the shape of the arrays, but all of the above must be backed by memory; see blockScanElems.

The callers in Futhark.CodeGen.ImpGen.GPU.SegScan.TwoPass and Futhark.CodeGen.ImpGen.GPU.SegRed pass arrays whose elements are the x and y parameters of the operator, so for them the copies that keep the two in step are redundant, but harmless.

blockScan splits the block into chunks of this size, which are scanned separately. The only requirement is that it should not be less than the square root of the block size. With 32, we will work on blocks of size 1024 or smaller, which fits every device Troels has seen. Still, it would be nicer if it were a runtime parameter. Some day.

Assign iterations of a for-loop to threads in the threadblock. The passed-in function is invoked with the (symbolic) iteration. For multidimensional loops, use blockCoverSpace.

peeledKernelLoop tid num_threads num_full_chunks n f invokes f on the iterations tid, tid+num_threads, and so on, below n, where num_full_chunks must be n quot num_threads. Only the final partial chunk is bounds checked, so the body is generated twice; see Note [Peeled block loops]. The body must contain thread-level code.

Various extra configuration of the kernel being generated.

The default kernel attributes.

Compute kernel attributes from SegLevel; including synthesising block-size and thread count if no grid is provided.

For many kernels, we may not have enough physical blocks to cover the logical iteration space. Some blocks thus have to perform double duty; we put an outer loop to accomplish this. The advantage over just launching a bazillion threads is that the cost of memory expansion should be proportional to the number of *physical* threads (hardware parallelism), not the amount of application parallelism.

Assign iterations of a for-loop to all threads in the kernel. The passed-in function is invoked with the (symbolic) iteration. The body must contain thread-level code. For multidimensional loops, use blockCoverSpace.

Iterate collectively though a multidimensional space, such that all threads in the block participate. The passed-in function is invoked with a (symbolic) point in the index space.

If we are touching these arrays, which kind of fence do we need?

Generate a constant device array of 32-bit integer zeroes with the given number of elements. Initialised with a replicate.

Retrieve a size of the given size class and put it in a variable with the given name.

Perform a Replicate with a kernel.

Perform an Iota with a kernel.

Is there an atomic BinOp corresponding to this BinOp?

Do an atomic update corresponding to a binary operator lambda.

Locking strategy used for an atomic update.

The mechanism that will be used for performing the atomic update. Approximates how efficient it will be. Ordered from most to least efficient.

A function for generating code for an atomic update. Assumes that the bucket is in-bounds.

Perform a scalar write followed by a fence.