Tensor types#
There are two core tensor types:
ndarray (
qd.ndarray)field (
qd.field)
See also tensor for a unified qd.tensor(...) factory that picks between the two via a backend= keyword.
In addition, when used from a qd kernel, fields can be:
referenced as a global variable
passed into the kernel as a parameter
For the remainder of this doc, we will compare three approaches, which we will refer to as:
“ndarrays”: always passed into kernels as parameters
“global fields”: referenced from kernel as global variable
“field args”: passed into the parameter as a kernel
Example of each tensor approach#
Let’s first give an example of using each:
NDArray#
Annotate ndarray parameters with qd.types.NDArray[data_type, number_dimensions], where data_type is the type of each stored element and number_dimensions is the number of array dimensions. For example, qd.types.NDArray[qd.i32, 1] describes a one-dimensional array of 32-bit integers. The dimension count is optional: qd.types.NDArray[qd.i32] accepts any number of dimensions.
import quadrants as qd
qd.init(arch=qd.gpu)
a = qd.ndarray(qd.i32, shape=(10,))
@qd.kernel
def f1(p1: qd.types.NDArray[qd.i32, 1]) -> None:
p1[0] += 1
Pass fill= to initialize an ndarray during allocation:
a = qd.ndarray(qd.f32, shape=(4, 5), fill=1.0)
directions = qd.Vector.ndarray(3, qd.f32, shape=8, fill=[0, 0, 1])
matrices = qd.Matrix.ndarray(2, 2, qd.f32, shape=4, fill=[[1, 0], [0, 1]])
A scalar fills every component. Vector and matrix ndarrays also accept lists, tuples, or Quadrants vectors and matrices matching the element shape; that value is repeated across the array. Vector and matrix types passed to qd.ndarray support the same initializers.
Only the array’s forward values are initialized; gradient buffers retain their defaults when needs_grad=True.
Global field#
import quadrants as qd
qd.init(arch=qd.gpu)
a = qd.field(qd.i32, shape=(10,))
@qd.kernel
def f1() -> None:
a[0] += 1
You can see that we access the global variable referencing the field directly from the kernel. No need to provide the field as a parameter.
Field args#
import quadrants as qd
qd.init(arch=qd.gpu)
a = qd.field(qd.i32, shape=(10,))
@qd.kernel
def f1(p1: qd.Template) -> None:
p1[0] += 1
In this case, we provide the field to the kernel via a parameter, with typing type of qd.Template.
Initializing fields#
Use fill= to give a field an initial value:
weights = qd.field(qd.f32, shape=16, fill=1.0)
Field allocation is deferred until Quadrants needs the storage. The initializer runs once at that point, before the first read or write through Quadrants. Declaring multiple fields with fill= lets Quadrants allocate them together. This avoids creating a separate allocation after each .fill() call. On Metal, Apple’s GPU API, a kernel can access only a limited number of separate memory buffers. Allocating fields together reduces the number of buffers it needs.
fill= also works with qd.Matrix.field and vector or matrix types passed to qd.field. A scalar initializes every component; a list, tuple, or Quadrants vector or matrix supplies per-component values. Mutable values are captured when the field is declared. Omitting fill or passing None preserves the existing allocation behavior. Only the field itself is initialized; its gradient and dual fields, which store derivatives for automatic differentiation, keep their default values.
“Sparse fields” track which elements are active, meaning enabled for storage and computation. On these fields, fill= fills only elements active when initialization runs. It does not change the default value of elements activated later.
A field declared without shape must still be placed before use: placement assigns its storage layout using qd.root and .place(). The axis identifier qd.i denotes the first axis, and dense(qd.i, 16) specifies storage for 16 elements along that axis. For example, declare values = qd.field(qd.f32, fill=1.0), then call qd.root.dense(qd.i, 16).place(values) to assign that layout to the field. See the manual placement example.
For ndarray initialization, see NDArray.
Comparison of tensor approaches#
Tensor approach |
Launch latency |
Runtime speed |
Resizable without recompile? [*1] |
Encapsulation?[*2] |
|---|---|---|---|---|
ndarray |
Slowest |
Slower |
yes |
Yes |
global field |
Fastest |
Fast |
no |
No |
field arg |
Medium |
Fast |
no |
Yes |
[*1] We’ll discuss this in ‘Under the covers’ below
[*2] Will be discussed in ‘Encapsulation’ below
Let’s define each of these column headings.
Under the covers summary#
When running a kernel, three things need to happen:
the kernel needs to be compiled
the parameters need to be sent to the GPU (contributes to kernel launch latency)
the kernel launch needs to be sent to the GPU (contributes to kernel launch latency)
Looking at kernel launch latency:
field args and ndarrays both are passed in to the GPU as parameters, and hence increase launch latency
ndarrays have more parameter processing than field args, and have the biggest launch latency
Each tensor type is bound to the compiled kernel in some way:
global fields are permanently bound to the kernel
to use the kernel with a different tensor, you’d need to copy and paste the kernel, with a new name
field args are permanently bound to the compiled kernel
however, as the typing
qd.Templatealludes to, you can call the kernel with different fields, and the kernel will be automatically recompiled to bind with the new field
ndarrays are only bound by:
the data type (
qd.i32vsqd.f32for example)the number of dimensions
Each call with an ndarray of a different data type or number of dimensions, that hasn’t already been compiled for, will trigger a recompile.
However, no recompilation is needed for passing in a different ndarray, that matches data type and number of dimensions.
Encapsulation#
Using global variables provides fairly poor encapsulation and re-use.
Both ndarrays and field args provide better encapsulation, and kernel re-use.
launch latency vs runtime speed#
For kernels that run for sufficiently long, the launch latency will be entirely hidden by the kernel runtime. Launch latency only affects performance for very short kernels.
Recommendations#
for maximum flexibility to resize tensors, use ndarrays
for maximum runtime speed, with good encapsulation, use field args
if the kernels are very short, for maximum speed you might need to use global fields, but this comes at the expense of poor encapsulation