Node reference

Nodes are grouped as in the add-node menu. Runs says when a node does its work; Implemented by names the Klartraum class or function behind it.

Sources

Node

Description

Implemented by

Runs

Scene

Loads a 3D Gaussian model from an .spz file into CPU memory, optionally mirrored across Y for Nerfstudio exports.

klartraum::loadGaussiansSpz

once, when the graph is built

Image File

Loads an image file (PNG, JPEG, …), resized to W×H, as a 1×3×H×W tensor with values in [0, 1].

klartraum::TensorElement

once, when the graph is built

Numbers (CPU)

Node

Description

Implemented by

Runs

Number

A constant number.

studio

every frame (live), once per run

Time

Seconds since the studio started, times a speed.

studio

every frame (live), once per run

Sine

amplitude · sin(frequency · 2π · x + phase) + offset, e.g. to swing something back and forth.

studio

every frame (live), once per run

Upload Number

Copies a CPU number into a GPU buffer before every frame, without rebuilding the graph.

klartraum::HostFloat

every frame (live), once per run

Gaussians (CPU)

Node

Description

Implemented by

Runs

Transform (CPU)

Scales, rotates and moves Gaussians in CPU memory; orientations and view-dependent colour turn along.

klartraum::transformGaussians

once, when the graph is built

Merge (CPU)

Combines two sets of Gaussians in CPU memory into one, rendered and sorted together.

studio

once, when the graph is built

Upload Gaussians

Uploads Gaussians into the GPU buffers a Gaussian Splatting reads.

klartraum::GaussianDataStandard

once, when the graph is built

Gaussians (GPU)

Node

Description

Implemented by

Runs

Make Transform

A GPU transform from translation, rotation (degrees about X, then Y, then Z) and scale. Inputs that are not connected take the node’s values, which apply without rebuilding.

klartraum::TransformBuffer

every frame (live) or every run

Transform (GPU)

Moves Gaussians on the GPU by a transform, every frame.

klartraum::GaussianTransform

every frame (live) or every run

Merge (GPU)

Combines two sets of Gaussians on the GPU, every frame.

klartraum::GaussianMerge

every frame (live) or every run

Rendering

Node

Description

Implemented by

Runs

Orbit Camera

Camera uniform buffer driven by an orbit camera.

klartraum::InterfaceCameraOrbit

every frame (live), once per run

Swapchain Target

The window’s swapchain images, rendered into directly. Cleared to black before they are rendered into, unless Clear to black is off.

klartraum::ImageViewSrc

every frame (live only)

Offscreen Target

An image of fixed size (W×H) to render into for further processing. Cleared to black like the Swapchain Target, unless turned off.

klartraum::OffscreenTarget

every frame (live) or every run

Clear Image

Clears a Swapchain or Offscreen Target to a color before it is rendered into.

klartraum::ClearImage

every frame (live) or every run

Composite

Draws an image onto a target: replace or over (by the image’s alpha), stretch, fit (whole image, aspect kept, the rest of the target stays) or fill (covers the target, cropped). It passes the target on, so renderers after it draw over the image. The image is only read; it may be a Run result.

klartraum::ImageComposite

every frame (live) or every run

Gaussian Splatting

Renders the Gaussians over its target, with the compute or raster backend and all klartraum::GsplatConfig settings. The target is a Swapchain or Offscreen Target, possibly after Clear Image or Composite.

klartraum::createGaussianSplatting

every frame (live) or every run

Draw Basics

Draws a triangle, a cube or axes with a camera over a rendered image, e.g. axes as a reference after a Gaussian Splatting. Only on images of the window’s size.

klartraum::RenderPass with klartraum::DrawBasics

every frame (live) or every run

Renderers draw over what their target holds: a target starts from black because it is cleared, not because the renderer clears it. They render only into targets, which have an image per frame in flight; to render over another node’s result, draw it onto a target with a Composite first.

Linking a tensor into an input that takes images but not tensors (e.g. a Composite’s Image or Present) puts a Tensor to Image node in between.

On MoltenVK, the Swapchain Target is an image per frame in flight standing in for the swapchain, which Present copies into the window’s image. MoltenVK binds swapchain images to compute shaders as they were when the graph was built, so a Composite or a resample writing them directly would sometimes draw into an image that is not shown. The compiled graph shows the stand-in and the copy.

Compute

Node

Description

Implemented by

Runs

Image to Tensor

Converts a rendered image into a 1×3×H×W tensor.

klartraum::GeneralComputation

every frame (live) or every run

Tensor to Image

Converts a 1×3×H×W tensor (values in [0, 1]) into an H×W image.

klartraum::GeneralComputation

every frame (live) or every run

Resample

Resamples an image to W×H, nearest or bilinear.

klartraum::ImageResample

every frame (live) or every run

ONNX Model

Runs an ONNX network. It has one tensor pin per model input and output, named like them; they follow the model file, and links to pins a new model no longer has are removed. The operators Klartraum executes include Conv, ConvTranspose, MatMul, Gemm, Softmax, the normalizations, elementwise arithmetic, Reshape, Transpose, Slice, Concat and Resize.

klartraum::OnnxNetwork

every frame (live) or every run

Tensor pins carry any tensor. Its shape and element type (float32 or int64) are checked where it is consumed: an ONNX model’s inputs must match the model’s, layers and sinks take float32, and the node shows each output’s shape (and element type, unless float32).

Layers

Single network layers on float32 tensors, built with the same klartraum::layers factories the ONNX models use.

Node

Description

Implemented by

Runs

Add, Subtract, Multiply, Divide

A op B elementwise; the shapes broadcast (NumPy rules). If B is not connected, the node’s b is used as a one-element tensor.

klartraum::layers::binary

every frame (live) or every run

ReLU, Sigmoid, Sqrt

Elementwise; the output has the input’s shape.

klartraum::layers::relu, unary

every frame (live) or every run

Softmax

Softmax over the last axis.

klartraum::layers::softmax

every frame (live) or every run

Stable Diffusion

Stable Diffusion 1.5 text-to-image generation, with the models that Klartraum’s scripts/sd15_onnx/export_denoiser.py exports. The models are fixed-size, so one export serves one image size. They are not part of the Klartraum repository. Export them with uv run python export_denoiser.py --size 256 --onnx-dir ../../data/onnx/sd15_denoiser_256 (see the script’s README). The Stable Diffusion 1.5 example expects them in that directory.

Node

Description

Implemented by

Runs

Prompt

A prompt and a negative prompt, tokenized with CLIP’s byte-pair encoding into two 2×77 int64 tensors, Ids (negative prompt first) and Mask (the attention mask). The Vocabulary is the export’s vocab.json, with merges.txt next to it.

klartraum::ClipTokenizer, uploaded into two klartraum::TensorElement

once, when the graph is built

Text Encoder

Built-in meta node: an ONNX Model (the CLIP text encoder) that turns the Prompt’s Ids and Mask into 2×77×768 embeddings. Exposes Model.

klartraum::OnnxNetwork

every run

Latent Noise

Gaussian noise of the latent size for a W×H image (1×4×H/8×W/8), from a seed. Optionally it reads raw float32 latents from a file instead, e.g. the export’s initial_latents_f32.bin to reproduce its reference image.

studio

every run

DDIM Sampler

Denoises the latents in Steps DDIM steps with SD 1.5’s scheduler. Each step runs the UNet once on the negative and the positive prompt, blends the two noise estimates with the Guidance scale and takes the DDIM step on the CPU.

klartraum::OnnxNetwork, submitted once per step

every run, once per denoising step

VAE Decoder

Built-in meta node: Multiply (undoes the latent scaling) → ONNX Model (the VAE decoder) → Multiply and Add (maps [-1, 1] to [0, 1]). Outputs a 1×3×H×W image tensor for a Preview or an Image File Writer. Exposes Model and Latent scale.

klartraum::layers, klartraum::OnnxNetwork

every run

The DDIM Sampler is a staged node and only runs with Run. Run executes the Klartraum graph of everything before it, reads the tensors it needs back and runs the UNet once per step. It then uploads the result for the nodes that follow, whose graph it executes next. The compiled graph of a run shows every stage’s graph together with the UNet. Run waits for all denoising steps; the overview lists each step with its duration.

Live graphs that use Run results

A live graph may use what Run computes, e.g. render Gaussians over a Stable Diffusion image (File → New from Example → Lantern over a Stable Diffusion image: a Composite draws the image onto the swapchain, the Gaussian Splatting renders over it, and a Draw Basics adds axes):

  • Nodes that depend on a staged node, but not on a camera, the swapchain or time, are run-only: they are computed by Run, not every frame. They get the green run ring.

  • Where a live node reads a run-only node’s output, the live graph reads the result of the last Run: one tensor or image that all frames in flight read (klartraum’s TensorElementSinglePath, klartraum::SinglePathImage), never write. Tensors of any element type and images can be kept this way.

  • When such results are needed and there are none yet, Run runs once on its own; after that only when you press Run (F5). Run waits for the frames in flight before it replaces the results. The live graph keeps running and only needs rebuilding when a result’s shape changes.

  • A staged node that depends on a camera, the swapchain or time is an error: it cannot run every frame.

Meta nodes

A meta node stands for a subgraph, its definition. It has the pins and parameters the definition exposes; each stands for a pin or a parameter of an inner node. Definitions can contain meta nodes, to any depth.

  • Built-in definitions come with the studio (Text Encoder, VAE Decoder) and appear in the add-node menu in their group. They are read-only; Duplicate into this graph in the opened definition makes an editable copy that the node then uses.

  • Group (Ctrl+G, or the node’s context menu) turns the selected nodes into a meta node with a new definition stored in the graph file. Links into the group become exposed inputs, one per outside output feeding every inner pin it fed; links out of it become exposed outputs. Ungroup replaces a meta node by its inner nodes.

  • Open (double-click, the context menu or the inspector) shows the definition’s inner graph; the path back is shown above the editor. Changes apply to every node using the definition. In the inspector of an inner node, Meta node interface exposes or hides its pins and parameters; links of instances follow exposed pins by name.

  • A meta node’s parameters are edited in its inspector; Reset returns one to the definition’s value.

Meta nodes are expanded into their inner nodes before the graph is checked and compiled, so they run exactly like the nodes they contain. Problems inside one are reported on the meta node, prefixed with the inner node’s title (“VAE Decoder / Decoder: …”), and its compiled elements belong to it.

Outputs

Node

Description

Implemented by

Runs

Present

Shows the image in the window every frame, stretched to the window’s size.

klartraum::ImageResample, added by the studio

every frame (live only)

Preview

Shows a 1- or 3-channel image tensor, or an image, when the graph is run.

studio (read back, shown as an ImGui texture)

every run

Image File Writer

Writes a 1- or 3-channel image tensor, or an image, to a PNG file when the graph is run.

studio (read back, written with stb_image_write)

every run