Rendering and Shaders #

sys.gl adds GPU shaders to the 2D canvas. With it you run fragment shaders over the whole screen or a rectangle, post-process what the canvas drew (the shader reads it as the u_canvas texture), render offscreen into render targets, and draw 3D meshes with vertex buffers, textures, and depth testing.

The API follows the immediate model of OpenGL: every call acts when you make it. It is the same on desktop, Android, and the web, and shaders are written once in GLSL ES 3.00.

How a frame reaches the screen #

Within a frame, canvas drawing and sys.gl calls run in the order you make them. What ends up on screen depends on the GL draws you make:

• With no GL draw to the screen, the canvas is shown as drawn.
• drawFullscreen covers the whole screen with the shader's output. The canvas appears only through what the shader does with u_canvas, which holds everything drawn on the canvas so far in the frame.
• Region and mesh draws to the screen go over the canvas: Budo puts the canvas on screen first, then draws them on top.

// Compile the program once, during setup.
const program = sys.gl.createProgram('shader.vert', 'shader.frag');

function frame() {
  // Draw the canvas content the shader will read as u_canvas.
  sys.canvas.clear('#05060a');
  sys.canvas.setFillColor('#ffffff');
  sys.canvas.drawText('Shader input', 32, 64, 28);

  // Replace the screen with the shader's output.
  sys.gl.drawFullscreen(program);

  // Keep drawing frames while the shader animates.
  sys.animation.requestFrame(frame);
}

// Start the frame loop.
sys.animation.requestFrame(frame);

While shader passes draw to the screen, end each frame with sys.animation.requestFrame, not waitForInput.

Shader programs #

sys.gl.createProgram(vertexPath, fragmentPath) compiles and links two shader files from the project. sys.gl.createProgramFromBuffer(vertexBytes, fragmentBytes) does the same from source bytes in an ArrayBuffer or typed array, as in the runnable examples on this page. Both return a numeric program handle and throw when compiling or linking fails. sys.gl.destroyProgram(program) frees one you no longer need.

Every shader starts with #version 300 es. Budo runs the same source on WebGL 2 and OpenGL ES 3, and translates it for the desktop OpenGL core profile.

// GLSL ES 3.00, the dialect Budo runs on every platform.
#version 300 es
// A default precision for the floats below.
precision mediump float;

// The fullscreen quad's clip-space corner positions.
in vec2 a_position;
// The matching texture coordinates, (0, 0) at the top-left corner.
in vec2 a_texCoord;
// Hand the texture coordinate to the fragment shader.
out vec2 v_uv;

void main() {
  // Forward the texture coordinate unchanged.
  v_uv = a_texCoord;
  // Emit the corner position.
  gl_Position = vec4(a_position, 0.0, 1.0);
}

// GLSL ES 3.00, the dialect Budo runs on every platform.
#version 300 es
precision mediump float;

// What the canvas drew this frame.
uniform sampler2D u_canvas;
// Seconds since the app started.
uniform float u_time;

// The interpolated texture coordinate from the vertex shader.
in vec2 v_uv;
// The color written for this pixel.
out vec4 fragColor;

void main() {
  // Read the canvas at this pixel.
  vec4 base = texture(u_canvas, v_uv);
  // A horizontal wave that moves with time.
  float wave = 0.5 + 0.5 * sin((v_uv.x + u_time) * 20.0);
  // Modulate the canvas color and keep its alpha.
  fragColor = vec4(base.rgb * (0.75 + wave * 0.25), base.a);
}

Fullscreen and region passes feed the quad through a_position and a_texCoord. Budo sets these uniforms before each draw, when the shader declares them:

• u_canvas, a sampler2D on texture unit 0: the canvas, or the source of the pass.
• u_resolution, a vec2: the size in pixels of the area being drawn (the screen, the render target, or the region).
• u_time, a float: the time in seconds.
• u_offset, a vec2, in region draws only: the region's top-left corner in canvas pixels.

Do not set these yourself, since each draw overwrites them.

Texture coordinates follow the canvas: a_texCoord, and the v_uv it becomes, is (0, 0) at the top-left corner, so texture(u_canvas, v_uv) reads the canvas upright, and so do canvas textures and render targets drawn by these passes. gl_FragCoord starts at the bottom-left, as in any GL shader; vec2(gl_FragCoord.x, u_resolution.y - gl_FragCoord.y) converts it to canvas orientation.

u_canvas holds premultiplied colors. To lay it over an opaque background color bg, write vec4(canvas.rgb + bg * (1.0 - canvas.a), 1.0).

GLSL compilers remove uniforms the shader never reads, and setting a removed uniform throws Uniform not found. Declare only the uniforms your shader uses.

Uniforms and textures #

Every per-program call takes the program handle first:

Setters act at once: a uniform takes its new value now and keeps it for the next draws of that program. To draw the same program twice with different values, set the value before each draw.

// Draw once with a subtle strength.
sys.gl.setUniform1f(program, 'u_strength', 0.2);
sys.gl.drawRegion(program, 40, 40, 160, 120);

// Change the uniform and draw again: the second rectangle is brighter.
sys.gl.setUniform1f(program, 'u_strength', 0.8);
sys.gl.drawRegion(program, 220, 40, 160, 120);

The bind* calls attach a texture to a sampler uniform on a texture unit, also at once. Unit 0 belongs to u_canvas; use units 1 to 7 for your own textures.

Programs, textures, buffers, layouts, and render targets are all plain numbers, so they fit in arrays and tables and pass through generic render code.

Region draws #

sys.gl.drawRegion(program, x, y, width, height) runs a shader inside a rectangle of the screen, over the canvas. Inside the shader, u_resolution is the region's size and u_offset its top-left corner, and v_uv goes from (0, 0) to (1, 1) across the region. A sixth argument, a render target, draws the region into that target instead.

function frame() {
  const input = sys.input.get();
  // Draw the canvas first; the region goes over it.
  sys.canvas.clear('#101116');
  // Send the pointer position, in canvas pixels, to the shader.
  sys.gl.setUniform2f(program, 'u_pointer', input.pointer.x, input.pointer.y);
  // Draw the shader inside a 320 by 180 rectangle.
  sys.gl.drawRegion(program, 32, 32, 320, 180);
  // Keep animating.
  sys.animation.requestFrame(frame);
}

In the fragment shader, u_offset + v_uv * u_resolution gives the pixel's canvas position, ready to compare with the pointer.

Render targets #

A render target is an offscreen texture that passes draw into and later passes read. Use targets to chain effects: draw a scene, blur it, then composite the result.

// Create the targets once, during setup.
const scene = sys.gl.createRenderTarget(512, 512);
const tinted = sys.gl.createRenderTarget(512, 512);

function frame() {
  // Pass 1: draw the scene shader into the scene target.
  sys.gl.drawRegion(sceneProgram, 0, 0, 512, 512, scene);
  // Pass 2: read the scene target and draw the tinted result into another target.
  sys.gl.bindTexture(tintProgram, 'u_input', scene, 1);
  sys.gl.drawRegion(tintProgram, 0, 0, 512, 512, tinted);
  // Pass 3: read the tinted target and cover the screen with it.
  sys.gl.bindTexture(finalProgram, 'u_result', tinted, 1);
  sys.gl.drawFullscreen(finalProgram);
  // Keep drawing frames while passes run.
  sys.animation.requestFrame(frame);
}

Read a target with sys.gl.bindTexture(program, name, target, unit). Do not let a pass read the target it draws into; effects that feed back on themselves, such as trails, alternate between two targets.

A target keeps its pixels from one frame to the next, and there is no separate clear call. A fullscreen pass, or a region pass that covers the whole target, overwrites every pixel; to clear a target, draw such a pass with a shader that outputs the clear color.

When the window size changes, resize targets with sys.gl.resizeRenderTarget(target, width, height) rather than creating new ones. sys.gl.destroyRenderTarget(target) frees a target you no longer need.

Depth and 3D scenes #

Pass true as the third argument to give a target a depth buffer, for 3D rendering:

// A render target with a depth buffer.
const target = sys.gl.createRenderTarget(width, height, true);

The depth buffer serves depth testing only; shaders cannot sample it. The first depth-tested mesh draw into the target in each frame clears its depth, as on the screen, so all the mesh draws of a frame occlude one another. You can draw a whole 3D scene into the target, then post-process it:

// Send the following mesh draws to the offscreen target.
sys.gl.bindRenderTarget(target);
// Draw the scene with depth testing.
sys.gl.drawMesh(sceneProgram, layout, { mode: 'triangles', count: indexCount, depthTest: true });

// Return to the screen.
sys.gl.bindScreen();
// Read the scene in the final fullscreen pass.
sys.gl.bindTexture(finalProgram, 'u_scene', target, 1);
sys.gl.drawFullscreen(finalProgram);

Canvas textures #

A canvas texture is an offscreen Skia canvas backed by a GPU texture. Draw 2D content into it, such as labels, badges, panels, or a glyph atlas, then read it from a shader. sys.graphics.createCanvasTexture(width, height) creates one, available in JavaScript; draw through its canvas property and bind it with sys.gl.bindCanvasTexture.

// A 512 by 256 offscreen canvas backed by a GPU texture.
const ui = sys.graphics.createCanvasTexture(512, 256);
// A program whose fragment shader reads the u_ui sampler.
const post = sys.gl.createProgram('shader.vert', 'post.frag');

// Clear to transparent, then draw light text.
ui.canvas.clear('#00000000');
ui.canvas.setFillColor('#f8fafc');
ui.canvas.drawText('Budo', 32, 96, 42);

function frame() {
  // Bind the texture to u_ui on unit 1; this finishes its pending drawing first.
  sys.gl.bindCanvasTexture(post, 'u_ui', ui, 1);
  // Cover the screen with the shader's output.
  sys.gl.drawFullscreen(post);
  // Keep drawing frames while passes run.
  sys.animation.requestFrame(frame);
}

Because binding finishes the texture's pending drawing, the shader always reads everything drawn into it so far. Its colors are premultiplied, like u_canvas.

The texture's canvas offers a subset of sys.canvas: clear, the shapes (drawRect, drawRoundRect, drawCircle, drawOval, drawLine, drawPoint, drawArc), text (drawText, measureText, measureTextRect, setFont, drawParagraph, measureParagraph), the paint setters, gradients, blend modes, image and color filters, and readPixels. It has no transforms, clips, layers, paths, SVGs, or rich text; draw those on the main canvas.

The handle also has width, height, resize(width, height), and destroy().

Textures #

Load an image file, PNG or JPEG, as a 2D texture and bind it to a sampler:

// Decode a project image and upload it as a texture.
const albedo = sys.gl.loadTexture2D('assets/floor.png');
// Bind it to the sampler u_albedo on unit 1.
sys.gl.bindTexture2D(program, 'u_albedo', albedo, 1);

sys.gl.loadTexture2DFromBuffer(bytes) decodes image bytes you already hold, such as a download:

// Download an image, then decode and upload its bytes.
fetch(url).then(response => {
  photo = sys.gl.loadTexture2DFromBuffer(response.arrayBuffer());
});

For pixels you compute, sys.gl.createTexture2D(width, height, format, pixels) creates a texture in one of three formats: 'rgba8' (4 bytes per pixel), 'rgb8' (3 bytes), or 'r8', a single channel that shaders read as red. pixels is optional; sys.gl.updateTexture2D(texture, x, y, width, height, pixels) replaces a rectangle later, and sys.gl.destroyTexture(texture) frees it.

// A 256 by 1 lookup table: one byte per pixel, read as .r in the shader.
const ramp = new Uint8Array(256);
for (let i = 0; i < 256; i++) ramp[i] = i;
const lut = sys.gl.createTexture2D(256, 1, 'r8', ramp);

Cubemaps load from six images, in the order +X, -X, +Y, -Y, +Z, -Z, and bind with bindTextureCube:

// Load six project images into a cubemap.
const sky = sys.gl.loadTextureCube([
  // X axis faces, positive then negative.
  'sky/+x.png', 'sky/-x.png',
  // Y axis faces, positive then negative.
  'sky/+y.png', 'sky/-y.png',
  // Z axis faces, positive then negative.
  'sky/+z.png', 'sky/-z.png',
]);
// Bind it to the samplerCube u_env on unit 2.
sys.gl.bindTextureCube(program, 'u_env', sky, 2);

sys.gl.loadTextureCubeFromBuffer([px, nx, py, ny, pz, nz]) does the same from six image buffers.

Mesh drawing #

For 3D, describe geometry with vertex buffers and a vertex layout, then draw it with drawMesh.

// Create a vertex buffer with three 3D positions.
const vbo = sys.gl.createBuffer('vertex', new Float32Array([
  // Bottom-left vertex.
  -0.5, -0.5, 0.0,
  // Bottom-right vertex.
   0.5, -0.5, 0.0,
  // Top vertex.
   0.0,  0.5, 0.0
]));

// A vertex layout describes how buffers feed the shader's attributes.
const layout = sys.gl.createVertexLayout();
// Location 0 reads 3 floats per vertex, not normalized, 12 bytes apart, from offset 0.
sys.gl.setAttribute(layout, 0, vbo, 3, 'float', false, 12, 0);

// Draw the three vertices as one triangle.
sys.gl.drawMesh(program, layout, {
  // Use triangle primitives.
  mode: 'triangles',
  // Start at the first vertex.
  first: 0,
  // Draw three vertices.
  count: 3
});

Budo binds attributes with these names to fixed locations, so a shader can use them without layout(location = N): a_position at 0, a_texCoord or a_uv at 1, a_normal at 2, a_color at 3, and a_tangent at 4. For other names, use layout(location = N) in the shader, or ask with sys.gl.getAttribLocation(program, name).

createBuffer(kind, data, usage) takes 'vertex' or 'index', a typed array, and a usage hint: 'static' (the default), 'dynamic', or 'stream'. updateBuffer(buffer, data, byteOffset) rewrites part of a buffer, for geometry that changes. setAttribute(layout, location, buffer, size, type, normalized, stride, offset, divisor) accepts the types 'float', 'byte', 'ubyte', 'short', 'ushort', 'int', and 'uint'; a divisor of 1 advances the attribute once per instance. To draw indexed geometry, attach an index buffer with setIndexBuffer(layout, buffer, 'u16') (or 'u32'), and count then counts indices.

drawMesh options:

• mode: 'triangles' (the default), 'triangle_strip', 'triangle_fan', 'lines', 'line_strip', or 'points'.
• first and count: the range of vertices, or indices, to draw.
• depthTest and depthWrite: both true by default.
• cull: 'none' (the default), 'back', or 'front'.
• blend: 'alpha' (the default), 'add', or 'none'.
• instanceCount: the number of instances to draw, 1 by default.
• target: a render target; by default, the one bound with bindRenderTarget, otherwise the screen. -1 forces the screen.

The first mesh draw to the screen in a frame puts the canvas on screen and clears the depth buffer, so the canvas is the background of your 3D scene. For 2D content over 3D, such as a HUD, draw the scene into a depth render target and composite it with the canvas in a fullscreen pass.

For the matrix math of 3D (perspective, look-at, rotations), sys.math works in place on Float32Array storage; see the API reference.

Passes over other sources #

The plain draw calls cover most needs. Three variants give finer control over where a pass reads and writes:

• sys.gl.drawFullscreenImmediate(program, source) runs a fullscreen pass that reads source, a texture handle or a canvas texture, as u_canvas, and draws into the currently bound framebuffer.
• sys.gl.drawRegionImmediate(program, x, y, width, height, target) takes the same arguments as drawRegion. Like it, it draws over the canvas when it targets the screen.
• sys.gl.drawMeshImmediate(program, layout, options) takes the same arguments as drawMesh.

Errors #

Every sys.gl call that fails throws an error whose message explains why: a shader that does not compile (with the compiler's message), a missing file, an unknown uniform, or an invalid handle. sys.gl.getLastError() returns the last error message, or an empty string.

try {
  // Compile and link a program from two files.
  const program = sys.gl.createProgram('missing.vert', 'missing.frag');
} catch (error) {
  // Print the compiler or file-loading error.
  console.log('Could not create shader: ' + error);
}

In shader-heavy tools, show these messages in your own interface so a typo in a shader is visible at once.

Performance #

Create programs, render targets, vertex layouts, buffers, and textures during setup, not in every frame, and resize targets instead of recreating them. Reuse typed arrays for geometry that changes, uploaded with updateBuffer. Keep readPixels out of the frame loop. A few clear passes in a well-named sequence of draw calls are easier to follow, and often faster, than a general render abstraction built too early.