Data, Files, and Network #

This page covers the four ways a Budo app reaches beyond its own code: SQLite for structured data, sys.files for bundled and saved files, fetch for HTTP, and sys.net.udp for datagrams. Each one works the same way on desktop, web, and Android, within limits that keep your app portable: file paths stay inside two known folders, and network access is off until app.json allows it.

SQLite persistence #

sys.db.open(name) opens a SQLite database, creating name.db if it does not exist yet. On desktop the file lives in the project directory; on web it persists in browser storage. Names may contain only letters, digits, underscores, and hyphens.

// Open notes.db, creating it on first use.
const db = sys.db.open('notes');

// Create the table once; later runs leave it untouched.
sys.db.execute(db, `
  CREATE TABLE IF NOT EXISTS notes (
    id INTEGER PRIMARY KEY,
    title TEXT NOT NULL,
    done INTEGER NOT NULL DEFAULT 0
  )
`);

execute(db, sql) runs a statement with no parameters and no results, which suits schema changes. For statements that change data, use run(db, sql, ...params): each ? in the SQL takes the next parameter, so values never need to be pasted into the SQL string. run returns the number of changed rows, and lastInsertId(db) returns the id of the row just inserted.

// Insert a note; the ? placeholder receives the title safely.
sys.db.run(db, 'INSERT INTO notes (title) VALUES (?)', 'Sketch the menu');
// Read the id SQLite assigned to the new row.
const id = sys.db.lastInsertId(db);
// Mark it done; true is stored as the integer 1.
sys.db.run(db, 'UPDATE notes SET done = ? WHERE id = ?', true, id);

query(db, sql, ...params) returns the matching rows as an array of objects keyed by column name.

// Fetch every note, newest first.
const rows = sys.db.query(db, 'SELECT id, title, done FROM notes ORDER BY id DESC');
// Each row is an object with one property per selected column.
for (const row of rows) {
  console.log(row.id + ': ' + row.title);
}

Close a handle with sys.db.close(db) when the app no longer needs that database.

// Release the database handle.
sys.db.close(db);

A failing statement throws, in JavaScript and in Lua. The exception carries the SQLite message, and sys.db.getError() returns the last one.

Files #

sys.files sees one virtual root with two folders:

• assets/ holds the files bundled with your app. It is read-only.
• files/ is the app's own persistent storage. It is the only place an app can write.

Every sys.files path starts with one of those two folders. Absolute paths and .. are rejected, so an app can never wander outside them.

// Read a bundled text file.
const level = sys.files.readText('assets/levels/intro.json');
// Read a bundled binary file as an ArrayBuffer.
const bytes = sys.files.readBinary('assets/images/title.png');

// Save settings into persistent storage; parent folders are created.
sys.files.writeText('files/settings.json', JSON.stringify({ volume: 0.8 }));

// Check that the file exists before reading it back.
if (sys.files.exists('files/settings.json')) {
  // size() reports the length in bytes.
  console.log(sys.files.size('files/settings.json') + ' bytes');
}

writeBinary(path, data) saves an ArrayBuffer or typed array, replacing any existing file, and appendBinary(path, data) adds to the end of one. Both, like writeText, return the saved path. Lua has writeText and writeBinary; appendBinary is JavaScript only.

list(path) returns the entries of a folder, each with a name, a type ("file" or "directory"), and a size for files. Hidden files are left out. list() with no argument returns the two root folders. exists(path) is true only for regular files; use isDirectory(path) for folders.

// List the bundled levels folder.
for (const entry of sys.files.list('assets/levels')) {
  // Print each entry's type and name.
  console.log(entry.type + ': ' + entry.name);
}

sys.assets is a read-only shortcut for the assets/ folder. It offers list, readText, readBinary, exists, isDirectory, and size, with paths written without the assets/ prefix.

// These two calls read the same bundled file.
const a = sys.files.readText('assets/levels/intro.json');
const b = sys.assets.readText('levels/intro.json');

Where the folders live #

On desktop, both folders resolve to the project directory, and budo run --file-root <path> moves files/ elsewhere. On web, assets/ holds the files preloaded with the page, and files/ persists in browser storage (IndexedDB); each write also triggers a browser download. On Android, assets/ reads the app bundle, and files/ is the app's own external-files directory.

On Android, an app that needs broad shared storage can set "filesystem": true in app.json. The packager then requests the all-files storage permission, and files/ points at shared storage once the user grants it. Most apps do not need this: the pickers below reach user files without it.

{
  "filesystem": true
}

Large files, a block at a time #

readText and readBinary load a whole file into memory. For a video, a sound library, a long log, or a model, open the file with sys.files.openRead(path) and read it a block at a time instead.

// Open the file for block reads; null means it could not be opened.
const file = sys.files.openRead('files/big.log');
if (file === null) throw new Error(sys.files.getError());

// Count the bytes, 64 KiB at a time.
let total = 0;
for (;;) {
  // read() returns an ArrayBuffer: empty at the end, null on a read error.
  const block = sys.files.read(file, 65536);
  if (block === null) throw new Error(sys.files.getError());
  if (block.byteLength === 0) break;
  total += block.byteLength;
}

// getReadInfo() reports { size, position } in bytes.
console.log(total, sys.files.getReadInfo(file));
// Jump back to the start to read the file again.
sys.files.seek(file, 0);
// Release the handle.
sys.files.closeRead(file);

read(handle, maxBytes) returns at most maxBytes, 64 KiB by default and 16 MiB at most. seek(handle, byteOffset) moves the next read and returns false for an offset outside the file. Up to 32 files can be open for block reads at once; close each one with closeRead. Block reads are available in JavaScript only.

Letting the user pick a file #

sys.files.pickText(callback, extension) lets the user choose a text file from anywhere they like: their documents, Downloads, an SD card, or a cloud provider. Because the user picks the file, no storage permission is needed.

// Ask the user for a .json file.
const opened = sys.files.pickText((file, error) => {
  // On success, file is { name, text }.
  if (file) console.log(file.name + ': ' + file.text.length + ' characters');
  // On cancel or failure, error explains why.
  else console.log(error);
}, '.json');

// false means no picker is available or one is already open.
if (!opened) console.log('No file picker available');

The picker is a native dialog on desktop, a file input on web, and the system document picker on Android. The optional extension filters the picker where the platform supports it and is always checked against the chosen file's name. Files are limited to 8 MiB.

sys.files.saveText(name, text, callback) is the matching Save As: the user chooses where the text goes, and the callback receives null on success or an error message.

// Offer to save a backup wherever the user chooses.
sys.files.saveText('backup.json', JSON.stringify({ volume: 0.8 }), (error) => {
  // error is null when the file was saved.
  if (error) console.log(error);
});

pickText and saveText are available in JavaScript only.

HTTP fetch #

Network access is off until app.json lists the domains the app may reach. A listed domain also covers its subdomains.

{
  "network": ["api.example.com", "cdn.example.com"]
}

"network": "*" allows every domain. It is convenient while prototyping; narrow it to real domains before you ship.

JavaScript gets the familiar global fetch, also available as sys.net.fetch. It returns a promise of a response.

// Load scores from an allowed domain.
async function loadScores() {
  // Send the request; the promise rejects on network or policy errors.
  const response = await fetch('https://api.example.com/scores');
  // Treat any non-2xx status as a failure.
  if (!response.ok) {
    throw new Error('HTTP ' + response.status);
  }
  // Parse the body as JSON.
  return await response.json();
}

The options object accepts method, headers, and a body given as a string or an ArrayBuffer.

// Post a JSON event to an allowed domain.
const response = await fetch('https://api.example.com/events', {
  // Choose the HTTP method.
  method: 'POST',
  // Declare the body format.
  headers: { 'Content-Type': 'application/json' },
  // Serialize the payload.
  body: JSON.stringify({ name: 'start' })
});

The response has the browser shape: ok, status, statusText, url, redirected, type, bodyUsed, headers, and the body readers text(), json(), and arrayBuffer(). As in browsers, the body can be read only once. On web, requests also follow the browser's CORS rules.

response.headers matches names case-insensitively and offers get, has, forEach, entries, keys, and values.

// Read one header; the name is case-insensitive.
const type = response.headers.get('Content-Type');
// Check for a header before using it.
if (response.headers.has('ETag')) {
  console.log('ETag: ' + response.headers.get('ETag'));
}
// Visit every header.
response.headers.forEach((value, name) => {
  console.log(name + ': ' + value);
});

In Lua, fetch takes a callback, called with the response table or with an error message. The global json_parse turns a JSON string into a Lua table.

-- Fetch scores from an allowed domain.
fetch('https://api.example.com/scores', function(response, error)
  -- error is set for network and policy failures.
  if error then
    console.log('Network error: ' .. error)
    return
  end
  -- Parse the JSON body into a Lua table.
  local data = json_parse(response.body)
  -- Log the HTTP status.
  console.log('Status: ' .. response.status)
end)

Lua also accepts an options table before the callback: fetch(url, options, callback). The response table holds status, statusText, ok, url, redirected, headers, body, and bodyLen.

UDP sockets #

sys.net.udp sends and receives datagrams on desktop and Android. Browsers offer no raw UDP, so it is unavailable on web; check sys.capabilities.udp.available before using it.

// Bind to port 0 to let the system choose a free port.
const socket = sys.net.udp.bind(0);
// bind() returns -1 when the port cannot be opened.
if (socket < 0) throw new Error('UDP bind failed');
// Read the port that was chosen.
console.log('Listening on ' + sys.net.udp.getPort(socket));

// Handle incoming datagrams.
sys.net.udp.onMessage(socket, (message) => {
  // message.host and message.port identify the sender.
  console.log('From ' + message.host + ':' + message.port);
  // message.data is an array of byte values.
  console.log('Bytes: ' + message.data.length);
});

// Send the bytes of "Hi" to a local endpoint.
sys.net.udp.send(socket, '127.0.0.1', 5000, [0x48, 0x69]);

send takes the payload as an array or typed array of byte values and returns true once the datagram is sent. Incoming datagrams are delivered once per frame. Each socket has one message callback: setting another replaces it, and null removes it. Close the socket when you are done.

// Release the socket.
sys.net.udp.close(socket);

Network MIDI sessions run over UDP but have their own API in sys.midi; see Audio and MIDI.

How errors are reported #

All Budo APIs follow one rule. Mistakes in your code throw: a wrong argument, an unsafe path, invalid SQL. Conditions that a user or a device can cause return a plain value instead: a missing file returns null, a busy port returns -1, a missing feature returns false. Namespaces that can fail this way offer getError() with a readable message.

// Try to read a file that may not exist.
const text = sys.files.readText('files/missing.txt');
// A missing file returns null rather than throwing.
if (text === null) {
  // getError() explains what went wrong.
  console.log(sys.files.getError());
}

Fix what throws during development; handle what returns null, false, or -1 in your app.