Introduction

STARK is a key-value based, B-tree implemented local database. It is specifically designed for mobile applications, desktop applications, and offline video games. With its speed, lightness, and easy syntax, STARK stands apart from other databases.

At its core, STARK is a C library with a C++ wrapper. It stores data on disk using a B-tree index and a flat data file, providing sub-microsecond lookups with a footprint of only 2–5 MB of RAM and under 1 MB on disk.

Architecture

STARK uses a two-file storage model. When you open a database named "mydb", two files are created:

The B-tree index keeps keys sorted, so range scans and existence checks are fast. The data file uses an append-only strategy, which makes writes safe and crash-resilient.

Key Types

STARK supports three kinds of keys, each stored and optimized separately:

ACID Properties

STARK guarantees ACID compliance — the four properties that ensure database reliability even in the event of crashes, power failures, or errors.

Why this matters for games: A player's progress must never be partially saved. If a power failure hits mid-save, STARK either fully applies the changes or leaves the database exactly as it was before.

File Format Details

Understanding the file format helps with backup strategies and debugging. You can safely back up both .idx and .dat files together. Never move or delete one without the other.

CLI Basics

The STARK CLI lets you interactively create and manage databases without writing any code. Open or create a database by running stark_cli with a filename.

COPY# Open or create a database in the current directory
stark_cli mygame

# If stark_cli is in your PATH, you can run it from anywhere:
cd ~/my_project/data
stark_cli savefile

# The prompt will change to show you're inside STARK:
stark> help
# Shows all available commands

When you first open a filename, STARK creates mygame.idx and mygame.dat in the current directory. If the files already exist, STARK opens and loads them.

General Commands

COPYstark> stats
# Output:
#   Keys:        15
#   Data size:   2048 bytes
#   Index pages: 1
#   B-tree height: 2

Numeric Key Commands

Numeric keys are 32-bit unsigned integers (0 to 4,294,967,295). They are ideal for IDs — save slots, player IDs, inventory item IDs. Values can be strings or numbers, stored as raw bytes.

COPY# Store a save slot
stark> addn 0 "slot_empty"
stark> addn 1 "Hero - Level 12"
stark> addn 999 "1000 gold coins"

# Read them back
stark> getn 1
# Output: Hero - Level 12

stark> getn 5
# Output: key not found

# Check existence
stark> existsn 1
# Output: true

stark> existsn 5
# Output: false

# Delete a key
stark> deln 0
stark> existsn 0
# Output: false

# Overwrite a key (same as add)
stark> addn 1 "Hero - Level 15"
stark> getn 1
# Output: Hero - Level 15

String Key Commands

String keys let you use text-based lookups. The string is hashed internally for storage, but you interact with it by its original text. This is useful for config keys, usernames, or named settings.

COPY# Store settings
stark> adds "username" "player_one"
stark> adds "theme" "dark"
stark> adds "music_volume" "85"

# Read them back
stark> gets "username"
# Output: player_one

stark> gets "fullscreen"
# Output: key not found

# Check and delete
stark> exist_str "theme"
# Output: true

stark> dels "theme"

Type Commands — User-Defined Schemas

Types let you define structured records — like a lightweight schema. Each type has named fields with fixed data types. This is how you store game entities, inventory items, or any multi-field data.

Supported Field Types

COPY# Define a player type
stark> define player {
    name    string(32)
    hp      int
    level   int
    gold    int
    class   string(16)
}

# View the type definition
stark> desc player
# Output:
#   Type: player
#   Fields:
#     name   = string(32)
#     hp     = int
#     level  = int
#     gold   = int
#     class  = string(16)

# Define an item type
stark> define item {
    name     string(64)
    damage   int
    type     string(16)
}

# Remove a type (deletes all records too)
stark> undefine item

Typed Data — Adding and Retrieving Records

Once a type is defined, you can add, retrieve, and update records. Each record is identified by a numeric key and contains field values in field=value format.

COPY# Add players (keys 1 and 2)
stark> add player 1 name="Hero" hp=100 level=5 gold=250 class="warrior"
stark> add player 2 name="Mage" hp=80 level=7 gold=450 class="wizard"

# Get a player
stark> get player 1
# Output: name="Hero" hp=100 level=5 gold=250 class="warrior"

# Non-existent record
stark> get player 3
# Output: player:3 not found

# Update a record (same key overwrites all fields)
stark> add player 1 name="Hero" hp=85 level=6 gold=300 class="warrior"

# Verify update
stark> get player 1
# Output: name="Hero" hp=85 level=6 gold=300 class="warrior"

# Check stats after adding data
stark> stats
# Output shows: total keys, data file size, index size, etc.

Transactions

Transactions group multiple operations into a single atomic unit. This is critical when you need to update multiple values that must stay consistent — like a player winning a fight and gaining both XP and loot.

COPY# Setup a player
stark> define player { id int hp int level int gold int }
stark> add player 1 id=123 hp=100 level=0 gold=50

# === SUCCESSFUL TRANSACTION ===
# Player wins a fight — update atomically
stark> begin
stark> add player 1 id=123 hp=45 level=1 gold=150
stark> commit
# All changes saved atomically

# === ROLLBACK EXAMPLE ===
# Suppose a transaction goes wrong
stark> begin
stark> add player 1 id=123 hp=0 level=0 gold=0
stark> rollback
# Player data restored to: hp=45, level=1, gold=150

# Verify rollback worked
stark> get player 1
# Output: id=123 hp=45 level=1 gold=150

Best practice: Always wrap related changes in a transaction. If you're updating a player's HP, XP, and inventory after a battle, put all three updates in one begin/commit block.

Complete Walkthrough — Building a Game Save System

This walkthrough shows how to use STARK to build a complete game save system. Follow along step by step.

COPY# ================================================
# STEP 1: Open the database
# ================================================
stark_cli mygame
stark>

# ================================================
# STEP 2: Define types for your game
# ================================================

# Define the player type
stark> define player {
    name    string(32)
    hp      int
    max_hp  int
    level   int
    xp      int
    gold    int
    class   string(16)
}

# Define the inventory type
stark> define item {
    name     string(64)
    damage   int
    defense  int
    type     string(16)
    # type: weapon, armor, potion, quest
}

# ================================================
# STEP 3: Store save data
# ================================================

# Save the main character (key = 1)
stark> add player 1 name="Aldric" hp=120 max_hp=120 level=8 xp=4500 gold=1250 class="warrior"

# Save inventory items (keys 1-3)
stark> add item 1 name="Iron Sword" damage=15 defense=0 type="weapon"
stark> add item 2 name="Steel Shield" damage=0 defense=20 type="armor"
stark> add item 3 name="Health Potion" damage=0 defense=0 type="potion"

# Store save metadata using string keys
stark> adds "save_version" "1.0"
stark> adds "play_time" "04:23:15"
stark> adds "chapter" "3"

# Use numeric keys for save slots
stark> addn 1 "Aldric - Level 8 - Chapter 3"
stark> addn 2 "Empty Slot"
stark> addn 3 "Empty Slot"

# ================================================
# STEP 4: Use a transaction for battle updates
# ================================================

# Player fights a dragon — wins but takes damage
stark> begin

# Player takes 40 damage
stark> add player 1 name="Aldric" hp=80 max_hp=120 level=8 xp=4500 gold=1250 class="warrior"

# Player gains 500 XP and 200 gold from the kill
stark> adds "dragon_kills" "1"

# Commit — all changes saved together
stark> commit

# ================================================
# STEP 5: Verify your save data
# ================================================

stark> get player 1
# Output: name="Aldric" hp=80 max_hp=120 level=8 xp=4500 gold=1250 class="warrior"

stark> get item 1
# Output: name="Iron Sword" damage=15 defense=0 type="weapon"

stark> gets "chapter"
# Output: 3

stark> desc player
# Shows all player fields with types

stark> stats
# Shows total keys, file sizes, B-tree info

# ================================================
# STEP 6: Save and exit
# ================================================

stark> exit
# All data persisted to mygame.idx and mygame.dat

Using STARK in C++

The C++ wrapper (stark::Database) provides a modern, exception-safe interface over the C API. After installation, just #include <stark.hpp> and link with -lstark.

COPY// main.cpp
#include <stark.hpp>
#include <iostream>

int main() {
    // Open database (creates "save.idx" and "save.dat")
    stark::Database db("save");

    // Save data
    db.add(1, "Hero");
    db.add(2, "100 HP");

    // Load data
    std::cout << db.get(1) << std::endl;  // Prints: Hero

    // Get with default value
    std::string val = db.get(99, "not found");  // "not found"

    return 0;
}

Lifecycle — Opening and Closing

// Open existing or create new
stark::Database db("mydb");

// Throws stark::Error if
// the database cannot be opened

// Move is supported
stark::Database db2 = std::move(db);

// Copy is deleted
// Database db3 = db2; // ERROR

// Destructor auto-syncs
// and closes the database

The database automatically calls sync() and stark_close() in the destructor. You don't need to manually close it. If you need explicit control, call db.sync() to flush changes to disk.

Numeric Key Methods

All numeric key operations throw stark::Error on failure. get() returns an empty string if the key is not found.

// Insert or update (key is uint32_t)
db.add(1, "Hero");
db.add(42, "anything");

// Get — returns "" if not found
std::string v = db.get(1);

// Get with default fallback
std::string v2 = db.get(99,
    "default value");

// Remove — returns true if deleted,
// false if key didn't exist
bool removed = db.remove(1);

// Check existence
if (db.exists(1)) {
    // key exists
}

String Key Methods

String keys use std::string arguments. Internally, keys are hashed, but you interact with them using the original text.

// Insert or update
db.put_str("username", "player_one");
db.put_str("theme", "dark");

// Get — returns "" if not found
std::string v = db.get_str(
    "username");

// Remove by string key
bool ok = db.remove_str(
    "username");

// Check existence
if (db.exists_str("theme")) {
    // key exists
}

Typed Data Methods

Define schemas with stark::Field objects, then add and retrieve structured records. Field type constants: TYPE_INT (1) and TYPE_STRING (2).

using namespace stark;

// Define a type with fields
db.define_type("player", {
    Field("name",  TYPE_STRING, 32),
    Field("hp",    TYPE_INT),
    Field("level", TYPE_INT),
});

// Remove a type (returns
// true if it existed)
bool removed = db.undefine_type(
    "player");

// Add a typed record
db.add_typed("player", 1,
    "name=Hero hp=100 level=5");

// Get typed record (returns "" if
// not found)
std::string v = db.get_typed(
    "player", 1);

// Describe type (returns vector
// of Field objects)
auto fields = db.describe_type(
    "player");

Field struct: Field(name, type, size) — name is the field name, type is TYPE_INT or TYPE_STRING, size is the string buffer size (default 4, ignored for int).

Transaction Methods

// Start transaction
db.begin();

// Make changes
db.add(1, "updated");
db.put_str("key", "val");

// Commit (atomic write)
db.commit();

// Or rollback (discard)
// db.rollback();

// Check if in transaction
if (db.in_transaction()) {
    // inside begin/commit block
}

Cursor / Iteration (C API)

The C API provides a cursor for iterating over all keys in order. The C++ wrapper does not yet expose this directly, but you can use it via the C functions by linking to stark.h.

Cursor Functions

COPY// C API cursor example
#include <stark.h>

stark_db_t* db = stark_open("mydb", 0);
stark_cursor_t* cur = stark_cursor_create(db);

if (stark_cursor_first(cur) == STARK_OK) {
    do {
        uint32_t key;
        char buf[4096];
        size_t size = sizeof(buf);
        stark_cursor_get(cur, &key, buf, &size);
        printf("key=%u value=%s\n", key, buf);
    } while (stark_cursor_next(cur) == STARK_OK);
}

stark_cursor_destroy(cur);
stark_close(db);

Using STARK in C

The C API provides direct access to all STARK functionality. Include <stark.h> and link with -lstark.

COPY// main.c
#include <stark.h>
#include <stdio.h>

int main() {
    // Open database
    stark_db_t* db = stark_open("save", 0);
    if (!db) {
        printf("Failed to open database\n");
        return 1;
    }

    // Add a key
    stark_add(db, 1, "Hero");

    // Get the value
    char buf[256];
    size_t size = sizeof(buf);
    stark_get(db, 1, buf, &size);
    printf("Value: %s\n", buf);

    // Close database
    stark_close(db);
    return 0;
}

Result Codes

All C API functions return a result code indicating success or the type of error that occurred.

Using STARK in Python

STARK now provides a complete Python binding (v1.2.0) using ctypes — no compilation needed. The Python API mirrors the C++ interface and adds ML-ready features: binary array storage with NumPy zero-copy access and Arrow column layout for Pandas/Polars DataFrames.

Installation

COPY# 1. Build the C library first
cd starkdb && mkdir build && cd build
cmake .. && make

# 2. Install the Python package
cd ../bindings/python
pip install -e .

# 3. Optional: install ML dependencies
pip install numpy pandas

Lifecycle — Opening, Closing, and Context Manager

# Open existing or create new
db = starkdb.Database("mydb")

# Creates mydb.idx + mydb.dat

# Close (flushes to disk)
db.close()

# Auto-closes on exit
with starkdb.Database("mydb") as db:
    db.put(1, b"auto-saved")
    # db.close() called automatically

The database automatically calls sync() and stark_close() when the context manager exits or close() is called. If you need explicit control, call db.sync() to flush changes to disk immediately.

Numeric Key Methods

Numeric keys are 32-bit unsigned integers. Values are bytes. The API supports dict-style syntax.

# Insert/update
db[1] = b"Hero"

# Get by key
val = db[1]
# b'Hero'

# Delete
del db[1]

# Check existence
if 1 in db:
    # exists

# Insert/update
db.put(1, b"Hero")
db.put(2, "variable".encode())

# Get as bytes
val = db.get(1)

# Get as string
text = db.get_text(1)

# Delete (returns True if existed)
db.delete(1)

# Check existence
db.exists(1)

Batch Operations

COPY# Insert multiple keys atomically
db.put_batch({1: b"A", 2: b"B", 3: b"C"})

# Get multiple keys at once
results = db.get_batch([1, 2, 3, 999])
# Returns {1: b'A', 2: b'B', 3: b'C'}

String Key Methods

String keys use text-based lookups. Internally, keys are hashed using the djb2 algorithm and stored in the B-tree.

# Insert or update
db.put_str("player", "Hero")
db.put_str("theme", "dark")

# Get as bytes
val = db.get_str("player")
# b'Hero'

# Get as string
text = db.get_str_text("player")
# 'Hero'

# Delete string key
db.delete_str("theme")

# Check existence
if db.exists_str("player"):
    # key exists

Iteration — Scanning All Keys

The Python API provides full iteration over the B-tree. Keys are returned in ascending order.

# Iterate over all keys
for key in db:
    print(key)

# Iterate over (key, value) pairs
for key, value in db.items():
    print(key, value)

# Get all keys as a list
keys = db.keys()

# Get all values as a list
values = db.values()

# Count keys
count = db.count()

Internally, this uses the B-tree cursor to traverse the tree in sorted order. The C cursor API (stark_cursor_create, stark_cursor_first, stark_cursor_next) is also available if you need finer control.

Typed Data — User-Defined Schemas

Define structured types with int and string fields, then add/get records using field=value syntax — just like the CLI and C++ API.

# Define a player type
db.define_type("player", [
    {"name": "id",    "type": 1, "size": 4},
    {"name": "hp",    "type": 1, "size": 4},
    {"name": "level", "type": 1, "size": 4},
])

# Remove a type
db.undefine_type("player")

# Add a typed record
db.add_typed("player", 1,
    "id=100 hp=50 level=10")

# Get typed record
result = db.get_typed("player", 1)
# 'id=100 hp=50 level=10'

Transaction Methods

Transactions group multiple operations into an atomic unit. All writes are buffered until commit(), or discarded on rollback().

# Start transaction
db.begin()

# Make changes
db.put(1, b"updated")
db.put_str("key", "val")

# Commit (atomic write)
db.commit()

# Or rollback (discard)
# db.rollback()

# Check if in transaction
if db.in_transaction():
    print("inside transaction")

Binary Array Storage — NumPy Zero-Copy

Direct Binary Storage: Arrays are stored as contiguous byte blocks with no JSON, string, or formatting overhead. This enables sub-microsecond reads and direct memory sharing with ML libraries.

How It Works

When you call store_array(), the array's raw bytes are written to the data file as a continuous block with a small header (dtype + count + element size). When you call load_array_zerocopy(), it returns the database's in-memory page address directly — NumPy wraps a view around it without copying.

import numpy as np
import starkdb

db = starkdb.Database("ml_data")

# Store float32 embeddings
embeddings = np.array(
    [0.1, 0.5, -0.3, 0.8],
    dtype=np.float32)
db.store_array(1, embeddings)

# Load back (safe copy)
loaded = db.load_array(1)

# Store int32 labels
labels = np.array(
    [0, 1, 0, 1],
    dtype=np.int32)
db.store_array(2, labels)

db.close()

# Zero-copy — no memory copy!
# Returns a numpy array backed by DB memory
# WARNING: Pointer invalid after DB writes
fast = db.load_array_zerocopy(1)

# Use directly in ML
prediction = model.predict(fast)

# Copy only when needed
safe_copy = fast.copy()

Why This Matters for ML

• Direct Binary Storage: Vector embeddings and model inputs are stored as raw byte arrays — no parsing overhead
• Zero-Copy Buffer Export: PyTorch, TensorFlow, and JAX can read millions of numbers per second directly from database memory
• Batch Processing: Store 1000s of embedding vectors and load them instantly for training loops

COPY# Store 1000 embedding vectors (768-dim each) for batch ML
db = starkdb.Database("vectors")
for i in range(1000):
    vec = np.random.randn(768).astype(np.float32)
    db.store_array(1000 + i, vec)

# Load one embedding with zero-copy
vec = db.load_array_zerocopy(1500)
# shape (768,) float32 — zero copy
db.close()

Arrow Column Layout — DataFrame Integration

Arrow Column Layout: Table-like data is stored column-by-column (not row-by-row), following Apache Arrow's columnar format. This allows Pandas and Polars to reconstruct DataFrames instantly without reformatting.

How It Works

When you call store_dataframe(), STARK stores:

1. Schema metadata — JSON describing column names and data types
2. Column data — Each column stored as a contiguous binary block

When you call load_dataframe(), STARK reads the schema, loads each column's data, and reconstructs the DataFrame column-by-column — matching Arrow's zero-copy philosophy.

import pandas as pd
import numpy as np
import starkdb

db = starkdb.Database("datasets")

# Create a DataFrame
df = pd.DataFrame({
    "name": ["Alice",
              "Bob",
              "Charlie"],
    "age": np.array(
        [25, 30, 35],
        dtype=np.int32),
    "score": np.array(
        [95.5, 87.2, 91.8],
        dtype=np.float32),
})

# Store as Arrow columns
db.store_dataframe("users", df)

db.close()

# Load back as DataFrame
loaded_df = db.load_dataframe(
    "users")

print(loaded_df.head())
#     name  age  score
# 0  Alice   25   95.5
# 1    Bob   30   87.2
# 2 Charlie  35   91.8

# Directly usable with pandas:
mean_score = loaded_df[
    "score"].mean()
print(mean_score)
# 91.5

db.close()

Arrow compatibility: While STARK does not output standard Arrow IPC format files, the column-oriented storage layout is compatible with Arrow's zero-copy philosophy. Each column is stored as a typed contiguous buffer with schema metadata, enabling Pandas/Polars to reconstruct DataFrames with minimal overhead.

Utilities, Statistics & Error Handling

# Flush to disk
db.sync()

# Get database statistics
stats = db.stats()
print(stats)
# {'keys_count': 10, 'btree_height': 1,
#  'data_size': 1024, 'page_count': 2}

# Get last error message
err = db.error()

# Exception classes:
#   StarkDBError — generic error
#   StarkDBNotFound — key not found
#   StarkDBFull — storage full
#   StarkDBClosed — database closed

Complete API Reference

Architecture Notes

• Direct Binary Storage: Arrays stored as contiguous bytes (no JSON/string overhead). Perfect for ML vector embeddings.
• Zero-Copy Buffer Export: load_array_zerocopy() returns a NumPy array backed by the DB's memory — PyTorch/TensorFlow can read directly from disk cache.
• Arrow Column Layout: Data stored column-by-column with schema headers, ready for Pandas/Polars DataFrame reconstruction.
• ctypes Implementation: No Python C extension compilation — pure Python FFI through ctypes.