Introduction

What is sciencekit-lua?

sciencekit is a Rust machine-learning library reimplementing scikit-learn’s estimators and transformers with native numerical kernels. neurono-ml/sciencekit-lua is the standalone Rust crate (sciencekit_lua) that packages sciencekit as a loadable Lua 5.4 module named sciencekit, consumed in scripts with require("sciencekit").

local sciencekit = require("sciencekit")

local features = { { 0.0, 0.0 }, { 1.0, 0.0 }, { 0.0, 1.0 }, { 2.0, 3.0 } }
local targets = { 1.0, 3.0, 4.0, 14.0 }

local model = sciencekit.linear_regression()
  :include_intercept(true)
  :backend("faer")
  :fit(features, targets)

local predictions = model:predict(features)   -- table of one-element rows
local r_squared = model:score(features, targets)

Embedded Lua 5.4 — no system Lua required

Native scripting bindings often fail at build time because the host machine lacks liblua, lua.h, or a matching pkg-config definition. sciencekit_lua eliminates that dependency by configuring mlua 0.10 with features = ["lua54", "vendored"]:

• Compiled from source: Cargo compiles the official Lua 5.4 C sources directly into the Rust build using a standard C compiler (cc). No system liblua, header package, or pkg-config file is needed to run cargo build --release or cargo test.
• Hand-written module entry point: Because mlua’s #[mlua::lua_module] attribute macro (module feature) refuses to coexist with vendored, src/lib.rs exports luaopen_sciencekit explicitly as an extern "C-unwind" function backed by mlua::Lua::entrypoint1. When a host lua5.4 interpreter loads the shared library, luaopen_sciencekit populates the module table against the caller’s lua_State.
• Self-contained Rust tests: The crate produces both cdylib (for require("sciencekit") in external Lua 5.4 processes) and rlib (so cargo test spins up fresh embedded Lua 5.4 VMs in memory without shelling out to a lua binary).

How the module wraps the Rust core

Rather than re-implementing estimator configuration, parameter validation, fitting, or binary serialization in a binding-specific layer, sciencekit_lua links sciencekit_ffi from the main sciencekit repository in-process via a Cargo git dependency. Every Lua call traverses the exact same C ABI handles, descriptors, and status codes that a C or Go consumer sees.

<line x1="191" y1="80" x2="231" y2="80" stroke="#64748b" stroke-width="2" marker-end="url(#arrow-arch)" />

<rect x="235" y="22" width="205" height="116" rx="14" fill="#ede9fe" stroke="#7c3aed" stroke-width="2" />
<text x="337" y="54" text-anchor="middle" font-size="14" font-weight="700" fill="#3b0764">sciencekit_lua</text>
<text x="337" y="76" text-anchor="middle" font-size="12" fill="#3b0764">SKLuaBuilder / SKLuaModel</text>
<text x="337" y="96" text-anchor="middle" font-size="11" fill="#4c1d95">Table ↔ Vec&lt;f64&gt; copy</text>
<text x="337" y="116" text-anchor="middle" font-size="11" fill="#4c1d95">SKLuaError + Drop / close</text>

<line x1="440" y1="80" x2="480" y2="80" stroke="#64748b" stroke-width="2" marker-end="url(#arrow-arch)" />

<rect x="484" y="34" width="175" height="92" rx="14" fill="#f0fdf4" stroke="#16a34a" stroke-width="2" />
<text x="571" y="68" text-anchor="middle" font-size="14" font-weight="700" fill="#14532d">sciencekit_ffi</text>
<text x="571" y="90" text-anchor="middle" font-size="12" fill="#14532d">Stable C ABI (in-process)</text>
<text x="571" y="109" text-anchor="middle" font-size="11" fill="#166534">SKFfiArray · SKFfiStatus</text>

<line x1="659" y1="80" x2="699" y2="80" stroke="#64748b" stroke-width="2" marker-end="url(#arrow-arch)" />

<rect x="703" y="34" width="141" height="92" rx="14" fill="#fef3c7" stroke="#d97706" stroke-width="2" />
<text x="773" y="68" text-anchor="middle" font-size="14" font-weight="700" fill="#78350f">Rust Core</text>
<text x="773" y="90" text-anchor="middle" font-size="12" fill="#78350f">faer · ndarray</text>
<text x="773" y="109" text-anchor="middle" font-size="11" fill="#92400e">Fitted estimators</text>

<rect x="235" y="154" width="424" height="40" rx="10" fill="#ffe4e6" stroke="#e11d48" stroke-width="1.5" />
<text x="447" y="179" text-anchor="middle" font-size="12" font-weight="600" fill="#881337">Failures return "sciencekit &lt;status-name&gt; (&lt;code&gt;): &lt;message&gt;" to pcall</text>

The binding layer has four focused responsibilities:

Scope and explicit non-goals

Installation

sciencekit-lua is a standalone Rust crate (edition = "2024", rust-version = "1.85") that pulls sciencekit_ffi from the main sciencekit repository automatically during cargo build and compiles Lua 5.4 from vendored C sources.

Prerequisites

• Rust 1.85 or newer (rustup toolchain install 1.85) and cargo
• A standard C toolchain (gcc or clang) so Cargo’s cc build script can compile the vendored Lua 5.4 sources
• Git (used by Cargo to fetch sciencekit_ffi)
• (Optional) A lua5.4 interpreter if you want to run standalone .lua scripts outside cargo test
• (Optional) LuaRocks 3.x + Lua 5.4 headers (liblua5.4-dev or LUA_INCDIR) if installing as a system/user rock

Option 1 — Build from source with Cargo

Clone the repository and build the release library:

git clone https://github.com/neurono-ml/sciencekit-lua.git
cd sciencekit-lua
cargo build --release

Run the embedded Rust test suite (which spins up embedded Lua 5.4 VMs in-process):

cargo test

On Linux, cargo build --release produces:

target/release/libsciencekit_lua.so

(On macOS the artifact is target/release/libsciencekit_lua.dylib; on Windows, target\release\sciencekit_lua.dll.)

Option 2 — Install with LuaRocks

The repository includes sciencekit-0.5.0-1.rockspec (rockspec_format = "3.0"), which builds the crate with cargo build --release and installs the compiled library as $(LIBDIR)/sciencekit.so:

luarocks --lua-version=5.4 make sciencekit-0.5.0-1.rockspec

Once installed, any lua5.4 process whose package.cpath includes the LuaRocks tree resolves require("sciencekit") directly:

eval "$(luarocks --lua-version=5.4 path)"
lua5.4 tests/run.lua

Loading with require("sciencekit") and package.cpath

Lua’s require("sciencekit") searches package.cpath for a shared object named sciencekit.so (not libsciencekit_lua.so) and invokes its exported luaopen_sciencekit symbol.

Renaming or symlinking into package.cpath

If you built with Cargo and want plain require("sciencekit") without LuaRocks, copy or symlink the artifact as sciencekit.so in a directory on package.cpath:

mkdir -p ./lua_modules
cp target/release/libsciencekit_lua.so ./lua_modules/sciencekit.so
LUA_CPATH="./lua_modules/?.so;;" lua5.4 -e '
  local sciencekit = require("sciencekit")
  print("sciencekit", sciencekit.version)
'

(Note the trailing ;; in LUA_CPATH — Lua expands ;; to the default search path instead of replacing it.)

You can also prepend to package.cpath inside Lua before calling require:

package.cpath = "./lua_modules/?.so;" .. package.cpath
local sciencekit = require("sciencekit")

Loading libsciencekit_lua.so directly with package.loadlib

During local development or CI runs, you can load target/release/libsciencekit_lua.so directly without renaming the file by calling package.loadlib on luaopen_sciencekit. This is the exact helper used by tests/run.lua:

local function load_sciencekit()
  local path = os.getenv("SCIENCEKIT_LUA_LIB")
  if path == nil or path == "" then
    return require("sciencekit")
  end
  local loader, message = package.loadlib(path, "luaopen_sciencekit")
  if loader == nil then
    error("cannot load the native module at " .. path .. ": " .. tostring(message))
  end
  return loader()
end

local sciencekit = load_sciencekit()

Run the smoke test suite against a fresh release build:

SCIENCEKIT_LUA_LIB=target/release/libsciencekit_lua.so lua5.4 tests/run.lua

Pinning the sciencekit_ffi revision

By default, Cargo.toml tracks the main branch of neurono-ml/sciencekit:

sciencekit_ffi = { git = "https://github.com/neurono-ml/sciencekit", branch = "main" }

To pin a specific upstream commit or release tag in downstream builds, replace branch = "main" with rev (or tag) and refresh Cargo.lock:

sciencekit_ffi = { git = "https://github.com/neurono-ml/sciencekit", rev = "<commit-sha>" }

Quickstart

Every example on this page is runnable as written against lua5.4 once sciencekit is on package.cpath (see Installation). All inputs and assertions come directly from README.md and tests/run.lua.

1. Check module and C ABI metadata

local sciencekit = require("sciencekit")

print("Module version:", sciencekit.version)          -- "0.5.0"

local abi = sciencekit.abi_version()
print(string.format("ABI version: %d.%d.%d", abi.major, abi.minor, abi.patch))
print("Feature bitmask:", sciencekit.features())
print("Live handles:", sciencekit.live_handles())     -- 0

2. Linear regression: builder → fit → predict → score

Fit ordinary least squares on the linear relation y = 1 + 2 * x1 + 3 * x2:

local sciencekit = require("sciencekit")

local features = {
  { 0.0, 0.0 },
  { 1.0, 0.0 },
  { 0.0, 1.0 },
  { 2.0, 3.0 },
}
local targets = { 1.0, 3.0, 4.0, 14.0 }

-- Configure the estimator with chainable builder methods, then fit:
local builder = sciencekit.linear_regression()
  :include_intercept(true)
  :backend("faer")

assert(builder.family == "linear-regression")
assert(builder.dtype == "f64")

local model = builder:fit(features, targets)
assert(model.family == "linear-regression")

-- predict() always returns a table of one-element rows: {{y1}, {y2}, ...}
local predictions = model:predict(features)
for index = 1, #predictions do
  print(string.format("row %d -> predicted %.6f (expected %.6f)",
    index, predictions[index][1], targets[index]))
end

-- score() returns the R^2 coefficient of determination as a number
local r_squared = model:score(features, targets)
assert(math.abs(r_squared - 1.0) < 1e-8)
print("R^2:", r_squared)

-- Release native handles deterministically (optional; GC also frees them)
builder:close()
model:close()

3. Standard scaler: fit and transform

Center each column to zero mean and scale to unit standard deviation:

local sciencekit = require("sciencekit")

local features = {
  { 0.0, 0.0 },
  { 2.0, 2.0 },
  { 4.0, 4.0 },
  { 6.0, 6.0 },
}

local scaler <close> = sciencekit.standard_scaler()
  :with_mean(true)
  :with_std(true)
  :fit(features)

local transformed = scaler:transform(features)
assert(#transformed == 4 and #transformed[1] == 2)

for row = 1, #transformed do
  print(string.format("row %d -> [%.4f, %.4f]",
    row, transformed[row][1], transformed[row][2]))
end
-- Symmetric rows sum to zero after centering:
assert(math.abs(transformed[1][1] + transformed[4][1]) < 1e-9)

Next steps

• Learn all builder options and prediction layouts in Linear Regression and Standard Scaler.
• Save fitted regression models to disk and reload them in Model Persistence.
• Branch on stable error codes with pcall in Structured Errors.
• Understand deterministic cleanup and leak assertions in Handle Lifecycle.

Linear Regression

sciencekit.linear_regression() builds an ordinary least-squares (f64) regression estimator backed by the sciencekit_ffi C ABI.

Constructing and configuring the builder

Call sciencekit.linear_regression() to allocate an opaque builder userdata. Configuration setters mutate the builder in place and return the same userdata (builder == builder:include_intercept(true)), so calls chain cleanly:

local sciencekit = require("sciencekit")

local builder = sciencekit.linear_regression()
  :include_intercept(true)
  :backend("faer")

assert(builder.family == "linear-regression")
assert(builder.dtype == "f64")
assert(not builder.closed)

Configuration methods

Fitting, predicting, and scoring

builder:fit(features, targets) validates and copies both tables into contiguous f64 buffers, fits the model via sk_ffi_fit, and returns a new fitted-model userdata (model.family == "linear-regression"). The builder itself stays open and reusable until closed or garbage-collected.

local sciencekit = require("sciencekit")

local features = {
  { 0.0, 0.0 },
  { 1.0, 0.0 },
  { 0.0, 1.0 },
  { 2.0, 3.0 },
}
local targets = { 1.0, 3.0, 4.0, 14.0 }

local model <close> = sciencekit.linear_regression()
  :include_intercept(true)
  :backend("faer")
  :fit(features, targets)

local predictions = model:predict(features)
assert(#predictions == #targets)
assert(#predictions[1] == 1)

for index = 1, #targets do
  assert(math.abs(predictions[index][1] - targets[index]) <= 1e-8)
end

local r_squared = model:score(features, targets)
assert(math.abs(r_squared - 1.0) <= 1e-8)

Target table formats

Both builder:fit(features, targets) and model:score(features, targets) accept targets in two shapes, provided #targets == #features:

1. Flat sequence of numbers: { 2.0, 4.0, 6.0 }
2. Column of one-element rows: { { 2.0 }, { 4.0 }, { 6.0 } }

local sciencekit = require("sciencekit")

local features = { { 1.0 }, { 2.0 }, { 3.0 } }
local column_targets = { { 2.0 }, { 4.0 }, { 6.0 } }

local model <close> = sciencekit.linear_regression():fit(features, column_targets)
assert(math.abs(model:score(features, column_targets) - 1.0) < 1e-8)

Omitting targets on linear regression (sciencekit.linear_regression():fit(features)) raises sciencekit invalid-argument (3): linear regression requires targets.

Prediction output shape

model:predict(features) always returns a table of one-element rows ({ { y1 }, { y2 }, ... }), matching the uniform matrix output convention used across the binding:

• #predictions equals the number of input sample rows.
• #predictions[row] equals 1.
• predictions[row][1] is the predicted number (f64) for features[row].

Scaling to larger tables

From src/large_tests.rs, fitting and predicting on a 2000 × 20 table in Lua exercises the full row-major copy and ABI round trip within double-precision tolerance:

local sciencekit = require("sciencekit")

local rows, columns = 2000, 20
local features, targets = {}, {}
for row = 1, rows do
  local values = {}
  for column = 1, columns do
    values[column] = ((row * column) % 97) * 0.25 + column
  end
  features[row] = values
  targets[row] = 2.5 * values[1] - 1.5 * values[2] + 0.25
end

local model <close> = sciencekit.linear_regression():fit(features, targets)
local predictions = model:predict(features)
assert(#predictions == rows)

local worst = 0.0
for row = 1, rows do
  local delta = math.abs(predictions[row][1] - targets[row])
  if delta > worst then worst = delta end
end
assert(worst < 1e-8)

Unsupported operations on linear regression

Calling model:transform(features) on a fitted linear-regression model raises sciencekit unsupported (4). Conversely, model:save(path) is supported on fitted linear-regression models — see Model Persistence.

Standard Scaler

sciencekit.standard_scaler() builds a column-wise feature standardizer (family == "standard-scaler", dtype == "f64") over the sciencekit_ffi C ABI.

Constructing and configuring the scaler

local sciencekit = require("sciencekit")

local builder = sciencekit.standard_scaler()
  :with_mean(true)
  :with_std(true)
  :backend("faer")

assert(builder.family == "standard-scaler")
assert(builder.dtype == "f64")

Configuration methods

Calling builder:include_intercept(...) on a standard-scaler builder raises sciencekit unsupported (4).

Fitting and transforming matrices

standard_scaler is unsupervised: call builder:fit(features) with a single rectangular matrix table, then call model:transform(features) to obtain a fresh table with the exact same row and column dimensions:

local sciencekit = require("sciencekit")

local features = {
  { 0.0, 0.0 },
  { 2.0, 2.0 },
  { 4.0, 4.0 },
  { 6.0, 6.0 },
}

local model <close> = sciencekit.standard_scaler()
  :with_mean(true)
  :with_std(true)
  :fit(features)

local transformed = model:transform(features)
assert(#transformed == 4)       -- one row per sample
assert(#transformed[1] == 2)    -- one column per feature

-- Symmetric first and fourth rows sum to 0.0 after centering:
assert(math.abs(transformed[1][1] + transformed[4][1]) <= 1e-8)
-- Identical input columns share the same scaled values:
assert(math.abs(transformed[1][1] - transformed[1][2]) <= 1e-8)

Centering without scaling (with_std(false))

Disabling :with_std(false) subtracts each column’s mean (here 3.0) while leaving unit steps unchanged (from tests/run.lua):

local sciencekit = require("sciencekit")

local features = {
  { 0.0, 0.0 },
  { 2.0, 2.0 },
  { 4.0, 4.0 },
  { 6.0, 6.0 },
}

local centered <close> = sciencekit.standard_scaler()
  :with_mean(true)
  :with_std(false)
  :fit(features)

local offset = centered:transform(features)
assert(math.abs(offset[1][1] - (-3.0)) <= 1e-8)
assert(math.abs(offset[4][1] - 3.0) <= 1e-8)

Unsupported operations on standard scaler

Fitted standard-scaler models implement model:transform(features) and model:close(). Calling regressor-specific methods raises sciencekit unsupported (4) (or io / unsupported from the C ABI):

local sciencekit = require("sciencekit")

local scaler <close> = sciencekit.standard_scaler():fit({ { 1.0 }, { 2.0 } })
local ok, err = pcall(function()
  scaler:predict({ { 1.0 } })
end)
assert(not ok)
assert(tostring(err):find("sciencekit unsupported (4)", 1, true))

Model Persistence

Fitted linear-regression models can be serialized to disk with model:save(path) and restored into any Lua 5.4 VM with sciencekit.load(path). Both entry points delegate directly to sk_ffi_model_save and sk_ffi_model_load in sciencekit_ffi, so files written by Lua are binary-compatible with C and Go consumers using the same ABI version.

Saving and loading a fitted model

local sciencekit = require("sciencekit")

local features = {
  { 0.0, 0.0 },
  { 1.0, 0.0 },
  { 0.0, 1.0 },
  { 2.0, 3.0 },
}
local targets = { 1.0, 3.0, 4.0, 14.0 }
local path = os.tmpname()

do
  local model <close> = sciencekit.linear_regression():fit(features, targets)
  model:save(path)
end

do
  local loaded <close> = sciencekit.load(path)
  assert(loaded.family == "linear-regression")
  assert(loaded.dtype == "f64")

  local prediction = loaded:predict({ { 2.0, 3.0 } })[1][1]
  assert(math.abs(prediction - 14.0) <= 1e-8)
end

os.remove(path)

Supported estimator scope

In sciencekit_ffi ABI 1.0 (sciencekit_lua 0.5.0), binary model persistence covers fitted f64 linear regression models:

• model:save(path) writes the fitted linear-regression parameters to path and returns nil.
• sciencekit.load(path) reads path and returns a new SKLuaModel userdata with loaded.family == "linear-regression" and loaded.dtype == "f64".
• Like any fitted model userdata, a loaded model owns a live C ABI handle (sciencekit.live_handles() increments by 1) and is released via loaded:close(), local loaded <close>, or garbage collection.

Persistence error conditions

All persistence failures raise structured Lua runtime errors (see Structured Errors):

Structured Errors

Every failure in sciencekit-lua — whether caught during Lua table validation or returned by sciencekit_ffi — is raised as a standard Lua runtime error formatted with a deterministic prefix:

sciencekit <status-name> (<code>): <message>

Because both <status-name> and <code> come from sciencekit_ffi::SKFfiStatus, scripts can match error categories reliably with pcall or xpcall without brittle dependencies on human-oriented prose.

Status code table

From src/error.rs and src/error_tests.rs, every error maps to one of seven stable status codes:

Pre-ABI validation vs. C ABI errors

Error reporting works in two layers that produce the exact same prefix format:

1. Table and argument validation before any ABI call: src/array/input.rs and src/engine/family.rs inspect Lua arguments first. Ragged rows and non-numeric cells never cross the FFI boundary, and error messages pinpoint the exact 1-based row and column:

   sciencekit shape (2): feature row 2 has 1 values, expected 2; a matrix must be rectangular
   sciencekit invalid-argument (3): value at row 1, column 2 must be a number, found string
   

2. In-process C ABI errors: When a valid call fails inside sciencekit_ffi (for example, model:predict({{1.0}}) on a model fitted with 2 columns), SKLuaError::from_last_error copies the thread-local error message recorded by sk_ffi_last_error_message() and prefixes it with sciencekit <status-name> (<code>):.

Catching and parsing errors in Lua

Use pcall and pattern-match the header sciencekit ([%w%-]+) %((%d+)%): (.*) to extract the status name, numeric code, and detail message:

local sciencekit = require("sciencekit")

local function parse_sciencekit_error(err)
  local text = tostring(err)
  local status, code, detail = text:match("sciencekit ([%w%-]+) %((%d+)%): (.*)")
  if status then
    return { status = status, code = tonumber(code), message = detail }
  end
  return nil
end

-- 1. Ragged matrix -> shape (2)
local ok1, err1 = pcall(function()
  sciencekit.linear_regression():fit({ { 1.0, 2.0 }, { 3.0 } }, { 1.0, 2.0 })
end)
assert(not ok1)
local info1 = parse_sciencekit_error(err1)
assert(info1.status == "shape" and info1.code == 2)

-- 2. Non-numeric cell -> invalid-argument (3)
local ok2, err2 = pcall(function()
  sciencekit.standard_scaler():fit({ { 1.0, "two" } })
end)
assert(not ok2)
local info2 = parse_sciencekit_error(err2)
assert(info2.status == "invalid-argument" and info2.code == 3)

-- 3. Wrong estimator family -> unsupported (4)
local ok3, err3 = pcall(function()
  local scaler <close> = sciencekit.standard_scaler():fit({ { 1.0 }, { 2.0 } })
  scaler:predict({ { 1.0 } })
end)
assert(not ok3)
local info3 = parse_sciencekit_error(err3)
assert(info3.status == "unsupported" and info3.code == 4)

(Note: Lua’s pcall may prepend a source location such as "script.lua:12: " before "sciencekit ..."; avoid anchoring the match with ^ unless you strip the location prefix first.)

Handle Lifecycle

Every builder (SKLuaBuilder) and fitted model (SKLuaModel) owns an opaque C ABI pointer (*mut SKFfiBuilder or *mut SKFfiModel). Because Lua values are garbage-collected while native estimator memory lives outside Lua’s heap, sciencekit-lua provides deterministic release alongside automatic GC cleanup, guaranteeing that every handle is freed exactly once.

<rect x="305" y="18" width="210" height="44" rx="10" fill="#f0fdf4" stroke="#16a34a" stroke-width="1.8" />
<text x="410" y="45" text-anchor="middle" font-size="12" font-weight="600" fill="#14532d">1. handle:close()</text>

<rect x="305" y="76" width="210" height="44" rx="10" fill="#ede9fe" stroke="#7c3aed" stroke-width="1.8" />
<text x="410" y="103" text-anchor="middle" font-size="12" font-weight="600" fill="#3b0764">2. local h &lt;close&gt; (__close)</text>

<rect x="305" y="134" width="210" height="44" rx="10" fill="#fef3c7" stroke="#d97706" stroke-width="1.8" />
<text x="410" y="161" text-anchor="middle" font-size="12" font-weight="600" fill="#78350f">3. Lua GC (Rust Drop)</text>

<line x1="235" y1="76" x2="301" y2="44" stroke="#64748b" stroke-width="1.8" marker-end="url(#arrow-life)" />
<line x1="235" y1="98" x2="301" y2="98" stroke="#64748b" stroke-width="1.8" marker-end="url(#arrow-life)" />
<line x1="235" y1="120" x2="301" y2="152" stroke="#64748b" stroke-width="1.8" marker-end="url(#arrow-life)" />

<line x1="515" y1="44" x2="581" y2="76" stroke="#64748b" stroke-width="1.8" marker-end="url(#arrow-life)" />
<line x1="515" y1="98" x2="581" y2="98" stroke="#64748b" stroke-width="1.8" marker-end="url(#arrow-life)" />
<line x1="515" y1="152" x2="581" y2="120" stroke="#64748b" stroke-width="1.8" marker-end="url(#arrow-life)" />

<rect x="585" y="50" width="215" height="96" rx="14" fill="#ffe4e6" stroke="#e11d48" stroke-width="2" />
<text x="692" y="80" text-anchor="middle" font-size="14" font-weight="700" fill="#881337">Closed Handle</text>
<text x="692" y="102" text-anchor="middle" font-size="12" fill="#881337">pointer = None (freed once)</text>
<text x="692" y="122" text-anchor="middle" font-size="11" fill="#9f1239">further ops → invalid-handle (1)</text>

Three ways a handle is released

1. Explicit builder:close() and model:close()

Calling :close() frees the underlying SKFfiBuilder or SKFfiModel immediately and sets .closed = true. :close() is idempotent: calling it a second or third time returns nil without touching freed memory.

local sciencekit = require("sciencekit")

local model = sciencekit.linear_regression()
  :fit({ { 1.0 }, { 2.0 }, { 3.0 } }, { 2.0, 4.0, 6.0 })

assert(not model.closed)
assert(tostring(model) == "sciencekit model (linear-regression)")

model:close()
model:close()   -- idempotent no-op
assert(model.closed)
assert(tostring(model) == "sciencekit model (linear-regression) [closed]")

2. Lua 5.4 to-be-closed variables (<close>)

Both SKLuaBuilder and SKLuaModel implement Lua 5.4’s __close metamethod (MetaMethod::Close). Binding a handle with local ... <close> guarantees immediate native memory release when control leaves the lexical block — whether normally, via break/return, or because an error unwound the stack:

local sciencekit = require("sciencekit")

do
  local builder <close> = sciencekit.standard_scaler():with_mean(true)
  local model <close> = builder:fit({ { 1.0, 2.0 }, { 3.0, 4.0 } })
  local scaled = model:transform({ { 2.0, 3.0 } })
  assert(#scaled == 1)
end
-- Both `model` and `builder` are already closed here, before GC runs.

3. Garbage collector (Drop)

If neither :close() nor <close> is used, Lua’s garbage collector eventually finalizes unreachable userdata and drops the Rust wrapper (impl Drop for SKLuaBuilderHandle / SKLuaModelHandle), which invokes self.close(). If you already closed the handle explicitly, Drop sees pointer == None and does nothing.

Post-close safety

Attempting any configuration setter, fit, predict, transform, score, or save on a closed handle fails immediately with sciencekit invalid-handle (1):

local sciencekit = require("sciencekit")

local builder = sciencekit.linear_regression()
builder:close()

local ok, err = pcall(function()
  builder:include_intercept(true)
end)
assert(not ok)
assert(tostring(err):find("sciencekit invalid-handle (1)", 1, true))

Auditing handle leaks with sciencekit.live_handles()

sciencekit.live_handles() returns the exact count of SKLuaBuilderHandle and SKLuaModelHandle instances currently live on the calling OS thread. Combine it with two GC sweeps in test suites to assert zero leaks (from tests/run.lua):

local sciencekit = require("sciencekit")

collectgarbage("collect")
collectgarbage("collect")
local baseline = sciencekit.live_handles()

do
  local model = sciencekit.linear_regression()
    :fit({ { 1.0 }, { 2.0 }, { 3.0 } }, { 2.0, 4.0, 6.0 })
  model:close()
  model = nil
end

collectgarbage("collect")
collectgarbage("collect")
assert(sciencekit.live_handles() == baseline, "every handle was freed once")

(Why does sciencekit.linear_regression():fit(...) create two handles briefly? One for the anonymous builder and one for the returned model! Binding local builder <close> or running collectgarbage("collect") releases the temporary builder too.)

Tables and Copy Semantics

Lua has a single composite data structure — the table — whose elements are dynamically typed tagged values managed by Lua’s garbage collector. Unlike Rust ndarray slices or C arrays, a Lua table is never a contiguous unboxed f64 array in memory.

sciencekit-lua states its memory contract plainly: every input matrix or vector is validated and copied into a contiguous f64 buffer, and every output is returned as a fresh Lua table of rows. No zero-copy aliasing is promised across Lua values.

<line x1="196" y1="82" x2="232" y2="82" stroke="#64748b" stroke-width="2" marker-end="url(#arrow-tbl)" />

<rect x="236" y="34" width="180" height="96" rx="14" fill="#ede9fe" stroke="#7c3aed" stroke-width="2" />
<text x="326" y="64" text-anchor="middle" font-size="13" font-weight="700" fill="#3b0764">SKLuaMatrix (Copy)</text>
<text x="326" y="86" text-anchor="middle" font-size="11" fill="#3b0764">row-major Vec&lt;f64&gt;</text>
<text x="326" y="106" text-anchor="middle" font-size="11" fill="#4c1d95">borrowed SKFfiArray</text>

<line x1="416" y1="82" x2="452" y2="82" stroke="#64748b" stroke-width="2" marker-end="url(#arrow-tbl)" />

<rect x="456" y="34" width="145" height="96" rx="14" fill="#f0fdf4" stroke="#16a34a" stroke-width="2" />
<text x="528" y="64" text-anchor="middle" font-size="13" font-weight="700" fill="#14532d">C ABI Call</text>
<text x="528" y="86" text-anchor="middle" font-size="11" fill="#14532d">predict / transform</text>
<text x="528" y="106" text-anchor="middle" font-size="11" fill="#166534">sk_ffi_array_free</text>

<line x1="601" y1="82" x2="637" y2="82" stroke="#64748b" stroke-width="2" marker-end="url(#arrow-tbl)" />

<rect x="641" y="34" width="163" height="96" rx="14" fill="#fef3c7" stroke="#d97706" stroke-width="2" />
<text x="722" y="64" text-anchor="middle" font-size="13" font-weight="700" fill="#78350f">Fresh Output Table</text>
<text x="722" y="86" text-anchor="middle" font-size="11" fill="#78350f">{{y1}, {y2}, ...}</text>
<text x="722" y="106" text-anchor="middle" font-size="11" fill="#92400e">no shared memory</text>

Scalar precision (f64)

Lua 5.4 numbers come in two subtypes: 64-bit IEEE-754 floats (Value::Number) and 64-bit signed integers (Value::Integer). During table conversion (src/array/input.rs), both subtypes are accepted and stored as f64:

-- Mixed integer and float literals convert cleanly to row-major f64:
local features = {
  { 1.0, 2.0 },
  { 3,   4.5 },
  { 5.0, 6.0 },
}

Any other Lua type (string, boolean, nil, table, userdata, function) — even a numeric string like "2.0" — is rejected with sciencekit invalid-argument (3) naming its 1-based coordinates.

Input matrices (features)

Every features argument (fit, predict, transform, score) must be a 1-indexed rectangular table of row tables:

• Outer length is read with raw_len (1 .. rows, ignoring metatables) and must be >= 1.
• Row 1 sets the expected column count (columns >= 1).
• Every subsequent row 2 .. rows must be a table with raw_len == columns.
• Values are packed in row-major order (row 1 col 1, row 1 col 2, …, row N col M).

Input target vectors (targets)

targets passed to builder:fit(features, targets) and model:score(features, targets) must match #features in length (expected_length = features.rows()) and may use either of two layouts:

Output tables

Mutating an input table after :fit(...) never affects the fitted model, and mutating a returned output table never affects subsequent calls.

Validation reference (from src/array/input_tests.rs)

Concurrency

Lua and Rust have complementary concurrency models, and sciencekit-lua respects both boundaries cleanly (tested in src/concurrency_tests.rs).

1. Lua states are single-threaded (!Send)

Standard Lua 5.4 states (lua_State / mlua::Lua) are not thread-safe and cannot be shared simultaneously across OS threads without external locking. Accordingly:

• mlua userdata (SKLuaBuilder and SKLuaModel) never move between Lua VMs directly.
• sciencekit.live_handles() reads a thread-local counter (thread_local! { static LIVE_HANDLES: Cell<usize> }), because every userdata created on a given OS thread’s Lua VM is closed or garbage-collected on that same OS thread.
• Lua coroutines (coroutine.create / coroutine.resume) run cooperatively on the same OS thread and lua_State, so using builders and models across Lua coroutines is completely safe.

2. Multi-threaded hosts: one Lua VM per worker thread

Host applications written in Rust or C (such as game engines, HTTP servers, or worker pools) achieve multi-core parallelism by spawning one independent Lua 5.4 VM per OS thread and loading sciencekit in each VM.

Inside sciencekit_ffi, handles are stored in a process-wide mutex-protected registry that hands each call its own Arc to the immutable fitted model:

• Fitting, predicting, transforming, and closing handles on Thread A never races with or disturbs handles on Thread B (concurrency_freeing_one_model_does_not_disturb_another).
• Multiple independent VMs can fit and score models concurrently in the same process (concurrency_independent_vms_fit_and_predict).

Sharing a trained model across worker VMs

Because Lua userdata values belong to a single lua_State, share a trained linear-regression model across worker VMs by saving it once and loading it in each worker:

1. Training phase: Fit the model on one VM (or offline) and call model:save("/path/to/model.state").
2. Worker initialization: Each worker thread starts its own lua5.4 VM, calls local model = require("sciencekit").load("/path/to/model.state"), and serves model:predict(features) requests concurrently without cross-VM contention.

3. Native thread safety of fitted models

At the Rust/C ABI layer (src/engine/model/handle.rs), SKLuaModelHandle implements Send + Sync:

• Every read operation (predict, transform, score, save) borrows &self immutably while sciencekit_ffi clones an internal Arc from the handle registry for the duration of the call.
• Releasing a handle (close / Drop) requires &mut self, preventing any concurrent read/free race in Rust.
• concurrency_shared_model_predicts_deterministically in src/concurrency_tests.rs verifies 8 OS threads running 25 concurrent predict loops each against a single Arc<SKLuaModelHandle> with bit-exact agreement.

API Reference

Complete reference for the sciencekit Lua 5.4 module (sciencekit_lua 0.5.0), grounded directly in src/lib.rs, src/userdata/builder.rs, src/userdata/model.rs, src/engine/family.rs, src/error.rs, and tests/run.lua.

local sciencekit = require("sciencekit")

require("sciencekit") always returns a plain Lua table (type(sciencekit) == "table"), never a callable proxy.

Module table (sciencekit)

Native symbol

Builder userdata (SKLuaBuilder)

Returned by sciencekit.linear_regression() and sciencekit.standard_scaler().

Read-only fields

Methods

All configuration setters mutate the builder in place and return the same builder userdata (self), enabling fluent chaining.

builder:include_intercept(enabled) -> builder

• Parameters: enabled: boolean
• Supported families: "linear-regression" (raises sciencekit unsupported (4) on "standard-scaler").
• Description: Configures whether linear regression fits an intercept term.

builder:with_mean(enabled) -> builder

• Parameters: enabled: boolean
• Supported families: "standard-scaler" (raises sciencekit unsupported (4) on "linear-regression").
• Description: Configures whether each column is centered by subtracting its training mean.

builder:with_std(enabled) -> builder

• Parameters: enabled: boolean
• Supported families: "standard-scaler" (raises sciencekit unsupported (4) on "linear-regression").
• Description: Configures whether each column is divided by its training standard deviation.

builder:backend(backend) -> builder

• Parameters: backend: string | integer
• Supported families: "linear-regression", "standard-scaler"
• Accepted values:
  • "faer" or 0
  • "matrix-multiply" (alias "matrixmultiply") or 1
  • "ndarray-linalg" (alias "ndarray_linalg") or 2
• Errors: Raises sciencekit invalid-argument (3) if given an unknown string, an integer outside 0..2, or any other Lua type.

builder:fit(features[, targets]) -> Model

• Parameters:
  • features: table — non-empty rectangular 1-indexed table of numeric rows (N × D).
  • targets: table | nil — required when builder.family == "linear-regression" (flat sequence { y1, ..., yN } or column { {y1}, ..., {yN} } with length N); omitted (nil) for "standard-scaler".
• Returns: A new fitted Model userdata (model.family == builder.family, model.dtype == "f64"). The builder itself remains open.
• Errors: Raises sciencekit shape (2), sciencekit invalid-argument (3), or sciencekit invalid-handle (1).

builder:close() -> nil

• Description: Releases the underlying SKFfiBuilder handle immediately and marks builder.closed = true. Idempotent: subsequent calls are no-ops returning nil.

Metamethods

Fitted model userdata (SKLuaModel)

Returned by builder:fit(...) and sciencekit.load(path).

Read-only fields

Methods

model:predict(features) -> table

• Parameters: features: table — rectangular N × D table of numbers matching the fitted column count D.
• Supported families: "linear-regression" (raises sciencekit unsupported (4) on "standard-scaler").
• Returns: Fresh N × 1 Lua table of one-element rows: { { y1 }, { y2 }, ..., { yN } }.

model:transform(features) -> table

• Parameters: features: table — rectangular N × D table of numbers matching the fitted column count D.
• Supported families: "standard-scaler" (raises sciencekit unsupported (4) on "linear-regression").
• Returns: Fresh N × D Lua table of transformed rows.

model:score(features, targets) -> number

• Parameters:
  • features: table — rectangular N × D table of numbers.
  • targets: table — flat sequence or N × 1 table of rows with length N.
• Supported families: "linear-regression" (raises sciencekit unsupported (4) on "standard-scaler").
• Returns: Coefficient of determination (R²) as a Lua number (f64).

model:save(path) -> nil

• Parameters: path: string — filesystem path without interior NUL bytes.
• Supported families: "linear-regression" (raises sciencekit unsupported (4) on "standard-scaler").
• Returns: nil.

model:close() -> nil

• Description: Releases the underlying SKFfiModel handle immediately and marks model.closed = true. Idempotent.

Metamethods

Error format & status codes

Runtime error strings always match:

sciencekit <status-name> (<code>): <message>

Frequently Asked Questions

Why does sciencekit-lua embed Lua 5.4 instead of linking the system liblua?

Linking dynamically against a system liblua forces every build machine and CI image to carry matching headers (lua.h), pkg-config metadata, and compatible symbol exports. Enabling mlua with features = ["lua54", "vendored"] compiles Lua 5.4 directly from source with Cargo’s cc driver so cargo build --release and cargo test work out of the box on a clean Rust toolchain.

If Lua is vendored, why does luarocks make still ask for Lua 5.4 headers?

That check belongs to the luarocks package manager itself, not to cargo build. Before invoking any rockspec’s build_command, LuaRocks validates its own local configuration by probing for lua.h in LUA_INCDIR (provided by packages like liblua5.4-dev). If you build with cargo build --release and copy libsciencekit_lua.so to sciencekit.so on package.cpath, no Lua headers are needed anywhere.

Does this module work with LuaJIT or Lua 5.1 / 5.2 / 5.3?

No. Lua 5.1–5.3 and LuaJIT have distinct C ABIs and lack Lua 5.4 features such as to-be-closed variables (local model <close> = ...). Those runtimes are explicit non-goals for sciencekit-lua.

Why does model:predict(features) return {{y1}, {y2}, ...} instead of {y1, y2, ...}?

Both predict and transform convert C ABI output descriptors into a uniform 2-dimensional Lua representation — a table of rows (src/engine/model/output.rs and src/array/output.rs). For single-target regression, each sample’s prediction sits at predictions[row][1]. Notably, that N × 1 table of one-element rows is also directly accepted as a targets argument by fit and score.

Why are tables copied instead of passed zero-copy?

In standard Lua 5.4, a table is a garbage-collected structure of tagged TValue slots rather than a dense IEEE-754 buffer. Passing data into BLAS/SIMD Rust kernels (faer, ndarray) requires contiguous row-major f64 slices, so copying and validating on entry — and copying outputs into fresh Lua tables before freeing ABI buffers — gives predictable performance and memory safety without pinning Lua GC objects.

Why is dtype always "f64"?

Every floating-point number in standard Lua 5.4 is already a 64-bit IEEE-754 double. Converting Lua numbers down to f32 and back to f64 on return would introduce precision loss without reducing the memory footprint of any Lua table.

Which estimators are available, and how are new ones added?

sciencekit-lua exposes every estimator family carried by the shared sciencekit_ffi C ABI — currently linear_regression and standard_scaler. New algorithms land first in the Rust core (neurono-ml/sciencekit), gain C ABI entry points in sciencekit_ffi, and are then exposed here as thin userdata wrappers without duplicating numerical logic.

Contributing

Contributions to neurono-ml/sciencekit-lua follow the same engineering standards as the main sciencekit repository. Everything durable in the repository — code, comments, tests, documentation, commits, issues, and pull requests — is written in English.

Repository layout

sciencekit-lua/
├── Cargo.toml                      # Standalone crate (Rust 1.85, edition 2024)
├── sciencekit-0.5.0-1.rockspec     # LuaRocks 3.0 packaging definition
├── src/
│   ├── lib.rs                      # Module table & luaopen_sciencekit entry point
│   ├── error.rs                    # SKLuaError formatting & SKFfiStatus mapping
│   ├── lifecycle.rs                # Thread-local live-handle counter
│   ├── array/                      # Pure dispatcher + Lua table <-> Vec<f64> copy
│   ├── engine/                     # Safe RAII wrappers over sciencekit_ffi handles
│   ├── userdata/                   # SKLuaBuilder and SKLuaModel mlua UserData
│   └── *_tests.rs                  # Companion Rust unit & integration test modules
├── tests/
│   └── run.lua                     # Standalone Lua 5.4 smoke suite
└── docs/                           # mdBook documentation site & SEO tooling

Code conventions

• No duplicated math: All numerical work, parameter validation, and model serialization live in sciencekit and sciencekit_ffi. This crate only converts Lua tables, maps statuses onto SKLuaError, and manages userdata lifetimes.
• Naming: Public and crate-internal Rust types use the SK prefix (SKLuaBuilder, SKLuaMatrix), and free functions use sk_ (sk_lua_matrix_from_table). Avoid abbreviations in identifiers.
• Folder modules are pure dispatchers: mod.rs files only declare submodules, re-export items, and wire companion #[cfg(test)] modules — never implementation bodies.
• Companion tests: Keep tests in sibling *_tests.rs files declared at the bottom of the module (lib_tests.rs, error_tests.rs, input_tests.rs, lifecycle_tests.rs, gc_tests.rs, operations_tests.rs, persistence_tests.rs, concurrency_tests.rs, large_tests.rs), plus end-to-end coverage in tests/run.lua.

Local verification gates

Run every CI gate locally before opening a pull request (matching .github/workflows/ci.yml):

# 1. Verify Cargo.lock is in sync
cargo metadata --locked --format-version 1 > /dev/null

# 2. Check formatting
cargo fmt --all --check

# 3. Run Clippy with warnings denied
cargo clippy --all-targets -- -D warnings

# 4. Build release cdylib + rlib and run embedded Lua 5.4 tests
cargo build --release
cargo test

# 5. Run the standalone Lua 5.4 smoke suite (when lua5.4 is installed)
SCIENCEKIT_LUA_LIB=target/release/libsciencekit_lua.so lua5.4 tests/run.lua

Building the documentation locally

Install mdBook (v0.5.2) and Node.js, then build the book and generate sitemap.xml:

mdbook build docs
node docs/scripts/postbuild.mjs

create-missing = false in docs/book.toml ensures any broken chapter path fails the build immediately. Preview locally with live reload using mdbook serve docs.

License

Licensed under the Apache License, Version 2.0.