Introduction
sciencekit_ffi C ABI
Machine learning for Lua, backed by Rust
sciencekit-lua compiles the sciencekit native Lua 5.4 module with an embedded Lua interpreter — chainable estimator builders, fitted models, model persistence, structured errors and deterministic handle lifecycle.
sciencekit_ffi
What is sciencekit-lua?
sciencekit is a Rust machine-learning library reimplementing scikit-learn’s estimators and transformers with native numerical kernels. neurono-ml/sciencekit-lua is the standalone Rust crate (sciencekit_lua) that packages sciencekit as a loadable Lua 5.4 module named sciencekit, consumed in scripts with require("sciencekit").
local sciencekit = require("sciencekit")
local features = { { 0.0, 0.0 }, { 1.0, 0.0 }, { 0.0, 1.0 }, { 2.0, 3.0 } }
local targets = { 1.0, 3.0, 4.0, 14.0 }
local model = sciencekit.linear_regression()
:include_intercept(true)
:backend("faer")
:fit(features, targets)
local predictions = model:predict(features) -- table of one-element rows
local r_squared = model:score(features, targets)
Embedded Lua 5.4 — no system Lua required
Native scripting bindings often fail at build time because the host machine lacks liblua, lua.h, or a matching pkg-config definition. sciencekit_lua eliminates that dependency by configuring mlua 0.10 with features = ["lua54", "vendored"]:
- Compiled from source: Cargo compiles the official Lua 5.4 C sources directly into the Rust build using a standard C compiler (
cc). No systemliblua, header package, orpkg-configfile is needed to runcargo build --releaseorcargo test. - Hand-written module entry point: Because
mlua’s#[mlua::lua_module]attribute macro (modulefeature) refuses to coexist withvendored,src/lib.rsexportsluaopen_sciencekitexplicitly as anextern "C-unwind"function backed bymlua::Lua::entrypoint1. When a hostlua5.4interpreter loads the shared library,luaopen_sciencekitpopulates the module table against the caller’slua_State. - Self-contained Rust tests: The crate produces both
cdylib(forrequire("sciencekit")in external Lua 5.4 processes) andrlib(socargo testspins up fresh embedded Lua 5.4 VMs in memory without shelling out to aluabinary).
How the module wraps the Rust core
Rather than re-implementing estimator configuration, parameter validation, fitting, or binary serialization in a binding-specific layer, sciencekit_lua links sciencekit_ffi from the main sciencekit repository in-process via a Cargo git dependency. Every Lua call traverses the exact same C ABI handles, descriptors, and status codes that a C or Go consumer sees.
f64 buffers and invokes sciencekit_ffi in-process.The binding layer has four focused responsibilities:
Wraps opaque C handles (SKFfiBuilder and SKFfiModel) in chainable Lua userdata with inspectable family, dtype, and closed fields.
Checks rectangularity and numeric cells up front, copies Lua tables into contiguous row-major f64 buffers, and returns fresh Lua tables on output.
Maps both pre-call table errors and C ABI thread-local messages into Lua runtime errors prefixed by sciencekit <status> (<code>):.
Frees native handles deterministically via :close() or Lua 5.4 <close> variables, with garbage-collector fallback and zero double-frees.
Scope and explicit non-goals
| Dimension | Current behavior in 0.5.0 |
|---|---|
| Estimator families | Ordinary least-squares linear regression (sciencekit.linear_regression()) and column standardization (sciencekit.standard_scaler()), matching the C ABI surface. |
| Persistence | model:save(path) and sciencekit.load(path) for fitted f64 linear regression models. |
| Scalar type | f64 only (dtype == "f64"). Lua 5.4 numbers are IEEE-754 doubles, so f32 descriptors would add rounding without saving Lua memory. Both Lua floats and integers are accepted as input cells. |
| Memory model | Explicit copy semantics in both directions. Lua tables are garbage-collected hash/array hybrids, so no zero-copy aliasing is promised across the Lua boundary. |
| Non-goals | LuaJIT, Lua 5.1–5.3, async Lua coroutines over Tokio, and GPU-specific Lua APIs. |