JSARToolKitNFT

Emscripten port of WebARKitLib to JavaScript. Modified and lighter version of JSARToolKit5.

Try the example !! www.webarkit.org/examples/artoolkitnft_es6_example

Features

Markers Types

JSARToolKitNFT support only this type of marker:

NFT (natural feature tracking) markers ✅ 🎉 🎨
Multi NFT markers !!!

Multi-marker tracking

Load several NFT markers with loadNFTMarkers() and every one in view is tracked at the same time: getNFTMarker fires once per visible marker each frame, with that marker's index and its own pose, and lostNFTMarker fires for each marker on its own when it leaves the view.

Detection — finding a marker that is not tracked yet — is far more expensive than tracking one that is: a detection pass costs the full detection time on the frame where it runs. So it follows a policy:

while no marker is tracked, detection runs on every frame;
while at least one marker is tracked and another loaded marker is not, it runs at most once per detection interval, counted from the end of the previous pass, so a marker entering the view is still picked up and every pass is followed by tracking-only frames, however long a pass takes;
while every loaded marker is tracked, it does not run.

Two setters on ARControllerNFT tune this:

setContinuousDetection(enabled) — default true. With false, no detection runs once any marker is tracked, until tracking is lost (the single-marker behaviour of 1.12.0 and earlier).
setDetectionInterval(ms) — the interval above, in milliseconds. Default 300 in the default and SIMD builds. 0 detects on every frame; negative values count as 0.

The threaded (Pthread) build detects on a worker thread, off the main thread, so its default interval is 0; both setters work there too, and an interval saves worker CPU. The Node.js build runs the same native code as the default build, so it tracks several markers and honours both setters, with the same 300 ms default interval.

Note for upgraders: getTransformationMatrix() now returns a fresh array for each frame a marker is found, instead of updating one array in place. Read it each frame rather than keeping a reference and expecting it to change.

Runtime settings: `configure()`

configure(patch) changes several settings at once, on ARControllerNFT and ARToolkitNFT, in every build. getRuntimeSettings() returns all of them, with the values in effect:

Setting	Default	What it does
`near`, `far`	`0.1`, `1000`	Projection planes. `getCameraMatrix()` follows them at once.
`continuousDetection`	`true`	As `setContinuousDetection()` above.
`detectionInterval`	`300` (`0` threaded)	As `setDetectionInterval()` above, in ms.
`filtering`	`true`	Pose filtering, as `setFiltering()`.
`logLevel`	`ARLogLevel.Info` (1)	As `setLogLevel()`.
`ar2`	see below	AR2 tracking: `trackingThresh` 5, `simThresh` 0.5, `searchFeatureNum` 16, `templateSize1` / `templateSize2` 6, `searchSize` 6 or 12, depending on the build and CPU count (the preset's choice).

ar.configure({ near: 1, far: 5000, ar2: { simThresh: 0.6 } });
ar.getRuntimeSettings().far; // 5000

The patch is merged into the current settings (ar2 field by field), checked, and applied in one step: if any value is wrong, nothing changes. An unknown key, a value of the wrong type or a non-integer for an integer setting throws a TypeError; this includes the square-marker keys threshold, thresholdMode, imageProcMode and debugMode, which do nothing on NFT. A value out of range throws a RangeError. The individual setters keep working as before.

Behaviour changes in this release:

getCameraMatrix() right after initialisation is built with the near plane 0.1, the value ARControllerNFT always reported, instead of 0.0001. setProjectionNearPlane() and setProjectionFarPlane() now rebuild the matrix at once, so recalculateCameraLens() is no longer needed.
The threaded build now filters poses by default, like the other builds, and setFiltering() takes effect there (#692). Before, it was ignored.

Log level at initialisation

initWithDimensions(), initWithImage() and customInit() take a last, optional { logLevel } argument. It is applied before the native core is created, so the start-up output follows it:

import { ARControllerNFT, ARLogLevel } from '@webarkit/jsartoolkit-nft'

const ar = await ARControllerNFT.initWithDimensions(640, 480, 'camera_para.dat', false, {
  logLevel: ARLogLevel.Error,
});

(In the Node build, which has no internalLuma argument, the options are the fourth argument.)

The version line is logged at ARLogLevel.RelInfo, so it still prints at Error. The camera parameter table that the native camera loader prints is not gated yet; it is tracked in webarkit/WebARKitLib#113. The level belongs to each controller, since each one runs in its own WASM module.

WASM

has WASM embedded in a single file!

ZFT

JSARToolKitNFT now supports loading NFT markers from .zft compressed files. This allows for faster loading times and reduced file sizes.

A URL that ends in .zft is downloaded directly, once. A URL without an extension (path/to/marker) is checked for a .zft with one asynchronous request, and falls back to the .iset, .fset and .fset3 files when there is none. Any HTTP error status on that check means "no .zft", so storage services that answer 403 for a missing object still work. A marker without a .zft logs one 404 in the console; pass probeZFT: false to skip the check (see below).

Loading NFT markers

loadNFTMarker and loadNFTMarkers return a Promise that resolves with the marker ids once the files are downloaded and registered, so tracking can start right after the await:

const [id] = await arController.loadNFTMarker('path/to/marker')
const ids = await arController.loadNFTMarkers(['path/to/a', 'path/to/b'])

A failure rejects with an NFTLoadError. Its kind is one of http, network, missing, format, rejected or argument, and it carries the url, the HTTP status and the cause when there are any. NFTLoadError is exported from every entry point. Test err.name === 'NFTLoadError' rather than instanceof, because each bundle carries its own copy of the class.

try {
  await arController.loadNFTMarker('path/to/marker')
} catch (err) {
  if (err.name === 'NFTLoadError' && err.kind === 'http') {
    console.error('Not found:', err.url, err.status)
  }
}

The callbacks still work but are deprecated. A call that passes a callback never rejects for a load failure (a throw from your own callback still rejects): a failure goes to onError once (or to console.error when there is no onError) and the Promise resolves [].

arController.loadNFTMarker('path/to/marker', (id) => { /* loaded */ }, (code) => { /* HTTP status, or -1 */ })

In the browser builds (default, simd, td) a fourth argument takes options. signal cancels the load, and AbortSignal.timeout() gives a timeout; an aborted load rejects with the signal's reason. probeZFT: false skips the .zft check for markers that have none.

await arController.loadNFTMarker('path/to/marker', undefined, undefined, {
  signal: AbortSignal.timeout(10000),
  probeZFT: false,
})

fetch replaces XMLHttpRequest and axios, so a file:// URL no longer loads. WebAR needs a secure context anyway, and Cordova and Capacitor serve over a custom scheme. The Node build reads local files and has no signal or probeZFT; its loadNFTMarker(s) return an already-settled Promise. The deprecated legacy APIs in build/ keep their callback-only signatures.

Tracker facade

createNFTTracker and createNFTTrackerWorker wrap the core in one small API: create a tracker, add targets, then call process(frame, timestampMs) once per camera frame. createNFTTracker runs on the calling thread, and every build exports it, Node included. createNFTTrackerWorker runs the same tracker in a dedicated worker. It is in the browser builds only, and every method returns a Promise. ARControllerNFT is unchanged and keeps working.

In both samples, imageData is a 320 × 240 frame of the camera, e.g. the video drawn (letterboxed) into a 320 × 240 canvas and read back with ctx.getImageData(0, 0, 320, 240); see Projection and the example below.

Same thread:

const tracker = await createNFTTracker({ width: 320, height: 240, cameraParam: 'Data/camera_para.dat' })
const [load] = await tracker.addNFTTargets(['DataNFT/pinball'])
const result = tracker.process(imageData, performance.now())

In a worker, the same calls, each returning a Promise:

const tracker = await createNFTTrackerWorker(
  { width: 320, height: 240, cameraParam: 'Data/camera_para.dat' },
  { workerBase: '/dist/' },
)
const [load] = await tracker.addNFTTargets(['DataNFT/pinball'])
const result = await tracker.process(imageData, performance.now())
if (!result.skipped) { /* use result */ }

examples/ARToolkitNFT_ES6_tracker_example.html is a complete page: camera, worker tracker and three.js. Run npm install first: the page imports @webarkit/camera-view, a devDependency, from node_modules/. Then serve the repository root with python python-server.py and open /examples/ARToolkitNFT_ES6_tracker_example.html.

Relative URLs (cameraParam, the target sources, a filter module) resolve against the page, with createNFTTrackerWorker as with createNFTTracker: the wrapper resolves them before it posts them to the worker.

Frames and results

process() takes an ImageData, an ImageBitmap or { data, width, height } (RGBA). The frame must be exactly width × height of the config, and any other size throws a RangeError. In the worker the frame is transferred, not copied, so the caller's buffer is empty afterwards. A data view of only part of its buffer (a frame inside a bigger buffer, or inside WebAssembly memory) or of a SharedArrayBuffer is copied instead, and the caller keeps its buffer.

timestampMs is your clock, normally performance.now(). It drives lostTimeout and the filters.

interface FrameResult {
  time: number                 // the timestampMs you passed
  processMs: number            // time spent in process()
  targets: { id: number; found: boolean; pose: Float64Array | null; error: number | null }[]
  transitions: { type: 'found' | 'lost'; id: number }[]
}

pose is a column-major 4×4 model-view matrix in millimetres, the same matrix as matrixGL_RH in ARControllerNFT's events. It is ready for Matrix4.fromArray() in three.js.

Found and lost. targets reports this frame's raw tracking. transitions is debounced:

found comes on the first tracked frame, at the start or after a lost;
lost comes once a target has been untracked for longer than lostTimeout (default 200 ms).

Between the two, in the grace window, the target has found: false and pose: null, but no lost has fired yet. To keep showing the last pose until lost, keep it yourself.

Adding targets

addNFTTargets(sources, { origin }) loads the sources one at a time and resolves to one result per source, in order. A failure stays local to its source, and the call itself does not reject for a load failure:

// { status: 'loaded', info: { id, url, widthPx, heightPx, dpi, widthMm, heightMm, origin } }
// { status: 'error', url, reason, message }

`reason`	When
`'network'`	An HTTP error, a network failure or a missing file.
`'format'`	A `.zft` that fails to decompress or decode.
`'capacity'`	The 20-target limit is reached.
`'unknown'`	The core refused the dataset (it may be unreadable, e.g. a corrupt `.fset`), or a bad argument.

The ids are the core's. They stay stable for the tracker's lifetime and are never reused.

origin: 'corner' (the default) is ARToolKit's own origin, a corner of the image. 'center' moves the pose's origin to the middle of the image, from widthMm / heightMm (mm = px / dpi × 25.4). That is the offset each app used to compute by hand.

probeZFT (browser only, default true): as with addNFTMarkers, a source without an extension is first checked for a .zft, which logs a 404 for targets that have none. Pass probeZFT: false to skip the check when your targets have no .zft.

dispose() cancels a load still in progress: its target is never registered, and the call rejects with the "disposed" error.

Projection: `projectionFor` and `FrameGeometry`

cameraMatrix is the projection for the processing frame. projectionFor(viewport, geometry?) maps it onto what you display:

camera.projectionMatrix.fromArray(
  tracker.projectionFor({ width: innerWidth, height: innerHeight, fit: 'cover' }, geometry),
)

fit is 'contain' (the default: the whole frame shows, letterboxed) or 'cover' (the frame fills the viewport, cropped). The worker tracker computes it on the page, with no round trip.

The tracker does not fit the camera video into the processing frame: the camera side does. FrameGeometry says where it drew the video, so that projectionFor can undo the letterbox and the portrait rotation:

interface FrameGeometry {
  frameWidth: number; frameHeight: number             // the processing frame
  videoWidth: number; videoHeight: number             // the source video
  x: number; y: number; width: number; height: number // where the video was drawn
  rotated: boolean                                    // a portrait video drawn rotated 90 degrees
}

@webarkit/camera-view produces it: CameraView.geometry, or toFrameGeometry(vw, vh, getProcessGeometry(vw, vh, false, true)), which the example uses. The last argument, floor, rounds the sizes down to integers, so the frame size matches the canvas and its ImageData exactly (CameraView does the same). Without a geometry, the processing frame itself is the source.

Running in a worker

createNFTTrackerWorker(config, { build, workerBase }) picks the build at run time:

td (threaded) when the page is cross-origin isolated (COOP and COEP headers);
otherwise simd when WebAssembly SIMD is supported;
otherwise default.

build: 'default' | 'simd' | 'td' forces one. workerBase is the URL of the folder that holds the worker files. Without it, that folder is the one of the script that loaded the bundle. An app that re-bundles the library normally passes it. Serve these files from that folder:

NFTTracker.worker.js
NFTTracker_simd.worker.js
NFTTracker_td.worker.js
NFTTracker_td.pthread.js, which the td worker loads itself.

Skipped frames. One process() runs at a time. A call made while another is in flight resolves { skipped: true } at once, and that frame is neither sent nor transferred. The frames are dropped, not queued, so feeding the tracker every camera frame never builds up latency. Ignore the skipped results.

dispose() waits at most 2 seconds for the worker to dispose its tracker, then terminates the worker anyway. Calls still pending reject with a "disposed" error. It never rejects, and every call returns the same Promise, which settles once the worker is terminated: after await tracker.dispose() a new tracker never runs alongside the old worker.

Threaded build limitation. On td, dispose() skips the native teardown, because the threaded detector's destructor joins a pthread and hangs (#695). The WASM memory is freed only when the worker, or the page for createNFTTracker, goes away. The worker tracker terminates its worker, so its memory is freed.

Vite dev server. Serving the UMD bundles (ARToolkitNFT*.js) through the Vite dev server transforms them: it rewrites the filter module's import() and turns the file into an ES module, which then fails as a <script>. Serve them from public/ or a CDN. See the comment in vite.lib.config.mjs.

Pose filters

The filter setting chooses how poses are smoothed:

`filter`	What it does
`'native'` (default)	The core's own filter. The poses match `ARControllerNFT`'s for a controller created with `internalLuma = true`, which is the luma path the tracker uses.
`{ type: 'oneEuro', minCutOff, beta, warmUp }`	A OneEuro filter per target, reset on `lost`. Defaults `0.0001`, `0.01`, `0`.
`'none'`	No filtering.
`{ module, options }`	Your own filter, loaded by URL (see below).
`{ create, options }`	Your own factory function, on the same thread only.

Every filter other than 'native' turns the core's filter off, so poses are never filtered twice. Set it with filter, at creation or with configure({ filter }). The filtering key throws a TypeError here, and so does a key the form does not take (a misspelt minCutoff, say). A { create } factory is called once when the filter is set, as a module's is, so a factory that throws fails there, not on a later frame.

The 'native' filter resets on the core's first untracked frame, not on the tracker's lost (lostTimeout). The tracker cannot change that. The JS filters (OneEuro and yours) reset on lost.

OneEuro uses @webarkit/oneeurofilter-ts, bundled into the library, with its units: time in milliseconds, and minCutOff and beta passed to it unchanged, as ARnft does. aframe-nft divides minCutOff by 1000 before passing it, so divide a value taken from aframe-nft's settings by 1000 to get the same filter here. OneEuro expects increasing timestamps, so pass performance.now(). A time that does not increase returns the previous output. warmUp passes the first warmUp poses after a reset through unfiltered, while still feeding them to the filter.

To compare the poses of two cores frame by frame, pin Date.now: AR2 seeds rand with time(NULL).

Writing a pose filter

A filter module is an ES module that exports createPoseFilter. The tracker creates one filter per target and calls reset() when that target is lost:

interface PoseFilter {
  filter(timeMs: number, pose: Float64Array): Float64Array // the 16 pose values in, filtered values out
  reset(): void
}
interface PoseFilterModule {
  createPoseFilter(options: unknown): PoseFilter
}

examples/filters/average-filter.js is a template to copy:

const tracker = await createNFTTrackerWorker({
  width: 320, height: 240, cameraParam: 'Data/camera_para.dat',
  // A relative path resolves against the page, here a page in examples/.
  filter: { module: 'filters/average-filter.js', options: { weight: 0.5 } },
})

The module is loaded with import(), so it must be an ES module reachable by URL.
A module that fails to load, has no createPoseFilter, or whose createPoseFilter throws rejects createNFTTracker, createNFTTrackerWorker or configure({ filter }) with an Error naming the URL.
COEP. On a cross-origin isolated page (needed by the td build), a module served from another origin needs CORS plus a Cross-Origin-Resource-Policy header. A same-origin module needs nothing.
{ create: (options) => PoseFilter } passes a factory directly, but only on the same thread: a function cannot be posted to a worker, so createNFTTrackerWorker rejects it with a TypeError.
filter() must return exactly 16 values, or process() throws a TypeError. The tracker copies the returned pose, so a filter may reuse one output array across calls.
A malformed filter config (an unknown string, an unknown type, an empty module) is refused with a TypeError before anything loads: createNFTTracker, createNFTTrackerWorker and configure({ filter }) reject with it.

ES6

❕From 0.8.0 version has ES6 feature 🎉 😻

Typescript

❕From 0.9.0 version has Typescript feature 💖 💣

Pthread

From 1.6.0 version has Pthread experimental feature 🎉 🎉 🎉

❕❕❕ ATTENTION: this feature is experimental, and it is not well tested yet. It is not recommended to use it in production. You need to set up a server with COOP and COEP headers to use this feature. Read this Emscripten article

Threaded build: deploying

The threaded build is two files, and the worker must be served next to the script that loads the bundle: with a plain <script> tag that is the bundle's own folder, with importScripts() it is the calling worker script's folder, and in re-bundled apps it is the folder of the last <script src> on the page (details below):

dist/ARToolkitNFT_td.js: the bundle you load;
dist/ARToolkitNFT_td.worker.js: the pthread worker (an ES module worker, fixed name), which the bundle loads from the folder above.

Serve them with the Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp headers.

With a <script> tag, keep the two files side by side and you are done. With importScripts() in a worker, the bundle URL is the URL of the worker script that calls importScripts (the same rule as webpack), so ARToolkitNFT_td.worker.js must sit in the folder of that calling worker script, not next to the bundle. If you re-bundle it with your own build (Vite, webpack, ...), the bundler copies ARToolkitNFT_td.js into your output but does not know about the worker, so copy it next to the output bundle yourself:

// Vite: vite.config.js, with vite-plugin-static-copy
import { viteStaticCopy } from 'vite-plugin-static-copy'

export default {
  plugins: [
    viteStaticCopy({
      targets: [{ src: 'node_modules/@webarkit/jsartoolkit-nft/dist/ARToolkitNFT_td.worker.js', dest: 'assets' }],  // = build.assetsDir
    }),
  ],
}

The worker is resolved next to the script that loads the bundle. For module or re-bundled apps that is the last <script src> on the page, which in a Vite production build is /assets/index-*.js, hence dest: 'assets'. (A public/ placement works only with build.assetsDir: ''.) In vite dev the location differs (under /src/), so test the threaded build with vite build && vite preview.

// webpack: webpack.config.js, with copy-webpack-plugin
const CopyPlugin = require('copy-webpack-plugin')

module.exports = {
  plugins: [
    new CopyPlugin({
      patterns: [{ from: 'node_modules/@webarkit/jsartoolkit-nft/dist/ARToolkitNFT_td.worker.js', to: '.' }],
    }),
  ],
}

The symptom of a missing worker is a 404 for ARToolkitNFT_td.worker.js in the network panel, and detection that never finishes.

InternalLuma with simd

Enable internal luma calculation with simd instructions. This feature is experimental and it is not well tested yet. Set the internalLuma flag to true in the ARControllerNFT constructor, or in the initWithDimensions / initWithImage static methods.

Using the library 💥

You can use raw.githack.com links:

WASM version of the library (deprecated it will be removed in a future release):

<script src="https://raw.githack.com/webarkit/jsartoolkitNFT/master/build/artoolkitNFT_wasm.js">

WASM version of the library as a Module:

<script src="https://raw.githack.com/webarkit/jsartoolkitNFT/master/build/artoolkitNFT_ES6_wasm.js">

WASM version of the library as a Module with new ES6 feature:

<script src="https://raw.githack.com/webarkit/jsartoolkitNFT/master/build/artoolkitNFT_embed_ES6_wasm.js">

NO WASM minified (deprecated it will be removed in a future release):

<script src="https://raw.githack.com/webarkit/jsartoolkitNFT/master/build/artoolkitNFT.min.js">

or (recommended) use the UMD library:

<script src="https://raw.githack.com/webarkit/jsartoolkitNFT/master/dist/ARToolkitNFT.js">

or you can install with npm and use as a module:

npm i @webarkit/jsartoolkit-nft

then:

import { ARToolkitNFT, ARControllerNFT } from '@webarkit/jsartoolkit-nft'

The package ships an exports map, so the right build is picked automatically per environment:

in a browser / bundler, @webarkit/jsartoolkit-nft resolves to the UMD build (dist/ARToolkitNFT.js);
in Node.js, it resolves to the Node build: dist/ARToolkitNFT_node.js (CommonJS) for require(), and dist/ARToolkitNFT_node.mjs (a thin ES module wrapper over it) for import.

// browser / bundler -> UMD build
import { ARToolkitNFT, ARControllerNFT } from '@webarkit/jsartoolkit-nft'

// Node.js, ES module -> Node build, named imports
import { ARToolkitNFT, ARControllerNFT } from '@webarkit/jsartoolkit-nft'

// Node.js, CommonJS -> Node build
const { ARToolkitNFT, ARControllerNFT } = require('@webarkit/jsartoolkit-nft')

You can also target a specific build through a named subpath:

Import	Build
`@webarkit/jsartoolkit-nft`	UMD (browser) / Node (Node.js)
`@webarkit/jsartoolkit-nft/simd`	SIMD WASM build
`@webarkit/jsartoolkit-nft/td`	threaded (pthread) build
`@webarkit/jsartoolkit-nft/node`	Node build

import '@webarkit/jsartoolkit-nft/simd'

The raw dist/* deep-imports (e.g. @webarkit/jsartoolkit-nft/dist/ARToolkitNFT_simd.js) still work, and <script> / importScripts URLs are not affected by the exports map. In Node both require() and a named import { ... } work: the ES module entry re-exports the CommonJS bundle, so the two share one instance of every class.

Note: All the examples in the repository are running the code inside a Worker (don't use it in the main thread!). So i you need to import the library in a worker you need to use the importScripts function.

// example of import in a worker with the wasm code lib
importScripts("../build/artoolkitNFT_wasm.js");
// or the dist lib
importScripts("../dist/ARToolkitNFT.js");

Downloads

You can download the build libs in the releases page. Starting from version 0.8.0 it is possible to download dist or build zip packages and from 0.9.6 version only single libs (no zipped).

or you can clone the repository with git, follow the instructions below:

Clone the repository 🌀

Clone this repository
Clone WebARKitLib project to get the latest source files. From within JSARToolKitNFT directory do git submodule update --init. If you already cloned WebARKitLib to a different directory you can:

create a link in the jsartoolkitNFT/emscripten/ directory that points to WebARKitLib (jsartoolkitNFT/emscripten/WebARKitLib) (Linux and macOS only)
or, set the WEBARKITLIB_ROOT environment variable to point to your WebARKitLib clone
or, change the tools/makem.js file to point to your WebARKitLib clone (line 32-33)

Documentation

You can build the documentation of the library. You need node and npm installed and then run these commands in a console:

npm install
npm run docs

At this point you have build the docs in the docs/ folder, you should run a server and then go to docs/ folder.

Using with React

Try react-three-arnft a specific project that uses JsartoolkitNFT with React and Three.js.

ARnft library

JSARToolKitNFT is used by ARnft a small library that helps developers to create WebAR apps.

Python bindings 🐍 (experimental)

❕❕❕ ATTENTION: the Python bindings are experimental. The API may change without notice and they are not yet recommended for production use.

JSARToolKitNFT also provides Python bindings via pybind11, wrapping the same WebARKitLib C/C++ core used by the JavaScript build. They expose a high-level ARControllerNFT class and a lower-level artoolkitnft_core extension module so that NFT marker detection can be driven from Python.

Install from PyPI:

pip install artoolkitnft

Supported platforms

platform	wheels
Linux x86_64	CPython 3.9-3.13
macOS Apple silicon (arm64)	CPython 3.9-3.13
Windows x86_64	CPython 3.9-3.13
macOS Intel (x86_64)	none

Wheels only — there is deliberately no source distribution, because the build compiles sources from outside the package directory and needs the WebARKitLib git submodule, neither of which survives an sdist. On an unsupported platform pip reports no matching distribution found rather than attempting a build that cannot succeed. Intel macOS is absent because GitHub retired its free Intel runner and removes x86_64 macOS entirely in August 2027; build from source if you need it.

TestPyPI carries rehearsal builds only, and should not be used to install the package:

pip install -i https://test.pypi.org/simple/ artoolkitnft

What works

Loading NFT marker datasets (.fset, .fset3, .iset)
KPM-based marker detection
AR2 tracking with pose matrix output
Projection near/far plane setters
Threshold and image-processing mode
Optional pre-computed grayscale input via setGrayData
Event listener for getNFTMarker / lostNFTMarker

Not yet implemented

Live camera capture example (the current example processes a single static image)
getKpmImageWidth / getKpmImageHeight (temporarily excluded from the build)
configure() / getRuntimeSettings() (see Runtime settings). The plane setters already rebuild the camera matrix, as in JS.

The bindings are built and tested on Linux, macOS (Apple silicon) and Windows via the Build and Test Python Bindings workflow, and released by Publish Python package, which publishes to PyPI through trusted publishing (OIDC) on a python/<version> tag.

For full build-from-source instructions, local development tips and the publishing workflow, see python-bindings/README.md.

Node.js 🟢 (experimental)

❕❕❕ ATTENTION: Node.js support is experimental and under active development. The API may change without notice and it is not yet recommended for production use.

JSARToolKitNFT ships a dedicated Node.js build (dist/ARToolkitNFT_node.js), compiled from the same TypeScript sources and WebARKitLib C/C++ core as the browser build. It lets you run NFT marker detection server-side on static image data, without a browser, camera or <canvas>.

When you install the package, Node automatically resolves to this build (see the exports map):

// ES module — named imports resolve to dist/ARToolkitNFT_node.mjs
import { ARControllerNFT } from '@webarkit/jsartoolkit-nft';
// CommonJS — resolves to the Node build in Node.js
const { ARControllerNFT } = require('@webarkit/jsartoolkit-nft');
// or the explicit subpath
const { ARControllerNFT } = require('@webarkit/jsartoolkit-nft/node');

process() takes raw RGBA pixel data, so decoding the image is up to you. No decoder is bundled with the package — install the one the example you follow uses:

# for the sharp example below
npm install @webarkit/jsartoolkit-nft sharp

# or, for the canvas example
npm install @webarkit/jsartoolkit-nft canvas

A minimal example decoding an image with sharp and feeding the RGBA pixels to the controller:

const { ARControllerNFT } = require('@webarkit/jsartoolkit-nft');
const sharp = require('sharp');

async function init() {
  const arControllerNFT = await new ARControllerNFT(2000, 1500, '/camera_para.dat');
  const ar = await arControllerNFT._initialize();

  // process() expects RGBA pixel data, so add the alpha channel.
  const data = await sharp('pinball-demo.jpg').ensureAlpha().raw().toBuffer();
  const imageData = new Uint8Array(data.buffer);

  ar.on('getNFTMarker', (e) => console.log('NFT marker detected: ', e));

  ar.loadNFTMarker('DataNFT/pinball', (id) => {
    ar.trackNFTMarkerId(id);
    // NFT tracking needs several iterations before it locks on.
    for (let i = 0; i < 10; i++) ar.process(imageData);
  });
}

init();

What works

Loading NFT marker datasets (.fset, .fset3, .iset). Camera and marker paths are read from disk relative to the working directory.
KPM-based marker detection and AR2 tracking with pose matrix output
Multi-marker tracking: load several markers with loadNFTMarkers(['DataNFT/pinball', 'DataNFT/kuva'], onSuccess, onError) and each one in view is tracked, with setContinuousDetection() / setDetectionInterval() as in the browser builds. Markers can also be added in later calls: ids continue from the markers already loaded, and a call that fails leaves them in place.
Event listeners for getNFTMarker and lostNFTMarker
Decoding image input via sharp or the canvas package (process() expects RGBA pixel data). Neither is a dependency of this package — install whichever you prefer.

Not yet implemented

Live camera capture (the examples process a single static image)

Runnable examples live in examples/node: example_dist.js (sharp + the Node build) and example_canvas.js (the canvas package). Run one with:

cd examples/node && node example_dist.js

Project Structure 📂

build/ (compiled debug and minified versions of JSARToolKitNFT)
dist/ (compiled UMD lib with ES6 of JSARToolKitNFT)
emscripten/ (C/C++ source code for ARToolKitNFT)
examples/ (demos and examples using JSARToolKitNFT)
js/ (api and workers of JSARToolKitNFT.js for the standard api)
python-bindings/ (experimental Python bindings — see section above)
src/ (source code of ARToolKitNFT with Typescript)
tests/ (the Vitest browser suite in tests/vitest/ and the Node suite in tests/node/ — see Running the tests)
tools/ (build scripts for building JSARToolKitNFT with Emscripten)
types/ (type definitions of ARToolKitNFT)

Running the tests 🧪

Install dependencies, then fetch the browser the test suite drives:

npm ci
npx playwright install chromium

That second step is required. The browser suite runs in a real Chromium supplied by Playwright, and without it you get a missing-executable error before any spec starts. It is the only browser the tests need.

npm test              # everything: the browser suite, then the Node suite
npm run test:vitest   # the browser suite only
npm run test:node     # the Node suite only
npm run test:coverage # the browser suite, with an lcov report scoped to src/

CI runs the browser suite with coverage and uploads coverage/lcov.info to Codecov. The report covers src/ only: the WebAssembly core and the committed bundles in build/ are not instrumented. Coverage is reported, not enforced, so a drop does not fail a PR.

npm run test:vitest:watch re-runs on change while you work. To run a single file, pass its path:

npx vitest run tests/vitest/legacy-min.test.ts

What the suites cover:

The TypeScript API (tests/vitest/ over src/): ARControllerNFT as consumers import it. This covers marker loading, process(), two markers detected in a real photo and marker-lost events, on the default, SIMD and threaded builds.
Every browser build (legacy-*.test.ts, embed-es6.test.ts, module-surface.test.ts): each committed artifact in build/ loads, detects the pinball print in examples/node/pinball-demo.jpg, and exports what src/ relies on (HEAPU8, FS, _malloc).
The published bundle (dist-bundle.test.ts): dist/ARToolkitNFT.js loaded with a script tag, the way a <script> consumer gets it.
The Node build (tests/node/), run with node --test.

Tests run against the committed build/ and dist/ artifacts. If you change anything under emscripten/ or tools/makem.js, rebuild before testing, or you will be testing stale WebAssembly. See AGENTS.md.

WebAssembly 👋

JSARToolKitNFT supports WebAssembly. The library builds WebAssembly artifacts during the build process, WASM is embedded in a single file. This is build/artoolkitNFT_wasm.js. To use it, include the artoolkitNFT_wasm.js into your html page like this:

<script src="../build/artoolkitNFT_wasm.js"></script>

As loading the WebAssembly artifact is done asynchronously, there is a callback that is called when everything is ready.

window.addEventListener('artoolkitNFT-loaded', () => {
    //do artoolkit stuff here
});

See the examples folder for details.

Build the project 🔨

Go to the wiki for more information. Note that you only need to build the library if you make changes to the C++ core or TypeScript source code. The TypeScript bundles in dist/ are built with Vite (npm run build-ts), which also runs tsc for the declarations in types/. There are three ways to build the project:

1. Natively on Host (Windows, macOS, Linux)

To build natively, you must have the following tools installed and available in your environment's PATH:

Node.js (v22.18+ or v24.11+; .nvmrc pins 24)
Emscripten SDK (v4.0.17+)

Once set up, run:

npm install
npm run build

2. Using Dev Containers (Recommended)

If you use Visual Studio Code, you can build and develop inside a pre-configured Docker container using the Dev Containers extension:

Install the Dev Containers extension.
Open this repository in Visual Studio Code.
Click the green button in the bottom-left corner of the window (or press Ctrl+Shift+P / Cmd+Shift+P and search for Dev Containers: Reopen in Container).
VS Code will build the Docker container and configure your environment with EMSDK, CMake, Ninja, Node.js, and all native compilation dependencies.
In the container terminal, run:
```
npm run build
```

3. Using Manual Docker Scripts

If you prefer running Docker manually from the command line, you can use the built-in npm scripts:

Start the background Docker container:
```
npm run setup-docker
```
This resolves host workspace paths automatically (cross-platform on Windows, macOS, and Linux) and starts a persistent Emscripten container.
Build the library inside the container:
```
npm run build-docker
```
(Or run npm run build-docker-no-libar if you want to skip building libar.o)

Notes

The jsartoolkitNFT npm package is served until version 0.9.4 from @kalwalt/jsartoolkit-nft. By 0.9.5 version from @webarkit/jsartoolkit-nft.

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

JSARToolKitNFT

Features

Markers Types

Multi-marker tracking

Runtime settings: configure()

Log level at initialisation

WASM

ZFT

Loading NFT markers

Tracker facade

Frames and results

Adding targets

Projection: projectionFor and FrameGeometry

Running in a worker

Pose filters

Writing a pose filter

ES6

Typescript

Pthread

Threaded build: deploying

InternalLuma with simd

Using the library 💥

Downloads

Clone the repository 🌀

Documentation

Using with React

ARnft library

Python bindings 🐍 (experimental)

Supported platforms

What works

Not yet implemented

Node.js 🟢 (experimental)

What works

Not yet implemented

Project Structure 📂

Running the tests 🧪

WebAssembly 👋

Build the project 🔨

1. Natively on Host (Windows, macOS, Linux)

2. Using Dev Containers (Recommended)

3. Using Manual Docker Scripts

Notes

Runtime settings: `configure()`

Projection: `projectionFor` and `FrameGeometry`