AlvaAR

AlvaAR is a realtime visual SLAM algorithm running as WebAssembly, in the browser. It is a heavily modified version of the OV²SLAM and ORB-SLAM2 projects. SLAM is the core building block of Augmented Reality applications focusing on world tracking.

Examples

The examples use ThreeJS to apply and render the estimated camera pose to a 3d environment.

Video Demo: A desktop browser version using a video file as input.
Camera Demo: The mobile version will access the device camera as input.

Run with http server

To run the examples on your local machine, start a simple http server in the examples/ folder:

$: python 2: python -m SimpleHTTPServer 8080 or
$: python 3: python -m http.server 8080 or
$: emrun --browser chrome ./

Then open http://localhost:8080/public/video.html in your browser.

Run with https server

To run the examples on another device in your local network, they must be served via https. For convenience, a simple https server was added to this project – do not use for production.

1) Install server dependencies

    $: cd ./AlvaAR/examples/
    $: npm install

2) Generate self-signed certificate

    $: cd ./AlvaAR/examples
    $: mkdir ssl/
    $: cd ssl/
    $: openssl req -nodes -new -x509 -keyout key.pem -out cert.pem

3) Run

    $: cd ./AlvaAR/examples/
    $: nvm use 13.2
    $: npm start

Then open https://YOUR_IP:443/video.html in your browser. If met with a ERR_CERT_INVALID error in Chrome, try typing badidea or thisisunsafe directly in Chrome on the same page. Don’t do this unless the site is one you trust or develop.

Usage

This code shows how to send image data to AlvaAR to compute the camera pose.

import { AlvaAR } from 'alva_ar.js';

const videoOrWebcam = /*...*/;

const width = videoOrWebcam.width;
const height = videoOrWebcam.height;

const canvas = document.getElementById( 'canvas' );
const ctx = canvas.getContext( '2d' );

canvas.width = width;
canvas.height = height;

const alva = await AlvaAR.Initialize( width, height );

function loop()
{
    ctx.clearRect( 0, 0, width, height );
    ctx.drawImage( videoOrWebcam, 0, 0, width, height );
    
    const frame = ctx.getImageData( 0, 0, width, height );
    
    // cameraPose holds the rotation/translation information where the camera is estimated to be
    const cameraPose = alva.findCameraPose( frame );
    
    // planePose holds the rotation/translation information of a detected plane
    const planePose = alva.findPlane();
    
    // The tracked points in the frame
    const points = alva.getFramePoints();

    for( const p of points )
    {
        ctx.fillRect( p.x, p.y, 2, 2 );
    }
};

Build

Prerequisites

Emscripten

Ensure Emscripten is installed and activated in your session.

    $: source [PATH]/emsdk/emsdk_env.sh 
    $: emcc -v

C++11 or Higher

Alva makes use of C++11 features and should thus be compiled with a C++11 or higher flag.

Dependencies

Build Dependencies

For convenience, a copy of all required libraries has been included in the libs/ folder. Run the following script to compile all libraries to wasm modules which can be linked into the main project.

    $: cd ./AlvaAR/src/libs/
    $: ./build.sh

Build Project

Run the following in your shell before invoking emcmake or emmake:

    $: [PATH]/emsdk/emsdk_env.sh

Then, run the following:

    $: cd ./AlvaAR/src/slam
    $: mkdir build/
    $: cd build/
    $: emcmake cmake .. 
    $: emmake make install

Roadmap

• Improve the initialisation phase to be more stable and predictable.
• Move feature extraction and tracking to GPU.
• Blend visual SLAM with IMU data to increase robustness.

License

AlvaAR is released under the GPLv3 license.

OV²SLAM and ORB-SLAM2 are both released under the GPLv3 license. Please see 3rd party dependency licenses in libs/.

Contact

Alan Ross: @alan_ross or [email protected]
Project: https://github.com/alanross/AlvaAR