An easy-to-use XR engine for building AR and VR applications with C# and OpenXR!
Another release wrapped up! This time around, I'm extremely thankful for last release's CI/CD, which made shipping preview builds joyful, and made this release much easier to put together! I'm also really thankful to @ponugotimanasaqc @austinbhale and @0y8w1x, who all contributed to this release.
This release we spent a lot of time working on UI and theming! SK's UI has never looked better, and grows increasingly more customizable by the day! A spatial anchor API was added in this release for those devices that support the MSFT anchor extensions (sorry Quest, next release!). We also did a lot of meaningful refactors to the platform code to enable the new SKSettings.mode options, as well as some exciting graphics features still down the road!
You can see the raw changelog here, but here's the high level overview:
UI.HSpace and UI.VSpace. UI.Space is now deprecated. #751UI.ButtonImg now has an overload that allows tinting the image. #811UI.Window elements that makes it easier to grab windows. #784UI.Push/PopGrabAura to turn the aura on and off, and UI.GrabAuraEnabled to check the current state.UI.SetElementColor, allowing you to override the default color category of UI elements. #795UI.SetElementSound, allowing customization of UI element sounds. #795UI.GenQuadrantMesh, a tool for generating more complex UI meshes easily.sk_eye_offset to global shader variables for determining which eye is currently being rendered. #862SKSettings.mode, this replaces SKSettings.displayPreference, which is now deprecated. This change comes with a big rework of platform code, and new ways of running StereoKit! AppMode.Window allows for a simulator free flat window, and AppMode.Offscreen allows for running StereoKit without a display surface at all.SKSettings.omitEmptyFrames, which tells SK to skip submitting a frame to the compositor if nothing has been drawn. This can be useful for performance of overlay applications that want to be listening for input, but don't always need to draw. #745Device.Runtime, for the current OpenXR runtime's name. #871Tex.R16 texture format. #841Tex.GetColorData for more format flexibility, this replaces Tex.GetColors, which is now deprecated. #859Sprites. Sprite.ArrowLeft/Right/Up/Down, Sprite.Backspace, Sprite.Shift, and Sprite.Close.Renderer.SkyMaterial for easy overriding of the skybox material. #863World.Tracked, to tell if the device itself is currently tracking. #851Platform.ForceFallbackKeyboard now applies even in the presence of a physical keyboard. #847Tex.R8g8 textures on OpenGL platforms. #868Font.FromFile when referencing fonts in the Assets folder. #853UI.ButtonImgAt binding calling the incorrect native function. #850Installation is via the dotnet templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html
If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.
It's finally here! After more than a year of development, v0.3.8 is ready at last! This release took way longer than expected, but includes a lot of really good work. Many thanks to all the contributors here with quite a variety of fixes and features!
New dotnet powered templates are now available, and are the recommended way to develop going forward! These templates work both for Visual Studio 2022 as well as CLI environments like VSCode. And most notably, the templates now include an official Android compatible template! The getting started guide on stereokit.net has also been updated to use these templates.
StereoKit now uses GitHub Actions for nightly builds, releases, and verification of PRs! This helps catch and prevent certain bugs and build issues before they happen, and helps get you features and fixes sooner. All changes on the /develop branch will be available the next day as a .preview build on NuGet! To get nightly builds, make sure to enable Include prerelease in your Visual Studio NuGet Package Manager.
To help facilitate this, StereoKit's build tooling was also completely overhauled to make it easier to build the SDK from scratch. If you're interested in contributing to the core, things just got a bit easier for you :)
IStepper Tools
Asset and UI.LayoutPushCut APIs.Device class.Material.Chain feature.SKSettings.origin, World.OriginMode and World.OriginOffset.Device info class. Name, GPU, DeviceTracking, DisplayBlend, DisplayType, ValidBlend, HasEyeGaze, HasHandTracking.Material.Chain for multi-pass materials.UI.LayoutPush, UI.LayoutPushCut, and UI.LayoutPop for more flexible UI layouts.UI.SetThemeColor that allow for specifying separate colors for different element states. #506UI.ButtonBehavior to provide the interacting hand.UI.Handle and UI.HandleBegin to allow configuration of the activating gesture. #601UI.SystemMoveType for configuring how SK provided UI windows move. #479UIVisual.ButtonRound, UIVisual.Panel, and UIVisual.Carat for customizing a few more UI elements. #484UI.Toggle overload with sprite arguments. #514UINotify property to UI.HSlider to change notification behavior between 'on changed' and 'on finalized'. #321UI.VSlider and UI.VSliderAt.UI.TextAt and UI.Text with size. #540UI.TextStyle for peeking the current style.UI.Settings.UIBtnLayout.None option to image button UI elements.UISettings.rounding and UISettings.margin. #556UIVis.SliderLineActive and SliderLineInactive.UI.ButtonBehavior that takes a depth and activation depth.UI.Radio, and added UI.RadioAt. #571UI.LastElementHandActive and UI.LastElementHandFocused.UI.PopupPose, a function for getting a good popup location between the user and what they're interacting with. #478
UI.PopupPose for their location. #478UI.LastElementHandUsed.Assets.All, Assets.Type, get/set properties for Id on assets, and all asset types now inherit from the IAsset interface. #432Mesh.SetData for setting vertices and indices at the same time.Mesh.GenerateCircle, and added doubleSided to Mesh.GeneratePlane, thanks jackdaus! #503calculateBounds parameter to Mesh.SetVerts.Tex.Mips, Tex.GetNativeSurface, and a new overload for Tex.GetColors. #466Tex.GenParticle.Tex.SetMemory, thanks Austin!. #574Tex.SetNativeSurface, thanks technobaboo! #459Material constructor that takes a shader filename.Material shader properties: GetTexture, GetFloat, GetInt, GetUInt, GetBool, GetMatrix, GetVector2, GetVector3, GetVector4, GetColor.Material.Chain for multi-pass materials.ModelNode.Info for storing per-node key/value pair data. GLTF also loads "extras" here #447Model.RecalculateBoundsExact, thanks Paul! #578Model.Draw that allows for overriding the Materials used.Sprite.Find.TextStyle.FromFont, a more consistent replacement for Text.MakeStyle.TextStyle.CharHeight.Solid asset. Reactphysics3d will be removed in v0.4. Please see this sample project for how to use a 3rd party library such as Bepu.Matrix.Transpose and Matrix.Transposed.Vec3.Direction and Vec2.Direction, #476 thanks Ethan Merchant!Quat.Delta and Quat.Relative, deprecated Quat.Difference, #479 thanks Ethan Merchant!Pose constructor that defaults to Quat.Identity for orientation.Matrix.Perspective overload for camera perspectives. #577, thanks Austin!Pose.LookAt.Backend.OpenXR.SystemId, Backend.OpenXR.OnPreCreateSession, Backend.OpenXR.OnPollEvent(#513, thanks jackdaus! ), and Backend.OpenXR.AddEndFrameChain.Backend.OpenXR.UseMinimumExts to only load critical extensions by default, and Backend.OpenXR.ExcludeExt to exclude specific extensions. #444Backend.OpenXR.SetHandJointScale for scaling joint sizes that come from the hand tracking extension, for #337Backend.OpenGL_WGL, OpenGL_GLX, and OpenGLES_EGL, along with related enum values to Backend.Graphics. #382Input.HandSource for figuring out what type of hand data is currently present.Input.Mouse.Ray.Input.HandSimPoseAdd, Input.HandSimPoseRemove, and Input.HandSimPoseClear. #149Input.KeyInjectPress/Release and Input.TextInjectChar to the public API. #630XR_EXT_hand_tracking_data_source. When present, this allows SK to properly handle controllers that pretend to be hands. #667XR_EXT_hand_interaction. Prevents an error popup on HoloLens 2. #490XR_EXT_palm_pose, controller simulated hands may show up with a 'better' orientation.SK.GetStepper and SK.GetOrCreateStepper.Renderer.Screenshot that takes a Pose instead of 2 Vec3s. #628Renderer.Scaling, Multisample, SKSettings.renderScaling and renderMultisample. May not work on some runtimes, use carefully! #483Renderer.Screenshot. #629Renderer.Screenshot that provide raw color buffers in custom formats. #618, #622, thanks Austin!RenderLayer.FirstPerson, ThirdPerson, AllFirstPerson, and AllThirdPerson enum values.RenderLayer is now RenderLayer.AllFirstPerson.Sound.Read / WriteSamples.Time.Frame.Time.Step and deprecated Time.Elapsed, and added time_total instead of time_get in the C API.backend_opengl_egl_get_config to C API. Thanks technobaboo! #451UI.Input now hides passwords. #552UI.Input now scrolls when typing or moving past the end of the visible area.UI.H/VSlider.UI.Toggle and UI.Radio.UI.Input now uses ButtonBehavior code, gaining backpress prevention. #600.UI.Slider's push confirm mode can now also be pinched.UI.Labels.UI.Panel's behavior.UI.Input, now supports carat navigation and keyboard-based text selection. No finger based selection or multi-line yet. #434SK.Initialize now picks an assembly based name if none is provided.() parameter lists were switch to the more appropriate (void).SK.Step, calling StereoKit functions in the loop instead of the callback is now valid.ApplicationIcon.SKCopyAssets true/false, SKAssetFolder path, and SKAssetDestination pathSKOpenXRLoader.SKOpenXRLoaderFolder and SKShaderStandardInclude to the SK MSBuild properties.SKG_LINUX_EGL option failing to compile, thanks technobaboo! #426Model.Find / PlayAnim returning a valid result on failure. #422Solid's deconstructor not releasing the asset properly.UI.ButtonImg's glitchy alpha fade. #440UI.ButtonImg with explicit size. #448UI.Input text entry no longer moves the camera in the Simulator. #365Platform.WriteFile adding an extra \0 character on Windows, #477UI.Panel now uses the correct visual element. #484ISteppers would sometimes Step before Initialize when added from a separate thread. #528material.Shader = null, now falls back to Unlit.UI.Text using the wrong depth.UI.Label sizing wrong when not using padding.Tex.SetColors. #528SK.ExecuteOnMain. #568dotnet run.render/skybox_mesh -> sk/render/skybox_mesh
render/skybox_material -> sk/render/skybox_material
platform/swapchain -> sk/platform/swapchain
platform/swapchain_zbuffer -> sk/platform/swapchain_zbuffer
Installation is via the dotnet templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html
If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.
Another chonky update, v0.3.6 weighs in at over 200 commits! The primary focus for this update was adding asynchronous asset loading. This applies only to textures for now, but this helps significantly when trying to decode large and expensive .png or .jpeg files! A staged loading process will display temporary textures, and smartly load textures in order of importance. All of this is automatic in the background, with no breaking API changes. But that's definitely not all that we've been up to, check out the Feature Highlights below for a list of notable items!
A big thanks to all the contributors that have features is this release, as well as the Discord community's valuable feedback and testing! StereoKit is much better thanks to all of your contributions :)
Just to highlight some related items here, there will be 3 StereoKit presentations at Mixed Reality Dev Days on June 8th! These won't be streamed, but will be made available online sometime after the event. If you'll be present at the event, do stop by and say hello! There will probably still be a few StereoKit stickers left :)
Also, Microsoft is hosting another MR hackathon that includes StereoKit! Plenty of prize money there too, so be sure to check it out if you have the time. We're very excited to see what you'll make!
UI.ButtonImg, UI.ProgressBar, and UI.PanelBegin/End.UI.LastElementFocused, UI.LastElementActive, and UI.LastElementHandUsed.Backend features for OpenXR, Android, and D3D11!Backend as an optional IStepper.XR_OCULUS_audio_device_guid extension.Assets class for managing asset loading.
Assets.CurrentTask
Assets.TotalTasks
Assets.CurrentTaskPriority
Assets.BlockForPriority
Assets.ModelFormats and Assets.TextureFormats for easy lists of supported asset types.Tex.AssetState to peek at the loading stage.Tex.OnLoaded event for load messages. This is tex_on_load & tex_on_load_remove in C.Tex.FallbackOverride for overriding the default fallback before load is complete.Tex.SetLoadingFallback and Tex.SetErrorFallback to give control over the default fallback textures.Tex.DevTex and Tex.Error, nice looking placeholder textures that help give a better sense of perspective on objects.Tex.From* now also take a loadPriority value as a parameter.Tex.SetNativeSurface to the public API. #86UI.ButtonImg and UI.ButtonImgAt for text + image buttons! #340UI.LastElementFocused, UI.LastElementActive, and UI.LastElementHandUsed. #272UI.ProgressBar and UI.ProgressBarAt. #326.UI.PanelBegin UI.PanelEnd and UI.PanelAt, a tool for visual grouping.UI.Get/SetThemeColor for granular manipulation of UI colors.Separator, SliderLine, SliderPush, and SliderPinch to UIVisual for UI.SetElementVisual.Backend.Graphics, and Backend.Platform for info about the current platform and graphics code backing the current session.Backend.D3D11 class with Backend.D3D11.D3DDevice, and Backend.D3D11.D3DContext for mucking about directly with Direct3D.Backend.Android class, for access to a number of low level Android interop variables.Backend.OpenXR.EyesSampleTime, #356, #358. Thanks @sandrist!Backend.OpenXR.AddCompositionLayer to work with the FB passthrough extension.ModelNode.Visible for hiding nodes on a Model.Mesh.VertCount and Mesh.IndCount.Mesh.Intersect overload that returns the index of the intersected tri. Thanks @Faolan-Rad! #293Mesh.GetTriangle for fast retrieval of just a tri. Thanks @Faolan-Rad! #293Bounds.Scaled and Bounds.Scale, #351. Thanks @Strepto!Color.Hex and Color32.Hex for using hexadecimal color values.Renderer.Projection, SetOrthoClip, and SetOrthoSize, thanks @slitcch!Text.Add overload that takes a color tint as a parameter.SK.AppFocus to tell the current focus state of the session/window.mesh_get_vert/inds, added a parameter to describe memory ownership.UI.HSlider pinch has better hit volumes now, should mean less window grabs. #335Tex.Rough is now (1,0,1,1) instead of (0,0,1,1).Material.Copy crashing when provided null.FilePicker failing to parse Unicode text. #374FilePicker would add a trailing \0 to C# strings.FilePicker would crash on cancel.SK.Initialize and SK.Run happen on separate threads, or execution is transferred to a blocking asset. #361Tex.GetColors returning black on separate threads.ui_quadrant_size_mesh.World.BoundsPose) not locating when stage bounds weren't available.Backend.OpenXR.GetFunction.UI.Push/PopPreserveKeyboard not working with elements downstream a UI.Input element.ModelNode.Mesh improperly returning empty Mesh objects. #302, Thanks @Strepto !Mesh references not releasing properly. #303, Thanks @Strepto!UI.HSeparator sometimes adding an extra empty layout line.static const for those compilers.Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html
If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.
v0.3.5 is a big, general improvement update composed of more than two months, ~185 commits, and code from 4 different people! This was also during the StereoKit hackathon, and many improvements here come from suggestions and reports from the participants. If you participated in the hackathon, thank you so much for the feedback! Everyone else should check out the awesome projects the hackers put together!
You won't find any huge features in this update, but there's a lot of smaller ones, and a ton of excellent fixes! I think everyone should find something in here to enjoy :) In particular, I'm pleased to have added SourceLink support! With a few easy tweaks, you can debug and step through StereoKit's source code, just from the NuGet package!
I'd also like to specifically thank @Faolan for their work on the fallback soft keyboard, among other things, @opl- for a couple of really great fixes, and @slitcch for catching a sneaky OpenXR input issue!
Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html
If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.
v0.3.4 brings skinned animation to StereoKit! I hope you'll enjoy using it as much as I've enjoyed making it :) Along with that comes a solid collection of bug-fixes, and some polish to the UI layout system. Plus a whole bunch more! A big thanks to @slitcch, @maxgolov, @sjando, and @aprilspeight, who all made contributions to this release!
In other big news, Microsoft is hosting a StereoKit hackathon for the month of November! It's a great moment to try StereoKit out, and maybe win some cash prizes along the way. Along with that, we'll be hosting some online social events and AMAs, so keep your eyes peeled for those, even if you aren't participating in the hack!
For those getting started with StereoKit, @aprilspeight has created a list of getting started videos! These are also embedded in the docs site, and should help ease new StereoKit devs into their first StereoKit application. Many thanks April!
And check out the new documentation guides for Drawing and Working with Materials! Keep your eyes peeled in November for more docs like these :)
UI.VolumeAt.UI.Text.UI.HSlider variant for doubles. By @slitcch #194Hierarchy.ToLocal and Hierarchy.ToWorld for Pose types.line_add_axis, model_draw and mesh_draw to the C API.Quat.Vec4 and Vec4.Quat
Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html
If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.
Another two months and over 100 commits! Some really awesome features for this update including Unicode support, revamped Model inspection API, UI updates, anti-aliased flatscreen, and a whole bunch more! For a high level look at this update with some flashy gifs, check out the accompanying blog post!
I also really wanted to thank @ClonedPuppy for their input during this update, a good number of features and fixes came from their feedback!
ModelNode API to the Model assets for detailed exploration of a Model's contents. Check here for some examples.Model.Copy.Model.Intsersect and a Ray.Intersect(Model) overload.Vec2.InRadius and Vec3.InRadius methods.Hand.pinchPt, a stable, non-sliding pinch location indicator.UI.Model.UI.PushSurface and UI.PopSurface for more control over placing UI layouts within the scene.UI.HSlider confirmation mode VariablePinch, for more fine tuned manipulation of slider values.UI.SetElementVisual to override UI Meshes and Materials, along with UI.QuadrantSizeVerts and UI.QuadrantSizeMesh to aid in working with 'quadrant sizing' meshes.Hand.pinchPt.Renderer.Blit renders upside-right now.C# methods and properties all have an equivalent C++ function, but here are some additional changes that are of interest specifically to the native developer.
_addref functions.color_hex, color32_hex, and color32_to_128.material_set_vector2, material_set_vector3, material_set_vector4.text_* and text_*_16 functions, as well as ui_* and ui_*_16.Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html
If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.
This update brings in support for a couple new extensions, as well as a steady march of general improvements! In particular, the recently added XR_MSFT_scene_understanding extension makes world occlusion and raycasting just a line of code away for HoloLens 2 devices. Also, the file picker is now integrated into StereoKit's core, and works properly with native pickers and permissions when available!
Many thanks to @slitcch and @technobaboo for their contributions to this update!
World's Raycast, RaycastEnabled OcclusionEnabled, and OcclusionMaterial. #145Platform.FilePicker, and the accompanying Platform.WriteFile and Platform.ReadFile. #147SKSettings.overlayApp.Model.GetName for identifying mesh subsets.Default.MaterialUnlitClip. This removes undocumented alpha clipping from the default Unlit material, and puts it in UnlitClip.Default.X assets can now also be accessed on their respective classes, such as Material.Default, or Mesh.Cube.SphericalHarmonics, and Tex.GenCubemap.TextAlign shortcuts for bitflag combinations, makes text alignment more terse.Tex.FromColors as a more consistent method of creating textures from color data.UI.TextConsume and UI.TextReset for working with text entry input the same way UI.Input does.tex_set_surface to the public C API.Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html
If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.
A new API to interact with controller input was added. Hands are still the primary input device, whether simulated or articulated, but you can now get detailed information about the controller directly! A lot of work was also done to audio: a new Microphone API, streaming audio support, and and API for sound play instances.
Another big visual change was a new default lighting! The previous one was a little dim, and had a lot of extra color in it. The new one is much brighter and a lot less colorful, making it much easier to make white objects look white! If you want the old lighting back, see the bottom of this page for a code snippet.
UI also got a bunch of polish, a lot of small details were fixed, improved, or made prettier! An initial 2-handed grab was added, and a number of useful new UI functions were added. The HSeparator and Push/PopTextStyle methods allow for some extra styling of your UI, and UI.Text enables you to display large blocks of text! UI.Input also got an overhaul that makes it look better, and behave far nicer!
During this update, Varjo also shipped a conformant OpenXR runtime! StereoKit worked with it out of the box, and the SKSettings.blendPreference option was added to hook into their passthrough AR capabilities. A pass was also done on Quest to make the experience there nicer, it's now more stable, supports hand tracking, and brings back a few features that had been temporarily disabled for the v0.3 release.
Enjoy :)
To see a number of these new features in action, check out the interactive release notes demo!
https://github.com/maluoi/StereoKitReleaseNotes/tree/main/v0.3.1
UI.HSeparator, UI.Text, UI.PushTextStyle, UI.PopTextStyle, UI.EnableFarInteract, and UI.ButtonBehavior.Renderer.RenderTo.Tex.FromFiles.Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html
If you already have a project using StereoKit, go to Project->Manage NuGet Packages, and update to this version! Do a full rebuild of your project afterwards.
If you need the old lighting/skybox back, stick this after SK.Initialize:
Renderer.SkyLight = new SphericalHarmonics(new Vec3[] {
new Vec3( 0.27f, 0.26f, 0.25f),
new Vec3( 0.07f, 0.09f, 0.11f),
new Vec3(-0.06f, -0.06f, -0.04f),
new Vec3(-0.06f, -0.04f, -0.01f),
new Vec3(-0.04f, -0.05f, -0.06f),
new Vec3( 0.15f, 0.16f, 0.16f),
new Vec3(-0.04f, -0.05f, -0.05f),
new Vec3( 0.05f, 0.05f, 0.04f),
new Vec3(-0.11f, -0.13f, -0.13f),
});
Renderer.SkyTex = Tex.GenCubemap(Renderer.SkyLight);
I would describe v0.3 as the platform compatability update! This big update has been a lot of work, but brings in initial support for Oculus Quest 1 & 2 on Android, as well as Linux! This meant a complete overhaul of the graphics pipeline, and even resulted in the development of a new single-header cross platform graphics library, sk_gpu.h!
There's a lot more, but I'll say these are most of the big items! This update has been in the works for 8 months, and you can see all the changes over in this commit :)
Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html
Exciting update! StereoKit now works on SteamVR's OpenXR runtime! Also, a number of features finally land as OpenXR 1.0.9 brings a couple cool extensions out of preview. Which means StereoKit also now supports OpenXR's hand tracking instead of UWP's, 3rd eye render for more accurate screenshots/video, as well as QR code support!
Installation is via Visual Studio templates and NuGet, please refer here for instructions! https://stereokit.net/Pages/Guides/Getting-Started.html