snow alpha-2.0 changelog

sign up sign in

Transitioning

This update has a lot of big underlying changes!

While redoing things, a whole lot of outstanding issues have been resolved! There is also a chance that there are new issues, oversights, or things that aren't 100% there yet – just like before – We're still in alpha, after all.

I'm fairly certain I have forgotten a number of major things and everything will explode shortly but hey. We'll sort it out in no time anyway :)

I've tried to ensure that there is at minimum parity with the day to day workflows in place already, and that things removed have a parallel immediate term as much as possible.

If you run into any issues - please take the time to report them on the repo.

This is unusual...

For onlookers and completeness: This type of large update cycle is not normal for us here - we don't typically work on large updates like this at all. While waiting for tests to run I drew a diagram of how this unique situation came about - essentially, I was messing around in the bushes and had to bring a couple developers into the bushes with me to make some releases happen well.

The image is a timeline, hopefully legible:

  • A - Throwing around random ideas of a rewrite of snow in a new repo, for fun
    • “hmm, I wonder how snow would look written from scratch”
  • B - THREES release
    • High DPI fixes, but can't use snow master since A and C were converging
  • C - The Westport Independent releasing soon
    • code from the A detour is going to be required
    • have to make the squiggling lines immediately usable
  • D - Today, merging all of this back into master

Anyway - onto the outcomes!


Changes

Woo, there's a lot to cover.

I'll state this right up top for clarity:
I believe it's important to understand how your tools are made, and what features they offer. A lot of the changes below won't break your game code dramatically or anything - But it's good to pay attention to the details. It's totally fine if a lot of this is stuff you'll never need to interact with directly.

It's also worth noting there are so many changes that I probably missed listing some here, or chose not to because the compiler will tell you about them. The most important stuff is covered below!

core changes

The most important change of course is that the snow core itself is now written entirely in Haxe! On native targets uses the linc libraries directly to handle the platform specifics as needed.

An important point of this is that all of the linc libraries that are dependencies of snow, are available at your disposable. For instance, you may now call SDL directly from Haxe, directly from your game. This is true for snow and luxe users of course. There are a number of new APIs available, and this aspect will continue to improve over time - but keep in mind that they change the nature of your code for the platforms you're targeting - this doesn't magically allow you to call SDL APIs on the web platform. The linc libraries are C++ target specific, and some of them get more specific by platform (like Desktop OpenGL calls on mobile are not viable). You have the tools to deal with that of course, but keep it in mind.

There's is some deep diving though process below the bullet lists if you're interested.

my favourite improvements

It's hard to even pick from the list because of the amount of changes and improvements that have gone in but here are some of my favourites:

Debugging Haxe native code via NDK tools in Android studio

With some minor configuration Android Studio lets you debug native code and java code at the same time. Since this is about Android Studio, it's worth mentioning the Gradle based build workflow, Android build specifics in the IDE, access to android frameworks and libraries and much more.

Zero runtime allocations by snow itself

Since the new version is significantly closer to beta than before - that goes for tightening up the performance profile of the core framework. Lots of performance improvements and consistency fixes all around.

Much tighter native integration

Having everything accessible makes a big difference, and allows much easier access to important native functionality. This is part of the much mentioned ongoing native features, and is what allowed us to easily use Android/iOS native services from Haxe without much effort at all. Here's a teaser of a (nearly ready) linc library that leverages this for example:

Common changes

Haxe version requirements

As mentioned in a previous dev log - maintaining legacy code during alpha and active development makes no sense in practice. From this update:

  • Haxe 3.2+ is required
    • Haxe 3.1.x is no longer explicitly supported
  • hxcpp versions above 3.2.x are required

This decision while pragmatic is also dictated by Haxe and hxcpp features, there are things required in the newer code that just don't run on older versions.

hxcpp compile cache
I'd also encourage you to try out the newer hxcpp cache features, which significantly speeds up C++ builds. They share files built by projects in a smart way, which means you only have to build files used by all your projects when they change and not every time they need to be built. Here's an example of a clean rebuild of a snow sample that took ~4 seconds, a lot of that time spent in the Haxe compiler on the code than the c++ build.

To use it, for now you have to set an environment variable pointing HXCPP_COMPILE_CACHE to a path to use. The path must exist before it will work, so create the folder manually first.

If you have issues using this (I haven't with months of use, across projects large and small) file an issue on the hxcpp repo with the necessary steps to reproduce.

You can set this up really easily inside of the hxcpp_config.xml file (Mac & Linux ~/.hxcpp_config.xml, Windows C:\Users\USER\.hxcpp_config.xml). Just add the path as a setenv node inside the vars section.

...
<section id="vars">  
    <setenv name="HXCPP_COMPILE_CACHE" value="/Users/sven/.hxcpp_cache/"/>
...

dependency changes

Since snow now depends on a number of linc libraries - you have to install these before using the new. Luckily installing them is easy, just run the following:

haxelib git linc_openal https://github.com/snowkit/linc_openal.git  
haxelib git linc_timestamp https://github.com/snowkit/linc_timestamp.git  
haxelib git linc_stb https://github.com/snowkit/linc_stb.git  
haxelib git linc_ogg https://github.com/snowkit/linc_ogg.git  
haxelib git linc_sdl https://github.com/snowkit/linc_sdl.git  
haxelib git linc_opengl https://github.com/snowkit/linc_opengl.git  

It's important to note that if you clone them directly, be sure to use git clone --recursive and if you already cloned without that run git submodule update --init in the folders. If you don't you'll get build errors as the submodules files are missing.

The setup process will be more automated soon ish.
EDIT: Here's a helper for installation and updating: snowfall

native setup stuff

snow no longer has any prebuilt components and includes the c++ directly into your app in a clean way. This removes the ndll folder and its set up step entirely, so don't worry about that any more.

If you ran into previous Visual studio/MSVC version inconsistency issues with snow previously this should be resolved now because of the above changes too.

added/fixed/changed

  • Added Desktop GL 1.0 to 4.5 APIs via linc_opengl
    • Call desktop GL directly from your Haxe code, now you can use more than just the ES2.0 portable implementation - but remember these are desktop only, for cross platform portability the best API is still the ES2.0 abstraction at snow.modules.opengl.GL like before
    • This includes GLEW API for querying versions and extensions
    • Further APIs will be added for GL ES, there might be endpoints that are incorrect and more - please see the linc_opengl readme for full details
  • Added gles profile to OpenGLProfile config for runtime checking of the rendering profile. Later this will be used to request ES on desktop like the other two (core/compatibility) are used now.
  • snow OpenGL is now fully native externs, reimplemented in linc_opengl, which is much faster, cleaner, and aligns closer to WebGL spec. It has some WIP endpoints but these will be addressed soon (nothing majorly blocking).
  • Added proper High DPI support and handling
    • ALL window related sizes in snow and luxe refer to rendering size in pixels, only
    • See the section on High DPI below for details
  • Implemented a whole new Audio API
    • added AssetAudio containing AudioData
    • implemented a new Web Audio implementation for web platform
    • very close to final API, chance of the API changing again at all are low
    • added AudioSource, AudioInstance, AudioData
    • added AudioHandle as a disposable Int for simplicity/performance
    • complete parity with native, including audio_data_from_bytes, and more
    • separated the query from the setting of attributes for clarity
    • simplified audio events, added new ones like when a source is destroyed
    • solves many issues on native, like
      • playing multiple instances from a single source
      • proper cleanup of data and sound buffers
    • encapsulates audio implementation details directly in the module
      • better portability in the abstractions
      • no more hoops and opaque types needed
      • allows modules to self contain their dependencies
    • cleaned up custom streaming interfaces
      • with added example for native
      • allows generative streaming sounds and synths efficiently
  • App clean up and consistency pass
    • moved control over main to user space
    • clarify naming of all app timing flags
    • added snow_no_main and snow_native_no_tick_sleep defines
    • made AppFixedTimestep a runtime flag, not compile time flag so you can now toggle and compare details while running
  • Leverage snow flow project config nodes
    • allows setting log levels, runtimes, modules, disabling the config.json file or changing it's name and so on
  • Clarified and fixed the update vs tick concepts framework wide
    • tick is the raw update event fired from the runtime
    • update is the app level event that respects the app timing flags
    • this allows handling tick updates that are relevant when the app is paused or not ready yet (like timers and so on) and gives you access to the runtime tick rate separate from the app update timings
  • Added initial snow.core.Extension interface
    • This is part of the ongoing snow sdk/extension implementation
  • Added snow.api.Emitter, moved lower from luxe
    • an efficient macro based event emitter, used in the core now
  • Added new snow samples for feature specifics
    • assets
    • audio
    • audio_custom_stream (native only)
    • config
    • window
  • Lots and lots of clean up and tweaking
    • There are no longer any runtime allocations in snow
    • Fixed consistency issues in many API endpoints and config nodes
    • Removed a lot of needless abstractions
  • Changed snow.system to snow.systems plural for consistency
  • Changed config.render.depth and config.render.stencil to Int
    • This removes the bool redundancy of asking for a depth buffer and having to specify the bits separately, if the bits are > 0 it will request it now.
  • Changed how easy it is to disable the config.json
    • set a blank name for it in the snow config node in the flow file

Platform specifics

Web

  • New cleaner, more customizable web runtime
    • redid windowing implementation
    • cleaned up gamepad implementation
  • New platform level Audio implementation
    • uses the native browser Web Audio API and HTML5 <audio>
    • gives you access to the Web Audio context to do advanced/custom processing
    • removes howlerjs dependency
  • Added ability to configure WebGL version and more
  • Fixed high DPI implementation details for the canvas/WebGL
  • Fix touch + mouse input bugs when page scrolls or zooms or has High DPI
  • Added ability to override fallback when WebGL isn't found
  • Added window_id and window_parent config as js.html.Element
    • allows setting the <canvas id="window_id"> explicitly.
    • default is now app instead of window1 (which was generated by the window id)
    • allows controlling which element the <canvas> is appended to

Android

  • Moved Android specific flow configuration to Android studio where it belongs
  • Migrated to the gradle build tool instead of older ant based
  • Migrated to Android Studio projects
    • Gives full access to all Android features directly
    • Allows more platform aligned workflow like it was with iOS
    • Build, edit and run from Android Studio
    • Manage signing and everything from the UI
  • Project specific Java integration with the IDE
    • Tools like library projects, jar files, code completion are first class

iOS

  • Moved iOS specific flow configuration to XCode where it belongs
  • Cleaned up the XCode project template further
    • depending on your project, it's better to just generate a new one
    • if you've made extensive changes, you can backup the real one (which is always a good idea), then generate a new one and run a diff to see the minor changes.

Linux

  • Because you're building SDL, OpenAL and OpenGL code directly now, you need dev libs to do so. Install the following packages:
    • libgl1-mesa-dev
    • libglu1-mesa-dev
    • libopenal-dev

Mac

  • Added XCode project generator
    • use flow build mac --generate-project
    • This allows debugging and profiling the app directly from XCode, much like iOS and allows tighter integration with the platform for later native integration.
    • Follows the same concept as iOS/Android projects (generate once)

Removed

Remember, most of these seem drastic but have 1:1 parallels from SDL or new linc libraries.

A lot of the pruning here is to finalize the API from it's exploratory roots. Portability, maintainability and parity are important to me (and I'm sure many users). Every API endpoint that exists in the core API is an implicit statement about it's support on all supported platforms and targets.

There were plenty endpoints where this wasn't true, and many cases where it was just impossible (like setting the mouse cursor position on a browser will never be allowed due to security risks). If the user was making assumptions would end up being mislead into disappointment but the fault would partly be on the API.

In line with the vision and goals of snow, the best option was to remove all of that down to the absolute minimum that can be delivered upon consistently - which this update definitely gets close to achieving - and to make every non portable API endpoint much more explicit like the original module/API separation. Now you can always tell whenever the code you are calling is intended to be portable or not, making your code that interacts with snow more imperative and easier to reason about.

  • Removed the NDLL folder, and project folder for new implementations in Haxe
  • Removed snow.api.Libs which was legacy handling for NDLL stuff
  • Removed file dialog endpoints
  • Removed file watch and drop events
    • Use SDL for the drop event since that's where it came from
    • Use linc_filewatch for the file watch API replacement, it's cleaner
  • Removed most windowing/desktop windowing APIs
    • Only essentials like fullscreen/grab remain, since that have full parity on almost every platform
    • SDL provides all the desktop APIs plus many more snow didn't expose, now you have much more to go on
  • Removed snow.api.File
    • replaced inside of the native io modules as file_handle and file_handle_from_mem and associated file_*, since the snow.api are typically the user facing cross platform API helpers
  • Removed Input/Windowing modules
    • app.input system API remains the same
    • This removes multiple window support from the core API but you can create multiple windows in exactly the same way using SDL directly, and have far more control over them with the full SDL API on hand. This concept didn't translate to web or mobile or consoles so it's much better now
    • The runtime handles windowing now and is at app.runtime / Luxe.snow.runtime
    • Window API calls are now app.runtime.window_*
    • The window handle can be reached via app.runtime.window but note that it is not a snow.system.window.Window, it is now a raw runtime handle. On SDL runtime this is an sdl.Window and on js a js.html.CanvasElement
  • Removed Native and Web config specifics
    • These are handled by the RuntimeConfig instead now, which is defined by the runtime itself and strong typed by Haxe magic
  • Removed howlerjs implementation and lib files on web
    • replaced by pure js APIs
  • Removed all flow nodes regarding mobile
    • To be more specific, things like project.app.mobile.orientation and project.app.mobile.android.permissions - NONE of that exists any more. You configure your project in the platform specific way via the project formats (manifest xml, info.plist, IDE settings etc)

Known short term issues

This is a short list of things that don't have parity or are new issues as of right now. I will update this list short term so you can check back as needed if they affect you, or keep an eye on the repo for the issues being filed + closed if they're a bug.

  • OpenAL dll stuff on Windows and Linux, see below
  • Android build workflow requires manually editing flow.gradle to include the --debug flag if it is required. This is a very short term pain point and will be resolved soon.
  • URL open API isn't implemented on native right now
    • This was missing android implementation before anyway
    • This will be replaced by linc_urlopen which will be pushed soon
OpenAL differences

Since we're now using openal-soft for OpenAL on Windows and Linux via linc_openal, it's important to understand that OpenAL has dynamic linking requirements. On Mac and iOS for instance there are system libraries, so this mainly affects Windows and Linux short term.

The details and specifics are listed on the linc_openal readme for more information, but keep in mind that there is a change here in how this works, and that if you're giving people builds that you've taken note. Don't worry though, this is really common for all games that use OpenAL.

Soon this won't be the case, as the default snow audio module will be moving away from OpenAL - but of course - the option to continue to use linc_openal and the snow audio module for it will be up to you when that happens. There will be more details about this in the near future.

luxe specific changes

Audio changes

Since the snow audio API has been properly redone and finalized its worth noting the changes to the existing workflow.

The main note is that it now matches the other resources with Luxe.resources.load_audio and Luxe.resources.audio.

The AudioResource that it loads contains a source: AudioSource property - this is the actual audio source that you use to talk to the play(sound.source) and loop(sound.source) functions.

The other difference is that audio is modelled correctly as "one source, many instances" and there are no longer Sound instances as classes/objects. This just means that play/loop returns a handle to an instance, created from the source you give it. The handle is just an int and is disposable, but is required if you want to change the audio properties of the handle. This keeps things optimal and cheap - but just as easy.

As before, the samples have the newest examples for reference and questions welcome on the chat.

Batcher limit changes

I've changed how the "max verts" limit works in phoenix short term, since it was getting in the way of normal usage a bit. By default the batcher has a smaller size (lowering default memory usage) but also has an option now when creating a batcher - You can now specify the max size per batcher, smaller or larger, and have control over this.

For the debug console batcher, it sets itself up with more than enough room for much bigger space so the majority of annoying issues with it are out of the way.

I've also exposed the name, enabled state, and more stats in the Renderer stats view.

While I was at it I cleaned up the resource list and made it easier to see the estimated usage of the resources. Audio resources are now properly listed and tracked as well.

GL core profile

Added default shaders and made sure the rendering initialized properly for GL 3 core profile usage. Part of the ongoing move to embers.

Internal design changes

runtime changes

The design of snow is maturing to my intent for it which is pretty exciting. In dev log 8 I referred to access, the new version of snow moves a significant amount forward on that front especially with regards to the runtime itself.

But what does this mean, what is the runtime?
The runtime is the entry point, the main loop, and the application life cycle.
It is also now responsible for the window and input from the platform.

In snow terms the runtime is the backbone of your application. On native platforms, SDL provides the runtime implementation details. On web, the runtime uses Haxe js APIs directly from the core runtime module. If you were to port snow to a new platform - say a console platform - the question of how would be answered with “replace the runtime module”. It sets up the window, and emits input events to the core which then processes them and forwards them onward.

To give a concrete example of a simple runtime, what if you wanted to make a CLI based game that didn't require SDL? That sounds like a bunch of build configurations and all sorts right? Now it's just a small change in your flow file and you point snow to use any class you want as the runtime.

A runtime is just an interface and has three main functions, run, ready, and shutdown. Inside run, you could imagine waiting for user input (like getch), and firing the updates from there and you have a MUD on your hands. Some examples of this will be added in future, but it should be pretty clear as the runtime is configured the same way for the default web and native runtime.

customizing the default runtimes
This is where it gets more interesting to framework developers - instead of having to implement your own runtime, what if you just wanted to change something small about how the default runtime works without having to hack code into snow anywhere?

You can now just extend the default runtime, override any function in particular you want to change, and then tell snow to use yours instead. This solves a loat of issues and gives a lot of options with regards to flexibility.

Along these lines, the default runtimes I provide also expose some common functions that are more likely to be overridden than others. For instance, the SDL runtime on native has a function called loop, which does the SDL pollEvent call and emits the events as needed. It's good to have access to modify the default behaviour, or even replace it entirely if you have specific needs – without having to reengineer the whole framework first.

Some more examples of this from the SDL and web runtimes: window_flags, create_window, create_render_context, post_render_context, apply_GL_attr. By default, snow creates an OpenGL window, what if you don't want that? What if you want to create the SDL software renderer and use SDL APIs? What if you wanted to use DirectX or need some really specific Windows API calls snuck in after the GL context is created. What if you wanted to configure the window externally and embed the GL context ... you get the idea. You have all the control you need now, all the way down.

runtime portability
There are some requirements for the runtime to fulfill of course.
In order to align with the resulting design goals of snow (and with sensible cross platform abstraction design in general) the runtime handles all the details with regards to the platform that it wants to run on. In this manner, ALL the platform specific details that are tied specifically to the platform are now encapsulated within the runtime.

To word that differently - previously there were modules for Audio, Input, Windowing, IO, Assets and so on. This has changed to just be Audio, Runtime and io and assets are similar for now.

Input from an OS typically comes from a window, these two systems are coupled on almost every single platform. The window is coupled to the platform itself, and on some platforms the window even controls the run loop. It's way better to have all of the code for a specific port or runtime to be encapsulated, allowing it to set up and maintain the runtime requirements without having to hack things into random corners of the code base. It also helps remove many layers of abstraction, since the runtime has access to all of the necessary variables, APIs and objects that it needs.

strong typed user specified types
One thing I had originally been trying to do was use strong types for the user specified types in the original module system before now. The newer version uses a feature of Haxe to allow this, which means all types that are defined by you end up being strong typed in the core itself as well. This means for example, app.runtime would be typed explicitly as the type specified, so it would always have the correct code completion and type safety as if it was defined internally that way.

notes
Now that all of the runtime specifics are in the user space, giving you control over these as needed and again for almost all users these details are already taken care of by snow and it's core and default runtime modules.

For the users that have more specific requirements or intentions like porting snow to a new platform or runtime, or for maintaining snow long term and keeping it minimal and reliable - this makes a big difference.

Looking forward

It should be clear from previous posts I didn't intend for this sort of large scale change to happen locally. We're back to incremental changes on master according to the versioning mentioned in dev log 10. Also as mentioned, we're going to reset the dev cycles a bit so expect more posts soon on all that.

Thanks for the patience and for helping each other out on in the community! I do notice, and it makes a big difference to how great snõwkit is for everyone.

I'm excited for the new and improved version to be in everyones hands and I can't wait to continue marching forward together. There's a lot of cascading benefits from the work put into this update.

I'll be around on the chat more frequently now that I'm out of the tunnels, see you there!

Other updates


You can follow the snõwkit community twitter, hang out in the chat and see what people are up to using #luxeengine or #snowkitcommunity. Don't forget to post here on snowkit.org as well!

Tags
devlogdevsnowluxechangeschangenotes