snõwkit dev log #8 (access)

sign up sign in

A look at snow development history, present, and future.

previous editions
- #1 “assets”
- #2 “history”
- #3 “modularity”
- #4 “data flow”
- #5 “phoenix”
- #6 “community”
- #7 “embers”
- #8 “access”


Let's start with what I've been doing outside of snõwkit and get that out the way.

I've said before, that when I'm sorting out things in life outside of the community – I have to spend what free time I do have carefully. It means that when I have to choose between writing dev logs or finishing important aspects of code; The code will always win. This is especially true during the alpha (it's why we are all here technically!).

The good news is that one thing that had a frequent tendency to stick it's head through my window and mess with my attention span is gone. We travelled to Toronto early November to finalize our permanent residency here in Canada (after moving from South Africa in 2013), so that is all behind us now.

Even though there have been a low number of commits in the last month~ish compared to usual, trust me I've still been busy!

embers and luxe alpha-3.0

I have to mention the current focus, because I am about to detail a bunch of stuff I've been working on that isn't related to the current focus.

This is twofold, the first thing to note is that a lot of what I'm going into has been in the works all along. The second thing to note is that sometimes (frequently) the thing you're working on becomes less important than something else, this is especially true when it is time sensitive.

If we had a long winded roadmap and were all rigid about it we would miss out on being adept and flexible, and we wouldn't be able to solve immediate problems when they mattered most. The workflow we use here (exploration based coding) gives us the benefit of being able to address the most important thing right now making it the current focus.

It also means that your projects aren't thrown into disarray because half complete code is plaguing your projects while I had something to solve, everything remains usable as it was before.

This all sounds pretty vague so let me narrow it down:

  • The Westport Independent will be releasing sometime. They're a team and project I courted way before luxe and snow were public. Me supporting their release as their technology provider is a part of the success of luxe/snow/snowkit. Their successful release is our success.
  • Releasing the game to a wide audience on a version of snow that's going away makes no sense. All the benefits of a release, like hardening the code in the wild, would be more appropriate for the work that's been in process for a couple of months.
  • The new snow backend also solves features required for release, especially on mobile, like native plugins (achievements etc). This ties into the native SDK I've mentioned before, and it's all connected.

Does that mean that embers is stopped or anything? Not at all! It just means that it gets much less time allocated immediate term by me specifically, while I finish up the required changes to snow.

If you wanted a succinct way to say it, the current focus is temporarily switched to snow and will be back to embers at full focus very soon. I've done all the harder snow stuff already (more details later), so it's basically too late to worry :p.

access?

A core design principle of snow is access.
It's the platform layer : it gives consistent and cross platform access to some fundamental API's (like assets, input, audio, etc).

The second tier of access in snow is access to the platform specifics. This is why instead of abstracting OpenAL behind the scenes internally only, you can import OpenAL directly and use to it's fullest extent. This was mostly true for OpenGL (WebGL spec), but there are still parts in the current code base that weren't (easily) accessible - like SDL API's.

Previously, while I was working on the snow native SDK, the goal was to expose the c++ backend stuff (like SDL) to the user space via a form of c++ library and Haxe API combo. Once I started it though, I soon realized that Haxe has the power to give us the type of access I would prefer – Haxe itself!

If you've been following along with recent progress you would have seen linc arrive. And if you're astute you probably have realized that this is what I'm talking about and where we have been headed for a long time with snow.

When I said months ago that “the ndll is going away” and that there won't be any more annoying native build disparity issues this is what I was referring to back then.

snow+linc

When I had this idea for linc + migrating snow to use it, there were still quite a few things hxcpp wasn't ready yet (to be more specific: the features were in active development in the 3.2 cycle and weren't usable by end users, and the rest was subsequently addressed).

It gave me time to explore and find a good path forward and especially: make sure that all the potential pitfalls of switching to this type of integration were resolved. The majority of that was solved by linc - it started as a proof of concept and soon I had entire libraries working smoothly with a drop in nature. It felt like Haxe feels all the time, it was like magic.

This is why linc started in private repos, I wanted to be sure that it was viable before everyone was excited about the concept. I really dislike the type of “false hope or wasteland of abandoned projects” style of development (which sadly is a common trend for developers of all kinds, their enthusiasm blinds them from sensible choices sometimes).

This is also true for the concept of replacing all of the snow c++ backend with linc libraries – I wasn't sure of a couple of critical things and needed to answer those questions by coding. It was at first, just an idea, an exploration, and it was best done in a place that wouldn't disrupt existing users or workflow. When current code changes are migrated to a second branch sometime this week hopefully, they'll be in an almost complete form.

This screenshot may look the same as before (a good thing): this is the existing snow samples running on the new linc based version of snow.

snowdev

snow modularity

In dev log #3 I spoke about the nature of snow and it's modules, how moving toward modularity was required for all of this.

It worked out really well, and in fact the backend itself (in c++) is modular too, so it was possible to use your own implementations of things as needed. Possible, but not as simple.

Once the exploration of linc libraries was complete and the proof of concept for snow being "all Haxe" had passed important workflow and stability tests, it was time to pursue it properly. A lot of work to be able to migrate easily was already done, it was just a matter of replacing some internal functions with Haxe functions and deleting everything that doesn't matter.

However, some things about snow that I didn't like (structurally) that came from my original ideas when creating lime, back in 2012/2013. It would make more sense to address them while restructuring now than later, so the new code includes a whole lot of changes.

snow core restructuring

One benefit of exploration based coding is that by this point I already had a good picture of what I wanted snow 1.0 to look like, and started moving toward the modules and replacing the c++ backend. Another part of that was simplifying the core structure of snow.

Around June I redesigned the internals of snow and started experimenting with empty repos (i.e starting snow from scratch, repeatedly) just to see what decisions I'd make again and which ones I'd throw out. I also tried to finalize how the final version of the runtime will look, and how platform level extensions will fit into the runtime and vice versa. It looked a bit like this at the time.

snowcore

It has changed since so don't rely on this for anything but it's really close to what I was thinking. The even more final version is even simpler. I can't wait to switch in fact, but like always, I'm interested in doing things right and at the right time.

incoming snow changes

Technically the code I'll be merging into master (it will first be in a second branch for testing) is actually started from scratch, there are changes headed in there too (to be expected during alpha). Most of them are really great if you work on the snow level, but the majority of luxe users won't know the difference when it changes.

Below is just a few important changes worth mentioning that are done and will be up soon. A lot of them are focused on removing unnecessary abstraction and putting more control over what the application is doing into your hands. When I say "user space" I mean “from within your flow project or Haxe code”. You'll have control over a lot more in easier ways in the new snow.

It's important to stress that all of these changes are internal, and the "sane defaults" and the existing way your luxe or snow apps work remains the same. There will be minor API changes and clean up but that's not going to render any existing project unusable unless you were relying on internal structures that were expected to change during the alpha.

In cases where important sounding API was removed, there is a parallel API endpoint that does the same thing.

linc backend
This is the biggest and most beneficial. This is specifically a boon to access, because it means that with the default configuration of snow - you will now have full access to all of the SDL API's from your app.

Android project revised
The current android project was based on the SDL project, which is based on older android workflows (ant and friends). The new Android Studio based projects use Gradle, and it significantly improves the android workflow and ecosystem. I've wanted to switch for some time, so we have.

This also changes to match how iOS works, once the project is generated it is yours to break use. This means you can run and modify your luxe/snow projects from within Android Studio and all android specifics in the workflow will be consistent with the platform workflow directly.

For existing workflow, when you run flow build android it generates a new folder and nothing else changes from before. If and when you need to change your android project stuff you can do so in your folder or via the Android IDE.

Modular core and runtime
The new version of snow also exposes the entire runtime and innards to the user space more directly. For people making their own frameworks or ports, it's all done directly from Haxe and is controlled from the flow project file.

This is especially useful for users that have special circumstances or on new platforms where the default assumptions would have gotten in the way.

A fun example: imagine telling snow to use your custom runtime, but that runtime is a CLI ascii main loop and your output is terminal based. In the new snow, you could do this and still have access to audio and assets (obviously without a window, rendering like OpenGL or window based input falls away).

Cleaned up bootstrapping
Technically it was possible before to customize the boot process, but it leaned on the flow build templating and various changes to get right. If you don't know what this means, the "SnowApp" and "LuxeApp" files that you find in your build process - they're gone in the new version.

These files did the job but they caused a lot of headaches, especially for systems like code completion in IDE's that require them to exist, which requires at least one build-like-thing to generate the file correctly.

The boot process in the new version is all code/Haxe side now, in user space. It's significantly simpler and also like below, strong typed.

Stronger module typing
In a few ways, the existing module system had some edge cases where typing got confused and there were completion errors. There were also pieces of the user specific workflow with snow that didn't have a strong type (like config.runtime in the config functions? Those were Dynamic because they came from JSON).

This is no longer the case in the new snow. It's all 100% deterministically strong typed, which makes code completion more consistent, and user specified modules and types are all strong typed internally too.

Here's an example, in image form, of the user runtime config from JSON but strong typed. (It's also now called config.user, a minor clarity change).

user config

API pruning
This one might sound worrying but I assure you that endpoints I've deleted have parity with the above mentioned access.

A lot of the api in snow was driven by the abstraction across platforms, where platforms have disparity with one another fundamentally (this is quite common and is not necessarily bad). An example would be app.windowing.display_native_mode. This is a function that truthfully only makes sense on desktops (3/5 primary targets) but has to exist and muddy the waters on web and mobile in the API.

To me, I prefer that the cross platform API is consistently the same, and anything superfluous be module level - which was partially true before - but not in a few places.

So, any API's that didn't make sense are gone, especially API's that were access related (i.e very important on desktop, but not on other platforms). These API's are now directly accessible, so having them in the snow API is redundant.

This leaves us with a really clean, explicit API that's easy to port, easy to maintain, and gives you with the freedom to make platform specific choices the way it should be.

Runtime structure
The last and probably biggest internal change is related to the runtime. I've mentioned it above and I'll go into more detail soon (and in the documentation).

Simply put, some modules have been collapsed into a single runtime module. This makes porting easier, and there is less API boilerplate or disparity around - like trying to false parity across platforms like mobile, web, desktop and game consoles.

snow plugins/sdk

This all wraps around to this important point, the new structure and projects make it much easier to access the native functionality of the platform (like on iOS and Android) without it being a fragile workflow.

As always though, I'm not only interested in solving this for snow/luxe users but for all Haxe/hxcpp users. This will need it's own post for detail, so once I push all this live a post about these plugins will follow.

Quite soon access to the iOS/Android services will be available, which I know a number of people have been waiting for.

Infinity

Before this post reaches this headline in length,
I'll have to leave you with some notes about what else I still have to mention.

  • A post mortem/case study of that project I was doing
  • A post detailing some new community initiatives
    • this includes some fun secrets
    • this includes a couple of proactive community members writing community posts with me, to keep things more visible and steady
  • A post detailing the abovementioned plugins
  • A post showing the new work done on embers so far

I'll be around on the chat and working my way through the issues as usual now that I'm back from travels and so on.

That's it for now.

For more immediate news, follow on twitter.
Join the chat, sign up and don't forget to check out and share what you're working on using #luxeengine or #snowkitcommunity hashtags on Twitter.

Tags
snowkitdevsnowkitdevlog