snõwkit dev log #3 (modularity)

sign up sign in

I've been spending some time breaking everything finalizing the underlying snõw API!

This means there's a lot to talk about, which is what we have dev logs for.

previous editions
- #1 “assets”
- #2 “history”
- #3 “modularity”


Short changelist

If you're here for the quick "what changed" :
Take note that unless you were interacting with the underlying api's directly, this shouldn't really affect you.

Mainly what has been changing is the underlying structure in snow, removing exploratory code in favor of sticking to a direction.

Changes:

  • import snow.render.opengl.GL; => import snow.modules.opengl.GL;
  • snow.Log => snow.Debug, use import snow.Debug.*;
  • import snow.assets.Asset*; => import snow.system.assets.Asset;
  • import snow.audio.openal.AL; => import snow.modules.openal.AL;
  • snow.platform.native/web => snow.core.native/web
  • snow.platform.native.io.IOFile => snow.io.File
  • SoundStream type was consumed by Sound type
  • all systems moved to snow.systems instead of the root
  • all modules moved to snow.modules instead of the platform and other folders
  • all platform code moved to the snow.core separation.

More details on the changes are below.

Fixes:

Thanks to Haxiomic helping me commit some quick fixes:

  • Fix iOS black screen due to new Promise code in startup
  • Fix GL getUniformLocation inconsistency across platforms

New things:

  • Added assert and assertnull macros to snow.Debug and luxe.Log
    • assertions only exist in debug by default (or with snow_assert and luxe_assert defined), i.e their code has no consequence or cost on runtime as they are stripped out by macros! This also means that null values will escape in non-debug builds, so test those.
    • assert(expr) is a regular assert, if the expression evaluates to false, it raises an exception
    • assertnull(value) will throw if value is null. i.e
      • assertnull(texture);
      • //use texture confident it's not null

solidifying

The larger purpose of the changes currently going into snow at the moment are finalizing the details of the code as it heads toward beta. It's all going to be about turning the ideas explored into concrete implementations, and removing the exploratory code. These changes start answering the final how, when we already know what we want from snow and have already explored it's use and implications in practice.

I am really glad that Casey Muratori has been doing a series called Handmade Hero, you should definitely watch it if you're interested programming what so ever.

This video in particular overlaps quite a bit on how I approach programming, it's called "Exploration Based Architecture". Casey elaborates on this idea for the first ~35 minutes, and a lot of my processes in snowkit libs should start to make more sense. You should really watch the entire series, though - It's invaluable.

cores | systems | modules

snow was designed around modularity, inside the c++ code as well as in the Haxe code. If you looked closely, you'd notice that all the code was sectioned out into folders by platform (web, native), and then further by provider (openal, sdl, opengl). In the c++ code, it goes as far as letting you switch out each section with a custom implementation (or using a different provider, like GLFW instead of SDL) but this was all largely exploratory - trying out approaches that best fit for what I wanted.

To finalize all this to concrete implementation, code has been clarified into the following three things:

  • systems
    • snow system api (audio, assets, io, windowing, etc)
    • previously, folders were in the root,
    • this mixed system implementation with user facing api (bad)
  • cores
    • core system implementation
    • vary on a fundamental level, enough to warant clean separation
    • i.e web vs native
  • modules
    • replaceable system api implementations
    • separated from the core code + systems
    • configurable by build configs

systems
To further clarify this, consider the audio system. It offers an api, which must be implemented somewhere. It must make some form of guarantee as to what functionality is available, and give you a fixed api across all platforms.

This is the snow system api for audio.

cores
Now, consider web audio vs native audio, very different - their core code path is going to differ, guaranteed. They'll differ enough that I'd want to keep their code separate. It must be easy to maintain them, where changes to the web assets don't accidentally break native and vice versa. There are many times some core code might be able to take advantage of the platform (say, with threading or other native specific affordances) - which should not be held back by another core implementation.

Unlike the differences between say, Android and iOS, which are both native, their code path is largely identical (also by picking cross platform api's like OpenAL). The platform code (i.e OpenAL, SDL, snow c++ code) handles the native differences internally, presenting a unified API for the higher level code, which is essentially a second lower level doing what we are doing above it.

These are different cores, they implement the essential details for the systems to work, and provide base implementations for modules to fulfill.

core modules
So now we have the user facing system api, the core separation, we still need a place to put the different code in a modular form. This is what modules are for.

There are two kinds of modules, core modules and modules. If you think about it, the system api offers some functionality, like play a sound. The cores split the differences internally for us, and then a core module within each one of those cores would carry out the work of playing a sound.

Like snow.core.native.audio.Audio is a core module. It fulfills the api provided to you, by snow.system.audio.Audio.

modules
And finally, modules should actually handle the implementation details. On web we use howlerjs, on native we use OpenAL - but this isn't set in stone.

If you would like to implement your own audio module, but keep the code the same on the using side, you can. Say you were writing an FMOD audio module for others to use, the user would be able to switch to using your FMOD module without changing their code (unless they used the OpenAL module directly).

Which brings us to the last and final important distinction - modules can and should and do implement more than what the system api offers you.

This includes the snow core modules. Since there are differences on web/native and even desktop/mobile, the modules can have custom api exposed.

To access these, you go directly via the module, not the system api. Take for example the snow OpenGL api, you can call it directly and do full GL code. Same with OpenAL, you can ignore the built in audio module implementation and just code OpenAL yourself directly - already.

Another example, the snow io module on native implements dialog_open, but it's specific to desktop platforms only. To access this function, you ask the module. Inside a snow app ready function, you'd call app.io.module.dialog_save, rather than app.io.dialog_save. This makes it clearer because 1) you're potentially using an api that doesn't exist on other platforms and 2) you're explicitly doing so 3) that api can be completely custom from a module, modules can provide full apis on the platforms they choose.

This is also true of the snow.modules folder, and I'd urge module developers to take this into consideration, that when you're reaching into the module or modules naming from user code that it's clear how to find where that code lives.

Along those lines, you can always query the type of the module at runtime, and with debug logging enabled in snow, it will print the modules that are actively assigned:

Here you can see that Assets and IO are using the core modules, and the Audio is using openal, the Input and Windowing are handling some SDL specifics from the c++ code. In the case of the SDL ones, they are actually only subclassing a single function (on_event) and deferring the rest to the core modules.

The new code allows more clarity in this regard, and especially allows you to completely replace the modules with your own, or another one that snow provides later. This is especially important for porting to new platforms, instead of having to deal with third party api's on the platform, you can leverage the platform api to its fullest extent, and just implement the required pieces to get your code across.

module interfaces
To deal with this flexibility, and to make it easier to know exactly what the core and systems want from you - I added the snow.modules.interfaces folder. These interfaces you can use with class MyAudio implements snow.modules.interfaces.Audio, and then the Haxe compiler will tell you what the minimum requirements are for the system api.

Easy huh!

luxe modules

You've heard me say this a bunch before probably about luxe modules, this stuff is paving the way, but not directly related. Only a few of the core modules in luxe make sense to replace (the rest make sense to disable) and luxe user modules I refer to in the luxe design are more built on top of luxe api's not as luxe api's. This will make sense when we get to them.

finalizing the finalization

While the new implementation is done, there are a few things that need finishing, like the web howlerjs audio module has it's fingers in the web core code, not what we want of course.

If you encounter bugs, please be sure to report them either on the issues list or in the chat, on twitter.

I'll probably write about the implementation details of this module system and how it works on my own dev blog because it's a good use case for how flexible Haxe is in practice.

If you're curious you can look at the code in the mean time, it's actually quite simple. The modules are set in the flow file, like the default openal module is enabled with the following line:

build:{ flags:['--macro snow.Module.set("Audio", "snow.modules.openal.Audio")'] }

And the default core modules are set in the same way:

snow_native : {  
  build: {
    flags:[
      '--macro snow.Module.set("Audio",     "snow.core.native.audio.Audio")',
      '--macro snow.Module.set("IO",        "snow.core.native.io.IO")',
      '--macro snow.Module.set("Input",     "snow.core.native.input.Input")',
      '--macro snow.Module.set("Windowing", "snow.core.native.window.Windowing")',
      '--macro snow.Module.set("Assets",    "snow.core.native.assets.Assets")'
    ],
  }
}

Along with these last details taking shape, the complete structure can now be documented in full - because they're not changing again.

Notes on documentation

This brings me to my next point - people have asked this on previous dev logs and in chat before and I wanted to just write this here in one place to refer to in future.

Starting with: documentation is absolutely paramount and a high priority for me, but never at the expense of actually finishing the code to be documented.

Read the above real world example, and you'll know that the documentation for the code before that point needs to be completely thrown out. Everything written about the systems before today would be useless.

When you've been coding this way for a long time, exploring first, finding what you want from the code, and then committing to it and progressing forward; you either learn the hard way or you learn the hard way. In both cases you do double, triple and sometimes infinite treadmills worth of work that you're throwing away. I've done this before, and I just don't do it anymore.

There are two main kinds of documentation to cover before calling it done: the code documentation (commenting the code for the API docs) and the guides + tutorials + samples.

  • Code documentation
    • Whenever I open a code file that is missing code comments, and I am committing changes to that file, I will document and reformat the file as well
    • If I encounter code that I know is going to change, I move along.
  • Guide/tutorials/samples
    • Each alpha includes a new sample that covers the FAQ during the prior focus sheet
    • Guides for systems that are not going away or changing are written already, or when completed
    • Guides for systems that I know are changing are a waste of time, and more importantly for me, create a false sense as to the state of things.
    • Personally, in my experience, when a person is reading “completed” documentation for something and they understand it, they internalize it. They start to form mental models. They start telling others and (with the best intentions) help others to learn these (incorrect) things, it could change the same day. This leads to times where people are "proven wrong" when they were actually justified because the documentation did actually teach them that - and then betrayed them. Sometimes the mental model break down can feel too much, and make the code seem too difficult or too complex. But it's important to me because losing trust in something at an early stage is a shame.

Simply put, it's what it says on the box. We're not done here - and that is ok.

It should be clear hopefully this far that I'm not just going to leave things undocumented. If there are missing guides for something there are good reasons, and they usually fall under one of two: A side effect of time, or a side effect of change. One leads to the other, I choose to spend the time on the code because that's what's we're here for in the first place, and that's what the documentation refers to. If it's not done yet, it's not time.

Either way - if you thought the docs and guides that are done were “neat”, just wait till you see what I have planned for the final documentation.


luxe alpha-2.0 progress

You may have thought the movement on luxe code side might be slow but there are reasons - firstly, I wanted to keep the Haxe 3.2.0 testing easier for everyone, if there were large scale sweeping changes in the engine code, it might be easy to miss something important.

Secondly, a lot of the changes that the assets need in luxe, rely on changes in the asset system in snow. Only recently (around dev log #2) did the new io code come into the master branches, and it's been tested in the field a bit first, before lugging some new code on it's back.

Like the iOS black screen bug which was caused by this very code I'm referring to, it's important to give us some breathing room to verify that the code is actually working as intended.

That said - things are still progressing well. The snow changes are nearing completion and the luxe asset changes should be in shortly, and there are some important things to note about alpha-2.0:

alpha-2.0 in split parts

Due to certain things taking longer than anticipated, implementing the new underlying IO code, a new Haxe release on the horizon and various other things - I've decided to push certain parts of the alpha-2.0 milestone forward into a different focus sheet. This is partly because of the thing mentioned above about changes needing time to be tested, and also to give users a chance to keep up.

More so,
When I posted in dev log #1 about the tagging and and reloading stuff, I was still in exploration around those systems and didn't have every detail down, so I held off on changing the issue list to reflect that. I'd rather that the tasks be discrete and clear and easy to follow, so you know when they're in and usable.

To that end, I'm going to redo the milestone tasks to better reflect the current more critical changes (the fetch+load separation, the promise based api, the new evented resource manager) and wrap that up ASAP. This gives us a small window for some other needed changes, mentioned below.

ludumdare 32 coming up

I love ludumdare! I've been involved as a friend, participant and spectator since around LD 16 (iirc). It's a great way to make stuff quickly, and learn a lot about your tools.

It's on April 17th, with the time and date specifics here http://ludumdare.com/compo/

I was pretty surprised (and pleased) by the output of the community during the last LD, and it would be silly of me not to facilitate it better - say by introducing a bunch of changes to be tested right before. Let's do the opposite, and make sure things are in a good place for the jam.

There are a few things that we'll try to get done ahead of time for ludumdare, which some of the snowkit devs have already basically done:

  • merge up alpha-2.0 asset changes sooner, pushing the other two bigger changes afterward
  • document some of the luxe modules that won't change much (states, input, etc) and the new ones (assets, random, etc)
  • potentially: add the alpha-3.0 sample, and maybe more entry tutorials
  • finalize the changes in hxcollision and merge them upward to luxe (nearly done, with PDeveloper, KeyMaster-)
  • fix the nape debug draw being slow and silly (done, KeyMaster-)
  • push existing changes to TextGeometry (done, for color and other markup)
  • make sure all the tests, api docs, and tutorials reflect the latest code

Community

As always, http://snowkit.org is a place to share your projects, it's your community. I'd really love to see more of the awesome work I see on twitter and the chat right here on the community home page as well.

Don't forget the http://snowkit.org/chat and can conveniently find news right here or on http://twitter.com/snowkitorg

I forgot to share this previously, but Bennett made a fun game for his students using luxe, which you can play by clicking the link below:

http://www.foddy.net/ice/

 


Also, don't forget to follow the #luxeengine hashtag and see all the cool stuff people are up to (which they should be sharing here hint hint).

Tags
snowkitdevlogsnowkitdev