snõwkit dev log #5 (phoenix)

sign up sign in

Out of the fog, a slightly delayed dev log appears (from the lands of deliberation).

previous editions
- #1 “assets”
- #2 “history”
- #3 “modularity”
- #4 “data flow”
- #5 “phoenix”


Busy, busy

As some have noted it does appear that I am pretty swamped the last few weeks. Aside from important things that I have to take care of outside of snõwkit, in life, I have also been finishing up a project using luxe. It was an opportunity worth seizing, so it took a bit of focus. It's really awesome, and that's all I can say about it for now.

What I mean to say is - I have been doing snõwkit related stuff, and I'll do a whole write up on it when I am able.

I did want to note - in general, even when I am swamped, I still have time to poke my head up for air, write dev logs, and keep things moving forward. The time i've been slow to or not replying on chat lately is mostly because the life-tasks I had to deal with were time sensitive, and I still have work to do aside from community stuff, and if I don't balance things out everything suffers.

As mentioned though, progress has and will be made - I use snõwkit libraries as a much needed distraction to keep my life balanced and stay sane. Nights I can't sleep and the moments to think are spent doing stuff (which sounds weird but hey, I'm weird), and are used to think over designs and make this the best it can be given the time frames.

Sometimes that means coding instead of writing dev logs!

Decisions, decisions

The bonus part of having something to keep my hands tied in between alpha-2.0 and alpha-3.0 on luxe is that I had a good amount of time to deliberate over the next milestone, and what it means to the engine. I had written up a post about alpha-3.0 and after many hours of thinking aloud with friends and users I reminded myself of a key part of moving forward.

The biggest work left in finishing luxe 1.0 all stem backward from the renderer - phoenix. Even mint hinges on it being focused on ideally. Anyway - As I've mentioned a number of times, phoenix is going away in favour of a new renderer, one that is a bit more intentional about it's purposes and goals - as the current iteration of phoenix was created as a stopgap for getting luxe usable.

When you look at an engine - there is a separation between the renderer and the higher level constructs (like Sprite doesn't really care what happens under the hood and doesn't always need to) - but that isn't really the case in details. The details are what matter most to me.

alpha-3.0?

The new renderer is the current focus going forward.
I'll make the dev log and focus sheets soon - there will be lots of fun stuff to be done for the community so if you're looking for tasks to help out on (on the new renderer AND in general) - get your coding gloves on and get involved.

A brief phoenix timeline

I've mentioned before but phoenix has been around for some time in various forms. Long ago it was a c++ 2D engine, which was then rewritten to be an experimental 2D rendering only library, which was refactored into phoenixCore - which was the basis for my previous engine.

That previous engine was written with a really thin c++ layer, and the rest of the game specifics were written in Javascript (where V8, JavascriptCore, SpiderMonkey etc were embedded as the scripting engine on various platforms). This also meant that phoenix took form of a scripted interface, and had more features on top of the phoenixCore library to integrate well with that engine.

That iteration was used in a lot of games by me and a few other developers over some years, and we came across a number of design choices that in practice, created problems. These we had made for ourselves of course, but they were good learning in what to fix in subsequent iteration.

Further along the road, I found Haxe, and immediately went to work on porting what was there in the previous engine to get started with luxe. This is where the "stopgap" phoenix came from. I knew that the structure and design of the phoenix I had was along the lines of what I wanted - but I wanted to take advantage of newer knowledge to better it, while at the same time having something usable short term. I took what I had and quickly ported it to Haxe, and it served us well up until now.

It's got some crazy in it

There are two major flaws (and many smaller ones) that cause the biggest problems in the stop gap phoenix. This section is more about explaining the process, more example of the exploratory approach behind luxe.

Forced batching

As some people waste their time trying to draw thousands of bunnies, they inevitably run into what is one of the major flaws with the current phoenix renderer - it gets really heavy really fast with a lot of items to draw.

It's really easy to explain why, too - the way the rendering works is by batching geometry together that can be batched, to reduce the number of draw calls sent to the GPU. The way it does this in the current version is 90% smart and 10% heavy handed, where the 10% is this flaw. In order to batch them, it has to have the vertices transformed and stored in a big list to send to the GPU.

It does this on every vertex that is batched, every frame.

I'm not a fan of bandaid solutions - especially not on temporary code - but you can still do a surprising amount of rendering using this (and the static geometry), plus you could always just throw geometry through the OpenGL calls yourself if you needed to but what was there had to go.

Fixed layout

This is the second major flaw. Most people don't need to worry about this but for 3D and customized performance profiles and effects it's pretty important - customizing the geometry layout that is sent to the GPU.

For example, the current geometry defines pos, color, tcoord as float4 (for alignment) and it used to (but has been commented out since) define normal as well. The “commented out” should be a red flag as it is - it shouldn't require internal code changes to change what is sent and how (aside from uniforms - you can send those easily already).

A more clear example is - in some 2D applications and engines - they may make more sense to send the geometry position as a short integer, in x and y only, transforming the 2D angle of rotation and stuff in the vertex shader. They build their application specifics, around the limitations that brings - but also the benefits. They may also decide to send 4 tcoord slots and do some kind of application specific masking and texturing in an optimized way. The key here is the "application" layer does the thinking and the engine allows the flexibility.

To add to both of these,
The positions are sent as world coordinates (due to flaw #1 above mostly) - rather then object local (which for 3D is pretty expected) and limits what you can do with the regular geometry.

Looking forward

This is where I've been deliberating - Do I make alpha-3.0 tasks the ones I had in mind (“Scene & Animation”) or do I make alpha-3.0 renderer specific? I could give some of the scene and animation tasks to the community to help with - they have been doing stuff on them in fact, plus there have been quite a few scene + animation fixes since alpha-2.0.

But that would reduce the amount of focus, increasing the time it takes to change over for good. Changing the renderer over is a big deal because it affects everything, so the sooner it happens the better.

I've mentioned that there are new transforms, vectors, matrices, quaternions and associated facilities for them - these come from the renderer. Aside from a few semantic changes (correcting the style of the methods and properties names) those won't really affect much user code. They will be in one swoop (see below) - and the most relevant change was previously mentioned - Delta will be used for tweening because of the changes.

Co-existing

And, as we know, the way that luxe is structured is around modules. One of the modules is phoenix - that's why it has it's own package, and that's why lot's of luxe types (in the luxe/ package) typedef to phoenix types, to create the connections needed. It's possible to change the underlying type quite easily, keeping the user code patterns the same.

What I want to do is put the new renderer alongside phoenix as another module, and not be in use directly by the default rendering until it's ready to be switched over.

luxe/
phoenix/
newrenderer/

This means that:

  • Switching will be "at once" with a clear, concise set of changes
  • Changes can be well pre-empted and you can switch sooner
  • Tests and feature tests will be isolated and alongside regular code
    • Which means that every user could run them safely and help find issues
    • No annoying branches, defines or flags getting in the way of dev or testing
    • Existing projects looking to ship soon (there are a couple!) can stick on phoenix as it works without worrying about bugs short term
  • allows the tests and code to be run at the same time as phoenix

This last one is especially critical when debugging a new renderer - having a way to reliably draw debug information, text, and more without every change being a potential failure - is invaluable. Keep that in mind when you write a new renderer, it will save you a million hours of pain.

When it's time, it's time

The switch will be clean cut, and phoenix will be binned as soon as the new renderer is ready. There will be plenty of warning - including this post - the important fact has been mentioned before, maintaining multiple veins of old code against an alpha library is a waste of time.

Iteration

The other note to take away here is that the constructs or rather, concepts in phoenix aren't gone away, I just won't suggest that no changes will happen in a switch to a completely new backend.

The new rendering module is an iteration on what is known to work well in phoenix over the years and will be way faster, simpler, and closer to the metal. This isn't going to affect the majority of users who work on the higher levels - but will drastically affect how the engine grows moving forward.

phoenix to ashes to ...

The next dev log will talk a little more about the new renderer, it's roots and goals and I'll tell you what it's called.

Until then!


community

Again - thanks to everyone that continues to make great things! Don't forget to post what you are working on, right here.

The http://snowkit.org/chat is always open, and you can find news on http://snowkit.org and on twitter at http://twitter.com/snowkitorg as it happens.

There is a lot to cover community side (since it's been a few weeks) but for now I'll just note that the 0010 team released their public alpha of The Westport Independent if you wanted to play it, now's your chance to give it a go.

There's always the hashtag and haxe roundups for a bunch of recent stuff.
Anissen also does a great job of keeping awesome-snowkit up to date which is a must see.


I'll make a bigger community section post on it's own to show all the stuff in one joyous post, as well as mention some things from wwx of relevance. Keep an eye out!

<3

Tags
snowkitdevlogsnowkitdev