Nothing much here, but I thought I’d share a handy little library to decode WebP in Flash. Here’s two demo’s : Lena and V for Vendetta, both encoded using the webp binaries provided here.

It’s a small wrapper around the WebP library ported with Alchemy. The port is pretty easy to do using Alchemy, aside from some issues with the Alchemy toolchain pretending to be a normal gcc toolchain; it enables some x86 asm code for cpu identification in the webp code, which obviously won’t work in Flash. After those issues are resolved getting up and running is easy.

However, I’d figure I’d post this as I have seen several requests for making webp work in Flash.

Although webp decoding isn’t there natively om the FlashPlayer, this should help you a long way. Read further for an example on usage and download of the swc.

As for this blog; due to time constraints I find myself not being able to update it regularly, even though I want to. Also, as I do a lot more tech then just Flash these days, I’m looking for a way to post experiments in other technologies, without the daunting task of making a full scale blogpost about it….more on that when I figured it out.

Download webplib.swc.

public class Main extends Sprite
{
	[Embed(source="V.webp", mimeType="application/octet-stream")]
	private const WebPImageClass:Class;

	public function Main()
	{
		stage.scaleMode = StageScaleMode.NO_SCALE;
		stage.align = StageAlign.TOP_LEFT;
		init();
	}

	private function init():void
	{
		WebPLib.init();//Only needs to be called once.
		var bitmapData:BitmapData = WebPLib.decode(new WebPImageClass() as ByteArray);
		addChild(new Bitmap(bitmapData));
	}
}

An image of ShadowGun a game made by Madfingers, using Unity.

If you wonder why it’s silent in here again; same old, same old; I’ve been busy. So many things happened this year, most of them very good. Workwise, I have joined the brilliant minds at Unity Technologies to work on further expanding the reach of Unity. Today is exiting, because over on the Unity blog, there is a first public sneak peek of our progress. Let us know what you think !

JFA Voronoi Paper by Guodong Rong and Tiow-Seng Tan.

WebGL / Canvas – Jump Flood Voronoi Experiment. from Ralph Hauwert on Vimeo.

Demo Video of WebM being decoded by Flash. from Ralph Hauwert on Vimeo.

This is a little side project I have been playing with. I’m porting google’s libvpx using Adobe Alchemy to run WebM in Flash Player 10, which doesn’t support the format natively yet. Because it’s not much more then a small and experimental side project, I quickly made a video to show it’s progress up until now. Monday the new part in the Particle Sessions will appear.

Part One of the Particle Session kicks off with looking at a system similar, but simpler then this Bloom particle experiment (music made for it by the fantastic artist Rich Bologna, thank you again for it, Rich!). We’ll be taking slow steps of optimizations and at the end have a get prepared for the next level. To be able to get up to speed with the working of the video particles, we’ll start from the beginning and work our way up. For that reason, we’ll start very simple and look at a basic unoptimized particle system with next to zero functionality, and modify the code for performance.

Basis : The Particle

The opening picture of this post is the album cover of The Strokes – Is This It. It being one of my favorite albums, I have listened to it many times, but never noticed what the cover really was. It’s amazing & inspiring what imagery one can get from science. Now put on your headphones and listen to that album while reading this post.

Over the last years I’ve more and more started enjoying introductory books about theoretical physics. In complete honesty, it wasn’t even that long ago that I didn’t know that the Boson in the Higgs Boson wasn’t the other guy discovering it, next to Mr.Higgs. You know, like the Hale-Bopp comet was discovered by Hale & Bopp.

Luckily for us and this post, we don’t need to deal with wave particle duality and quantum entanglement, but we can achieve seeming complexity by using simplicity.

Getting started : Simplicity & Complexity

The simple core particle we will be using in this session get its complexity from its display of large numbers of them, not from per instance properties. To illustrate that, let’s start off with a simple particle object in AS3

package com.unitzeroone.particlesessions
{
	import flash.display.BitmapData;

	public class Particle {
		public var x:Number;
		public var y:Number;
		public var velX:Number;
		public var velY:Number;

		public function Particle(x:Number, y:Number, velX:Number, velY:Number) {
			this.x = x;
			this.y = y;
			this.velX = velX;
			this.velY = velY;
		}
	}
}

As you can see, this particle object is pretty simple, and it contains the core properties we really need to make them move. Its position stored in x and y, and its velocity stored in velX and velY. For the case of this post it’s more then enough to do what we want.

I do want to point out two things about the simplicity of this. First of all, we express all values in absolute x and y. For the positions, this would be called a cartesian system. In some cases it might be handier to express velocity as a polar coordinates, but performance reasons we’ll keep them like this.

OOP (s)

It’s very common within the AS3 industry to “objectify” everything in nicely wrapped classes first and think about performance later. In the case of our particle and a classic “update” loop as seen in game engines, the most natural feeling way to implement the particle’s moving behaviour for the particle is as follows.

package com.unitzeroone.particlesessions
{
	import flash.display.BitmapData;

	public class Particle {
		public var x:Number;
		public var y:Number;
		public var velX:Number;
		public var velY:Number;

		public function Particle(x:Number, y:Number, velX:Number, velY:Number) {
			this.x = x;
			this.y = y;
			this.velX = velX;
			this.velY = velY;
		}

		public function updateAndDraw(toBitmapData:BitmapData):void
		{
			this.x += velX;
			this.y += velY;
			toBitmapData.setPixel(this.x,this.y,0xFFFFFF);
		}
	}
}

This is a very logical thing to do and is suitable for most things, especially in larger code bases, it certainly helps in readability. It also matches many things you’ll see in OOP and AS3 books. Like car.drive() or lasership.shoot() or avatar.updatePosition(). From a pure OO perspective, it makes sense. From a performance perspective, this isn’t as good though. We’ll tackle that in a bit.

Main

Now, let’s say we’d have a very simple Main class to run this system, as such :

package {

import flash.display.StageScaleMode;
import flash.display.StageAlign;
import com.unitzeroone.particlesessions.Particle;
import flash.display.Bitmap;
import flash.display.BitmapData;
import flash.display.Sprite;
import flash.events.Event;

	public class MainOne extends Sprite {
		private var bitmap : Bitmap;
		private var bitmapData : BitmapData;
		private var particles:Array;

		public function MainOne(){
			init();
		}

		private function init():void
		{

			//Create a canvas to draw too
			bitmapData = new BitmapData(500,400,false,0x000000);
			bitmap = new Bitmap(bitmapData);
			addChild(bitmap);

			//Create a set of random particles with a random velocity
			particles = new Array();
			for(var i:int = 0; i<=1000; i++){
				particles.push(new Particle(Math.random()*500,Math.random()*400,Math.random()*2-1,Math.random()*2-1));
			}
			//Wait to be added to stage
			addEventListener(Event.ADDED_TO_STAGE, onAddedToStage);
		}

		private function onEnterFrame(event : Event) : void
		{
			//Clear the stage
			bitmapData.fillRect(bitmapData.rect,0x000000);

			//Lock the bitmapdata, while we are updating it.
			bitmapData.lock();

			//Draw particles
			var p:Particle;
			var l : int = particles.length;
			for(var i:int = 0; i<l; i++){
				p = particles[i];
				p.updateAndDraw(bitmapData);
			}

			//We are done updating, unlock the bitmapdata
			bitmapData.unlock();
		}

		private function onAddedToStage(event : Event) : void
		{
			//If added to stage, set stage properties and start listening to the global enterframe.
			stage.frameRate = 30;
			stage.align = StageAlign.TOP_LEFT;
			stage.scaleMode = StageScaleMode.NO_SCALE;
			stage.addEventListener(Event.ENTER_FRAME, onEnterFrame);
		}
	}
}

For clarity sake I’ve omitted a couple of things, but this code runs. If you run it, you’ll see bunch of particles moving at a constant speed, moving offscreen. Our simplest of simple particle system works, so now we can start looking at performance and why we need to change the structure for performance.

Performance, the small issues

If you aware of performance caveats within Flash, you’ll immediately notice a couple of things, amongst which the lesser problems like for instance;

bitmapData.fillRect(bitmapData.rect,0x000000);

A lesser known fact about this piece of code is that the property rect of the bitmapdata is not a static read-only rect, but that every time you query this property a new rect is instantiated. Instantiation is rather expensive in the AVM2, but in this case, it’s called once per frame, thus not a major issue. Still it’s unneeded. Keep this in mind when you do many things per frame with bitmapdata.rect. In general, keep in mind that instantiation & allocation is expensive.

If you ever had one of those swf’s which seemed to hickup every now and then, chances are that it is the garbage collector kicking in. It doesn’t have to be one specific place where you are instancing to many objects, it might be more then one. So keep things clean from the beginning. Also keep in mind that many small performance losses will result in one big one.

The larger performance costs

There’s multiple things we can optimize in this piece of code. For now let’s keep it a bit simple and look at the larger issues, where we get the most gain. One of the main issues is how we store and access the individual particles.

particles = new Array();

We currently store them in an (as always in ActionScript) untyped array. If we’d like to keep the “storage” an array like structure, we would be better off using a Vector for storage, but we can get it even a bit faster by using a linked list instead. For the purposes of this simple particle system, that data structure is enough, as we are only going to linearly iterate over the particle set in one direction. This is called a singly linked list.

Let’s have a look at how that works.

Linked List

First, we modify our Particle class, to contain a link to the next particle in the list, by adding this to its properties.

public var next:Particle;

Then we modify the Main class to instanciated the particles like this :

var particle:Particle;
var previousParticle:Particle;
for(var i:int = 0; i<=1000;i++){
	particle = new Particle(Math.random() * 550, Math.random() * 400, Math.random() * 2 - 1, Math.random() * 2 - 1);
	particle.next = previousParticle;
	previousParticle = particle;
}
firstParticle = particle;

For every particle we create we store a reference to the previous particle in that particle. Then when we are done with the loop we keep a reference to the last created particle. That particle refers to the particle created before that, and so on, until we reach the first particle which has no reference to any particle.

We now update the drawloop to do the following;

var p : Particle = firstParticle;
while (p) {
	p.updateAndDraw(bitmapData);
	p = p.next;
}

Final(ly) ?
Using a linked list instead of an Array helped, but we can even more performance for free, by adding the “final” keyword to the particle class.

final public class Particle {

You would use the final keyword normally to prevent classes from being subclassed, but there is an interesting piece about final in Flash AS3 and AVM2 performance tuning, which by now is an aged document. In this case it indicates that sealing classes using the final keyword helps the AVM because the properties are already resolved and bound.

I find that in Flash Player 10.1 the advantage is less noticable then 10 and before, but it still has a slight performance gain on my machine, although I’ve heard it been debated more then one time. It seems to me also that some platforms/architectures benefit more then others. My naive way of thinking about it is that if you’re not going to subclass it anyway, you might as well finalize it. Simplicity.

Function calling
It may seem obvious to call a method of an object (like with car.drive()), but if you want maximum performance you should avoid function calls. With this in mind, let’s take another look at our draw loop :

var p : Particle = firstParticle;
while (p) {
	p.updateAndDraw(bitmapData);
	p = p.next;
}

The updateAndDraw call here is the costliest thing in the main loop. Not because what it does, but because it is a function. In general, function calling is always going to have some form of overhead, but in the case of Flash / AVM2 / AS3, the cost is exceptionally high. Although this situation has improved with Flash 10.1, the overall hit of just calling a function inside the loop is way to high to be acceptable if we want to achieve high numbers of particles.

Inlining

Inlining is something a compiler does for for you to replace a method call with the actual functionality in place of the function call.

AS3 does not have an inline keyword natively, and the compiler wouldn’t know what to do with it. Joa Ebert’s Apparat tool suite, written in Scala,provides amongst other incredibly handy things a way to do inlining in AS3. I highly recommend using it for larger projects where code style and performance are important. I actually also would highly recommend it for any project where performance is a priority. For this instance though, let’s go to a simpler solution first.

We’ll remove the updateAndDraw method from the particle class and make our main loop look like this.

var p : Particle = firstParticle;
while (p) {
	p.x += p.velX;
	p.y += p.velY;
	bitmapData.setPixel(p.x,p.y,0xFFFFFF);
	p = p.next;
}

Yay. For now you’ll have to take my word for it, but this is faster. (later on in this series will prove this with benchmark tests, I wanted to avoid this post just being a bunch of charts and performance graphs).

Yet another function call.

While looking at this new snippet, immediately you’ll notice another function call in our mainloop.

bitmapData.setPixel(p.x,p.y,0xFFFFFF);

The bitmapData.setPixel is yet another function call being called as many times as there are particles. What’s important to note though is that Flash native API calls are faster then calls to methods in non “api” classes. So although the bitmapdata.setpixel call is costing us more then it should, it still not costs as much as calling a method on one of your objects.

Still, there is cost associated to this function call, and we can avoid it by inlining the method again. setPixel() is an incredibly handy utility function though, so we need to replace its functionality by a piece of code which mimicks its behaviour. Also, we need to write to the bitmapdata’s data by hand.

I’m not going to jump into using Alchemy / FastMemory just yet, so for now we’re going to use the next fastest accessible type that is available in AS3. Vector.

Drawing to a Vector instead

One can avoid drawing to a bitmap using setPixel and use a faster way to do so using a Vector object. While certainly not the fastest way to write to a bitmapdata, it’s probably the fastest way to do it in pure actionscript without any external tools. In comparison to using setPixel in our main loop, it should be a very good performance improvement.

One of the things setPixel did for us was bounds checking. Since we won’t use it anymore, we need to do it ourselves, so that we don’t write to a position in the vector that is outside of the bounds of the vector. At the same time, we’ll use this to resolve another issue. The particles were randomly flying offscreen. Instead of that we’ll respawn them at their starting position. We’ll add these properties to the particle class.

final public class Particle  {

		public var x:Number;
		public var y:Number;
		public var velX:Number;
		public var velY:Number;
		public var spawnX:Number;
		public var spawnY:Number;
		public var next:Particle;

		public function Particle(x:Number, y:Number, velX:Number, velY:Number) {
			this.spawnX = this.x = x;
			this.spawnY = this.y = y;
			this.velX = velX;
			this.velY = velY;
		}
}

And here is our main renderloop :

private function onEnterFrame(event : Event) : void
		{
			// Clear the stage
			bitmapData.lock();
			bitmapData.fillRect(bitmapRect, 0x000000);
			var vec:Vector.<uint> = bitmapData.getVector(bitmapRect);

			var w : int = bitmapRect.width;
			var h : int = bitmapRect.height;
			// Access particles and draw them
			var p : Particle = firstParticle;
			while (p) {
				p.x += p.velX;
				p.y += p.velY;
				if(p.x > w || p.x < 0){
					p.x = p.spawnX;
				}
				if (p.y > h || p.y < 0) {
					p.y = p.spawnY;
				}
				vec[int(w*int(p.y) + int(p.x))] = 0xFFFFFFFF;
				p = p.next;
			}
			bitmapData.setVector(bitmapRect, vec);
			bitmapData.unlock();
		}

While this setup is still far from optimal from a performance perspective, we’ve got a reasonably fast way to render 10 to 20k particles to screen.

One note here is the bitmapData.getVector and setVector.
This still isn’t that optimized – the function call “BitmapData.getVector” creates a new Vector object every time we call it, and as you know, creating objects takes CPU and memory allocation. Ideally the function should allow us to pass it a pre-existing vector object for it to populate with data. There are a couple of examples of this throughout the Flash API’s, and I truly hope Adobe will address these in future versions of the Flash Player. The benefit here of using getVector is that we can quickly clear the entire vector. Or if you would play around with this, you could not fillrect the bitmapdata and blur it slightly every frame.

For reference, what we have now looks like this, and can be download as source here: MainOneVersionOne (576)

So far, simple enough, right ?

I’ve kept everything quite simple so far to illustrate a couple of mostly obvious performance hints, and to set a frame of reference for the follow up article, part two. Technically this solution is far from the optimal solution to run particles. But for experimenting with moving the particles in a more interesting fashion, its good enough for Part One of this series. The benefit is that we can still express all the code in ActionScript. We’ll add a little bit of PixelBender in the mix to get comfortable with it and use it as an example of manipulating data with PixelBender.

Velocity Field ?

Visually this example is completely boring, so let’s make it a bit more interesting. We’ll start by making the particles move to a background pattern. We’ll use some perlin noise and pixelbender to generate a vector field for the particles to read from.

We’ll use a vector similar to the vector we are using to draw to the bitmapdata we did earlier, and mix in some perlin noise to convert grayscale perlin noise to velocities to be quickly looked up by the particles. We start by adding some code to the Main class, let’s start with the init method.

private function init():void
{
	//Get some perlin noise in grayscale to read in our pixelbender filter.
	var perlinMap:BitmapData = new BitmapData(WIDTH, HEIGHT, false, 0x000000);
	perlinMap.perlinNoise(128, 128, 6, 123456, true, false,7,true);

	//Load the embedded shader bytearray into a new shader.
	var velocityConverterShader : Shader = new Shader(new VelocityConverter() as ByteArray);
	var velocityConverterFilter : ShaderFilter = new ShaderFilter(velocityConverterShader);

	//Create a new temporary bitmapdata to write the filter result too.
	var perlinVelocityMap : BitmapData = perlinMap.clone();
	perlinVelocityMap.applyFilter(perlinMap, perlinMap.rect, perlinMap.rect.topLeft, velocityConverterFilter);

	// And get the result into a fastly readable vector
	velocityVector = perlinVelocityMap.getVector(perlinVelocityMap.rect);

	//Nicely dispose of the memory used by the two bitmapdata's.
	perlinVelocityMap.dispose();
	perlinMap.dispose();

	//Create a canvas to draw too
	bitmapData = new BitmapData(WIDTH, HEIGHT, false, 0x000000);
	bitmapRect = bitmapData.rect;
	bitmap = new Bitmap(bitmapData);
	addChild(bitmap);

	//Initialize the particles and keep them as a linked list.
	var particle:Particle;
	var previousParticle:Particle;
	for(var i:int = 0; i<=1000;i++){
		particle = new Particle(Math.random() * WIDTH, Math.random() * HEIGHT, Math.random() * 2 - 1, Math.random() * 2 - 1);
		particle.next = previousParticle;
		previousParticle = particle;
	}
	firstParticle = particle;

	//Wait to be added to stage
	addEventListener(Event.ADDED_TO_STAGE, onAddedToStage);
}

As you can see we added some code and a new property to our mainclass, to hold a reference to a “velocity vector”. This Vector will contain values much like colors. There’s many ways to get pixelbender to output data. To be able to read the data quickly in our mainloop, we need a vector again. So the easiest way to get a vector with uint’s from a pixelbender is to actually let it render to a bitmapdata.

The reason we are using 2 bitmapdata’s is that it’s significantly cheaper to instanciate them yourself and pass them, then to pass 1 bitmapdata for pixelbender to use as input and output, as it will allocate a block of memory for the output in any case. This way it’s faster and we have more control.

After the pixelbender filter has done it processing, we simple do a getVector on the bitmapdata to get the result into a Vector.<uint>.

A tiny bit of pixelbending

To get familiar with pixelbender kernels, we’re going to use one to calculate the data in our velocity field with. The pixelbender filter is one of a very simple kind, let’s have a look at it. It only has one input and it samples four pixels for every pixel it evaluates. Sampling from data within the Pixelbender is quite heavy, but for now we will only run this filter during the initialization of our swf. It might be fast enough to run every frame even.

<languageVersion : 1.0;>

kernel VelocityConverter
<   namespace : "com.unitzeroone";
    vendor : "UnitZeroOne";
    version : 1;
>
{
    input image4 src;
    output pixel4 dst;

    void
    evaluatePixel()
    {
        const float halve = 0.5;
        const float3 offsets = float3(0.0,1.0,-1.0);//We'll swizzle these around in our sampling.
        float4 left = sampleNearest(src,outCoord()-offsets.zx);
        float4 right = sampleNearest(src,outCoord()-offsets.yx);
        float4 top = sampleNearest(src,outCoord()-offsets.xz);
        float4 bottom  = sampleNearest(src,outCoord()-offsets.xy);

        //We'll store the slopes in green and blue, the two lower bytes in the vector.
        dst.r = 0.0;// We don't need to set r, but on some older flash players this caused bugs.
        dst.g = ((left.b-right.b)*halve)+halve;//Get the slope value between left and right, and normalize to 0-1
        dst.b = ((top.b - bottom.b)*halve)+halve;//Get the slope value between top and bottom and normalize to 0-1
        dst.a = 1.0;//We're using four channels, because we use shaderfilter.

    }
}

What this filter does is that it calculates the so called central difference for every pixel, using its connecting neighbours and store them back into the color channels. Let’s look at its input and output to have a visual reference before discussing it further.

The input & output.

Although that wikipedia page about the central difference might have scared you off, all in all what it does is very simple. For every pixel you look at the connecting neighbours in both directions and you look at the difference between that and store that. Note that we are normalizing the value, so that the maximum negative value of -1 and the maximum value of +1 fit in a range 0 to 1. This way we can read these values later on in our mainloop. We store the result for the difference on the horizontal axis in the green channel and the result for the vertical axis in the blue channel, hence the blue green teint of the image.

It’s good to note that PixelBender really makes these types of image calculations fast and simple. Long before PixelBender we had to do this using combinations of filters and copy channels to do this quickly enough. My first experiment with fast bumpmapping from a long, long time ago used that way of doing it. When I started working on Papervision3D it resulted in this experiment.

Making things move

Now that we have this new set of data, how do we make our particles dance to it ? The reason we wrote too a vector uint is so that we can read from it quickly. But we store the data in two different channels, being blue and green. Let’s take that data out and manipulate our particles velocities using it.

Our main loop will look something like this;

while (p) {
	//Add the current velocity to the particle position
	p.x += p.velX;
	p.y += p.velY;

	//Keep the particles in bounds
	if(p.x > w || p.x < 0){
		p.x = p.spawnX;
	}
	if (p.y > h || p.y < 0) {
		p.y = p.spawnY;
	}

	//Calculate the new position index;
	pos = (w*int(p.y) + int(p.x));

	//Draw the pixel
	vec[pos] = 0xFFFFFFFF;

	// Read the velocityfield at the new particles position
	var velocities : int = velocityVector[pos];
	var velXU : int = ((velocities>>8)&0xFF)-127.5;
	var velYU : int = ((velocities)&0xFF)-127.5;

	//Add a fraction of the position velocity to the current velocity
	p.velX += velXU*.073;
	p.velY += velYU*.073;

	// Dampen the current particle velocity
	p.velX = p.velX*.99;
	p.velY = p.velY*.99;

	p = p.next;
}

This bit of actionscript :

var velXU : int = ((velocities>>8)&0xFF)-127.5;
var velYU : int = ((velocities)&0xFF)-127.5;

If your unfamilar with masking and bitwise operations, this might look slightly strange. But if you have worked with colors using setPixel before, you might be familiar with the pattern here. Basically we extract the “color” as output by PixelBender from the green and blue channel. I put color in quotes there, because it’s not what we use it for. We simple use the byte packing into uint’s as a way to quickly access one value in a vector (which is the heaviest), and then unpack it into two of its components. A uint in Flash is 32 bit, so it contains 4 bytes. In theory we have 2 bytes left for storing other data. More on that in part two.

And it does almost exactly the opposite of what this did in the pixelbender loop :

dst.g = ((left.b-right.b)*0.5)+0.5;
dst.b = ((top.b - bottom.b)*0.5)+0.5;

The only difference is that pixelbender treats the color value as a range between 0-1 and that our actionscript values will now be filled with a value between -127.5 and 127.5. We handle putting the particle velocities to reasonable values by multiplying them by a rather small value. Play around with the values if you like.

To assure that the particles don’t pick up an infinite speed we dampen it by multiplying by a value near to one, as to only slightly lower it every loop.

p.velX = p.velX*.99;
p.velY = p.velY*.99;

So is it magical yet ?

If we look at what it looks like now with a thousand particles, we see this:

So, no it’s still rather uninteresting. Although our particles do seem to have to taken up a life on their own.

Adding complexity by adding the numbers

The particles seem to settle in areas, and the randomness of their movement and location is worsened by the fact that they are moving over perlin noise, where the noise part of that implies it’s somewhat random.

As I said at the beginning of this post, part of the percieved complexity of a particle system is in its numbers. What happens if we replace the perlin noise by a picture and add a lot more particles, let’s say 20.000 ? And then tweak the values a bit ?

Click here to see what that looks like.

Get the source here : Session One Final Source (665)

Closing and what will follow

The length of this simple introductory Session One already got way out of hand, so it’s time to close for this session, which means that unfortunately Alchemy is saved for next week. We’ve covered the basics of a reasonably performant particle system in pure, directly compilable AS3.

To continue this series and make the example code do a lot more then it does now, we need to start diving much deeper in PixelBender and the Alchemy-memory within Flash Player. We’ll be able to increase the number of particles to a much higher count then 20.000 of them using these techniques and hopefully be able to fully understand why and how. Along the way we’ll be able to look at how these technologies function inside the Flash Player.

Hopefully, we’ll polish up this example to a point where it’s starting to become interesting to look at. You’d be surprised to see how close we are to achieving something like the Particle Reactor with the current source code. But to be able to fully explain ways of using PixelBender and Alchemy, we need about another time the length of this post.

Until then, I would love to see your experiments based upon the source code above. If you do happen to do anything with it, please send it over and if there is enough response, maybe we’ll add it to next the post. I’ll give you some hints; dynamic vector fields, colors, trails.

I would like to thank Seb Lee-Delisle for reviewing this post and helping me to get it cleaner, better and more to the point. Another big thank you to Tim Knip for reading and pointing out the missing explanations.

I would also greatly appreciate any feedback on this article from you. I’m new to writing large articles / tutorials like this, and would love to hear what you think and what you would like to see.

(people who want to know more would refresh after seeing the pieces in full)

Over the last days I’ve spent some time twittering back and forth with Eugene. His supershape demo is very cool and we spent some time ‘tweeting’ about shaders. This lead me to upload and twitter one of my experiments which I used as part of my FOTB / MAX presentation. Judging by the huge number of retweets it seems people enjoyed it a lot, so I figured that it was worth a blog post. I started working on this experiment a while back after toying around with OpenGL, OpenFrameworks and audio reactive shapes and that inspired me to do something similar in Flash. This is an unfinished demo as it lead me to pursue something else, at which point I stopped working on it.

In the session where this came from, specifically this part, I shared information about the path of yet another demo, where this experiment led me too. Some of you who attended Flash on the Beach, might recall it from the Adobe keynote, where it was aptly named “milky ball”.  I’ll present on it one more time on the 3rd of December at the AUG XL event, in Amsterdam. I’ll post more of the session’s demos with explanation, when I have the time.

I do want to clarify that the code base I’ve created for this, which I currently call Triangle3D, is not to be a new Papervision3D, Away3D, Alternativa3D, Infinity3D, Sandy3D, Null3D or Yogurt3D (I’m probably missing a couple, sorry, have your way in the comments). It’s my experimental playground of classes to play around, with no care for architecture restrictions or a generalized API. It’s all about me playing and enjoying. Pure experimentation of what can be achieved with Flash 3D and hoping to find something beautiful in it.

Orignal unfinished demo

Demo with adjusted lightmaps from my new mirrorball.

(here’s the shot of the mirrorball, on flickr).

The music track is Sixtyten by Boards of Canada.

My favorite Adobe User Group does it again! And how! At the threshold of end of the year, on the 3rd December, the dutch AUG in co-operation with Adobe presents to you a full XL event. With the popularity of the normal meetings rising and rising and these event being of an exceptional level for any type of usergroup, this time, the lid is fully blown off!

The XL event will bring some great national and international speakers to the stage; Dave August of OOOii will talk about his use of Flash in JJ Abram’s Star Trek, as well as his creative input to movies such as Fast & Furious. Tobias Mannewitz of Karakter will be there and with work on projects for Spellbound, EA Games and SCEE this promises to be a session not to miss! Anita and Geoff from Champagne Valentine will amongst other great things talk about an interactive music video they created for Placebo.  Adobe will give you insights in their products and tooling with such international renowned speakers / evangelists as Serge Jespers, Jason Levine and Paul Burnett. If that’s not enough, Niko Stumpo of Hanazuki will be talking about the studio’s new work together with Mark Barzinski of Barzinski and Jean Jean. Dirk Groten of Layar fame will be talking about the Layar platform, something I’m looking forward to find out more about. Last but hopefully not least, I was honored to get an invitation to speak, and I will be talking about my latest forays into realtime flash.

This lineup is already stellar, with more names to come and be added to the list! Combine that with the friendly atmosphere and a great place to extend your network, learn and enjoy, this is an event not to be missed. Even if this is run by the dutch usergroup, I wouldn’t be surprised if people come in from all over, as this XL event truly is XL. Be there! And if you do, make sure you say hi!

Get more information at the XL webpage, follow the AUG on twitter, and order the insanely low priced tickets here.

With the addition of the 2.5d API in Flash 10, lot’s of developers now have even easier access to basic 2.5D environments. But with those new capabilities and API’s new challenges for developers arise. Famously, Flash 10 doesn’t do the Z-Sorting natively. When Lee Brimelow asked me if I could send him an easy solution for that, I sent him the SimpleZSorter. Which he in turn used to paste my profile picture on as much as possible planes, but more significantly, use for this tutorial.

Althought the name Simple would imply something a bit away from a PerfectZSorter (which, technically, has some challenges considering the 2.5D API and the player performance, don’t ask), the thing which astounded me was the amount of feedback. Sure, more people had asked me for such a solution and this simple little tool did it for them most of the time.

But considering the simplicity of the SimpleZSorter and the amount of feedback I got on it (even if I didn’t publish on it myself) made clear that although the 2.5D API of Flash 10 opened up loads of opportunities for developers and designers a like, some of the simpler issues aren’t addressed by it, whist there is clearly a need for it. That need can be addressed and with this post I’m hoping to do a another small part.

The ArcBall

One of the questions which people have been asking me about for in relation to using the 2.5D API, is (loosely combined) “How do I rotate things in 3D correctly with the mouse”.As this question came more and more, I’ve also realized how fundamental this is to our community. With Flash 10 we have native realtime (2.5D) 3D, but with realtime comes interactivity. And what use is it for users to have a 3D interface, while the controls are not intuitive ? At that point 3D becomes a worse experience, rather then a better. While this solution won’t address every issue with that, it most certainly addresses one, 3D rotation dragging.

Native 3D interaction might be new to Flash, but it certainly isn’t new to Flash 3D, or 3D in general for that matter. And the problem of having a user rotate an object with a mouse has been addressed before in numerous different ways. So luckily the wheel doesn’t need any inventing. The wheel I chose for to address this problem is called an ArcBall, first proposed by Ken Shoemake.  Now if you want to read that paper go ahead, but let me define this in flashers layman terms.

The Problem

Let’s first have a look at the problem as I’ve seen examples of code of people and derived it to this “problem” example. Drag the cube to rotate it.

[kml_flashembed publishmethod="static" fversion="10.0.0" useexpressinstall="true" movie="http://www.unitzeroone.com/labs/arcball/wrong/example/Main.swf" width="430" height="430" targetclass="flashmovie"]

[/kml_flashembed]

Although the mouse dragging does offer some feeling of control, you’ll find it difficult to rotate it in a natural intuitive way, to let’s say to the front of one of those colored planes. Here’s what’s happening here in that code, and what I’ve been seeing most.

if(isDragging){
boxSprite.rotationX = mouseY;
boxSprite.rotationY = mouseX;
}

This is example is the most common and is responsible for the example above. Now, the problem here is that it doesn’t take into account the local object transformation, as well as the dragging coordinates. As the object rotates around it’s axis, so do it’s axis change. Also, as the dragging goes on, we don’t take into account the 2D coordinate versus it’s 3D transformation dragging start and final output. All in all it ends up being a bit messy.

The Solution

Here’s the solution using the ArcBall class. Drag to rotate.
[kml_flashembed publishmethod="static" fversion="10.0.0" movie="http://www.unitzeroone.com/labs/arcball/correct/example/Main.swf" width="430" height="430" targetclass="flashmovie"]

[/kml_flashembed]

The solution, the complex part

As you can see / feel, object rotations now feel much more direct and natural. But what is this magic ? First off all, the class is fully Flash 10, and uses it’s native methods and features, where available for this solution. To explain it as simple as possible, as you click and start dragging on the cube, a sphere is established. This virtual sphere, around the object is then used to establish a point in 3D space on the sphere. As you are dragging, your actually rotating other points on the sphere. From these angular differences on the sphere, we can derive a new transform matrix for the object.

The hard part is in the establishing the rotational angles from those 2 coordinates on a sphere. A well proven way to do that (and the only form I have seen in ArcBall implementations, so far), is using Quaternions (go ahead, read it now, all of it, I dare you). Quaternions are complex things (pun intended). Now Quaternions are a bit daunting to start at and at this point I would be lying if I said I fully understand them. I’m not the only one who finds them scary, quoting Keith Peters / Bit-101 ; “Oh no. Vector3D’s w is the second most scary w in the world.” (I wonder what the first is, Keith ?).

Quaternions and Flash 10

If you are looking for the clearcut solution and implementation of my class for it, skip this part, otherwise, read on, enjoy.

The reference to the w in that quote is also a reference to the complex component real component of the quaternion, where the xyz are complex numbers. Yes, complex numbers (I dare you again) and their appliance is where stuff does get scary for most of us. The great thing about the Flash 10 3D and geom API is that there is a partial native support for quaternions, next to having a vector with x,y,z and w components. For instance, this:

matrix3D.decompose(OrientationStyle.QUATERNION);

Which will return a Vector of 3 Vector3D’s, which are in order translation, rotation, and scale. When using the quaternion orientation style, the rotation will be filled with a Vector3D with it’s x,y,z,w values nicely filled; a quaternion. While this is something we could have handled ourselves (decomposing a matrix to it’s components using quaternions), it’s great Flash 10 has it. Less code, less size and it’s faster then doing it with your own defined functions.

This ArcBall implementation relies heavily on that piece of code and some added code to do some additional stuff with the quaternions (multiply doesn’t seem to be available native). Now while I’m not enough of an authority to try and explain quaternions, the applications for them in 3D rotation are numerous, most famously, the first thing which will get mentioned when talking about the topic of Gimbal lock, which is a problem most encountered by astronauts and people developing 3D applications (see! us developers do have something in common with astronauts, we’re now doing rocket science!).

That being said, on a wider look on Quaternions, for example, Both Andy Zupko and John Lindquist have done some nice articles on the subject and it’s appliances in Papervision3D. Papervision3D, running from Flash 9 and higher, has it’s own Quaternion implementation, as Flash 9 doesn’t support it natively.

Deriving an easy solution for all

While the solution to ArcBalls using Quaternions isn’t even that hard, I still felt the problem should be abstracted a bit more, with an easy one stop solution for all. After all, the reason I got that much feedback on the SimpleZSorter, is clearly because people actually use it. Why do people use it ? My estimate is, it addresses a real problem and it’s simple to implement. And while that might cause a little overhead, and make it less flexible, it works for people looking for a simple solution. Here’s how that was implemented.

SimpleZSorter.sortClips(boxSprite,true);

Simple enough right ? So for the abstraction of the ArcBall something similar made sense.
Using the ArcBall class (and getting your correct dragging rotations up and running in no time) works like this.

arcBall = new ArcBall(boxSprite);

Now you can drag this object, and get nice, intuive rotations from it. There are a bunch of options to use;

arcBall = new ArcBall(boxSprite, useRadius, radius);

If you set the useRadius argument to true, and set radius to a specific value for that object (think about a sphere which completely encapsulates the object you have, it’s bounding sphere, see my beautiful artistic rendering at the top of this post), the bounding sphere used per dragging action is the same size. When that option is left set to false, the point on the sphere is derived from the click in 3D space on that object and thus speed might vary, depending where the user start the drag.

Because the ArcBall class automatically adds listeners to the object, and automatically enables dragging on it, you need a way to switch it off when it’s not wanted.

arcBall.enabled = false;
//To enable it again
arcBall.enabled = true;

So, that’s it. I hope it helps you in getting a better experience for the end user. Since this is an abstraction for ease of use and much more can be done, the entire code base for both examples and the ArcBall under the MIT License; essentially, do with it what you want, if it breaks anything, it’s not my fault. Let me know what you think.

Lastly a mention of sources which helped me with getting this right; rainwarrior, NeHe, processing. Sharing works!

Sources

ArcBall (just the sources for use).

RotationWrong (sources for the first example, incorrect drag rotation)

RotationArcBall (sources for the second example, correct dragging using the ArcBall class)

[Update 8/9/2009]

Little update on the quaternion explanation.

I normally don’t delve into book reviews or endorsements; and I won’t go there just yet with this post. But I do want to get your attention for the first released Papervision3D book; Papervision3D Essentials.

Written by Jeff Winder and Paul Tondeur, this elaborate, 13 chapter book looks great for anyone wanting to learn or deepen their knowledge of the Papervision3D API and Engine. Congratulations to both authors for getting it released! Great job!

Although I haven’t recieved my ([total disclosure] free, thank you guys), copy for review yet, the book looks promising, if you can judge it by these two free online chapters, about Lines3D and VectorVision.

As one of the core members of Papervision3D and having architectured and written rather large portions of the Papervision 2 Engine, for me personally it’s amazing to see a project like this. It’s documenting code that the team has been writing over the years and getting it on paper, along with an explanation and examples is something huge to me, as well as being huge for people wanting to start or already experienced with the engine! Having recieved multiple mails full of questions and additional bug fixes (as have other members of the team) from Paul and Jeff, I can assure they have been thorough in reviewing & examplifying the Papervision code base.

In a way, that’s a very odd feeling; an obscure function which might have been thought up and written at 2pm at night (which during the initial dev of Papervision3D 1.7, 1.9 and 2.0 was rule, rather then exception), might now have gotten an explanation in the book.

One thing the project has been lacking the most though, was a paper resource to look things up; and that has now arrived!

I will blog a more thorough review after having recieved and read the book. Can’t wait to see this book as a piece of tangible material and actually hold it!

[UPDATE]

Another free chapter has emerged as PDF. Chapter 8; modelling and importing to Papervision3D.