Part One of the Particle Session kicks off with looking at a system similar, but simpler then this Bloom particle experiment (music made for it by the fantastic artist Rich Bologna, thank you again for it, Rich!). We’ll be taking slow steps of optimizations and at the end have a get prepared for the next level. To be able to get up to speed with the working of the video particles, we’ll start from the beginning and work our way up. For that reason, we’ll start very simple and look at a basic unoptimized particle system with next to zero functionality, and modify the code for performance.

Basis : The Particle

The opening picture of this post is the album cover of The Strokes – Is This It. It being one of my favorite albums, I have listened to it many times, but never noticed what the cover really was. It’s amazing & inspiring what imagery one can get from science. Now put on your headphones and listen to that album while reading this post.

Over the last years I’ve more and more started enjoying introductory books about theoretical physics. In complete honesty, it wasn’t even that long ago that I didn’t know that the Boson in the Higgs Boson wasn’t the other guy discovering it, next to Mr.Higgs. You know, like the Hale-Bopp comet was discovered by Hale & Bopp.

Luckily for us and this post, we don’t need to deal with wave particle duality and quantum entanglement, but we can achieve seeming complexity by using simplicity.

Getting started : Simplicity & Complexity

The simple core particle we will be using in this session get its complexity from its display of large numbers of them, not from per instance properties. To illustrate that, let’s start off with a simple particle object in AS3

package com.unitzeroone.particlesessions
{
	import flash.display.BitmapData;

	public class Particle {
		public var x:Number;
		public var y:Number;
		public var velX:Number;
		public var velY:Number;

		public function Particle(x:Number, y:Number, velX:Number, velY:Number) {
			this.x = x;
			this.y = y;
			this.velX = velX;
			this.velY = velY;
		}
	}
}

As you can see, this particle object is pretty simple, and it contains the core properties we really need to make them move. Its position stored in x and y, and its velocity stored in velX and velY. For the case of this post it’s more then enough to do what we want.

I do want to point out two things about the simplicity of this. First of all, we express all values in absolute x and y. For the positions, this would be called a cartesian system. In some cases it might be handier to express velocity as a polar coordinates, but performance reasons we’ll keep them like this.

OOP (s)

It’s very common within the AS3 industry to “objectify” everything in nicely wrapped classes first and think about performance later. In the case of our particle and a classic “update” loop as seen in game engines, the most natural feeling way to implement the particle’s moving behaviour for the particle is as follows.

package com.unitzeroone.particlesessions
{
	import flash.display.BitmapData;

	public class Particle {
		public var x:Number;
		public var y:Number;
		public var velX:Number;
		public var velY:Number;

		public function Particle(x:Number, y:Number, velX:Number, velY:Number) {
			this.x = x;
			this.y = y;
			this.velX = velX;
			this.velY = velY;
		}

		public function updateAndDraw(toBitmapData:BitmapData):void
		{
			this.x += velX;
			this.y += velY;
			toBitmapData.setPixel(this.x,this.y,0xFFFFFF);
		}
	}
}

This is a very logical thing to do and is suitable for most things, especially in larger code bases, it certainly helps in readability. It also matches many things you’ll see in OOP and AS3 books. Like car.drive() or lasership.shoot() or avatar.updatePosition(). From a pure OO perspective, it makes sense. From a performance perspective, this isn’t as good though. We’ll tackle that in a bit.

Main

Now, let’s say we’d have a very simple Main class to run this system, as such :

package {

import flash.display.StageScaleMode;
import flash.display.StageAlign;
import com.unitzeroone.particlesessions.Particle;
import flash.display.Bitmap;
import flash.display.BitmapData;
import flash.display.Sprite;
import flash.events.Event;

	public class MainOne extends Sprite {
		private var bitmap : Bitmap;
		private var bitmapData : BitmapData;
		private var particles:Array;

		public function MainOne(){
			init();
		}

		private function init():void
		{

			//Create a canvas to draw too
			bitmapData = new BitmapData(500,400,false,0x000000);
			bitmap = new Bitmap(bitmapData);
			addChild(bitmap);

			//Create a set of random particles with a random velocity
			particles = new Array();
			for(var i:int = 0; i<=1000; i++){
				particles.push(new Particle(Math.random()*500,Math.random()*400,Math.random()*2-1,Math.random()*2-1));
			}
			//Wait to be added to stage
			addEventListener(Event.ADDED_TO_STAGE, onAddedToStage);
		}

		private function onEnterFrame(event : Event) : void
		{
			//Clear the stage
			bitmapData.fillRect(bitmapData.rect,0x000000);

			//Lock the bitmapdata, while we are updating it.
			bitmapData.lock();

			//Draw particles
			var p:Particle;
			var l : int = particles.length;
			for(var i:int = 0; i<l; i++){
				p = particles[i];
				p.updateAndDraw(bitmapData);
			}

			//We are done updating, unlock the bitmapdata
			bitmapData.unlock();
		}

		private function onAddedToStage(event : Event) : void
		{
			//If added to stage, set stage properties and start listening to the global enterframe.
			stage.frameRate = 30;
			stage.align = StageAlign.TOP_LEFT;
			stage.scaleMode = StageScaleMode.NO_SCALE;
			stage.addEventListener(Event.ENTER_FRAME, onEnterFrame);
		}
	}
}

For clarity sake I’ve omitted a couple of things, but this code runs. If you run it, you’ll see bunch of particles moving at a constant speed, moving offscreen. Our simplest of simple particle system works, so now we can start looking at performance and why we need to change the structure for performance.

Performance, the small issues

If you aware of performance caveats within Flash, you’ll immediately notice a couple of things, amongst which the lesser problems like for instance;

bitmapData.fillRect(bitmapData.rect,0x000000);

A lesser known fact about this piece of code is that the property rect of the bitmapdata is not a static read-only rect, but that every time you query this property a new rect is instantiated. Instantiation is rather expensive in the AVM2, but in this case, it’s called once per frame, thus not a major issue. Still it’s unneeded. Keep this in mind when you do many things per frame with bitmapdata.rect. In general, keep in mind that instantiation & allocation is expensive.

If you ever had one of those swf’s which seemed to hickup every now and then, chances are that it is the garbage collector kicking in. It doesn’t have to be one specific place where you are instancing to many objects, it might be more then one. So keep things clean from the beginning. Also keep in mind that many small performance losses will result in one big one.

The larger performance costs

There’s multiple things we can optimize in this piece of code. For now let’s keep it a bit simple and look at the larger issues, where we get the most gain. One of the main issues is how we store and access the individual particles.

particles = new Array();

We currently store them in an (as always in ActionScript) untyped array. If we’d like to keep the “storage” an array like structure, we would be better off using a Vector for storage, but we can get it even a bit faster by using a linked list instead. For the purposes of this simple particle system, that data structure is enough, as we are only going to linearly iterate over the particle set in one direction. This is called a singly linked list.

Let’s have a look at how that works.

Linked List

First, we modify our Particle class, to contain a link to the next particle in the list, by adding this to its properties.

public var next:Particle;

Then we modify the Main class to instanciated the particles like this :

var particle:Particle;
var previousParticle:Particle;
for(var i:int = 0; i<=1000;i++){
	particle = new Particle(Math.random() * 550, Math.random() * 400, Math.random() * 2 - 1, Math.random() * 2 - 1);
	particle.next = previousParticle;
	previousParticle = particle;
}
firstParticle = particle;

For every particle we create we store a reference to the previous particle in that particle. Then when we are done with the loop we keep a reference to the last created particle. That particle refers to the particle created before that, and so on, until we reach the first particle which has no reference to any particle.

We now update the drawloop to do the following;

var p : Particle = firstParticle;
while (p) {
	p.updateAndDraw(bitmapData);
	p = p.next;
}

Final(ly) ?
Using a linked list instead of an Array helped, but we can even more performance for free, by adding the “final” keyword to the particle class.

final public class Particle {

You would use the final keyword normally to prevent classes from being subclassed, but there is an interesting piece about final in Flash AS3 and AVM2 performance tuning, which by now is an aged document. In this case it indicates that sealing classes using the final keyword helps the AVM because the properties are already resolved and bound.

I find that in Flash Player 10.1 the advantage is less noticable then 10 and before, but it still has a slight performance gain on my machine, although I’ve heard it been debated more then one time. It seems to me also that some platforms/architectures benefit more then others. My naive way of thinking about it is that if you’re not going to subclass it anyway, you might as well finalize it. Simplicity.

Function calling
It may seem obvious to call a method of an object (like with car.drive()), but if you want maximum performance you should avoid function calls. With this in mind, let’s take another look at our draw loop :

var p : Particle = firstParticle;
while (p) {
	p.updateAndDraw(bitmapData);
	p = p.next;
}

The updateAndDraw call here is the costliest thing in the main loop. Not because what it does, but because it is a function. In general, function calling is always going to have some form of overhead, but in the case of Flash / AVM2 / AS3, the cost is exceptionally high. Although this situation has improved with Flash 10.1, the overall hit of just calling a function inside the loop is way to high to be acceptable if we want to achieve high numbers of particles.

Inlining

Inlining is something a compiler does for for you to replace a method call with the actual functionality in place of the function call.

AS3 does not have an inline keyword natively, and the compiler wouldn’t know what to do with it. Joa Ebert’s Apparat tool suite, written in Scala,provides amongst other incredibly handy things a way to do inlining in AS3. I highly recommend using it for larger projects where code style and performance are important. I actually also would highly recommend it for any project where performance is a priority. For this instance though, let’s go to a simpler solution first.

We’ll remove the updateAndDraw method from the particle class and make our main loop look like this.

var p : Particle = firstParticle;
while (p) {
	p.x += p.velX;
	p.y += p.velY;
	bitmapData.setPixel(p.x,p.y,0xFFFFFF);
	p = p.next;
}

Yay. For now you’ll have to take my word for it, but this is faster. (later on in this series will prove this with benchmark tests, I wanted to avoid this post just being a bunch of charts and performance graphs).

Yet another function call.

While looking at this new snippet, immediately you’ll notice another function call in our mainloop.

bitmapData.setPixel(p.x,p.y,0xFFFFFF);

The bitmapData.setPixel is yet another function call being called as many times as there are particles. What’s important to note though is that Flash native API calls are faster then calls to methods in non “api” classes. So although the bitmapdata.setpixel call is costing us more then it should, it still not costs as much as calling a method on one of your objects.

Still, there is cost associated to this function call, and we can avoid it by inlining the method again. setPixel() is an incredibly handy utility function though, so we need to replace its functionality by a piece of code which mimicks its behaviour. Also, we need to write to the bitmapdata’s data by hand.

I’m not going to jump into using Alchemy / FastMemory just yet, so for now we’re going to use the next fastest accessible type that is available in AS3. Vector.

Drawing to a Vector instead

One can avoid drawing to a bitmap using setPixel and use a faster way to do so using a Vector object. While certainly not the fastest way to write to a bitmapdata, it’s probably the fastest way to do it in pure actionscript without any external tools. In comparison to using setPixel in our main loop, it should be a very good performance improvement.

One of the things setPixel did for us was bounds checking. Since we won’t use it anymore, we need to do it ourselves, so that we don’t write to a position in the vector that is outside of the bounds of the vector. At the same time, we’ll use this to resolve another issue. The particles were randomly flying offscreen. Instead of that we’ll respawn them at their starting position. We’ll add these properties to the particle class.

final public class Particle  {

		public var x:Number;
		public var y:Number;
		public var velX:Number;
		public var velY:Number;
		public var spawnX:Number;
		public var spawnY:Number;
		public var next:Particle;

		public function Particle(x:Number, y:Number, velX:Number, velY:Number) {
			this.spawnX = this.x = x;
			this.spawnY = this.y = y;
			this.velX = velX;
			this.velY = velY;
		}
}

And here is our main renderloop :

private function onEnterFrame(event : Event) : void
		{
			// Clear the stage
			bitmapData.lock();
			bitmapData.fillRect(bitmapRect, 0x000000);
			var vec:Vector.<uint> = bitmapData.getVector(bitmapRect);

			var w : int = bitmapRect.width;
			var h : int = bitmapRect.height;
			// Access particles and draw them
			var p : Particle = firstParticle;
			while (p) {
				p.x += p.velX;
				p.y += p.velY;
				if(p.x > w || p.x < 0){
					p.x = p.spawnX;
				}
				if (p.y > h || p.y < 0) {
					p.y = p.spawnY;
				}
				vec[int(w*int(p.y) + int(p.x))] = 0xFFFFFFFF;
				p = p.next;
			}
			bitmapData.setVector(bitmapRect, vec);
			bitmapData.unlock();
		}

While this setup is still far from optimal from a performance perspective, we’ve got a reasonably fast way to render 10 to 20k particles to screen.

One note here is the bitmapData.getVector and setVector.
This still isn’t that optimized – the function call “BitmapData.getVector” creates a new Vector object every time we call it, and as you know, creating objects takes CPU and memory allocation. Ideally the function should allow us to pass it a pre-existing vector object for it to populate with data. There are a couple of examples of this throughout the Flash API’s, and I truly hope Adobe will address these in future versions of the Flash Player. The benefit here of using getVector is that we can quickly clear the entire vector. Or if you would play around with this, you could not fillrect the bitmapdata and blur it slightly every frame.

For reference, what we have now looks like this, and can be download as source here: MainOneVersionOne (576)

So far, simple enough, right ?

I’ve kept everything quite simple so far to illustrate a couple of mostly obvious performance hints, and to set a frame of reference for the follow up article, part two. Technically this solution is far from the optimal solution to run particles. But for experimenting with moving the particles in a more interesting fashion, its good enough for Part One of this series. The benefit is that we can still express all the code in ActionScript. We’ll add a little bit of PixelBender in the mix to get comfortable with it and use it as an example of manipulating data with PixelBender.

Velocity Field ?

Visually this example is completely boring, so let’s make it a bit more interesting. We’ll start by making the particles move to a background pattern. We’ll use some perlin noise and pixelbender to generate a vector field for the particles to read from.

We’ll use a vector similar to the vector we are using to draw to the bitmapdata we did earlier, and mix in some perlin noise to convert grayscale perlin noise to velocities to be quickly looked up by the particles. We start by adding some code to the Main class, let’s start with the init method.

private function init():void
{
	//Get some perlin noise in grayscale to read in our pixelbender filter.
	var perlinMap:BitmapData = new BitmapData(WIDTH, HEIGHT, false, 0x000000);
	perlinMap.perlinNoise(128, 128, 6, 123456, true, false,7,true);

	//Load the embedded shader bytearray into a new shader.
	var velocityConverterShader : Shader = new Shader(new VelocityConverter() as ByteArray);
	var velocityConverterFilter : ShaderFilter = new ShaderFilter(velocityConverterShader);

	//Create a new temporary bitmapdata to write the filter result too.
	var perlinVelocityMap : BitmapData = perlinMap.clone();
	perlinVelocityMap.applyFilter(perlinMap, perlinMap.rect, perlinMap.rect.topLeft, velocityConverterFilter);

	// And get the result into a fastly readable vector
	velocityVector = perlinVelocityMap.getVector(perlinVelocityMap.rect);

	//Nicely dispose of the memory used by the two bitmapdata's.
	perlinVelocityMap.dispose();
	perlinMap.dispose();

	//Create a canvas to draw too
	bitmapData = new BitmapData(WIDTH, HEIGHT, false, 0x000000);
	bitmapRect = bitmapData.rect;
	bitmap = new Bitmap(bitmapData);
	addChild(bitmap);

	//Initialize the particles and keep them as a linked list.
	var particle:Particle;
	var previousParticle:Particle;
	for(var i:int = 0; i<=1000;i++){
		particle = new Particle(Math.random() * WIDTH, Math.random() * HEIGHT, Math.random() * 2 - 1, Math.random() * 2 - 1);
		particle.next = previousParticle;
		previousParticle = particle;
	}
	firstParticle = particle;

	//Wait to be added to stage
	addEventListener(Event.ADDED_TO_STAGE, onAddedToStage);
}

As you can see we added some code and a new property to our mainclass, to hold a reference to a “velocity vector”. This Vector will contain values much like colors. There’s many ways to get pixelbender to output data. To be able to read the data quickly in our mainloop, we need a vector again. So the easiest way to get a vector with uint’s from a pixelbender is to actually let it render to a bitmapdata.

The reason we are using 2 bitmapdata’s is that it’s significantly cheaper to instanciate them yourself and pass them, then to pass 1 bitmapdata for pixelbender to use as input and output, as it will allocate a block of memory for the output in any case. This way it’s faster and we have more control.

After the pixelbender filter has done it processing, we simple do a getVector on the bitmapdata to get the result into a Vector.<uint>.

A tiny bit of pixelbending

To get familiar with pixelbender kernels, we’re going to use one to calculate the data in our velocity field with. The pixelbender filter is one of a very simple kind, let’s have a look at it. It only has one input and it samples four pixels for every pixel it evaluates. Sampling from data within the Pixelbender is quite heavy, but for now we will only run this filter during the initialization of our swf. It might be fast enough to run every frame even.

<languageVersion : 1.0;>

kernel VelocityConverter
<   namespace : "com.unitzeroone";
    vendor : "UnitZeroOne";
    version : 1;
>
{
    input image4 src;
    output pixel4 dst;

    void
    evaluatePixel()
    {
        const float halve = 0.5;
        const float3 offsets = float3(0.0,1.0,-1.0);//We'll swizzle these around in our sampling.
        float4 left = sampleNearest(src,outCoord()-offsets.zx);
        float4 right = sampleNearest(src,outCoord()-offsets.yx);
        float4 top = sampleNearest(src,outCoord()-offsets.xz);
        float4 bottom  = sampleNearest(src,outCoord()-offsets.xy);

        //We'll store the slopes in green and blue, the two lower bytes in the vector.
        dst.r = 0.0;// We don't need to set r, but on some older flash players this caused bugs.
        dst.g = ((left.b-right.b)*halve)+halve;//Get the slope value between left and right, and normalize to 0-1
        dst.b = ((top.b - bottom.b)*halve)+halve;//Get the slope value between top and bottom and normalize to 0-1
        dst.a = 1.0;//We're using four channels, because we use shaderfilter.

    }
}

What this filter does is that it calculates the so called central difference for every pixel, using its connecting neighbours and store them back into the color channels. Let’s look at its input and output to have a visual reference before discussing it further.

The input & output.

Although that wikipedia page about the central difference might have scared you off, all in all what it does is very simple. For every pixel you look at the connecting neighbours in both directions and you look at the difference between that and store that. Note that we are normalizing the value, so that the maximum negative value of -1 and the maximum value of +1 fit in a range 0 to 1. This way we can read these values later on in our mainloop. We store the result for the difference on the horizontal axis in the green channel and the result for the vertical axis in the blue channel, hence the blue green teint of the image.

It’s good to note that PixelBender really makes these types of image calculations fast and simple. Long before PixelBender we had to do this using combinations of filters and copy channels to do this quickly enough. My first experiment with fast bumpmapping from a long, long time ago used that way of doing it. When I started working on Papervision3D it resulted in this experiment.

Making things move

Now that we have this new set of data, how do we make our particles dance to it ? The reason we wrote too a vector uint is so that we can read from it quickly. But we store the data in two different channels, being blue and green. Let’s take that data out and manipulate our particles velocities using it.

Our main loop will look something like this;

while (p) {
	//Add the current velocity to the particle position
	p.x += p.velX;
	p.y += p.velY;

	//Keep the particles in bounds
	if(p.x > w || p.x < 0){
		p.x = p.spawnX;
	}
	if (p.y > h || p.y < 0) {
		p.y = p.spawnY;
	}

	//Calculate the new position index;
	pos = (w*int(p.y) + int(p.x));

	//Draw the pixel
	vec[pos] = 0xFFFFFFFF;

	// Read the velocityfield at the new particles position
	var velocities : int = velocityVector[pos];
	var velXU : int = ((velocities>>8)&0xFF)-127.5;
	var velYU : int = ((velocities)&0xFF)-127.5;

	//Add a fraction of the position velocity to the current velocity
	p.velX += velXU*.073;
	p.velY += velYU*.073;

	// Dampen the current particle velocity
	p.velX = p.velX*.99;
	p.velY = p.velY*.99;

	p = p.next;
}

This bit of actionscript :

var velXU : int = ((velocities>>8)&0xFF)-127.5;
var velYU : int = ((velocities)&0xFF)-127.5;

If your unfamilar with masking and bitwise operations, this might look slightly strange. But if you have worked with colors using setPixel before, you might be familiar with the pattern here. Basically we extract the “color” as output by PixelBender from the green and blue channel. I put color in quotes there, because it’s not what we use it for. We simple use the byte packing into uint’s as a way to quickly access one value in a vector (which is the heaviest), and then unpack it into two of its components. A uint in Flash is 32 bit, so it contains 4 bytes. In theory we have 2 bytes left for storing other data. More on that in part two.

And it does almost exactly the opposite of what this did in the pixelbender loop :

dst.g = ((left.b-right.b)*0.5)+0.5;
dst.b = ((top.b - bottom.b)*0.5)+0.5;

The only difference is that pixelbender treats the color value as a range between 0-1 and that our actionscript values will now be filled with a value between -127.5 and 127.5. We handle putting the particle velocities to reasonable values by multiplying them by a rather small value. Play around with the values if you like.

To assure that the particles don’t pick up an infinite speed we dampen it by multiplying by a value near to one, as to only slightly lower it every loop.

p.velX = p.velX*.99;
p.velY = p.velY*.99;

So is it magical yet ?

If we look at what it looks like now with a thousand particles, we see this:

So, no it’s still rather uninteresting. Although our particles do seem to have to taken up a life on their own.

Adding complexity by adding the numbers

The particles seem to settle in areas, and the randomness of their movement and location is worsened by the fact that they are moving over perlin noise, where the noise part of that implies it’s somewhat random.

As I said at the beginning of this post, part of the percieved complexity of a particle system is in its numbers. What happens if we replace the perlin noise by a picture and add a lot more particles, let’s say 20.000 ? And then tweak the values a bit ?

Click here to see what that looks like.

Get the source here : Session One Final Source (665)

Closing and what will follow

The length of this simple introductory Session One already got way out of hand, so it’s time to close for this session, which means that unfortunately Alchemy is saved for next week. We’ve covered the basics of a reasonably performant particle system in pure, directly compilable AS3.

To continue this series and make the example code do a lot more then it does now, we need to start diving much deeper in PixelBender and the Alchemy-memory within Flash Player. We’ll be able to increase the number of particles to a much higher count then 20.000 of them using these techniques and hopefully be able to fully understand why and how. Along the way we’ll be able to look at how these technologies function inside the Flash Player.

Hopefully, we’ll polish up this example to a point where it’s starting to become interesting to look at. You’d be surprised to see how close we are to achieving something like the Particle Reactor with the current source code. But to be able to fully explain ways of using PixelBender and Alchemy, we need about another time the length of this post.

Until then, I would love to see your experiments based upon the source code above. If you do happen to do anything with it, please send it over and if there is enough response, maybe we’ll add it to next the post. I’ll give you some hints; dynamic vector fields, colors, trails.

I would like to thank Seb Lee-Delisle for reviewing this post and helping me to get it cleaner, better and more to the point. Another big thank you to Tim Knip for reading and pointing out the missing explanations.

I would also greatly appreciate any feedback on this article from you. I’m new to writing large articles / tutorials like this, and would love to hear what you think and what you would like to see.

Nexus One running Flash Player 10.1 from Ralph Hauwert on Vimeo.

This is a video of the Nexus One with a pre-release version of Adobe Flash Player 10.1. It does not represent the final quality of the Flash Player on this specific device. But it’s pretty good as is!

This is the Nexus One running 2 demos. First a Nexus one specific version of the “Milkyball“, which is adjusted in resolution and screensize.

Second a non-adjusted version of FlashModPlug, an experimental project I’m currently working on. It’s a port and wrapper of LibModPlug, running inside the Flash Player through Adobe Alchemy.

Conclusion 10.1 on the Nexus is really, really fast!

(go to vimeo for the full resolution version). (link to nexus demo)

So, here’s a post where I am asking you, the Flash community, for a hand. Any of it is truly appreciated and will hopefully get some attention and maybe even a solution to a problem which has been in around in Flash since as long as I can remember; screen tearing.

First, what does tearing mean in this context ? Read this wikipedia article. In this case, it’s a visual split along a vertical line in a Flash movie. It’s been a known issue with Flash since quite some time (always as far as I know), with improvements over several Flash player versions, but I haven’t had to deal with it much lately….until now. I’m in a project where this visual artefact is a major distraction from the content and the experience.

All in all, this seems to be the cause (at least, my analysis of it) of tearing in this case; Flash internally rasterizes a screen, and pushes it to the screen buffer. Internally, this should be synced to your display refresh (60hz or 72hz, for instance), so that the buffer doesn’t get updated halfway through the videocard building up the frame. But on some browsers and platforms it does update during the video card frame refresh, resulting in two rendered frames being displayed at once (or better; 1 frame being updated to the next while drawing to screen), with an optically visible vertical split visible between the two frames. This problem has been around for a long time. It’s more prone to show up in content with “tall” items in it, fast frame rates with large portions of the screen moving, and during horizontal scrolling of displayobjects.

The picture above shows the problem in action. It’s a picture taken using a regular camera pointed at the screen, as this problem doesn’t show up in screencapture software. It shows two different frames of the testing movie; both a black and white square, updated halfway through the screen. While the position of where the slicing happens changes, in many single frames, 2 frames will be displayed at the same time.

I’m seeing problems with the latest 10 Player on Windows XP (Firefox), Mac OS X (Firefox, safari doesn’t but “stutters), and Windows 7. All in all, this is a graphical issue which has been around for a long time and should be solved. It’s definitely a Flash Player bug and with FlashPlayer 10.1 around the corner, I would really love to see this fixed. As well as being hopeful for finding a solution for current player/browser configurations. But testing results are varying and that’s where I am asking you for help, by running 2 flash movies and posting the results to my comments.

Over the last days I’ve been frantically testing several configurations and options, I would hope force vsync with the flash screen buffer, amongst which the following :

• Adding a video at 30fps to the stage (no result)
• Adding system text to the display list (seems to work on windows xp)
• Several configurations of stage.invalidate() and Event.RENDER (no result)
• Different drawing / caching methods.
• Wmode=”direct” for flash player embed options. (result across the board!)

The last option is the only thing which seems to work on all testing configurations, but comes at a cost. Switching to wmode=”direct” seems to force vsyncing and this would adhere to what has been said about direct wmode:

“This mode tries to use the fastest path to screen, or direct path if you will. In most cases it will ignore whatever the browser would want to do to have things like overlapping HTML menus or such work. A typical use case for this mode is video playback. On Windows this mode is using DirectDraw or Direct3D on Vista, on OSX and Linux we are using OpenGL. Fidelity should not be affected when you use this mode.” [Link to Tinics blog]

“Hardware offscreen buffer
Software renderer, perfect fidelity” [link to Jim Corbett's presentation (ppt)]

As opposed to the wmode=”gpu” flag, which moves rasterization to the videocard (at a hight cost in most content cases, also altering the appearance of your content depending on the videocard), this option still uses the Flash Software Rasterizer to build up content for the screen. But on both my Mac and PC testing this comes at a performance cost of several frames per second, up to about 15fps on larger screens (1280×768). Also, I’m not sure about the results of this on machines with no gpu. (mostly, cheap on board graphic chipsets, which share system memory). While most of the time, your content type might vary in terms of results with either GPU enabled wmodes, in this test that shouldn’t be an issue.

Obviously, abusing wmode=”direct” to force vsyncing on the Flash Player isn’t ideal, but it seems to work. In an effort to survey the results of this, I’ve built a very simple test swf. What I’d like you to do is run both swfs and copy the text field of both and paste them in my comments. Although I’m already in contact with Tinic about this, there’s currently no JIRA bug describing the problem. With the results you have, I’d like to put up an official bug there, and use your results to describe what systems are impacted.

Firefox seems to be the main culprit. Since FF 3 there have been a number of big issues with Flash and Firefox; for instance, in some cases a right click on the flash player will cause the updated areas in the Flash movie to render as an opaque rectangle with the background color. In another case, Flash doesn’t respond to rollover events and doesn’t even render without a mouse click occuring.

Please help me and hopefully Adobe identify and fix this problem, by running these tests. Here’s what you should do :

• Run both movies
• Look for tearing. Typically, in these test, instead of steady flickering, you’ll see a moving vertical tear between the black and white area. It is essentiall you identify tearing correctly, so make sure you know what it looks like (picture above is a still). The effect of tearing shows even better on my mac, when you move the firefox window so that the flash movie intersects your screenborder, halfway splitting vertically through the flash movie).
• Copy and paste the textfield contents when the test is done, on both tests to my comments. Fill in the required 2 fields, and if you can, what type of graphics hardware your machine has.
• Send this link to other people who are willing to test (and be part of the solution, making Flash a better place!).

Test with wmode opaque

Test with wmode direct

[WARNING : if you are prone to visually induced epileptic fits, this might induce one. Don't run it then]

(don’t forget to post to the comments).

Over the last days I’ve spent some time twittering back and forth with Eugene. His supershape demo is very cool and we spent some time ‘tweeting’ about shaders. This lead me to upload and twitter one of my experiments which I used as part of my FOTB / MAX presentation. Judging by the huge number of retweets it seems people enjoyed it a lot, so I figured that it was worth a blog post. I started working on this experiment a while back after toying around with OpenGL, OpenFrameworks and audio reactive shapes and that inspired me to do something similar in Flash. This is an unfinished demo as it lead me to pursue something else, at which point I stopped working on it.

In the session where this came from, specifically this part, I shared information about the path of yet another demo, where this experiment led me too. Some of you who attended Flash on the Beach, might recall it from the Adobe keynote, where it was aptly named “milky ball”.  I’ll present on it one more time on the 3rd of December at the AUG XL event, in Amsterdam. I’ll post more of the session’s demos with explanation, when I have the time.

I do want to clarify that the code base I’ve created for this, which I currently call Triangle3D, is not to be a new Papervision3D, Away3D, Alternativa3D, Infinity3D, Sandy3D, Null3D or Yogurt3D (I’m probably missing a couple, sorry, have your way in the comments). It’s my experimental playground of classes to play around, with no care for architecture restrictions or a generalized API. It’s all about me playing and enjoying. Pure experimentation of what can be achieved with Flash 3D and hoping to find something beautiful in it.

Orignal unfinished demo

Demo with adjusted lightmaps from my new mirrorball.

(here’s the shot of the mirrorball, on flickr).

The music track is Sixtyten by Boards of Canada.

After the overwhelming response of posting the sources of 300.000 pixels with Alchemy and Pixelbender, I thought it would be nice to post some more from my session at FITC Amsterdam.

First let’s take a look at what people have done in response of the previous sources.

Yonatan Offek – Sierpinski Particles

Peter Nitsch – Alchemy Particles

David Lenaerts – Smoke, Milk and Ink

Dennis Ippel – Incredibly fast Plasma

Joa Ebert – Massive amounts of 3D particles without Alchemy and Pixelbender

All the work in that list is incredible, but for this post, I’m most interested in the last 2. Joa’s work on getting AS3 up to speed with my Alchemy demo (Joa’s demo is actually faster then the Alchemy version on my machine) is incredible. And it also proves the one thing I tried to convey in my blog post Adobe Alchemy, is it ActionScript Heresy ?.

Joa’s optimization relies mostly on a linked list implementation to iterate over the particles, and using the Flash 10 Vector type to allow for faster access to write to screen. Another speed improvement over the Alchemy version is not using Pixelbender, but having both rotation and projection done in the same loop as drawing to screen. All in all, his result on this is pretty awesome.

Again, writing highly optimized ActionScript should enable you to do the same speedwise as Alchemy compiled code. There’s at least one definitive advantage which Alchemy has in that case. The first advantage in Alchemy lies is the fast ByteArray access (which currently only Alchemy and Haxe can use).

The second advantage is why I think we should have a better AS3 compiler. It is the use of a more advanced compiler. This reduces the need to *highly* optimize your code, although I’d always recommend doing that in any case. My demo’s here are meant of examples on how to use Alchemy, rather then to get the last bit of performance out. That, we’ll leave to PapervisionX work.

Now, as I said, for this post we would be looking at two of the responses on my earlier Alchemy demo, the second one being Dennis Ippel’s Fast Plasma effects. Plasma effects and Lookup Table efffects aren’t too different. Ricardo Cabello did some experiments with them earlier, using Pixelbender. I worked on taking this type of effect and implementing it using Alchemy. Since that I had already done some experiments with them earlier, I thought it would be nice to release the sources, and see what you guys do with them. Maybe Joa feels like another shot of doing the same with ActionScript ?

Example

Source

Play around, let me know what you do with it, and make sure you check out Mr. Quilez his webpage, specifically, the article on Plane Deformations, where part of this code is derived from. If you want to optimize more, you *could* read Kebby’s article F***ing learn to code again.

One more observation  : the difference between running this on Firefox on a Mac and just running it in the FlashPlayer is huge, even bigger then normal. Could someone from Adobe finally explain this ?

Pushing around +300.000 3D particles, realtime, on screen, using Flash ? No problem, if you are using Adobe Alchemy & PixelBender to compile and run your code!

During my session “professionally pushing pixels” at FITC Amsterdam this year, amongst other things, I talked about how to best utilize parts of the Flash Player to get top performance. This is one of the examples I showed. What you are seeing in this example, is +300.000 particles being 3D transformed, projected and draw to 2D. And it does so at quite a good framerate (well, it’ll depend on your machine too).

So, how do we achieve this ? The answer is a combination of PixelBender and Alchemy.

First, let’s look at the 3D transformation and projection.

Flash 10 has a number of native features to allow for 3D transformation and projection. You’ll find that this is a combination of using the Vector, Vector3D, Matrix3D, PerspectiveProjection, etc. Although these features are great, we can’t use them in combination with Alchemy easily. I’ll explain why later, for now, let’s look at an alternative method to do the projection.

Where oh where in the Flash Player do we have a method of doing very fast math ? The answer is; pixelbender! Although Pixelbender is normally used for image based manipulation, you can make it do any type of number-crunching which is able to be executed in parallel and without loops.

To calculate rotations and projecting our 3D data, we use Pixelbender in “ShaderJob” mode. When using pixelbender in image based mode, it operates in 8 bits per channel. Thankfully, when using it with a ShaderJob, it allows 32 bits precision per channel for the data processing. Since 8 bit precision wouldn’t be enough for this example, we use a shaderjob.

The VertexProjector pixelbender kernel, included with the source is a simple way of transforming and projecting vertices (representing particles, in this case) in 3D space. We feed this kernel a bytearray of x,y,z paired data, and execute the shaderjob. It then returns the data as a bytearray, in px, py, pz format.

Drawing things to screen.

Now we have all the 2D projected 3D data, we need to draw things to screen, and we have to do so as quickly as possible. This step is traditionally called rasterization. In AS3, you’re most likely to use getPixel when drawing on a per pixel basis. Doing so in a loop for 300.000 pixels turns out to be very slow. The solution for this would be to optimize that loop as much as possible. Either by writing your own bytecode, or maybe writing your own post-processor for you code, before you compile. But we don’t have too, since Adobe Alchemy exists.

As you can read in my earlier post about Adobe Alchemy, I openly questioned why it was so speedy, as compared to regularly compiled ActionScript 3 code. Although the answer is rather complex, the combination of C Based code, the LLVM compiler and “Alchemy Virtual Memory” are the base of this. The large difference between Alchemy compiled actionscript and regular compiled Actionscript can be further explained by the regular AS3 compiler not doing any optimisation. This example shows off those performance increases.

One thing to worry about when using Alchemy in your ActionScript projects is marshalling. You can read Branden Hall’s post on Alchemy for more info on that. Since we wouldn’t be able to marshal 300.000 vertices from a Vector.<Number> in AS3 to our alchemy code, we need to find a better solution. This is exactly why we are using Pixelbender and more-over, the bytearray data.

It is possible to manipulate the memory Alchemy uses in the runtime. This memory is represented as an AS3 ByteArray object. If we directly write and get our data from this memory block, no marshalling is needed. Although this means not all things can be done this way, for some things, this can be very useful. For instance, getting large blocks of data, like images and bytearrays of coordinates.

Getting all these 3D particles to screen is simply 1 inner loop. While we would normally call setPixel for that, in Alchemy code, we don’t have that luxury. Instead of that, we write directly to our screenbuffer memory, which is represented as a set of int’s. Here, one more problem comes into play. Endianess, defines the byte ordering for a set of data. Alchemy uses little-endianess for it’s internal memory representation. Specificall, it uses a small class called LEByteArray. This class extends ByteArray and ensures no changes are made to the endianess of the memory. Makes sense, since otherwise your code would blow up.

Writing to the screen is then a piece of cake. We take the alchemy processed data from it’s memory, and write it to a bitmapdata using the formerly much less usable setPixels() command. It’s amazing to see how fast this is.

Look at the example here, and download the full sourcecode here. As you can see from the example, the difference between doing this with regular ActionScript versus Alchemy nears a 5 fold speed increase.

Thanks to Keith Peters, for providing me with the 3D Strange Attractor code! And additional thanks to Mr.Doob for the stats object.

In future I’ll be posting more demos of the technology. Amongst which there will be one appliance for the future version of Papervision3D, PapervisionX.

It’s been silent here. Next to tripping to Tokyo to speak at Adobe MAX in Januari and doing some small jobs, I’m doing a very, very cool Flash 10 Project utilizing the upcoming Papervision for Flash 10. For some optimization parts I’ve been looking into using Alchemy as an easy way to write optimized bytecode. As opposed to what some people took from my previous post on Alchemy, I don’t hate Alchemy. My point there was that if Alchemy can perform that well, so should ActionScript be able to, with the standard compiler.

Now, that being said, all work and no play makes jack a dull boy. With no time to spare during the morning, daytime and evening, I took the laptop with me to bed last night, and decided to do something I didn’t do before, port something with Adobe Alchemy. I decided upon a long time thing on my wish list; SID Playback in Flash. What is SID ? Well, SID stands for Sound Interface Device, referring to the MOS6581 and it’s a programmable audiochip as used in the Commodore 64 and Commodore 128. I grew up with the Commodore 64, and the audio it produces is still something I hold dearly. You are quite likely to have heard a SID chip before, if you listen to the radio, as it is quite a possible you’ve heard one of these baby’s scream. Good recent examples are :

Bastian – You’ve got my love. (the beat in this song comes from Jeroen Tel’s Rubicon).

Nelly Furtado – Do It (controversy all around, Timbaland blatantly stole Glenn Rune Gallefoss’s version of Acid Jazzed Evening).

De Jeugd van Tegenwoordig – Hollereer (Produced by Bastian, C64 lover himself).

My interest was more playing back the old C64 tunes from games and demos. Keep in mind, an average SID file is about 2kilobyte to 10kilobyte; a perfect Small Web Format for all your music during preloading needs . Now, this is something I have been working on before in Flash 9, using the FC64 project, the amazing Flash C64 emulation effort by Claus Wahlers and Darron Schall. Last year, I even made a start with porting FC64 to Flash 10, but that needed to be dropped in favor of actual client work. SID Playback is something different from playing back .mod .xm or .mid files, as the SID chip doesn’t do much on it’s own. Essentially, a .SID file is an executable meant to be ran on the 6510, the Commodore 64′s CPU. So to playback the file, you not only need to emulate the SID chip, but also the 6510 and the needed registers / memory.

Software emulation of the SID chip has been around for a while, and all modern platforms have a way of playback SID files. Winamp plugins, command line utilities for conversion, it’s all there. As is there opensource code to do this. There’s libraries for most major languages. So, after a bit of research, I decided upon the Linux version of TinySid. This library is small enough and well optimized. It looked simple to port, so I went at it. All in all, 4 hours later, I had this baby up and running! (And an unhappy girlfriend who isn’t into the nerd thing, nor is she into listening to chiptunes whilst trying to sleep). Considering my very rusty C and a little hack around I had to do for the audio output, Alchemy shows off it’s feathers here. Many kudo’s to the Alchemy and Flash Player team for this effort. Just now I spent half an hour building a tiny interface using Keith Peter’s Minimal Components, and collected some songs; and here it is.

TinySidFlash (open link, requires Flash 10).

My favorites : Politik and Science (A C64 version of Coldplay’s the Scientist), Morphing and Turbo. Yes, I love Dane’s tunes!

There’s no code yet, as this is still very rough and unfinished work. I’m pretty sure that when I get time again, I’ll have a go at LibSidPlay instead, since the overall emulation in that library is a lot better.

A smooth shaded elephant.

Behind the scenes I’ve been insanely busy for the last months. But just to keep you posted on some progess on my side, here’s some demos of me playing around with Flash Player 10′s 3D API’s.

I’ll keep this short, here’s some demos.

Some tests with shading : 1, 2, 3, 4, 5, 6

You’ll notice that the speed slows down when you go up in the number of examples. The amount of triangles increase per example. Since shading is still one of the most heavy appliances of any 3D engine, I thought it would be good to start out testing that, and see how it would work in a realistic 3D engine environment.

Next, this is not adviced to look at if you get car-sick easily…..a pure speed test.

A bit slower then it could run using any of the new wmodes, but for speeds sake, we’re getting there.

Source code not available yet. First I’m looking at how to optimize. And yes, I’m working hard on a new version of Papervision3D, with the rest of the team. Soon I’ll post some better examples, including a BSP example.

Intro

As promised in my previous post, here’s a small opensource project from my side. It’s tiny, really, but I hadn’t seen an ActionScript implementation of any form of Image dithering before.

Dithering….what ?

Just a brief explanation what dithering in this context means. From wikipedia :

“Dither is an intentionally applied form of noise, used to randomize quantization error, thereby preventing large-scale patterns such as contouring that are more objectionable than uncorrelated noise.”

Words to take note of in that sentence are quantization, and “intentionally applied form of noise”. In our case quantization is the removal or swapping of colors. The form of noise applied differs over the several implementations and in so called “ordered dithering” it’s hardly noise, rather a carefully chosen threshold matrix.

Dithering, why ?

Dithering imagery has been around for some time. Put a close eye to any black and white printed newspaper’s pictures and you’ll see the effect.

It’s basically been around a lot longer then I have in my 28 years of life…that’s for sure. I first learned terms like “ordered dithering” and “Floyd-Steinberg Error Diffusion” in my young and early days on the Amiga hardware. See, in those days computer hardware wasn’t capable of displaying the huge arrays of colours like now-a-days. Since you then had few colours to spare (a typical amiga workbench ran at 16 to 128 colours or so), you needed to be creative to get yourself a nicely pimped desktop image, whilst still sparing colours for your icons.

A better example of a dithering implementation for our industry is probably that checkbox when you want to save for web in Photoshop, using GIF. Or when you print something on a pure black and white printer.

Conclusion; dithering is normally used to create the illusion of tones on a device which is otherwise not capable of displaying it. So why port this to our ARGB/32Bit enabled Flash Player ? Part of the reason why I worked on this is because I just like the aesthetic of the effect. It just brings back wonderful memories of pimping my Amiga desktop.

Dithering in AS3

So, the algorithms for dithering are really quite simple. I was playing around with hydra Pixel Bender and had some ideas on converting some old algorithms to have them run in realtime. Then I thought I could make this run in realtime on Flash 9. So, first I did a version of the Algo in pure AS3. I played around with converting it to something fast enough to run at at least 20 frames per second on an average machine…and painfully failed. The remains are the AS3 version (hydra Pixel Bender version still in progress)….and I decided to clean that up a bit, and have it released as an OS project for anyone to use.

Currently it only contains so-called error-diffusion based ditherers. They make the most sense anyway, since the ordered ditherers really have nasty visual sideeffects, like Bayer’s crosshatches. Check the variants out in this little demo application using the class.

The Demo : try and use “No Dithering” first to see the effects of regular palette conversion.

The Source

Usage :

ImageDithering.dither(bitmapData, type, levels, grayscale);

Where :

BitmapData is the image to be manipulated.

Type is the form of dithering, currently supported :

1. ImageDitheringType.FLOYD_STEINBERG
2. ImageDitheringType.FALSE_FLOYD_STEINBERG
3. ImageDitheringType.STUCK
4. ImageDitheringType.NO_DITHER

Levels is the amount of colour levels to quantize to per channel.

Grayscale is a boolean indicating whether to convert the image to grayscale before the process is ran.

Right click and viewsource on the example to see how it’s implemented in Flex.

Quick update : Mario Klingemann, working on Aviary’s Peacock did a quick test with it. My effort had some use anyway!

You always have projects lying around that are to be finished…but probably never have the time for…

Today Exey Panteleev posted some creative use of Papervision3D and the Shaders in 2.0. Accompanying to his post he asked if it would be possible to run water simulation with Papervision3D….

Actually, yes…
Water Ball Experiment 1 Water Ball 2(cpu burner alert!)

As one of those things I never finished and probably don’t have time to finish within the next month or so, I have two examples of Papervision3D running interactive water on top of a sphere.

Disclaimer : these things are hacked together experiments from a couple of months ago…as a result performance isn’t to good. I’m sure that given some time they can run smooth, without burning your cpu to the ground. Hope you enjoy anyway