Demo Video of WebM being decoded by Flash. from Ralph Hauwert on Vimeo.

This is a little side project I have been playing with. I’m porting google’s libvpx using Adobe Alchemy to run WebM in Flash Player 10, which doesn’t support the format natively yet. Because it’s not much more then a small and experimental side project, I quickly made a video to show it’s progress up until now. Monday the new part in the Particle Sessions will appear.

Hello there, and welcome again. So, this blog post at the beginning of 2011 is a result of my New Years resolution, which is to revive my blog to a fully functional and lively place. I apologize for it being mostly deserted. I left it alone since May 2010. Most of the reason behind the blog falling behind was that year’s fault really, not mine.

It was a good year. I started doing a project for Aviary and was honored to speak at many great conferences of the web industry. One of the most note worthy events for me personally was also to be part of Adobe FlashCamp Tokyo and Seoul. It was a great trip and a privilege to hang out with such heroes as Mike Chambers, Mario Klingemann, Lee Brimelow and Erik Natzke. You can see Lee’s video report of that trip in Flasher Magazine 3 on Vimeo. (Doug McCune is alive and well, despite what that video says).

Looking ahead, this year promises to be an equally great year. In the Flash space, one of my long time wishes, GPU support for Flash is on the shortlist for a public beta in this year. More on that in a later blog. Privately, many things are happening and I’m loving it. Let’s just say that good times are ahead.

As for the reasoning for why this year my blog will get updated on a regular basis; this year I’ll take it easy on the conferences. I’m not saying I won’t be speaking or going to any, but this year has so many things going on for me (both work and private) that I decided to at least take it easy for a while. How long that “while” is, I haven’t decided.

I’ll miss all the great people, but at the same time this will give me more time to focus, work and enjoy life. Also for this reason, I’ve decided the blog will be the main form of sharing information and hopefully share useful work and thoughts.

As I started thinking about topics, I realized I still had quite a backlog of things still to put out there publicly and decided to tackle them one by one. My last talk, aptly named “The Discontinuity” at Flash on the Beach in Brighton has ran exactly one time, with some still in progress work, which I’d like to talk about. Also, things like FlashModPlug and my GNURocket implementation are still to be publicized. About that GNURocket client implementation, 2010 was also the year of a true demoscene contribution, in the form of Area Unstable (video) (pouet download). This was made together with Sander.Focus (graphics) and Gloom303 (music,sync).

I’ve also branched out to doing iOS development and working on experiments using both OpenFrameworks and Cinder. So there’s many things still to blog about this year. But, one thing at a time, so I decided to work through a backlog and start with some Flash particle experimentation. My next post, due tomorrow, will outline the working and technicalities about this thing; Rendering Video with White Particles (Flash).

[Update, the blog gave up on me, and I had to reinstall it. Here's my blog in it's new form].

Long time no see! I’ve been busy, sorry. First off all, I want to congratulate you on the great show today. Wow, that iPad is truly one of those things everyone is going to want to have, like your previous successes with the iPod and iPhone. I take my hat off for that! And wow, what a lead up to the show. Probably the first hype of 2010. And do you guys know how to keep a secret and then give a great show unveiling it.

That being said, I felt as an avid Apple products user as well as your friend, (you know I own a total of 3 Macbooks, a plethora of Airports and have converted nearly my entire family to buy one of those nice machines of yours) (actually it was Grant who got me into buying one) it’s my duty to inform you that I think something went wrong during your presentation. When you were showing the web capabilities of the iPad, something was missing in it’s browser (see screen-shot above). As your keynote and product presentations are normally flawless (ah, well, maybe not always), I think you might have missed this one.

You touted the iPad’s (great product name, btw) web capabilities as being amazing, perfect, you know, the regular Apple thing. But during the presentation I couldn’t help but notice that little “missing plugin” logo we all know from the iPhone. Now, I was thinking that this might have been one of those very exotic plugins of back in the day, like Director or Realplayer. But, as it turned out, the missing plugin was the Flash Player.

As you’ve started the introduction to the iPad with a reference to Netbooks, and we can safely say that most netbooks support Flash, I think we are both safe to assume that to be on par with those netbooks, the minimum you should have is a fully featured browser?
I mean, this is not some kind of small screen device with limited capabilities in terms of performance and graphics. It’s a fully featured machine ? Be it with that amazing iPhone touch interface. And be it in a slightly new package intended for true casual and business use. (Honestly, you kind of confused me, games and reading books ? I thought the gaming generation didn’t read and vice-versa, ah we’ll talk about that after you write me back on this letter).

Anyway, I think it must have been a demonstration error, because it seemed like the iPad didn’t support Flash?

Mythhackers.

Ok, now, who am I kidding. I know you’re a perfectionist. And you’re a man of subtle remarks too. Between you and me, you can admit it. We both know that little missing plugin logo wasn’t an accident. I think we both know why you did it. It must have been your kind of humor, getting back at that little prank  Kevin and Johnny of Adobe pulled on you at Adobe MAX. (see picture above too). You have to admit, that was funny. Man, we all laughed about that one. But hey, I always think that if they’re teasing you, they must want your attention. All in all, they didn’t mean any harm.

(I do agree with you that they should’ve spent some more cash on the explosion effect in the video, being the maker of some of the most premiere video effect software out there, but hey, it was a good prank video, executed very well).

Now, given some of the easter eggs in Mac OS X, which essentially is your baby, through that other thing you did, with that next-cube thing you were so enthousiastic about back in the day, I take it you have a fair amount of subtle humor in you. It always reflects in your letters and I always have to laugh when I think about how you made that joke about a mouse only needing one button. But, as always, there’s a time for fun and a time for seriousness. And you seriously can’t mean that the full featured web browser doesn’t support the biggest plugin out there?

If you do mean it, I can understand why. I’ve never really bought that, “Flash is too slow for the iPhone” story you told everyone at that party. I also know for a fact that RIM, Nokia, Samsung, NVidia and all those guys also didn’t believe it.

And yeah, I know that little stunt Adobe pulled with compiling directly from Flash to iPhone isn’t the same as running it in the browser, but hey, it is the output of Flash running on an iPhone. And surely there’s a lot of content which can now be readily available in the App store. And if that puny little ARM processor in an iPhone can, then surely that incredible piece of custom silicon you so proudly called the Apple A4 can. (it’s ARM too, right?).

Your nightmare ?

Ahh, going off track, I was saying that I understood why you would not want Flash. I know having rich, web enabled apps which don’t go through the App store can be a scary thought, food for nightmares. I know you’ve had a thing for nightmares, especially after Monster’s inc. But Steve, I have to tell you. A device which is intended for casual and easy use with all the slickness Apple always brings, but doesn’t support so much web content? Come on, what will the people think? Those netbooks you called slow and PC-ish, can run Flash, but Apple’s state of the art technology can’t?

Yeah, I know it’s hard to have that piece of proprietary software running so much content essentially being a separate platform, while having no control over it. But that’s why Kevin and Johhny where throwing those jokes at you. They want to help you, really. And again between you and me, isn’t it true that you’re always selling computers when those guys at Adobe come up with a new version of their creativity software?

I would like to also come with a more serious note, as your friend, I know I can remind you of this one. Do you remember that when I was still using a PC, we were always talking sh*t about those guys at Microsoft for dominating the market? About how they pulled that monopoly thing on Netscape with that “lovely” (I still remember the word you had for it) browser we all still despise, Internet Explorer. And what they did to Real by bundling that Mediaplayer? Well, as it turns out, they kind of ran into some anti-trust issues with the European Union. I hear it was quite costly for Bill. While I agree the iPhone was a completely different thing, you can’t bring out a computer and expect everyone to think it’s just a big iPhone and then pull all that stuff off that our friend at Microsoft once did, right?

Yeah, yeah, I know what you’re going to say. Flash is proprietary too and there should be an open alternative to it. But honestly, in terms of open initiatives competing with it, it seems those HTML5 guys are still not ready for it, and although having H.264 video support native to the browser is nice, it hardly replaces what Flash can do right now. Not to mention all that content already being offered by Flash, people can’t see on the magnificent iPad. If pure HTML5 content is even 1% of that, we both know that would be a lot. I do feel you should be open to any plugin, honestly, so the guys at MS/Silverlight and Unity3D, for instance, also get a shot. All in all, it’s your product’s users who are going to suffer for it otherwise. Now, as I’ve made my living building Flash apps, of course I am biased, but doesn’t every developer deserve a fair shot?

You're the man!

You were always this idealist, but this thing is making you look like you are just plain greedy, at the cost of your end users, not being able to access all web content. Give me back the idealist Steve, I love that guy! (I’ve included an old picture of you to remind you of that man).

Now, you know I do admire that you’re not like all those others and you’re doing things your way (and it seems to work), I think you’re going down the wrong path on this one, or this might have been just an error. If it was, just fix it before release, and we’ll keep it between you and me.

Sincerest Regards, your friend,
Ralph.

(quick note, can you make Johnathan stop saying the word Magical, every time he’s talking about one of the products he designs ? Frankly, it’s a bit annoying. And he’s not designing unicorns or leprechauns.)

(quick note 2 : if you do manage to talk to Kevin or Shantanu at the next party, ask them if their engineering people can talk to your engineering people about that video hardware support and performance issues, I think it would make many of our friends happy).

Milkyball, a further extension of my Triangle3D experimentation, used in my presentation of end 2009. Click on the shape to see the magic vertexmap.

PixelBender + Flash raytracer. Work in progress.

I posted both of these earlier on my twitter (@unitzeroone), but they didn’t make the blog as of yet. More details in future posts.

Pushing around +300.000 3D particles, realtime, on screen, using Flash ? No problem, if you are using Adobe Alchemy & PixelBender to compile and run your code!

During my session “professionally pushing pixels” at FITC Amsterdam this year, amongst other things, I talked about how to best utilize parts of the Flash Player to get top performance. This is one of the examples I showed. What you are seeing in this example, is +300.000 particles being 3D transformed, projected and draw to 2D. And it does so at quite a good framerate (well, it’ll depend on your machine too).

So, how do we achieve this ? The answer is a combination of PixelBender and Alchemy.

First, let’s look at the 3D transformation and projection.

Flash 10 has a number of native features to allow for 3D transformation and projection. You’ll find that this is a combination of using the Vector, Vector3D, Matrix3D, PerspectiveProjection, etc. Although these features are great, we can’t use them in combination with Alchemy easily. I’ll explain why later, for now, let’s look at an alternative method to do the projection.

Where oh where in the Flash Player do we have a method of doing very fast math ? The answer is; pixelbender! Although Pixelbender is normally used for image based manipulation, you can make it do any type of number-crunching which is able to be executed in parallel and without loops.

To calculate rotations and projecting our 3D data, we use Pixelbender in “ShaderJob” mode. When using pixelbender in image based mode, it operates in 8 bits per channel. Thankfully, when using it with a ShaderJob, it allows 32 bits precision per channel for the data processing. Since 8 bit precision wouldn’t be enough for this example, we use a shaderjob.

The VertexProjector pixelbender kernel, included with the source is a simple way of transforming and projecting vertices (representing particles, in this case) in 3D space. We feed this kernel a bytearray of x,y,z paired data, and execute the shaderjob. It then returns the data as a bytearray, in px, py, pz format.

Drawing things to screen.

Now we have all the 2D projected 3D data, we need to draw things to screen, and we have to do so as quickly as possible. This step is traditionally called rasterization. In AS3, you’re most likely to use getPixel when drawing on a per pixel basis. Doing so in a loop for 300.000 pixels turns out to be very slow. The solution for this would be to optimize that loop as much as possible. Either by writing your own bytecode, or maybe writing your own post-processor for you code, before you compile. But we don’t have too, since Adobe Alchemy exists.

As you can read in my earlier post about Adobe Alchemy, I openly questioned why it was so speedy, as compared to regularly compiled ActionScript 3 code. Although the answer is rather complex, the combination of C Based code, the LLVM compiler and “Alchemy Virtual Memory” are the base of this. The large difference between Alchemy compiled actionscript and regular compiled Actionscript can be further explained by the regular AS3 compiler not doing any optimisation. This example shows off those performance increases.

One thing to worry about when using Alchemy in your ActionScript projects is marshalling. You can read Branden Hall’s post on Alchemy for more info on that. Since we wouldn’t be able to marshal 300.000 vertices from a Vector.<Number> in AS3 to our alchemy code, we need to find a better solution. This is exactly why we are using Pixelbender and more-over, the bytearray data.

It is possible to manipulate the memory Alchemy uses in the runtime. This memory is represented as an AS3 ByteArray object. If we directly write and get our data from this memory block, no marshalling is needed. Although this means not all things can be done this way, for some things, this can be very useful. For instance, getting large blocks of data, like images and bytearrays of coordinates.

Getting all these 3D particles to screen is simply 1 inner loop. While we would normally call setPixel for that, in Alchemy code, we don’t have that luxury. Instead of that, we write directly to our screenbuffer memory, which is represented as a set of int’s. Here, one more problem comes into play. Endianess, defines the byte ordering for a set of data. Alchemy uses little-endianess for it’s internal memory representation. Specificall, it uses a small class called LEByteArray. This class extends ByteArray and ensures no changes are made to the endianess of the memory. Makes sense, since otherwise your code would blow up.

Writing to the screen is then a piece of cake. We take the alchemy processed data from it’s memory, and write it to a bitmapdata using the formerly much less usable setPixels() command. It’s amazing to see how fast this is.

Look at the example here, and download the full sourcecode here. As you can see from the example, the difference between doing this with regular ActionScript versus Alchemy nears a 5 fold speed increase.

Thanks to Keith Peters, for providing me with the 3D Strange Attractor code! And additional thanks to Mr.Doob for the stats object.

In future I’ll be posting more demos of the technology. Amongst which there will be one appliance for the future version of Papervision3D, PapervisionX.

The Dutch Adobe User Group is the fastest growing dutch community for creative professionals and I’m proud to announce I’m now officially an ambassador for the Adobe User Group.

For those who know me a bit already, you’d know I used to be in the board for the User Group, and was amongst the original founders, after the Macromedia UG was rebooted as the Adobe User Group back in 2007.  In all honesty, I wasn’t able to be active beyond that point. A schedule of work, conferences and workshops has put me into a great place in my life, but I wasn’t contributing to this amazing community initiative, nor working as hard as my fellow board members. These guys have lifted an huge amount of work and enthousiasm to create an amazing User Group. The work they did has put this User Group at the top of the world wide lists of user groups.

Really ? Let’s talk some numbers. These days, the Dutch Adobe User Group has grown into a group of 1700 people, every themed meeting sees an average of 200 visitors. Simply amazing. This is an event anyone can join into, see great sessions, meet new people and have a general great time. All of that, for free. Make sure you subscribe as a member. The presentations given at any meeting are of an amazing quality; amongst the big names are AKQA, Wieden + Kennedy and dutch agency Achtung, who have been taking the visitor behind the scenes of their productions.

A personal highlight for me over the last year was the arrival of the “German meeting”, where Powerflasher/FDT, Mario Klingemann and Andre Michelle came to show off their magic. You’ll note that all the videos for this session are AVAILABLE ONLINE ! Next to sessions, there are Awards where people can send in their work, discounts for creative magazines and sponsored networking drinks afterwards. The Dutch AUG is a great platform to raise your knowledge, inspiration and network. And again, all of this is free! How’s that possible ? Thanks to a wide range of corporate sponsors formed by leading industry companies.

Now, back to my role in all of this. My personal opinion was that they should’ve kicked me out of the management a long, long, long time ago. Instead of that, I feel honored they asked me to become their ambassador. In this role I’ll try and spread the word about this user group, and hope I can be a utility to extend the success.

None of this would have been possible without these great management members:

Bert Hagendoorn (Adobe Benelux) – Manager
Niels de Keizer (Nyls.nl) – Co-manager
Mark de Jong (Streaming Media Partners Europe) – Secretary
Ben Smeets (Logica) – Treasurer
Owen van Dijk (BGP) – Founder, Program Director en Events

The next meeting will be on the 13th of May and will be themed “Games”.

I’m extremely happy they asked me to fulfill the role of ambassador, and I’m not the only one, as Nalden is one too!

It’s been silent here. Next to tripping to Tokyo to speak at Adobe MAX in Januari and doing some small jobs, I’m doing a very, very cool Flash 10 Project utilizing the upcoming Papervision for Flash 10. For some optimization parts I’ve been looking into using Alchemy as an easy way to write optimized bytecode. As opposed to what some people took from my previous post on Alchemy, I don’t hate Alchemy. My point there was that if Alchemy can perform that well, so should ActionScript be able to, with the standard compiler.

Now, that being said, all work and no play makes jack a dull boy. With no time to spare during the morning, daytime and evening, I took the laptop with me to bed last night, and decided to do something I didn’t do before, port something with Adobe Alchemy. I decided upon a long time thing on my wish list; SID Playback in Flash. What is SID ? Well, SID stands for Sound Interface Device, referring to the MOS6581 and it’s a programmable audiochip as used in the Commodore 64 and Commodore 128. I grew up with the Commodore 64, and the audio it produces is still something I hold dearly. You are quite likely to have heard a SID chip before, if you listen to the radio, as it is quite a possible you’ve heard one of these baby’s scream. Good recent examples are :

Bastian – You’ve got my love. (the beat in this song comes from Jeroen Tel’s Rubicon).

Nelly Furtado – Do It (controversy all around, Timbaland blatantly stole Glenn Rune Gallefoss’s version of Acid Jazzed Evening).

De Jeugd van Tegenwoordig – Hollereer (Produced by Bastian, C64 lover himself).

My interest was more playing back the old C64 tunes from games and demos. Keep in mind, an average SID file is about 2kilobyte to 10kilobyte; a perfect Small Web Format for all your music during preloading needs . Now, this is something I have been working on before in Flash 9, using the FC64 project, the amazing Flash C64 emulation effort by Claus Wahlers and Darron Schall. Last year, I even made a start with porting FC64 to Flash 10, but that needed to be dropped in favor of actual client work. SID Playback is something different from playing back .mod .xm or .mid files, as the SID chip doesn’t do much on it’s own. Essentially, a .SID file is an executable meant to be ran on the 6510, the Commodore 64′s CPU. So to playback the file, you not only need to emulate the SID chip, but also the 6510 and the needed registers / memory.

Software emulation of the SID chip has been around for a while, and all modern platforms have a way of playback SID files. Winamp plugins, command line utilities for conversion, it’s all there. As is there opensource code to do this. There’s libraries for most major languages. So, after a bit of research, I decided upon the Linux version of TinySid. This library is small enough and well optimized. It looked simple to port, so I went at it. All in all, 4 hours later, I had this baby up and running! (And an unhappy girlfriend who isn’t into the nerd thing, nor is she into listening to chiptunes whilst trying to sleep). Considering my very rusty C and a little hack around I had to do for the audio output, Alchemy shows off it’s feathers here. Many kudo’s to the Alchemy and Flash Player team for this effort. Just now I spent half an hour building a tiny interface using Keith Peter’s Minimal Components, and collected some songs; and here it is.

TinySidFlash (open link, requires Flash 10).

My favorites : Politik and Science (A C64 version of Coldplay’s the Scientist), Morphing and Turbo. Yes, I love Dane’s tunes!

There’s no code yet, as this is still very rough and unfinished work. I’m pretty sure that when I get time again, I’ll have a go at LibSidPlay instead, since the overall emulation in that library is a lot better.

Take your time, this is a lengthy post. Before I jump in the matter, I want to go back a bit.

The magic of C64 Assembler

One of my first experiences with computer programming was on the C64. My dad bought my uncles C64…… Of course we played games at first, but at some point I started to look at all the books that my uncle had gaven with it. One of the first books I opened up and started playing with was called “Basic for Beginners”. I remember thinking, wow, so this is how you make games ? I started playing with existing programs and later on began writing my own, such brilliant programs as “My cassette collection” and “Adding Numbers”. But what I really wanted was to write my own games.

How come my C64 basic programs where always so slow ? Surely, there must be something I was doing wrong. With nobody around to teach me, or tell me anything about this machine, I dived in the pile of books that came with the computer once again. Two books took my interest. “C64 Programmers Reference Guide” and a home printed & binded book that had written on it “C64 Kernel listing”. I had two problems with these, being 1, my ability to read english and understanding the magic listings this book had. It took me a long, long time to grasp what these books where saying, but these 2 books, together with a dutch book on C64 programming, I slowly learned how to do 6510 assembly language for the C64.

This was the realm of pure magic. All of a sudden things went a lot faster. Moving a sprite along the screen could be done fast and smooth. But things where a lot more complicated. 6510 (or actually 6502, for the purists) asm, although especially now a days, quite simple, isn’t exactly humanly readable, nor was it clear to me how to structure full programs or games. I learned how to do a lot of tricks, like getting more then the 8 hardware supported sprites on screen at the same time (something then called multiplexing), playing sound while still having being able to do other things at the same time, getting sine waving color bars on screen and screen border, but I never managed to build a full game. What I learned about that was that interpreted basic was a lot slower then assembler and if you wanted to get real speed you’d have to target the platforms native code. Assembler wasn’t magic, it was just a better way to target the platform.

Back to current day.
I’ve been working with Flash for almost 9 years now. I dare to say I know quite a lot of Flash as a platform, and have been doing many projects and experimentation with it. Last year at MAX, when FlaCC or now, Alchemy was shown it looked like pure magic! Sure, I understood the basic principles, but running Quake in the Flash VM was nothing short of magic. Running Quake within the Flash VM using a port of the C code was quite an achievement, but the speed was more incredible. This was nothing short of pure magic, like turning lead into gold!

It’s an ironic name Adobe chose for FlaCC. Let’s look at the word and dabble into the occult. There’s a clear distinction between alchemy and science. Alchemy is most known for trying to create noble metals from otherwise less-valuable materials. Even Sir. Isaac Newton spent a lot of time in the field, surprisingly, even more time then he spent working on subjects such as optics and physics, for which he is most known. In a way, Alchemy is the forerunner for what now is chemistry. From the occult to exact sciences. So, with that name, is Adobe suggesting Alchemy is pure magic ? Turning lead into gold ?

Is it witchcraft ?
After the initial amazement I had with seeing Quake run within Flash, I started to question some things and started asking around with people who might know more of this witchcraft. Was this really Flash Player 9 running this ? From multiple sources I heard it was a slightly modified version of the Flash Player 9. Something to do with reserving a block of memory. Later on, after finding Scott Petersen his LLVM session on this subject, this was confirmed. In this session, Scott also shows an emulator running on top of the Flash VM. Having some experience in the FC64 (a commodore 64 emulator for flash) project, again, the speed was pure magic.

What I ended up finding most interesting about Alchemy, wasn’t the ability to take C code and compile it to Flash (although that’s an incredible achievement on itself), but the pure raw speed it seemed to harness. And with that, I also got a bit disappointed in Flash. It boils down to this.

The first magic bullet in Flash speed came in the form of the ActionScript Virtual Machine 2. With JIT compilation of ActionScriptByte code the platform just became a whole lot faster. With the coming of the AVM2, lot’s of things have changed. For me personally, one of the displays of the game changer was Papervision3D, in which I got involved quite early on. John Grden’s early demo using Papervision3D running on AS3 was an incredible testament to the speed of the AVM2. Now, that demo was built using ActionScript 3 as the language to be compiled down to ABC. ActionScript 3 is the language which was intended to run on top of this platform and one might thus derive that the language and it’s compiler is most optimized for the AVM2. So logically, one might asses that the best way to write code for the AVM2 is using ActionScript.

Heresy I say! Heresy! Burn the witch!
Alchemy is proof that this is not the case. Here’s a bit of pure heresy in the Adobe church : Alchemy is a testament to the underachievement of the ActionScript compiler. Ouch. Surely, this can’t be the case, right ? Well, yes and no.

Let’s start with the “no” part. That reference of a modified player running Quake was partially true. This was an early demo of Flash 10, in a way. As Nicolas Canasse points out in this blog post about Alchemy virtual memory opcodes, it seems like Alchemy is cheating a bit. Cheating ? While, yes. This part of the API is not available from within ActionScript itself. So, the platform (AVM2) now has features which are not supported by the language which is native to it ? Wow. I find this a very significant move by Adobe. Moving to a more flexible compiler makes total sense, but is this the first symptom caused by the EcmaScript4 draft debacle ?

To achieve the speed as displayed by Alchemy, I have thus have to move to using C code, or HaXe to compile for the AVM ? This is odd, to say the least. Nicolas his work on HaXe is nothing less then brilliant. The features this compiler has are those which would greatly help the development of a 3D engine, such as Papervision3D. I’ve always had a great deal of interest of porting Papervision3D to HaXe, purely to be able to implement things like inlining easily, which will safe us function calls and thus speed up the process of rendering 3D to the screen. The downside to that is, that the code base would have to change to HaXe, and thus become incompatible with the compilers of Flash and Flex. We (Papervision3D) could provide with a SWC built with HaXe, but in a way, I’m still a bit weary of that.

Now we come to the yes part, of my earlier “yes and no”. Next to the virtual memory opcodes that are currently unsupported from within the AS3 language, Branden Hall states something very disturbing on his blog about Alchemy, “Understanding Adobe Alchemy“.

“In fact, because LLVM does a lot of optimizations, whereas Flash and Flex have no optimization step built into their compilers, Alchemy code manages to overcome a lot of it is overhead just on its own.”

Ok, that is down right disturbing. The AVM2 was built with ActionScript in mind and visa versa. But compiling C using LLVM with ActionScript as a target produces better faster ActionScript (that’s actionscript, not bytecode) then the actual compiler built for the platform does ? Ouch. Now, moving to C development for the ActionScript platform to utilize this “magically” acquired speed seems somewhat illogical to me ? Joa Ebert already showed us some work he had done on the compiler, allowing for inline bytecode to be used. Nicolas shows us he can harnass the new Virtual Memory Opcodes, and his work on HaXe also shows us that inlining is very, very nice.

I’d love someone from adobe shedding some light on this issue. I love the Flash Platform, I love the work all the people at Adobe do, but this leaves me somewhat confused on how to develop for the platform I hold so dearly. If we have to let go of ActionScript to harness the pure speed Alchemy proofs is possible, then tell us. What is the future of our beloved language and our compilers for it ?

Intro

As promised in my previous post, here’s a small opensource project from my side. It’s tiny, really, but I hadn’t seen an ActionScript implementation of any form of Image dithering before.

Dithering….what ?

Just a brief explanation what dithering in this context means. From wikipedia :

“Dither is an intentionally applied form of noise, used to randomize quantization error, thereby preventing large-scale patterns such as contouring that are more objectionable than uncorrelated noise.”

Words to take note of in that sentence are quantization, and “intentionally applied form of noise”. In our case quantization is the removal or swapping of colors. The form of noise applied differs over the several implementations and in so called “ordered dithering” it’s hardly noise, rather a carefully chosen threshold matrix.

Dithering, why ?

Dithering imagery has been around for some time. Put a close eye to any black and white printed newspaper’s pictures and you’ll see the effect.

It’s basically been around a lot longer then I have in my 28 years of life…that’s for sure. I first learned terms like “ordered dithering” and “Floyd-Steinberg Error Diffusion” in my young and early days on the Amiga hardware. See, in those days computer hardware wasn’t capable of displaying the huge arrays of colours like now-a-days. Since you then had few colours to spare (a typical amiga workbench ran at 16 to 128 colours or so), you needed to be creative to get yourself a nicely pimped desktop image, whilst still sparing colours for your icons.

A better example of a dithering implementation for our industry is probably that checkbox when you want to save for web in Photoshop, using GIF. Or when you print something on a pure black and white printer.

Conclusion; dithering is normally used to create the illusion of tones on a device which is otherwise not capable of displaying it. So why port this to our ARGB/32Bit enabled Flash Player ? Part of the reason why I worked on this is because I just like the aesthetic of the effect. It just brings back wonderful memories of pimping my Amiga desktop.

Dithering in AS3

So, the algorithms for dithering are really quite simple. I was playing around with hydra Pixel Bender and had some ideas on converting some old algorithms to have them run in realtime. Then I thought I could make this run in realtime on Flash 9. So, first I did a version of the Algo in pure AS3. I played around with converting it to something fast enough to run at at least 20 frames per second on an average machine…and painfully failed. The remains are the AS3 version (hydra Pixel Bender version still in progress)….and I decided to clean that up a bit, and have it released as an OS project for anyone to use.

Currently it only contains so-called error-diffusion based ditherers. They make the most sense anyway, since the ordered ditherers really have nasty visual sideeffects, like Bayer’s crosshatches. Check the variants out in this little demo application using the class.

The Demo : try and use “No Dithering” first to see the effects of regular palette conversion.

The Source

Usage :

ImageDithering.dither(bitmapData, type, levels, grayscale);

Where :

BitmapData is the image to be manipulated.

Type is the form of dithering, currently supported :

1. ImageDitheringType.FLOYD_STEINBERG
2. ImageDitheringType.FALSE_FLOYD_STEINBERG
3. ImageDitheringType.STUCK
4. ImageDitheringType.NO_DITHER

Levels is the amount of colour levels to quantize to per channel.

Grayscale is a boolean indicating whether to convert the image to grayscale before the process is ran.

Right click and viewsource on the example to see how it’s implemented in Flex.

Quick update : Mario Klingemann, working on Aviary’s Peacock did a quick test with it. My effort had some use anyway!

“Pixel Bender” ? Really ?

[Fill in my excuse for not posting in such a long time]. There’s so much to tell, hence the weird title. Let’s start of with todays news; Adobe renames Hydra to Pixel Bender. They actually did it….now…it might be me…but wasn’t Hydra a way cooler name then “Pixel Bender” ? At least we’ll be able to make Bender references all the time now…adding to nerd-status. The cool thing ? Bender’s brain is a MOS6502 … the same cpu which for instance, also powered the best machine of all times.

Puppet Interview

So, moving on from that news. Over the last months I’ve done several interviews. I was reluctant to post them, but come to think of it, why ? You are reading this page so you’re at least interested in what I have to say. The best interview as of yet was with Gideon, one of the worlds best designers. In all honesty, “jet-lagged” as I was, I wasn’t at my best, so these interviews might entertain you more; Joshua Davis, Hugh Elliot, Moock&Gmunk, Seb-Lee Delisle.

FITC & rePhlex and Flash Player 10

Of course, all of this was recorded at the FITC Toronto 2008 conference. First off all, thanks to the FITC Organisation…you guys did an awesome job. The conference had an excellent vibe … (except for that annoying drunk dude at the front door on Tuesday…you know who you are), and I had a great time hanging with you all. I did a sold-out workshop on day 1, day 2 a panel, with Adobe’s Justin, Richard, Lee, Joa from Hobnox and Mr. Paperworld: Trevor.

On day 3 I did my session…I was shocked to find the room literally packed with people….and was somewhat nervous throughout some moments in my session, but as it seems by the feedback, people really enjoyed it. The reviews I found in my incoming links are at least very positive.During this session I also got to talk about rePhlex again, my new toy project. rePhlex is meant to be a lightweight, realtime image processing and syncing/eventing library for site/example and demo purposes. It’s been overgoing some heavy changes and refactoring…you’ll all just have to wait a bit more.

Another thing I really enjoyed doing in my session was that I was allowed to show some hydra Pixel Bender running inside Flash Player 10. Yes….you missed a sneak of Flash Player 10′s awesome power.

On day 4 of the conference I had a taped conversation with Keith Peters, which, next to the obligatory joke he made at my cost, to get back at me for sleeping a bit to long, we both hoped won’t bore you too much…you put to actionscript / math / graphic nerds on a couch…what do you think happens ?

I saw some sessions myself, but was really unlucky with my taste vs the schedule. Things I can quickly remember seeing; Seb-Lee Delisle, Mario Klingemann, Keith Peters, Robert Hodgin, Erik Natzke and many others. Robert’s session (the last of the conference) absolutely blew me away. And I do love Radiohead!

So, that would wrap up FITC Toronto (and Amsterdam too).

Upcoming Conferences

I’m looking forward to speak at this months Multimania. In June there’s going to be FITC Chicago…later on this year many more, amongst which for instance; Singularity 08. I’m really unlucky to have to miss out on this years FlashBelt…..hope you all have a good time there anyway.

Goodbye Earthmine ! Hello new works ?

As some of you, attending one of my sessions or workshops earlier this year might know, most of my commercially available time went to the Earthmine project. That work ended for me last week and after working on one project for such an extended period of time (7 months+) and some rest, I’m now really looking forward too and open for new jobs and work..I’m not sure what I’ll do yet and am still contemplating on what I really want to do as the “next” thing. In the mean time I’ll be working on smaller freelance jobs. And if you have something you’d like me to work on….ping me.

Playing with Hydra and image algorithms

Another post coming up today will explain some more. With my work on rePhlex and Hydra, I’m having a blast doing image manipulation algorithms….one of my experiments was image dithering and half-toning…with as a result a new open source project. More on that in my next post…which will appear on the all new and revamped AXNA / Adobe Feeds! Welcome back !