Read Part One: Motto’s and Percentages. Read Part Two: Byte Code, Control Tags, Renderers… oh my!

Size Was King

Prior to Flash Player 9, one of the complaints from the Flash development community was the overall performance of the ActionScript language. Performance not only for animation but also for manipulation of large data sets. This issue was most apparent with Flex 1 and 1.5, which were developed on ActionScript 2. The challenge the Player team was facing with ActionScript was that the ActionScript Virtual Machine (AVM) had reached its performance limit.

Until Flash Player 9, size was always the most important factor for the development team. These were the days when modems still ruled the earth and ATM to the door was a fantasy dream utopia. To help facilitate adoption, the Flash Player had to be small and compact so that users wouldn’t flinch when it was time to install or upgrade. During my stint at the company, I remember hearing of huge battles for just adding 10k to the player size.

With Flash Player 9, the landscape had changed a lot. High Speed Internet was more of a reality and therefore size became less of the driving factor and more of an desirable goal. Performance was now the new king. At the time, Macromedia decided to start the process from moving the Flash Player from a plugin and into the realm of a platform. In our planning meetings we started to talk about the Flash Platform and what it could achieve. To make this a reality the player had to speed up and handle large data sets with grace and ease. With this goal; in stepped AVM+.

My Two AVMs

AVM+ was the new ActionScript Virtual Machine. It was designed from the ground up for screaming performance. The goal was an overall 10x performance increase, which was a really serious task for the Player team. Not only was AVM+ about speed but it also opened the doors for new features such as Just In Time (JIT) compiling, cleaning up the existing and often confusing API set, updating the ActionScript language to a first class citizen and finally exposing granular control over how content is accessed and managed.

One of the drawbacks with AVM+ is due to the radically different design, there was no way it could be backward compatible. This meant that if the Player only had AVM+ then the team would not be honoring their motto of “Don’t break the web.”. To resolve this, Flash Player 9 contains AVM-, or the original AVM from Flash Player 8 (plus some fixes). Having two AVMs does a few things that are not beneficial:

1. Dramatically increases the file size of the Flash Player.
2. Means that mixed code, such as SWFs that use AS3 and SWFs that use AS2 in the same application, are not able to directly talk to each other because the code is being run in different sandboxed AVMs

What did we get?

AVM+ created support for ActionScript 3, the first implementation of the ECMA 4 Spec. ECMA is the standard that JavaScript is based on and ECMA 4 is the basis for the new JavaScript 2. Right now there is a tremendous amount of fighting going on about ECMA 4, JS 2 and Browser support. The Ajax community is all aflutter right now but one of the benefits is that when JS 2 is finally approved and in the wild, AS3 developers will have a good leg up on the language and its structure.

At the release of Flash Player 9, ECMA 4 was still very much in draft form and meant that the Player team had to solve language issues without the guidence or approval of the ECMA board. The issue with this is that when the language format is finalized new syntax will need to be added to ActionScript. My guess is that AS4 is not that far away… mmm “let is the new var”.

As an old school Flash developer, one of the coolest additions to the AVM+ is the ability to directly access and manipulate the display list. In previous versions of Flash if we wanted to dynamically add content we need to either load it into a MovieClip using the old attachMovie(), or create an empty MovieClip to shove things into. Now that the display list is accessible developers can spawn new instances of objects and attach them directly to the display list. The display list is a tree based on a parent/child hierarchy that enables branches to be moved around while maintaing their structures. This is great for when you want to build a specific layout, pop it off of one part of the display list and then add it somewhere else. Again, this is a deep topic that I may write a whole article about.

Debugging an application has become a lot easier with AVM+ because a robust Exception handling system has been put into place. This is great for building apps because its now possible to catch the exception or see the full call stack when the exception is thrown. One of the challenges with exception handling is that if you don’t catch the error and your user is running a debug player then they will see the exception. Right now uncaught exceptions are a big problem with Flex apps.

AVM+ now supports Loading classes, which may not seem a big deal but it offers a lot of flexibility. In the previous players, when you loaded content the item that was loading the content was removed and replaced by the loaded content. Now a loader class can be used to pull in assets and then either displayed via the loader or the content can be referenced and used elsewhere. This also means that if the data is graphical (PNG, GIF, JPEG, Video, etc.) and coming from a crossdomain supported location you can use the bitmap clone() feature and then clone Bitmaps from a single loaded item and use it in multiple places. This is pretty powerful stuff if used correctly.

Finally, for this post at least, we come to the open source aspect of the AVM. Adobe did a very interesting, and personally brilliant, move. They open sourced the AVM+ to the Mozilla foundation. As I mentioned earlier, the AVM+ is for ActionScript 3 which is an ECMA 4 language, just the same as JavaScript 2. Mozilla is now using AVM+ aka “Tamarin” as their core VM for JavaScript 2. As Tamarin is updated by Mozilla, Adobe pulls in the new build of the VM and uses it in the Flash Player. If Adobe makes a fix to the VM, they submit it back to Mozilla. This helps keep the languages (ActionScript and JavaScript) in sync with the ECMA standard and opens the door so developers can truly see what is going on under the VM hood.

Enough for now… next time I will dig into the world of JIT and Garbage Collection.

Update: Read Part Four: Just-in-time (JIT) compilation

Read Part One: Motto’s and Percentages.

Byte’ing the Hand That Feeds

When an application is compiled from a FLA or mxmlc the generated SWF file has two main components that control how the application behaves: Byte Code and Control Tags. The Byte Code is the generated ActionScript. Control Tags are currently only produced from Flash Authoring.

The generated Byte Codes are loaded into the Player via the SWF and are sent to the Flash Player ActionScript Virtual Machine (AVM) for processing. The AVM processes the Byte Codes and then manipulates the current Display List based on the commands within the code. The Display List is a tree based structure that represents an asset hierarchy within the SWF. These assets are MovieClips (Sprites / Components), Audio elements, Video elements, etc. The assets are added, updated and removed from display list by the AVM. In the following example, the code is compiled down into a Byte Code set that at run time informs the AVM to create a new Button and then add it to the display list.

var myButton:Button = new Button();
Application.application.addChild(myButton);

Control Tags skip the Byte Code and AVM step and directly access the Display List. Control Tags were briefly mentioned but from what I recall when I worked at Macromedia/Adobe, the timeline’s keyframes are compiled into Control Tags and when processed by the Player they update the stack according to how the keyframe interaction was defined.

Render Me This

There are two types of renderer’s in the Flash Player, immediate and retained. The vector engine that Flash is famous for is a retained renderer. The example that was used was that we request two boxes to be drawn in the same render pass (or frame): a red rectangle and a black rectangle. The black rectangle is 40px by 40px, is located at 10,10 on the stage and is requested first. The red rectangle is 200px by 200px, is located at 0,0 and is drawn second.

The retained renderer is a smart rendering system and sees that the black rectangle is covered by the red rectangle and therefore the rendering system skips drawing the black rectangle. This process helps memory usage and other system gains but it is a more time consuming process because all the drawing requests have to be made before the render pass is completed.

As I mentioned earlier, there are two renderer’s in the player. The second renderer is the Bitmap renderer which is an immediate render and happens off stage. This means that as data is requested to be drawn in the Bitmap renderer buffer, the update is done at the time of the request. This makes Bitmap rendering much, much faster and is one reason why changing animations and other assets to bitmaps can offer an increased performance gain. For more information about Bitmap Caching performance check out Guy Watson’s “Using Bitmap Caching in Flash”.

I was going to delve into the AVM deeper in this post, but looking at what I have to talk about I am going to save that for part three!

Update: Read Part Three: The ActionScript Virtual Machine
Read Part Four: Just-in-time (JIT) compilation

You break it you buy it

First off, Jim and Lee dropped the Flash Player motto on us: “No breaking the web”. This is a really important concept for developers as they evolve apps over time. As a lot of oldschool Flash devs know there are a lot of hacks we used to take advantage of. Accessing objects via hidden flags so we can get to the data that we really need. The issue is that some of these hacks were bugs in the player that we used to our advantage. What happens when the Flash Player team finds and then fixes the bug we built upon?

For example, we built our SWF in Flash 6 and used a hack to make our application work. In Flash 7 they fix the bug/hack. This would mean that if our SWF was run in Flash Player 7 the app would break… and this violates the Flash Player teams motto.

So, what they have done is made a kind of progressive fallback sandbox. If it worked in Flash Player 6 and your movie was made for Flash Player 6 then your SWF will be run in the FP6 version of the sandbox and your hack will work. If the developer wants to leverage the fix in Flash Player 7, then the SWF needs to be recompiled for that version.

This ability goes all the way back to the first Flash version. I am making the assumption this is based on major releases and not dot releases. I also assume that this is only for hacks that are not considered a security risk. I really wish I had a chance to ask about that… but my brain was full by the time the session was done.

How deep is deep?

One of the questions clients almost always ask is: “How deep is the Flash Player x.x penetration?”. If you go to the Adobe website you will see that Flash Player 9 has 93.3% of the mature market (as of Sept. 2007). Okay, but the trick is what about the .x? Does 9.0.47 really have 93.3%? When “Moviestar” (the newest player dot release code name) rolls out when will it have the full share of the pie? Jim and Lee dropped some great estimation numbers on us. These are based on the percentage of adoption from the release date of the new dot release (or player too).

@3 months = 30 – 40% @6 months = 55 – 65 @9 months = 80 – 85% @12 months = 90+%

What it gets down to is that if you need the newest dot release you need to consider the penetration estimates just like you would with a new full version update. For a great breakdown of how this works check out Emmy Huang’s blog about the topic (where I re-poached the values since I couldn’t type fast enough to get them all).

Okay, so that’s post one and I just scratched the surface… next up will be Byte Code, Control Tags, AVMs… oh my!

Update: Read Part Two: Byte Code, Control Tags, Renderers… oh my! Read Part Three: The ActionScript Virtual Machine Read Part Four: Just-in-time (JIT) compilation

I still have a lot of notes from Max that I want to post, the one topic that I really want to dig into is the Flash Player internals presentation that I attended. Lots of great stuff and I want to sit down and get it up. I may try to later today.

I have heard rumor that sessions from Max were recorded and may become available. I hope they do, specifically for the Inspire Session that was presented by Michael Gough. Michael co-heads the Adobe XD design team and he did an amazing session talking about design challenges and goals for current and future web applications. Brilliant stuff and I want to spend more time thinking and talking about what he hit on.

The Adobe Share application is a Flex based tool used for your repository management. You an add files, remove files, update files, manage user permissions and other basic functionality you would need.

With the new repository Adobe is also releasing a public API to allow developers to build custom tools around the repository. The API services provides hooks into the Share system to manage users, file tracking, file upload, usage tracking, etc.

The API is REST based using a token system that most services have been adopting. Developers request a token for their repository and then use it in the connection to identify the account. The key not only pairs you to the repository, it also prevents abuse and allows Adobe to provide service throttling.

Currently the accounts are throttled to 10,000 calls an hour and require a 500ms delay between calls from a client. Clients are tracked by session so the time is not tracked for the whole account, just the specific client. The 10k limit is set to the account.

One interesting feature that was hinted at during the session is the ability to have a MD5 hash as a unique key for your application. This key allows you to verify that the application you developed is the only one that can access your repository and helps provide a much more secure application. There are currently two SDK libraries available for Adobe share: Java and Flex. They didn’t mention if the SDK for Flex is truly Flex or if it is AS3. My impression is that it was Flex and leveraged HTTPService, but I can not confirm this.

Currently, media files are not supported (MP3, FLV, MOV) but DOC, PDF, ZIP are. One feature they are looking to add is the abiltiy to do document conversion so that a PDF can be converted to a SWF or potentially a Word document to PDF, etc.

The second half of the presentation was about the upcoming Adobe Connect public service APIs and SDK. One note about the following content is that this feature set is still in development and may drastically change from what was presented.

First off, Adobe Connect is being rebuilt entirely in Flex (probably 3 since its still in development). One of the benefits of this is that a lot of the core classes and components that are being developed for connect will be released with the SDK. The focus of the redesign is to refactor the UI to move a lot of basic tools and options out of the pod module and move them to a more unified control panel/UI that allows easier access from any view.

One reason that Adobe wants to release an SDK is that there are a lot of basic requirements for Collaboration applications, such as Push/Pull of data, AV streams, user identity (who the users are), user presence (who has joined/left), permissions (who can control the presentation) and document/media sharing. Many of these tasks are often not thought of developers, are tedious to build or can become complex very quickly. Adobe wants to help streamline development by providing solutions to these tasks, that is where the SDK comes in.

The SDK is currently called CoCoMo and provides the foundation classes used in Adobe Connect. The SDK also includes component modules/pods that support webcam, mic, chat, etc. These components can then be extended to make custom versions or developers can take the foundation classes and build their own from the ground up.

As I mentioned there is a public API that is REST based that will allow developers to leverage the existing Adobe Connect network and build applications using this network. This is important because developers can build large scale collaboration applications and not worry about having to build a custom hosting solution to manage the server environments.

An interesting MXML tag that Adobe is working on for the Connect SDK is the <mx:connectSession> tag which handles connection to the servers. This tag points to a URL redirect that allows for automatic failover if the current server goes down, reconnection, connection identification management, etc. This tag also handle child creation only after the connection has been established. This makes development a lot easier because developers don’t have to worry about async management and load order. This functionality can be overridden if needed but it seems very straightforward and easy to use.

What Adobe really wants is Real Time Collaboration (RTC) out of the box. This includes: Async management, room management, user presence/management, disconnect/reconnect automation, state persistence(“late comer” data is managed for you), recording/playback, external addressability (new clients that talk to the service).

Check out the Adobe blog: blogs.adobe.com/collabmethods. They will use this as a forum to talk about the API / REST technology.

Currently the API and SDK is not available but the goal is to at first launch a free but throttled version and then roll out purchasable enterprise solutions.

I am at Adobe MAX 2007 being held in Chicago and it was kicked off with the General Session hosted by Kevin Lynch. As with previous MAX’s the general session is part sneak peek and part highlights of the new technologies/applications that Adobe and their customers are creating.

I am not going to focus too much on the general session, but more post notes about the sessions that I am attending. I do want to mention a few things I learned at the general session but for now I be posting information I gleam from the sessions…

One of the cooler things I did see at the general session was a peek at the new Flash Player version 10, aka “Astro”. Two features they are focusing on are text managment/manipulation and effects. The new text features have much more granular control of text objects and full support for right to left languages. This is a huge update because one of the hardest things to work with in the Flash Player is textual content. In some of the projects I have worked on the solution was often to not let Flash handle the text and move the control back to the browser for displaying complex layout.

The other great thing about Astro is that they are rolling out a new language to allow custom filters and effects that can be applied to the Flash content. These filters are similar to the existing filters such as blur, drop shadow, etc. The language used is the same that drives AfterEffects effects and this will allow for much more high performance effects that are small and lightweight.

Internet stability has been spotty so I will try and post as I get steady connections. There may be multiple posts at a time depending on when and where my connections actually talk with the network.