The battle for making riches out of RIA platforms is now really heating up between Adobe’s Flash/Flex platform and Microsoft’s Silverlight. This is an important area for the future of web applications, and is bound to give the winner a critical flow of developer bucks.
Up until now I’ve been an advocate of Flex through-and-through. This was partly, I admit, because I’ve always had the feeling that Silverlight has just been a copy of the Flex/Flash environment. But in the real world, it doesn’t matter so much who got there first, and it’s the current productivity and functionality benefits that count. And so it’s only come to dawn on me recently that there’s one big problem with Flex that Adobe has yet to face up to…if they can. And it’s potentially a killer.
The problem
At first the problem seems a like not such a big deal. In fact, for many RIAs out there today, it probably makes not once single bit of difference. But I think it really is a fairly vital feature that Microsoft offers out of the box, but Adobe has somehow managed to over-look. And I think it’s only because of the relative immaturity of RIAs, and the strong event focus of Adobe’s Flash platform that has caused it to be overlooked.
The problem comes down to one word: threading. In short, Silverlight has it, and Flash doesn’t. We’ll talk a bit about why Adobe haven’t provided this below, but first let’s examine why it’s an issue.
The scale of the problem
How important is multi-threading in the RIA space? Well, of course, that rather depends on what it is that you’re trying to build. As I have already said, I don’t think this is a problem for many of the RIAs out there today. On the other-hand, it certainly is for some. As RIA apps grow more complex and libraries become sophisticated and nested, it’s going to be more of a problem.
Synthesizers
Take the fantasic Noteflight. If you haven’t yet seen this app then you really need to check it out. This Flex RIA app lets you drag around musical notation to compose scores, which may then be played-back using a synth that overlays sounds loaded from .wav files on noteflights servers. It’s a fantastic app, and I’m pretty sure that tools like this will bring to many musicians what mp3s have brought to music listeners (i.e. mass, systematic theft of music, which has, somehow, to be properly monetized).
But ‘.wav’ files? What’s going on there? Surely nobody would notice if they noteflight used mp3s for thier sound samples, and wouldn’t the download time be a million times better? Well, as it turns out, noteflight.com used wav files because “an ActionScript3-based decoder is inevitably going to be slow (and consume development cycles too)“.
Now maybe I’m reading between the lines a little bit, but here is what I suspect is going on. If you do a method call to decode an mp3 file in ActionScript then it’s going to take quite a long time to complete. During that time, because Flash is single threaded, absolutely nothing can happen in the UI. Nothing. Progress indicators can’t update, mouse events can’t be processed, and the user will generally be left with an impression that the RIA is temporarily dead. This isn’t at all good.
For Silverlight, on the other hand, I believe it would be a doddle to spawn a worker process to sort this out, and meanwhile still remain responsive. Just like what you’d expect out of a modern-day platform.
Image processing
For a chronic example, look at the quick image resizer. This site is pretty nifty because it lets you resize a jpg, directly from your local filesystem, without once transferring an image between you and the RIA’s server.
But my-oh-my have you seen what happens when you resize a large image taken with a digital camera? Nothing….for a long while, and then…..your image. This simply won’t do.
Other apps
It’s not hard to extend this to a whole host of other applications that need threading. Doesn’t a word-processor need to spell check innocuously? Doesn’t any form of editor need to be able to process it’s content for saving, without blocking the UI? Is there any point in Flash Player 10 having support for audio processingPixelBender aside, which is kind of a cop out for certain flavours of processing, and besides, do most developers really want to have to code this stuff in a different language catered for mass parallel processing)?
Why is Flash like this?
The short answer to this is that Flash has grown organically and hasn’t always been intended for the sophisticated RIA applications that people are now starting to build with it. In fact, ActionScript was first created with much smaller ambitions: i.e. animated vector graphics. The Flash event model was ideal for this, queuing up messages for processing via a single thread. Now these original platform intentions have been hijacked to support the Flex libraries that enable such more complex embedded web content, but the underlying threading model didn’t change.
In contrast, Silverlight has been built from the ground up with more ambitious targets in mind: fully fledged applications.
How can the problem be solved?
Instead of moving in the direction of Silverlight, there are a few other steps that Flex developers can take to get around this problem. Unfortunately though, none of them are particuarly pleasant!
Splitting tasks up
One approach is to simply split up big processing tasks into small blocks so that other events can be processed between each block. If this is possible then it’s certainly an approach, albeit an awkward one. It goes without saying, however, that it is a complete pain to manually divide up sequential composition into Flash’s event model.
Spawning another VM
Another approach is to get all your work done in a completely seperate Flash virtual machine. Flash’s LocalConnection interface is probably the best way to communicate with such a seperate process. Despite again being an obvious pain to code, this method is also likely to be pretty slow at transferring large amounts of data between the VMs, and there are all sorts of synchronization problems when you cater for the user having the same app open in multiple browser windows etc.
Adobe extending the Flash Player and ActionScript
It should be possible for Adobe to cater for multi-threading in a single virtual machine, but it’s likely to be a lot of work for them. Obviously, many of the libraries (including the Flex ones) would need to be rewritten to add thread-safety. Events would still have to be processed one at a time by a single thread to maintain backwards compatibility, since existing apps will depend on this.
Adobe’s stance on this is a little disappointing, as Alex from Adobe explains on his very useful blog:
As we hear more from our users about the importance of threading, the higher its priority will be.
While it’s very nice that Adobe is taking our concerns into account, I can’t help thinking that this isn’t the best way for them to do business. Wouldn’t it be better if problems could be anticipated, rather than waiting until developers are upset about them? In this case, Microsoft seems to have managed this pretty well.
Any reason you didn’t include JavaFX in your comparison? This also allows threading and I’m willing to wager that it will have better cross platform runtime support than Silverlight by the end of the year.
@Sharebear - Not mentioning JavaFX is purely out of my own lack of experience with it (for instance, I have no idea what tools there are for graphical designers which integrate with JavaFX), plus the fact that it does seem to be less of a contender judging by other discussions that I have seen. Of course, this may all be rather circular, and you may well turn out to be right
Wow, your argumentation have logic. i would like to see a response from Adobe flex developers…
Many of the places where threading is needed (processing data of some sort) can actually benefit from Alchemy:
http://labs.adobe.com/technologies/alchemy/
Alchemy has implemented some of the C standard library on top of the Flash Player - including threading (afaik).
Another option which is still experimental is to use Pixel Bender for data processing:
http://elromdesign.com/blog/2009/02/09/using-pixel-bender-to-do-heavy-lifting-calculations-makes-flash-player-multi-thread/
The big downside to both of these options is that they currently require you to write the code in a language other than ActionScript. It would certainly be beneficial to have threading just built-in but until then there are some workarounds.
-James (Adobe)
Thanks for this article. Helps to know that we made the right choice for our upcoming release. If it helps anyone else out there I know our dev group has found some great Silverlight tips at mostlydevelopers.com
Thanks James. By the look of it Alchemy somehow includes some sort of thread management in the generated bytecode, but still ends up being faster than normal ActionScript overall, which does seem somewhat magical. I guess it must gain some compilation help from the use of C. Please up the support for AS3 multi-threading if you can
It’s frustrating as a rich app developer and a video game developer watching slow moving organizations such as Adobe or even the W3C labor at building out the support for the richer interfaces we all want to build. With Flash it is ridiculous to see that PaperVision3D has to even exist - that 3d vector graphics aren’t a natural part of flash by now. Consider the opposite case: IPhone Applications act like web applications in that they’re delivered over the web but they go down to the metal to exploit the limited hardware resources - and in many ways these small apps deliver vastly superior interfaces than most web apps running on far more powerful computers. Sun and Java were intended to be the heirs to this throne but they seem to have failed as well. The whole thing is a disaster. If you want to get a good sense of how much this has slowed the quality of web applications consider the difference in quality between the average video game, manipulating 100k lit shaded polygons in real time, with heads up just in time displays, shared many-participant worlds, with a responsiveness measured in milliseconds - and the average web application such as say google maps.
One other thing… Everyone who wants this should go vote for this feature request:
https://bugs.adobe.com/jira/browse/ASC-3222
-James
for long running tasks in AS just like in JS, you can use setInterval() and that essentially can give you simplified multithreading.
why don’t you stop making online versions of things that should be local applications?
A music composition program is a moronic thing to make online
This article is almost as mute as the one that states that Flex/Flash isn’t ready for the enterprise yet because it does not render HTML.
Flash/Flex does have multi-thread support; you don’t have to have your UI lock-up. For starters Flash runs a number of background worker threads to help with things such as network loading, etc. But most importantly, PixelBender allows you to run complex calculations (such as image manipulation) outside on the main loop thread. The example you gave above about the image resizing app locking up is because the developer did not choose to use PixelBender. http://help.adobe.com/en_US/AS3LCR/Flash_10.0/flash/display/ShaderJob.html
@ed Alchemy works via breaking up methods/loops into chunks (”chunking”) that are scheduled. You can do this without Alchemy. http://tv.adobe.com/#vi+f15384v1023 26mins-in
@TRS: Could you explain why this is a moronic idea? Something like Wordpress lets us hold collaborative discussions, so why shouldn’t something like Noteflight let us hold collaborative composition? Guitar tabs have been a very popular feature of the internet, whereas classical music scores have yet to get to this point.
@Alex: Thanks for that. I’m sorry if this wasn’t clear, but I meant explicit threading support in the language. In my opinion, including background worker threads is like saying that javascript is multithreaded because a browser is. This lets you do some work in pseudo-parallel, such as resizing an HTML image, but the expressiveness of the parallelism is weak.
Why is Flash like this?
Calling UI code from an arbitrary thread is not thread safe, that’s why it was never introduced. Threads are limited in Silverlight and they would be limiting in Flash as well, which is why they never introduced them. I really wish the Flash player allowed threads as well, but I don’t think its the elephant you make it out to be.
Any USER code block that would be valid for execution in a secondary thread, is also going to work well when split up across frames. The problem that flash has is that most of their internal apis were not designed to be multithreaded, or provide a way for asynchronous callbacks to the main thread. This includes AMF/ByteArray deserializing, JPEG/PNG decompression, and audio decompression. If you have code hanging outside of those calls, then you’re doing something wrong.
All this sounds great until you are *forced* to use Silverlight with WCF. And then somewhere way down in the stack, an error is thrown. And then that error bubbles up to the top, not as the original error, but as a 404 or a 500 error.
And we won’t even discuss the auto-update Microsoft forced on machines between Beta1 and Beta 2 which broke some xaml code but still allowed applications to compile.
As adopters of Silverlight back to Beta 1, the experience has been quite mixed. Some things great, some things awful. Xaml is painfully verbose. Xap file sizes can easily get out of control. If I could go back and start over, I have to say, I would probably give Flex a try. I’m not sure the ability to spawn threads would be enough to entice me to stick with it.
@Jasper: It’s interesting that you’ve had bad experiences, although I think MS are justified to not to maintain compatibility between beta versions.
Anyone choosing to develop a substantial application in either Flash or Silverlight is asking for a world of pain. Microsoft have a long inglorious history of building up technologies and then throwing them in the dumpster, Flash just plain sucks. Either choice will very probably lead your application into a technical cul-de-sac where it will stagnate and die.
The world has chosen html, css and javascript and it will improve evolve and grow to include all the capabilities of these other technologies.
While there are features Flash and SIlverlight both have the other doesn’t, both provide more than adequate RIA platforms. I think the thing Adobe should be worried about are developers like me. I come from a computer science background, not a designer background. Adobe and Flash are definitely not the first names that come to mind when I think of web programming. This is a product of them marketing to designers instead of developers, and Flex’s relative newness. There has also been some general confusion about Flex and Flash in the traditional programmer community. I have had to explain the relationship of Flash & Flex to smart, talented programmers on more than one occasion.
It should also be noted that there are developers like me that have been using Visual Studio for years. My entry cost to get into Silverlight development is $0.00. I don’t think I have had a computer in the last 10 years that hasn’t had a version of Visual Studio installed. My entry cost to do something in Flex is $249.00, from what I understand. There is also the environment. I have been programming in C# since 2000. There are a lot of C# and a lot of VB.net programmers out there that now have an easy entry point into RIA.
@Ryan,
I am not sure what specific technologies to which you refer when you state “Microsoft have a long inglorious history of building up technologies and then throwing them in the dumpster”. I would doubt that Microsoft abandons Silverlight anytime soon. The .Net development environment rivals anything MS has ever done. It is about 10 years old and is still getting better. Microsoft is actually writing Visual Studio 2010 using .Net and WPF. I would be shocked to see MS dump .Net for another technology in the near future (a la COM). I think we can count on it through 2015 at least. Silverlight is the new .Net UI mechanism for the web. Silverlight will probably grow in size from its current 4 MB download size to provide more desktop-like functionality, and will probably be one of the ways MS delivers Office, etc, to the web.
@Joe: Actually the Flex (MXML) compiler and Flex libraries are open source, but I think development is pretty tough without the $249 Flex Builder.
Actually I stay with JavaFX.
This is something that I have been preaching for a LONG time and this is a HUGE benefit of Silverlight over Flash.
I wrote two articles on how you can improve your UI and computational processes using multiple threads. It is a very signifigant performance improvement. Keep in mind that most new decent workstations are either a dual core or quad core and 6 cores are coming this year from Intel. Parallelism will kill Flash if they don’t fix it soon.
My Silverlight UI multithreading article:
http://silverlighthack.com/post/2008/09/01/Sliverlight-Multithreading-with-Control-by-Example.aspx
My Silverlight computational processes article:
http://silverlighthack.com/post/2008/09/07/Silverlight-MultiThreading-with-a-Computational-Process-(Counting-Primes).aspx
Note: You know its sad that when you search for some things on Google that include “flash and multithreading” and my SILVERLIGHT article is comes out on the first page.
Thanks for the informative article. I chose Silverlight as the development platform for my games site http://avenscorner.com. I must say, as a c# developer I am very impressed with the potential that Silverlight has.
Thanks for this post, I just added a comment and my vote for User experience over at http://blogs.adobe.com/cgi-bin/mt-comments.cgI
I’ll read the comments on this post next …
On, http://blogs.adobe.com/aharui/2008/01/threads_in_actionscript_3.html I wrote:
“Well said donynam, it’s about performance.
Alex, it’s all about the user experience, right?
A non responsive UI really sucks for users … say . when they are trying to manipulate a large image.
My code could not be more optimized, so what’s a developer to do?
Yep, cut back the feature, make it less engaging to get the job done.
In short, sacrifice User experience to ensure the responsiveness of the app.
Not good for the User, not good for the developer and not good for Adobe.
Found your blog from: http://effectgenerator.com/blog/?p=35
“Alex responds:
As we hear more from our users about the importance of threading, the higher its priority will be.”
Please count this as a vote from me to help me stay away from Silverlight … I like Adobe much better, but I like my Users best so I’ll go with whatever platform gives them the best … experience.”"
Go with open source and you’ll never be locked out. Go with SMIL. Not happy? You can make it better?
Everything has its advantages and disadvantages. Choose what is best suited for your requirements!!
The elephant in the room, and the biggest hurdle for Silverlight, is Microsoft itself, and the fact that Silverlight is tied to the Microsoft Windows platform.
Today there are more Linux users than Mac users, and do you think Silverlight is ever going to work there? I know there is Mono’s Moonlight, but that isn’t supported by Microsoft at all. I just tested it with the examples mentioned above and some basic examples elsewhere (like a tic tac toe game) and none of them worked at all.
The java platform is coming around, adopting more of the end-user niceties of flash, like quicker loading, easier in-browser installing, etc. Not to mention it is open source and completely cross-platform.
Gosh, where to start? There is a lot of ideology flying around this thread.
At Noteflight, I’d like to think we’re not particularly ideological. Silverlight and Flash/Flex are both great platforms, but one of them presently has a 95+% installed base on consumer machines. Guess which one? Our decision to use Flash was (as it should be for anyone wishing for mass adoption of their app) primarily driven by the fact that it is already installed on our user’s machines, not by its development merits. Even if Flex sucked (which it patently does not) we would still have just gritted our teeth and coded up Noteflight in AS3. And if Silverlight takes over (which I’m not betting on, but have no say in) well, we’ll just have to port our app to Silverlight, won’t we? We are building an app for the real world out there, not for ourselves.
As for WAV files: the lack of threading played absolutely no role. What turned out to be more important in the end was that MP3 is a lossy compression algorithm, and we need a lossless approach in order to download and accurately loop sample fragments whose decompressed signal must be absolutely identical to the original. I note that Flash 10 does in fact include a native MP3 decoding API.
All this aside, as a programmer I would of course prefer to have access to a thread construct in AS3 and would find threads immensely convenient for many aspects of Noteflight. I’m simply saying that as a platform issue it is really, really far down the list of things that matter.
One last point about Noteflight - the author’s post says that our site will deliver “mass, systematic theft of music, which has, somehow, to be properly monetized”. I would encourage everyone to look and see what is actually going on on our site, and what is not. Noteflight’s aim is to foster a global musical community, particularly in service of education. Our business plan is built around people teaching and learning music, not around a site that attracts millions of people who don’t want to pay for copyrighted songs.
One more technical point that I didn’t really clarify: in general, the work involved in breaking up an audio synthesis or decoding/encoding operation has not been especially difficult for us (although threading would have simplified it greatly). When you play back a piece in Noteflight, audio synthesis takes place in short 50 ms chunks, in between which all pending UI events are handled and the app remains completely responsive. Is this fun to program? Not really. Is this hard to program? Not really. I would say that if you take a worker thread and structure it so that it can report on its progress and produce consumable output along the way, you have already done a very good fraction of the work required to avoid the use of threads at all.
It is only a matter of time before you can make async calls to a local JVM from a running Flex/Air application.
Then multi-threading will be super easy from Flex. And cross platform. And you will have access to the entire Java library.
Artemis showed us the way.
As for Silverlight… I live in Redmond.
I see more open positions for Flash developers at Microsoft then I see for Silverlight developers every time I look.
MS will probably abandon it when it fails, just like every internet “technology” they have ever attempted.
But besides that I find the fact that flash doesnt thread pretty sad. especially when I can write an app for the iphone that will thread perfectly fine.
Perhaps the push to replace flash isnt about flash, but about the fact that we have moved on to a point where noone cares what its called.
Still, when prompted to download silverlight on Vista I steadfastly refuse out of a misplaced sense of pride.
I admit, you are right, but so was the post about the html rendering in flex not being optimal and lacking things like tinymce. Because of this reason flex wouldn’t be suitable for enterprise applications.
This post is a simular, a good point but only for 10-15% of the applications. It’s a big enough percentage to care, but doesn’t really affect the needs of most applications.
Flex has been getting a lot of exposure and I think it is time for adobe to step up now while they still can, to do all the necessary “base” work like threading because this could impact many of the applications and then go to the nice but really not needed stuff like directly exporting UI designs from photoshop, skins, etc.
This is incorrect, AFAIK. I’m not an expert on this, but I think:
- You could convert the wav to mp3 in the background by putting it in an interval timer (and letting it execute only once, of course).
- Possibly you could also let this happen server-side with AJAX, if the data needed to create the mp3 (e.g. a wav file) is already stored servver-side.
Or is there anything I’m missing here which makes this impossible?
bluestix your comment it’s very funny, sorry for that , but..If flex would be able to access java libraries…then why we need to use Flex? then it’s better JavaFX, don’t think the same?…but until JavaFX change his way to do things and take out that horrible script …many years had gonne.
for now Silverlight is ok.
@Kris,
It is like you say Adobe has to step up “while they still can”.
Forget it when Microsoft starts integrating Silverlight into every product horizontal and vertical. You are already seeing that with: Windows Azure, Virtual Map SDK, SharePoint, IIS Smooth Streaming etc. Adobe simply doesn’t have anything like that…their dominance right now is: market share and UI…once Silverlight becomes respectable in those areas…game over.