As the web begins to shift from the more traditional HTML interfaces most common today to using rich Internet application technologies such as Adobe Flash and Ajax, developers need to be aware of how the mechanics of these technologies impact the way dynamic interfaces should be tested. For example, one important distinction about Flash-based interfaces is that they change their appearance dynamically and often process user input locally in real time—a very different model from the server-centric "page" metaphor that traditional HTML interfaces utilize, which requires a trip back to the server for almost every user interaction. Another important difference is that Flash content is rendered by a common engine across all platforms and browsers. By contrast, the variety of web browsers render HTML content slightly differently across platforms.
Despite these differences, many people assume that testing a Flash-based interface is simply a matter of applying the same techniques and success criteria they are accustomed to doing for HTML interfaces. This often results in an interface that works exactly as expected during testing but exhibits buggy behavior when accessed by real-world users.
In this article, I present four practices that you should follow when testing a Flash-based interface and describe the issues that these practices are designed to expose. If you follow these practices faithfully, you will ensure solid, consistent performance of your Flash-based interfaces and better user experiences for your visitors.
STEP 1: TEST OVER MULTIPLE CONNECTION SPEEDS
This step exposes the following:
- Download performance problems
- Streaming animation glitches
- Code logic flaws, including:
- Accessing dynamic data before it loads
- Invoking methods on objects before they are initialized
- Incorrect assumptions about a fixed execution order for asynchronous operations
- Jumping to Timeline frames before they have loaded
Testing over multiple connection speeds is the most important practice to follow when testing Flash interfaces. It's also the one that traditional HTML developers are likely to dismiss. Testing HTML interfaces in this manner accomplishes primarily one purpose: confirming that download performance is acceptable for all users. With Flash, testing over various connection speeds reveals much more than just download performance problems.
If you test only from your local hard drive—or a web server on your local network—you are in danger of masking problems that might occur only under real-world connection conditions. When you test locally, things like server communication, image loading, and SWF streaming happen almost instantaneously. Under these conditions, it is easy to inadvertently write code that relies on this instant response time without realizing it.
An example helps illustrate this concept. If you use the statement
gotoAndPlay(20) on the first frame of your movie's timeline, it will most likely work as expected when testing locally or even over broadband. However, if you test the same file over a modem connection, your movie will probably behave incorrectly. This is because the streaming nature of Flash content allows the code on frame 1 to run before the later frames of your timeline have finished loading. An attempt to send the playhead to a frame that hasn't loaded will fail.One very helpful way around this problem is to make frequent use of the built-in Simulate Download feature in Flash, which you can find in the View menu of the SWF window that appears after choosing Control > Test Movie (see Figure 1). Because the Simulate Download feature allows you to preview the performance of your movie over various connection speeds, it is a good way to spot speed-related problems early.
But beware. The Simulate Download feature isn't perfect; you still need to test your movie over real connections of various speed. The most important thing to remember is that bugs related to connection speed are far easier to address if you find them early. The mantra "Test early, test often" is particularly relevant to this practice.
STEP 2: TEST ACROSS BROWSERS
This step exposes the following:
- Problems with embedding Flash Player
- Problems with relative URLs
- Problems with Flash-to-browser communication
Although cross-browser testing is essential with traditional HTML, the reasons for doing it with Flash content are very different. Cross-browser testing with HTML reveals page-display and client-side scripting problems. Because Flash Player handles both of these aspects for Flash-based interfaces, the browser has no influence over them.
The browser does, however, influence the way Flash content is embedded in the page. Different browsers require different HTML markup code in order to load and display Flash files. Additionally, the way Flash Player resolves relative URLs when attempting to load external assets like images and XML data—or when jumping to new web locations using
getURL()—varies from browser to browser.Testing across various browsers confirms that Flash content has been embedded properly on the page and that all URLs embedded in the Flash file are working consistently across all browsers. Of course, if your Flash interface communicates with the browser using JavaScript, cross-browser testing is also necessary to expose any differences between the way various browsers handle this form of communication.
Note: For tips avoiding relative URL problems, refer to the following Flash TechNote: Relative URLs are not referenced correctly in a Flash movie.
STEP 3: TEST ACROSS FLASH PLAYER VERSIONS
This step exposes the following:
- Publishing mistakes
- Flash detection problems
Because Flash Player handles nearly all aspects of the display and functionality of a Flash-based interface, you will need to test your interface in various versions of Flash Player. It is usually adequate to test in just two versions: the version of the player which you've defined as your minimum required version and the version of the player that preceded the required version.
For example, if your interface requires Flash Player 6, then you should test in the earliest release of that version (Flash Player 6r21) and in any release of the previous version (Flash Player 5r30). If your interface requires a specific later release of a particular version—for example, Flash Player 6r65—then you should still test in the versions mentioned previously as well as the required release.
In our example this would mean testing in Flash Player versions 5r30, 6r21, and 6r65. By testing across Flash Player versions, you will be able to expose any publishing mistakes (such as publishing for Flash 7 when you meant to publish for Flash 6), as well as any problems with any Flash Player detection functionality you may be using.
Note: You can find an archive of old player versions in the following Flash TechNote: Archived Flash players available for testing purposes.
STEP 4: TEST ON "WEAK" MACHINES
This step exposes the following:
- Issues with playback performance
- "This script appears to be running slowly" alerts
Displaying traditional HTML pages takes very little computer power. As a result, web developers have hardly ever had to consider how the power of the end user's computer influences the quality of the user's web experience. This is not the case with Flash content, which behaves much more like a CD-ROM or video game. The user's hardware—processor, RAM, and video card—impacts the experience significantly.
Much like CD-ROMs and video games, Flash-based interfaces should be designed with a specific minimum hardware requirement in mind. As you design and build your interface, repeatedly check the performance of your work on a machine that matches that minimum specification to ensure that you are not overtaxing the hardware with your graphics, animation, or coding choices.
Source: http://www.adobe.com/devnet/flash/articles/flash_interface_testing.html
