TweetFollow Us on Twitter

Using Virtual User
Volume Number:12
Issue Number:9
Column Tag:Quality Assurance

Software Testing With Virtual User

A cost-effective proving ground for software

By Jeremy Vineyard

Introduction

It is continually frustrating to have new applications crash soon after being installed. Such products give the impression that insufficient work was done before shipping to ensure their quality.

In software companies today, it is standard procedure to test a product to ensure its quality and reliablity before shipping. Quality Assurance (QA) is a common name for such testing. QA is becoming increasingly important as applications take on ever-increasing size and complexity. There can be many thousands of variations in the ways a user might interact with a software product, and a QA tester must check these to verify that everything works correctly. So QA testers need the tools to get their job done effectively.

There are numerous ways to improve the software testing process, but one of the most effective has been the automated testing tool, whereby QA testers can set up specific tests and suites of tests to be run by the computer. This frees them from many hours of drudgery, thus saving both time and money by allowing them to refocus their energies on tasks that need more human interaction, spreading the range of the testing eventually completed. This also leads to more reliability in the software.

You don’t have to have a dedicated QA department to be able to test your products. Even if you are a small developer, it is a good idea always to verify the quality of your products before they are shipped, and automated tools can help. One of the most popular automated testing tools for the Mac is Virtual User.

What is Virtual User?

Virtual User (VU) is an automated testing tool that allows a computer to emulate a human user, performing actions such as clicking the mouse and typing keys. This computer acts as a host, and controls other computers just as a person would. One or more target computers act as agents receiving instructions from the host.

Figure 1. Virtual User

The VU environment consists of an application that compiles and run scripts. VU scripts must be edited separately in a text editor, such as MPW or BBEdit.

VU links computers together over an AppleTalk network. The minimum setup for a VU testing system is the VU software package and two computers (one host and one target).

VU is developed by Apple Computer, Inc., and is available for under $100 through most Macintosh developer catalogs. [It comes for free if you’re already subscribing to the ETO CD. Apple lists the price as $150 on their Web site, at http://dev.info.apple.com/TPC/VU.Datasheet.html, but says $79 in the printed APDA catalogue. Go figure. - man]

Limitations Of Virtual User

There are many things that an automated tool cannot do, and it is important to realize these limitations before planning your automated test suites. An automated tool cannot tell when something “looks right” on the screen. VU can’t tell you when an icon or window is pretty or ugly or misaligned.

VU is fairly unintelligent. You can tell it to “Move window ‘My Window’ to (x,y)”, but you can’t tell it to “Open icon for hard drive in Finder.” Also, if the target machine crashes, VU doesn’t know about it and will keep trying to run the script. The script can be paused and the machine restarted to allow the script to resume. Proper debugging techniques are essential for determining the events that led up to the crash. Some of these techniques will be explained later in the article.

Advantages of Virtual User

VU is most effective at highly repetitive tests that may have many variations. One example might be to select every control in every dialog in the application, making sure that each click produces the appropriate dialog, action, etc. This would be a very tedious task for a human to accomplish, but the computer doesn’t care about tediousness, making it the ideal tester for the project.

Another use of VU is to write a test that can be run after every internal build of a product (development, alpha, beta, final candidate) to verify that nothing was broken by the code changes made to the previous version. VU can also be used to set up automatic bug verification, by acting out the steps necessary to reproduce a bug. With this capability, the automatic bug verification can be run on every new release of a product to ensure that the bug is fixed and that it doesn’t sneak back into the code.

Understanding Virtual User Scripts

Each automated script has an entry point specified by the script statement in VU:

script TestControlsAndWindows()
begin
    # Do automated tests here.
end;

From then on, the VU script is much like the actions of a person. You tell it to click on windows or buttons, move the mouse, type keys, and more. Here is an example of a script that will test some menu items:

script TestMenuItems()
begin
    # Select the menu item “Show Script Window” from the “Windows” menu.
 select [menuItem title:"Show Script Window" 
 menu:[menu title:"Windows"]];

    # Wait 10 seconds for the window to appear.
 wait(10);

    # Verify that the “Script Window” window appears.
 if match [window ordinality:1 title:"Script Window"]
 println("The window 'Script Window' 
 opened correctly.");
 else
 println("ERROR! The window 'Script Window'
  did not open.");
end;

In addition to scripts, VU supports functions that act as extensions to the script. These are called tasks. A task is a procedure with a list of parameters and an optional return value.

task DoTheRightThing(var1 := "default value", var2)
begin
    # Do the right thing here.
 
    # Return a value (optional).
 return "result";
end;

VU collects information about the target computer’s environment using (as we have already seen) the match statement. The match statement looks for a specific environment element, using descriptor traits such as the element’s title or ordinality.

VU can “see” any menu or window and any control that is implemented with the Control Manager and stored in the window’s list of controls. However, because the List Manager doesn’t provide a standard API for accessing the contents of a list, VU cannot see a list or the items inside it. VU can also see dialog items such as user items, static text, edit text, icons, and pictures.

# Find the menu called “Testing”.
theMenu := match [menu title:"Testing"];

# Find the frontmost window.
theWindow := match [window ordinality:1];

The collect statement is similar to match in that it collects all the elements of a certain type into a list.

# Keep a list of all the open windows.
windowList := collect [window];

VU then interacts with the environment, using such keywords as select, drag, close, type, and click. VU accesses common Macintosh objects such as windows, buttons, scroll bars, and menus.

# Select the menu item.
select [menuItem title:"Show Clipboard" 
 menu:[menu title:"Edit"]];

# Move the window.
drag [window title:"Clipboard"] relative:{30, 30};

# Type characters into the window.
type keystrokes:{"This is some text"};

# Select a button.
select [button title:"OK"];

# Close the window.
close [window ordinality:1 title:"Clipboard"]!;

Tip: If possible, implement your application windows as modeless dialogs. VU recognizes the user item element in a dialog, allowing user items to indicate to VU where non-standard user interface elements are located.

Debugging Virtual User Scripts

VU provides a log file feature, by which information from within the script can be written out, providing the scripter with information about the current run-time state of the script. One of the most effective ways to debug an automated script is to log with the println statement anywhere anything important happens or changes. (The {} syntax substitutes the actual values of the variables for the variable names in the string.)

task MyTask(var1, var2, var3)
begin
 println("MyTask({var1}, {var2}, {var3})");

 println("Starting batch processing.");
 StartBatchProcessing();
end;

Tip: Because there is no type checking in VU, it is a good idea to log the parameters that are input into every task to make sure that the correct values were passed.

Extending Virtual User with Libraries

One of the most useful features of VU is the ability to split commonly used tasks into reusable files called libraries. To use all of the tasks in a library, the Libraries statement is used as shown:

# This statement makes all the tasks in the file called “Special Tools.vulib” 
# accessible to this script.
Libraries "Special Tools.vulib";

Libraries are commonly used to group tasks with common actions into smaller and more manageable files, such as Finder Tools.vulib, My App Tools.vulib, etc.

Tip: Because large projects may have dozens of library files, it is a good idea to establish a naming convention for both the names of the library files and the names of the tasks within the files. One might, for instance, append “Tools.vulib” to the name of every library file, and use a unique two-letter prefix for every task in the library.

# This task is in the library “Finder Tools.vulib”.
task FT_EjectDisk()
begin
    # Eject the disk with a command-key combination.
 pressKey keystrokes:{commandKey};
 type keystrokes:{'E'};
 releaseKey keystrokes:{commandKey};
end;

Extending Virtual User with Globals

Another useful feature of VU is support for global variables. To declare a global variable, simply place the global keyword before the variable name when you define it.

# Once this variable is defined, it can be accessed from anywhere.
global gMyGlobal := 1000;

To access a previously defined global variable, again attach the global keyword before using the variable.

Here is an example of how using global variables can save considerable time and effort. This script:

task MA_DoThis(appVersion := '1.0', var1, var2, var3)
begin
end;

task MA_DoThat(appVersion := '1.0', var1, var2)
begin
end;

script MyScript()
begin
 MA_DoThis('1.0', 10, 20, 30);
 MA_DoThat('1.0', "test", "do");
end;

can be simplified to this:

task MA_DoThis(var1, var2, var3)
begin
    # Now the variable ‘gAppVersion’ is using the global value.
 global gAppVersion;
end;

task MA_DoThat(var1, var2)
begin
    # Now the variable ‘gAppVersion’ is using the global value.
 global gAppVersion;
end;

script MyScript()
begin
 global gAppVersion := '1.0';

 MA_DoThis(10, 20, 30);
 MA_DoThat("test", "do");
end;

The advantages may seem small in this simple example, but once you start writing scripts with hundreds or even thousands of separate tasks being called, it can be difficult to pass variables several levels deep through the chain. Using globals allows variables to be accessed from anywhere within the script.

Tip: For applications whose user interface is changing rapidly, insteading of searching all of your scripts and libraries every time the text for a menu item, window, or button is changed, use a global variable that is declared at the beginning of every script, to hold the current state of the interface item.

# This task must be called before the globals can be used.
task MA_DeclareGlobals()
begin
 global toolsMenu := [menu title:"Tools"];

    # If the name of the paint tool menu item ever changes, we have only to
    # change it in one place, and all other scripts will be using the correct
    # value. This keeps us from having to replace the string everywhere it
    # occurs in all of the scripts.
 global paintToolMenuItem := 
 [menuItem title:"Painter" menu:toolsMenu];
end;

task MA_SwitchToPaintTool()
begin
 select global paintToolMenuItem;
end;

Extending Virtual User with External Tools

If you find that you have reached the limitations of the VU language, it is possible to write an external tool for VU. An external tool is an application that communicates with VU by sending and receiving Apple events. Because the external tool can be written in a more powerful language such as C/C++ or Pascal, the VU language can be extended. Templates and examples for creating external tools come with VU, eliminating much of the work.

Additional Features of Virtual User

VU has many of the features of a modern programming language, including if-else statements, for/while loops, list- and string-processing operators, and more.

VU has a built-in regular expression matching system that allows you to write scripts that match environment elements based on regular expressions, rather than simple strings. This is useful for multiple interface elements that may have similar names, but are not exactly the same. “Untitled-1”, “Untitled-2”, etc., can be represented by the regular expression “/Untitled- /”.

When running against multiple target machines, agents can communicate with each other by sending and receiving messages. This allows synchronization of events for software that can be run on multiple systems at the same time. This is useful for network software products that must be installed on more than one machine to communicate with one another.

The best place to learn about the features of VU is to look in the VU Language Reference manual. It is well written, and thoroughly describes the capabilities of the VU language.

Summary

Automated testing tools will never be able to entirely replace human testers in QA, nor should they. Most of what QA does is about discerning how a user might interact with a product, and a computer cannot always predict these actions or define a good way to test them. However, used properly, automated tools can greatly increase the productivity of QA, saving labor costs, increasing consistency, and shortening time to market. In today’s highly competitive environment, automated tools can provide you with the edge you need to succeed.

[There is a great deal of VU-related material on the Tool Chest editions of Apple’s Developer CD. For instance, in the May CD, which was the current Tool Chest CD as this issue went to press, the Testing & Debugging folder inside the Subject Index folder held aliases to various folders containing VU material, including a tutorial and a host of tools that extend VU’s abilities in valuable ways (note, in particular, Ivy, which allows VU to “capture and compare screen images”). - man]

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

NetNewsWire 6.1.1 - RSS and Atom news re...
NetNewsWire is the best way to keep up with the sites and authors you read most regularly. Let NetNewsWire pull down the latest articles, and read them in a distraction-free and Mac-like way. Native... Read more
ScreenFlow 10.0.9 - Create screen record...
ScreenFlow is powerful, easy-to-use screencasting software for the Mac. With ScreenFlow you can record the contents of your entire monitor while also capturing your video camera, microphone and your... Read more
OnyX 4.3.8 - Maintenance and optimizatio...
OnyX is a multifunction utility that you can use to verify the startup disk and the structure of its system files, to run miscellaneous maintenance and cleaning tasks, to configure parameters in the... Read more
MacFamilyTree 10.2 - Create and explore...
MacFamilyTree gives genealogy a facelift: modern, interactive, convenient and fast. Explore your family tree and your family history in a way generations of chroniclers before you would have loved.... Read more
Viber 19.7.0 - Send messages and make fr...
Viber lets you send free messages and make free calls to other Viber users, on any device and network, in any country! Viber syncs your contacts, messages and call history with your mobile device, so... Read more
HoudahSpot 6.3 - Advanced file-search to...
HoudahSpot is a versatile desktop search tool. Use HoudahSpot to locate hard-to-find files and keep frequently used files within reach. HoudahSpot is a productivity tool. It is the hub where all the... Read more
Transmit 5.9.2 - Excellent FTP/SFTP clie...
Transmit is an excellent FTP (file transfer protocol), SFTP, S3 (Amazon.com file hosting) and iDisk/WebDAV client that allows you to upload, download, and delete files over the internet. With the... Read more
TeamViewer 15.40.8 - Establish remote co...
TeamViewer gives you remote control of any computer or Mac over the Internet within seconds, or can be used for online meetings. Find out why more than 200 million users trust TeamViewer! Free for... Read more
ffWorks 3.3.5 - A Comprehensive Video Co...
ffWorks, focused on simplicity, brings a fresh approach to the use of FFmpeg, allowing you to create ultra-high-quality movies without the need to write a single line of code on the command-line.... Read more
Arq 7.19.11 - Online backup to Google Dr...
Arq is super-easy online backup for Mac and Windows computers. Back up to your own cloud account (Amazon Cloud Drive, Google Drive, Dropbox, OneDrive, Google Cloud Storage, any S3-compatible server... Read more

Latest Forum Discussions

See All

Out Now: ‘Brotato’, ‘Slime Labs 3’, ‘Ter...
Each and every day new mobile games are hitting the App Store, and so each week we put together a big old list of all the best new releases of the past seven days. Back in the day the App Store would showcase the same games for a week, and then... | Read more »
SwitchArcade Round-Up: Reviews Featuring...
Hello gentle readers, and welcome to the SwitchArcade Round-Up for March 29th, 2023. In today’s article, we briefly go over that fancy Tears of the Kingdom preview from yesterday then head right into a review of the action bop Kraino Origins. After... | Read more »
‘Terra Nil’ Review – A Netflix Games Ess...
When Terra Nil (Free) from Devolver Digital and Free Lives was revealed, the striking aesthetic and premise had my attention. Devolver is known to publish interesting games, even if I don’t enjoy every release from them, but Terra Nil looked like... | Read more »
A Look Back at the ‘Final Fantasy’ Pixel...
Ooh, he said the thing. No, my dearest of long-time readers, the RPG Reload isn’t making a regular comeback. But with Square Enix’s Final Fantasy Pixel Remaster series about to make the hop to consoles sometime in the next month or two, I thought it... | Read more »
Smilehate and VA Games announce upcoming...
It is exciting times for mobile RPG fans, as Smilegate and VA Games have unveiled the brand page and first look at its upcoming game Outerplane. With a tentative global launch at the end of May, we can get our first look at the characters and... | Read more »
‘Skullgirls Mobile’ Major Update 5.3 Out...
Following the December version 5.2 update, developer Hidden Variable pushed out a major update for Skullgirls Mobile (Free) a few hours ago. Skullgirls Mobile 5.3 brings in Black Dahlia’s full release, XP boosts, fighter tuning, free gifts, and a... | Read more »
Classic Sports Game ‘Baseball Stars Prof...
Following last week’s ACA NeoGeo mobile release of Stakes Winner, Hamster and SNK have released the classic sports game Baseball Stars Professional on iOS and Android worldwide. Baseball Stars Professional debuted in 1990, and the classic sports... | Read more »
“Age of Falling Towers” Major Update Arr...
Well we’re about 9 months out from the launch of Diablo Immortal in June of last year, and at the time of its launch I was pretty heavily into the game for the first month or so. Then I just sort of churned out and haven’t really been keeping tabs... | Read more »
SwitchArcade Round-Up: Reviews Featuring...
Hello gentle readers, and welcome to the SwitchArcade Round-Up for March 28th, 2023. In today’s article, we’ve got full reviews of both MLB The Show 23 and Atelier Ryza 3: Alchemist of the End and the Secret Key. After that, we’ve got a handful of... | Read more »
Minecraft Dungeons & Dragons Collabo...
During the Dungeons & Dragons Direct, new Minecraft ($6.99) collaboration was revealed. If you’ve not kept up with Minecraft recently, the upcoming 1.20 is now called the Trails and Tales update and it is set to arrive later this year with a... | Read more »

Price Scanner via MacPrices.net

New low price: Apple AirPods Pro for $194, sa...
Verizon has Apple AirPods Pro on sale for $194.99 on their online store for a limited time. Their price is $55 (22%) off Apple’s MSRP, and it’s the lowest price currently available for AirPods Pro.... Read more
Open-box 13-inch M2 MacBook Pros available fo...
QuickShip Electronics has open-box return 13″ M2 MacBook Pros in stock and on sale for $300-$350 off MSRP on their eBay store right now, each with free express delivery. According to QuickShip, “The... Read more
Take $100 off the price of an iPad with Apple...
Apple will take $100 off 12″ M2 iPad Pros, $50-$100 off 11″ M2 iPad Pros, $50 off iPad Airs, $50 off 8.3″ iPad minis, & $20-$40 off 10″ iPads for all teachers, students, and staff of any... Read more
Deal Alert! Apple Studio Display with Nano Gl...
Amazon has the Apple Studio Display with Nano-Texture Glass (Tilt-Adjustable Stand) on sale for $400 (21%) off MSRP for a limited time. Shipping is free: – Studio Display (Nano glass): $1499 $400 off... Read more
Clearance 2020 13″ M1 MacBook Pros available...
Apple has clearance, previous-generation, 13″ M1 MacBook Pros available in their Certified Refurbished section for $1059. These are the cheapest 13″ MacBook Pros for sale today at Apple, and all... Read more
Amazon continues to offer $799 13-inch M1 Mac...
Amazon has Apple 13″ M1 MacBook Airs on sale for $200 off MSRP, only $799.99. Their prices are the lowest available for new MacBooks today among the retailers we track. Stock may come and go, so... Read more
Find the lowest prices on Apple iPads using o...
Our Apple award-winning iPad Price Trackers are the best place to find the latest information on iPad sales and deals. Current sales, as of this post, range up to $200 off MSRP depending on the model... Read more
Apple’s Reality Pro VR headset one step close...
Mark Gurman, in this weeks’s Power On newsletter, stated that last week, Apple held an important assembly of its highest ranking executives at the Steve Jobs Theater in Cupertino. The gathering,... Read more
Apple 16-inch M2 Pro MacBook Pros on sale for...
The first major sales on Apple’s 16-inch M2 Pro MacBook Pros arrived this month. B&H Photo has Space Gray 16″ M2 Pro MacBook Pros in stock and on sale today for $200 off Apple’s MSRP, starting at... Read more
Apple 14-inch M2 Pro MacBook Pros on sale for...
B&H Photo has Apple 14″ M2 Pro MacBook Pros in stock today and on sale for $100-$200 off MSRP, each including free 1-2 day delivery to most US addresses. Their prices are the among the lowest... Read more

Jobs Board

MacOS X / *Apple* Support Engineer - Royal...
MacOS X / Apple Desktop Support Engineer, on-site in New York, NY The Desktop Support Group is looking for an endpoint engineer with a focus on supporting MacOS and Read more
Wireless Device Portfolio Manager - *Apple*...
…in our Retail Wireless journey. The successful Device Portfolio Manager - Apple will work cross-functionally to develop, oversee and execute a device roadmap Read more
Omnichannel Associate - *Apple* Blossom Mal...
Omnichannel Associate - Apple Blossom Mall Location:Winchester, VA, United States (https://jobs.jcp.com/jobs/location/191170/winchester-va-united-states) - Apple Read more
Operations Associate - *Apple* Blossom Mall...
Operations Associate - Apple Blossom Mall Location:Winchester, VA, United States (https://jobs.jcp.com/jobs/location/191170/winchester-va-united-states) - Apple Read more
Cashier - *Apple* Blossom Mall - JCPenney (...
Cashier - Apple Blossom Mall Location:Winchester, VA, United States (https://jobs.jcp.com/jobs/location/191170/winchester-va-united-states) - Apple Blossom Mall Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.