Volume Number: 24 (2008)
Issue Number: 04
Column Tag: AppleScript

Test-Driven Development Using AppleScript

Using testing frameworks to create more robust AppleScript applications

by Andy Sylvester

AppleScript can be used to automate many tasks on the Mac. However, compared to other scripting languages such as Perl, Python, and Ruby, it can seem somewhat simple and not as applicable for writing larger applications. These other languages also have testing frameworks that can be used for building and testing applications, and have good support for object-oriented programming. A testing framework for AppleScript called ASUnit has been developed, which provides a way to test AppleScript functions. Although AppleScript does not natively support OOP in the common way of Perl, Python, and Ruby, you can structure scripts to provide much of the same functionality. This article will introduce the concepts of test-driven development and demonstrate the use of the ASUnit testing framework.

One of the significant changes in software development techniques in the past ten years has been a technique called "test-driven development". When using this development technique, the software developer writes a small amount of source code to test a function or feature in the application being developed. However, the software developer writes the test code before writing the actual application code. A summary of the technique is as follows:

Write some test code

Run the test and see that it fails

Write the source code that implements the feature

for the test

Run the test again and see that it passes

Refactor or clean up source code

By writing test code before writing application code, the software developer creates a suite of tests that can be used at any time to check the functionality of the application. In addition, if changes are made to the application, the test suite can be run to see if the existing functions have been affected by the changes. To support test-driven development, many testing frameworks have been developed to assist software developers in creating, running, and managing tests. Kent Beck's work on testing frameworks for Smalltalk (http://www.xprogramming.com/testfram.htm) led to the development of the Junit testing framework for the Java language (http://www.junit.org). Since the appearance of Junit, testing frameworks have been developed for all programming and scripting languages, and even for web-based application development (see http://www.xprogramming.com/software.htm for a list of available frameworks). The second part of this article will introduce a testing framework for AppleScript called ASUnit. Before learning about this framework and how to use it, let us first look at some examples to see how writing tests can help with application development.

One way to add tests to an application is to use dialog boxes to give information on if a test passes or fails. A simple test to check a math operation could be written as follows:

display dialog "Test #1"
if 1 + 1 is equal to 2 then
   display dialog "Test 1 passes!"
else
   display dialog "Test 1 fails!"
end if

For this example, a set of dialog boxes would be presented, first announcing the title of the test, then the test result. However, this could get awkward quickly, having to click on buttons to allow the program to run. Also, it is difficult to perform test-driven development, since the test code is intertwined with the application logic. Another approach is to create functions that contain application logic and then run the functions with test data. This would make it easier to test individual features without affecting other functions. Following the test-driven development philosophy, we will write a test function for application logic that checks for a specific input string. The following code demonstrates this technique:

script myNameGame
   CheckForMyName("Andy")
end script
run myNameGame

When this script is run in the Script Editor, a dialog box appears with the error message "«script myNameGame» doesn't understand the CheckForMyName message". This is to be expected, since the function CheckForMyName does not exist. So far, we are following the checklist of steps given above. Next, we add application logic for the CheckForMyName function:

on CheckForMyName(testGuess)
   if testGuess is equal to "Andy" then
      return "Your guess is correct!"
   else
      return "Nope! Try again!"
   end if
end CheckForMyName
script myNameGame
   CheckForMyName("Andy")
end script
run myNameGame

The script myNameGame calls the function CheckForMyName with an input string. The results from the test appear in the Results area of the Script Editor window. When this script is run, the expected result appears (Your guess is correct!). We have completed all but the last step of the our test-driven process (refactor and clean up). Since this logic is pretty simple, we will move on to adding some new features. Once again, we start with adding a test. The new feature to be added is that the function should check for the string "Andy" or "Bill". We can modify the myNameGame test to use "Bill" as the test string instead of "Andy". When we run the script, we get the failure result (Nope! Try again!). We can now add logic to check for both strings. Our function now looks like this:

on CheckForMyName(testGuess)
   if (testGuess is equal to "Andy") or ¬
      (testGuess is equal to "Andy")  then
      return "Your guess is correct!"
   else
      return "Nope! Try again!"
   end if
end CheckForMyName
script myNameGame
   CheckForMyName("Bill")
end script
run myNameGame

When this script is run, the successful result appears again (Your guess is correct!). Now we can keep adding feature after feature using this test-driven development technique and make sure that the application logic is working correctly.

As more functions are developed, though, this simple test structure could be a burden to maintain. If each function to be tested had to be in its own file, this could result in many intermediate files for development. Also, the above structure is set up to only run one test at a time on a function. If running multiple tests were desired, the test script would have to be edited each time. Now that we have demonstrated the basics of test-driven development, let us look at a testing framework that can address the maintenance aspects of developing and running multiple tests on multiple functions.

ASUnit is contained in a single AppleScript file. To install the program, download the latest version from the ASUnit website (http://nirs.freeshell.org/asunit/), unzip the file, and copy the file ASUnit.scpt to the Scripts folder in your Library folder or the Library/Scripts folder on the startup disk for your Mac. The remaining files are the README file for the application and a web page containing the ASUnit documentation (this page can also be found on the ASUnit website).

When creating an ASUnit test script, you must include a path to the location for ASUnit.scpt. Here is an example:

property parent : load script file¬
   (("Sylvester HD:Library:Scripts:") & "ASUnit.scpt")

Note that the path to the file needs to use the POSIX form with colons.

Next, we need to create a test fixture, or a framework, that will contain the tests that we wish to write. In ASUnit, this test fixture is an AppleScript which will contain other scripts within the main script. We can use the following structure:

script |accessing list|
    property parent : registerFixture(me)
    property empty : missing value
    property notEmpty : missing value

At the beginning of this script, the registerFixture script object is called. This script object is included in ASUnit.scpt. The accessing list script creates a property using the parent reference (which was included in the script file as the first line) to be able to access the registerFixture script object. Using the me keyword tells registerFixture that the accessing list script object is the fixture.

The last two lines define properties for objects that will be used in the test scripts that follow. For this example, two lists will be used (one that has no elements, and one that has at least one element).

When running ASUnit test scripts, there is a setUp script object which can be overridden with a local version to perform setup of objects for each test. This script will be called for each test script within the test fixture. For our example, we will create the two lists as follows:

on setUp()
    set empty to {}
    set notEmpty to {"foo", 1}
end

To perform a test, add another script which contains logic to perform some operation on the test objects. In this example, the objects we are manipulating are the lists empty and notEmpty.

script |add item|
    property parent : registerTestCase(me)
    set end of empty to "bar"
end

At the beginning of this script, the registerTestCase script object is called. This script object is also included in ASUnit.scpt. The add item script creates a property using the parent reference (which was included in the script file as the first line) to be able to access the registerTestCase script object. Using the me keyword tells registerTestCase that the add item script object is the test case to be registered. Next, the word "bar" is added to the end of the list empty, so that there is now an element within that list.

ASUnit provides a method, should, to check for a positive result for a condition. The method takes an AppleScript expression as the first argument, and an error message as the second argument. If the result of the expression is false, the error message will be displayed, otherwise the text "ok" will be displayed for that test. Add the following line to the add item script:

should(empty contains "bar", "no bar?!")

Since the previous line added the word "bar" to the list, this test should pass.

The final addition to this script will be logic to display the test results. ASUnit provides a script object called makeTestSuite to collect a number of test scripts into a single suite to be run. For this example, we only have one script. However, this function can be used with another ASUnit script object called makeTextTestRunner which will run all of the tests in a suite and create a window with the output of the tests. After adding logic to call these functions, our first ASUnit test script will be as follows in Listing 1:

property parent : load script file ¬
   (("Sylvester HD:Library:Scripts:") & "ASUnit.scpt")
property suite : makeTestSuite("My Tests")
script |accessing list|
   property parent : registerFixture(me)
   
   property empty : missing value
   property notEmpty : missing value
   
   on setUp()
      set empty to {}
      set notEmpty to {"foo", 1}
   end setUp
   
   script |add item|
      property parent : registerTestCase(me)
      set end of empty to "bar"
      should(empty contains "bar", "no bar?!")
   end script
end script
run makeTextTestRunner(suite)

After typing the above AppleScript code in a Script Editor window, click the Run button to execute the tests. You should see a new window open and display the following text:

My Tests
accessing list - add item ... ok
Ran 1 tests in 0 seconds.  passed: 1  skips: 0  errors: 0  failures: 0
OK

The one test (add item) passed - hooray! But what if a different word than "bar" had been added to the list empty? If we change "bar" to "bat" in the line adding the text to the list, we get the following test results:

My Tests
accessing list - add item ... FAIL
FAILURES
———————————————————————————————————
test: accessing list - add item
message: no bar?!
———————————————————————————————————
Ran 1 tests in 0 seconds.  passed: 0  skips: 0  errors: 0  failures: 1
FAILED

Now you know what a test failure looks like. Once you have looked at the test results, you can click on the red Close button to close the window, then click on the "Don't Save" button to complete closing the window.

Now that a test fixture has been created, you can add more tests. Also, within a test, you can have multiple "should" statements to perform more than one check. Listing 2 shows an addition to the previous script:

property parent : load script file ¬
   (("Sylvester HD:Library:Scripts:") & "ASUnit.scpt")
property suite : makeTestSuite("My Tests")
script |accessing list|
   property parent : registerFixture(me)
   
   property empty : missing value
   property notEmpty : missing value
   
   on setUp()
      set empty to {}
      set notEmpty to {"foo", 1}
   end setUp
   
   script |add item|
      property parent : registerTestCase(me)
      set end of empty to "bar"
      should(empty contains "bar", "no bar?!")
   end script
   
   script |add same item|
      property parent : registerTestCase(me)
      set end of notEmpty to "foo"
      should(last item of notEmpty is "foo", "first foo vanished?!")
      should(first item of notEmpty is "foo", "where is last foo?!")
   end script
end script
run makeTextTestRunner(suite)
Running this script give the following results:
My Tests
accessing list - add item ... ok
accessing list - add same item ... ok
Ran 2 tests in 1 seconds.  passed: 2  skips: 0  errors: 0  failures: 0
OK

Since both of the should statements were satisfied, the single "ok" message was printed. If either or both conditions had failed, their respective error messages would have been printed.

Now that we have a working test script, we can start experimenting with adding some functions to test. In AppleScript, we can organize functions and data within a script to be able to create classes and objects like other languages. As an example, let's look at creating a class for storing calendar dates. Add the following script to Listing 2

script CalendarDate
   — CalendarDate has three properties - day, month, and year.
   property calendarDay : 0.0
   property calendarMonth : 0.0
   property calendarYear : 0.0
   — Sets the calendarDay property to the value passed to it.
   on SetDay(theDay)
      set calendarDay to theDay
   end SetDay
   — Sets the calendarMonth property to the value passed to it.
   on SetMonth(theMonth)
      set calendarMonth to theMonth
   end SetMonth
   — Sets the calendarYear property to the value passed to it.
   on SetYear(theYear)
      set calendarYear to theYear
   end SetYear
   — Returns the value of the calendarDay property.
   on GetDay()
      return calendarDay
   end GetDay
   — Returns the value of the calendarMonth property.
   on GetMonth()
      return calendarMonth
   end GetMonth
   — Returns the value of the calendarYear property.
   on GetYear()
      return calendarYear
   end GetYear
   on InitializeDate(myDay, myMonth, myYear)
      SetDay(myDay)
      SetMonth(myMonth)
      SetYear(myYear)
   end InitializeDate
end script

We can imitate a class by creating a top-level function, declaring properties for object data, and creating functions to set the properties and get their values. To create script objects derived from this class description, we can use the copy command to make copies of the scripts, which will make copies of all of the functions and properties of CalendarDate. We can write the setUp function as:

   on setUp()
      copy CalendarDate to firstDate
      copy CalendarDate to secondDate
      — Set the values for the first date.
      tell firstDate to InitializeDate(21, 9, 2007)
      — Set the values for the second date.
      tell secondDate to InitializeDate(25, 9, 2005)
   end setUp

Now that we have two objects (firstDate and secondDate) and have initialized them, we can use the functions of the CalendarDate class to check the values of the two objects.

   script |CheckSameMonth|
      property parent : registerTestCase(me)
      set p to firstDate's GetMonth()
      set d to secondDate's GetMonth()
      should(p is equal to d, "month not equal!")
   end script
   
   script |CheckDifferentMonth|
      property parent : registerTestCase(me)
      set p to firstDate's GetMonth()
      set d to secondDate's GetMonth()
      should(p is not equal to d, "month is the same!")
   end script

Our finished script now looks like this (Listing 3):

property parent : load script file ¬
   (("Sylvester HD:Library:Scripts:") & "ASUnit.scpt")
property suite : makeTestSuite("My Date Tests")
script |DateTests|
   property parent : registerFixture(me)
   
   property firstDate : missing value
   property secondDate : missing value
   
   script CalendarDate
      — CalendarDate has three properties - day, month, and year.
      property calendarDay : 0.0
      property calendarMonth : 0.0
      property calendarYear : 0.0
      — Sets the calendarDay property to the value passed to it.
      on SetDay(theDay)
         set calendarDay to theDay
      end SetDay
      — Sets the calendarMonth property to the value passed to it.
      on SetMonth(theMonth)
         set calendarMonth to theMonth
      end SetMonth
      — Sets the calendarYear property to the value passed to it.
      on SetYear(theYear)
         set calendarYear to theYear
      end SetYear
      — Returns the value of the calendarDay property.
      on GetDay()
         return calendarDay
      end GetDay
      — Returns the value of the calendarMonth property.
      on GetMonth()
         return calendarMonth
      end GetMonth
      — Returns the value of the calendarYear property.
      on GetYear()
         return calendarYear
      end GetYear
      on InitializeDate(myDay, myMonth, myYear)
         SetDay(myDay)
         SetMonth(myMonth)
         SetYear(myYear)
      end InitializeDate
   end script
   
   on setUp()
      copy CalendarDate to firstDate
      copy CalendarDate to secondDate
      — Set the values for the first date.
      tell firstDate to InitializeDate(21, 9, 2007)
      — Set the values for the second date.
      tell secondDate to InitializeDate(25, 9, 2005)
   end setUp
   
   script |CheckSameMonth|
      property parent : registerTestCase(me)
      set p to firstDate's GetMonth()
      set d to secondDate's GetMonth()
      should(p is equal to d, "month not equal!")
   end script
   
   script |CheckDifferentMonth|
      property parent : registerTestCase(me)
      set p to firstDate's GetMonth()
      set d to secondDate's GetMonth()
      should(p is not equal to d, "month is the same!")
   end script
end script
run makeTextTestRunner(suite)

After adding these tests, we get the following results:

My Date Tests
DateTests - CheckSameMonth ... ok
DateTests - CheckDifferentMonth ... FAIL
FAILURES
———————————————————————————————————
test: DateTests - CheckDifferentMonth
message: month is the same!
———————————————————————————————————
Ran 2 tests in 0 seconds.  passed: 1  skips: 0  errors: 0  failures: 1
FAILED

Since both firstDate and secondDate had the same value for the calendarMonth property, the CheckSameMonth test passed while the CheckDifferentMonth failed (since both month values were the same). We can add additional tests to check the day and year values as follows:

   script |CheckSameDay|
      property parent : registerTestCase(me)
      set p to firstDate's GetDay()
      set d to secondDate's GetDay()
      should(p is equal to d, "day not equal!")
   end script
   
   script |CheckDifferentDay|
      property parent : registerTestCase(me)
      set p to firstDate's GetDay()
      set d to secondDate's GetDay()
      should(p is not equal to d, "day is the same!")
   end script
   
   script |CheckSameYear|
      property parent : registerTestCase(me)
      set p to firstDate's GetYear()
      set d to secondDate's GetYear()
      should(p is equal to d, "year not equal!")
   end script
   
   script |CheckDifferentYear|
      property parent : registerTestCase(me)
      set p to firstDate's GetYear()
      set d to secondDate's GetYear()
      should(p is not equal to d, "year is the same!")
   end script

Running all of the tests together gives the following results:

My Date Tests
DateTests - CheckSameMonth ... ok
DateTests - CheckDifferentMonth ... FAIL
DateTests - CheckSameDay ... FAIL
DateTests - CheckDifferentDay ... ok
DateTests - CheckSameYear ... FAIL
DateTests - CheckDifferentYear ... ok
FAILURES
———————————————————————————————————
test: DateTests - CheckDifferentMonth
message: month is the same!
———————————————————————————————————
test: DateTests - CheckSameDay
message: day not equal!
———————————————————————————————————
test: DateTests - CheckSameYear
message: year not equal!
———————————————————————————————————
Ran 6 tests in 2 seconds.  passed: 3  skips: 0  errors: 0  failures: 3
FAILED

We can see that the test results show that the month is the same for the two objects firstDate and secondDate, but that the day and year are not the same.

To conclude this example using ASUnit, we will show how to separate the program code from the test code. Copy the script CalendarDate to another file and call it Date.scpt. Next, save the above script as DateTest.scpt, delete the CalendarDate script and add another property at the top of the script to load the program code from another file. Finally, we need to modify references to CalendarDate in the test script to include the property added at the top of the file. After making these changes, the test script looks like this:

property parent : load script file ¬
   (("Sylvester HD:Library:Scripts:") & "ASUnit.scpt")
property lib : load script file ¬
   (("Sylvester HD:Test:") & "Date.scpt")
property suite : makeTestSuite("My Date Tests")
script |DateTests|
   property parent : registerFixture(me)
   
   property firstDate : missing value
   property secondDate : missing value
      
   on setUp()
      copy lib's CalendarDate to firstDate
      copy lib's CalendarDate to secondDate
      — Set the values for the first date.
      tell firstDate to InitializeDate(21, 9, 2007)
      — Set the values for the second date.
      tell secondDate to InitializeDate(25, 9, 2005)
   end setUp
   
   script |CheckSameMonth|
      property parent : registerTestCase(me)
      set p to firstDate's GetMonth()
      set d to secondDate's GetMonth()
      should(p is equal to d, "month not equal!")
   end script
   
   script |CheckDifferentMonth|
      property parent : registerTestCase(me)
      set p to firstDate's GetMonth()
      set d to secondDate's GetMonth()
      should(p is not equal to d, "month is the same!")
   end script
   script |CheckSameDay|
      property parent : registerTestCase(me)
      set p to firstDate's GetDay()
      set d to secondDate's GetDay()
      should(p is equal to d, "day not equal!")
   end script
   
   script |CheckDifferentDay|
      property parent : registerTestCase(me)
      set p to firstDate's GetDay()
      set d to secondDate's GetDay()
      should(p is not equal to d, "day is the same!")
   end script
   
   script |CheckSameYear|
      property parent : registerTestCase(me)
      set p to firstDate's GetYear()
      set d to secondDate's GetYear()
      should(p is equal to d, "year not equal!")
   end script
   
   script |CheckDifferentYear|
      property parent : registerTestCase(me)
      set p to firstDate's GetYear()
      set d to secondDate's GetYear()
      should(p is not equal to d, "year is the same!")
   end script
end script
run makeTextTestRunner(suite)

When we run the test script DateTest.scpt from the Script Editor, we get the same test results as the combined file in the last section.

Using the concepts of test-driven development, you can build and test your application as you go to make sure that it works the way you want. Using the ASUnit testing framework, you can create a suite of tests to serve as a check of your application logic whenever you make updates. The tests you create give you the freedom to refactor your code while still being able to ensure that your application works as expected. In the second part of this series, we will develop a complete application using test-driven development and the ASUnit testing framework.

Andy Sylvester is an aerospace engineer who has worked in software development for over twenty years. His interests include web applications, the use of computers in music composition, and software development techniques. He has a weblog at www.andysylvester.com where he writes on these subjects. You can reach him at andy@andysylvester.com.