Grokking Cocoa

[Updated the troubleshooting section 10/4/2009]

The unit tests are finally set up for the iPhone. You can start doing some Test Driven Development, but one is failing. I've read Apple's documentation, but how do I debug the blasted thing?

NSLog messages can be scattered throughout the code. Their output is sent to the console, but this is a pain. There must be a better way.

This is one area that needs to be improved in Xcode. There is not even documentation on how to debug unit tests.

Here's how to do it.

The following assumes that you have set up the unit tests using the templates in Xcode and follows along with Apple's example in their iPhone Development Guide. If you have not already set up your units test then the iPhone Development Guide is a good place to start. I am also using Xcode 3.2.

Three steps that need to be performed.

    1. Setup a target that contains the unit tests, but does not run them.

    2. Setup the otest executable to run the tests.

    3. Setup the otest environment so that otest can find your unit tests.

Setting up the Target

When the target is set to run unit tests, Xcode runs a script that runs the unit tests. We need a similar target, but one that does not run the script.

The easiest thing to do is duplicate the existing target (in Apple's examples LogicTests). Right click (or control left click) on the LogicTests target and select Duplicate from the context menu. This will create a new target with a name like LogicTest copy. Now in the LogicTests copy target, delete the Run Script build phase (select the Run Script item and press the delete key).

The target can also be setup as if creating a new unit test target. Just set it up and delete the Run Scripts build phase. I find it easier to copy the existing Unit Tests. This makes it easier to duplicate any changed build settings and files/frameworks that were added. If I am feeling good I then change the LogicTests copy to LogicTest Debug wherever I find it.

Important: This step will create a duplicate Product that will have to be renamed. For example, after the above duplication, there will be a two LogicTests.octest. Rename the second one to LogicTests coy.octest in the Product group in the project window.

Now to setup otest.

Setting up otest

Xcode places the unit tests in a bundle then uses otest to run them. What needs to be done is to create a Custom Executable that will run otest with our LogicTests copy bundle. The tricky part is finding the correct otest.

If you go to /Developer/Tools, you will find that there is an otest there. This is the wrong otest. This is one is for the Mac. We need one for the iPhone simulator. 

Execute the following command in a Terminal window to find the correct otest.

    find /Developer -name otest

This will hopefully give you a list of otest found. For example, in my development environment the correct otest is:

/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator3.1.sdk/Developer/usr/bin/otest

What a mess. Not as nice as /Developer/Tools/otest

Now a custom executable needs to be set up.  In your projects project window select Executables and bring up the context menu. Select add then New Custom Executable... I put otest as the executable name, but most importantly place the path to the above found otest. Again in my environment:

/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator3.1.sdk/Developer/usr/bin/otest

Press Finish and a panel will pop up. We need to configure this panel so that otest will use our unit test bundle.

Setting up the otest Environment

The environment needs to be set up so that otest can find our unit test. In the otest info panel, select the arguments tab. We need to place the name of our unit test bundle as an argument to otest. In our example, "LogicTests copy.octest" (quotes are needed if there are spaces in the name).  Go ahead and click the little plus sign and add this to the arguments list.

You can do man otest in a Terminal window to find out more on otest.

Now we need to set up a mess of environment variables. I have not checked if this is the minimum set, but it works. Add the following to the environment variables list.

Double check these values and make sure they are correct for your environment.

Running/Debugging the Unit Test

Now you can run the unit test. Select the LogicTests copy as the active target. Select the otest executable as the active executable. Now you should be able to run, debug, and set breakpoints like any other normal executable.

Don't forget if you add any files to the unit tests, add them to both targets, i.e. LogicTests and LogicTests copy.

Happy unit testing and TDD.

Troubleshooting

If you get errors like, 

SenTestingKit/SenTestingKit.h: No such file or directory
cannot find interface declaration for 'SenTestCase', superclass of 'LogicTests'

this is due to Xcode putting extra \'s before quotes in Framework Search Path build setting. Remove any backslashes in this setting. Select the LogicTest copy item and bring up its info panel (⌘I). Select the build tab and change the Framework Search Path for all configurations.

If you get an error like,

objc[16428]: '/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator3.1.sdk/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation' was not compiled with -fobjc-gc or -fobjc-gc-only, but the application requires GC

objc[16428]: '/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator3.1.sdk/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation' was not compiled with -fobjc-gc or -fobjc-gc-only, but the application requires GC

objc[16428]: *** GC capability of application and some libraries did not match

the wrong otest was selected. Make sure the path to the correct iPone simulator is selected in the General tab of the otest info panel.