Most of my experience with Sikuli has been in two application domains... testing iOS applications and writing crawlers to download things from the web where cURL wasn't working for me well. Trying to test iOS applications using Sikuli has presented a wide variety of challenges that others might be looking for solutions to. Here's my collection starting with some of the easy ones and moving onto the more esoteric. If you're new to Sikuli and haven't seen it already take a look at my introduction post.

Don't Turn off the Display

So one of the first problems I ran into is that once I had about an hours worth of tests the power saving feature on my mac would turn off the screen... and then all the tests would fail. To work around this I added a line of code to disable this to my testrunner script so I don't have to remember to change it back when tests aren't running. So if you've got your tests running as part of an automated system this is a really handy thing to have.

#disable display sleep
sudo systemsetup -setdisplaysleep Never
#set display sleep for 5 minutes  
sudo systemsetup -setdisplaysleep 5

I recently needed to do this on windows as well and found a solution (for Windows Vista+) on StackOverflow.

Starting up the Simulator Using iphonesim

Since you want to actually run your iOS app inside the simulator... there are two steps to starting up your application for testing. This could have been done using Sikuli's App.open and some Sikuli code, but since I need to load updated versions of my application on a regular basis (every time we test we grab the latest build) I opted for another approach. I ran across this tool called iphonesim that lets you launch your application and a particular version of the iPhone simulator in an easy way. Basically you do this:

iphonesim launch <absolute path to simulator.app>

''Note: This needs to be an absolute path or it won't work''

And you'll have your app up and running shortly. (There's a bit more to this, part discussed here and part will be in a future post about the testrunner I mentioned in the intro post).

Why Does the Simulator Hate Me! (slow motion)

This was probably the most vexing problem when it came to using Sikuli as a testing tool for iOS. After we had managed to write over a hundred tests we still couldn't run them reliably. For some reason the iPhone simulator would get into one of its debugging modes and the entire thing would then be running at half speed. Having your application running super slow is not good for the reliability of your Sikuli tests. We spent months researching this. We even found out that somebody else was having the same problem with Sikuli, though their solution (adding some wait()s) didn't work for us (since this would happen at random we didn't know where we needed to wait... and putting them everywhere slowed down running the tests making them significantly less useful).

One morning I got inspired and started doing some more research into the problem. It turns out that some versions of the iPhone Simulator didn't include a way to enable this slow motion mode. So somebody had figure out how to enable it again. Looking at the code I was thinking to myself... "So... can I make this do the opposite with a minor change?" And it turned out that I could. So if you're running into this problem you can work around it by getting the slowmo sources from github and then making the following change:

- [[(id<SlowmoMonitorController>)[NSClassFromString(@"MonitorController") sharedInstance] monitorWindow] setAllowTripleShiftSlowMotion:YES];  
+ [[(id<SlowmoMonitorController>)[NSClassFromString(@"MonitorController") sharedInstance] monitorWindow] setAllowTripleShiftSlowMotion:NO];

Once you've got it compiled... you need to start up the simulator and then apply a run time patch to the application. The slowmo tool provides a script which does this. Just find the "slowmo.sh" script in the project sources. It's very simple. This will completely disable the slow motion mode for the duration of the simulator session and let you get back to the real work of testing.

Conclusion

That's pretty much it for iOS specific stuff when it comes to Sikuli. The rest of the stuff is mostly generic stuff I've been doing as part of my testrunner (future post blah blah). If you're running into something that you think is specific to iOS and you need someone to ask... feel free to get in contact.