Franck Dernoncourt wrote a Sikuli Script for playing Angry Birds on Google Games.
The script is simple but efficient: it achieves high-scores that are very hard to beat. It tries different launching angles and beats Sergey Brin’s high-score on map 1-4 (36710 vs 29920).
The demo of this script can be seen on youtube.
The full script can be downloaded at here.
Raymond Cheung has created an open-source Magic Online Trading Bot with Sikuli to buy and sell digital products. Magic Online deals in thousands of digital cards that are bought and sold for with real money. Therefore the bot must be as accurate as possible.
The logic takes information from the Sikuli scripts that read the screen for product information and then, after processing that information, uses Sikuli scripting to interact with the Magic Online interface to finish the transaction.
Read the README on his GITHUB for further details. There is a full demonstration of the bot in the youtube video below.
Sikuli 0.10 has added tons of cool new features. Now it is easier to write smarter scripts to do more interesting and useful tasks than you can ever imagine. Here is one of my favorites. The original idea of this script was contributed by RaiMan, one of the most active Sikuli users. I simplified it a little bit and made a video to show how it runs in a real environment.
The script to download a bunch of files.
The goal of this script is to download all ZIP files whose file name starts with “Sikuli-IDE-osx” in a web page. To find all ZIP files is easy, because they have an identifiable icon. However, we can’t simply click on the icon. We need to click on the hyperlink next to the icon, instead. The really tricky thing is, how to know a hyperlink that starts with “Sikuli-IDE-osx”?
The simplest solution is capturing a screenshot of the text “Sikuli-IDE-osx”, and putting the screenshot as the parameter of click(). Unfortunately, this way is not really robust since the images of text only have tiny differences. Sikuli’s fuzzy image matching engine may treat “S1kul1-1DE-esx” as the same as what we want. So, a right way to do this is – extracting the text from the web page.
You may recall that there was an ancient technology named OCR (Optical Character Recognition), which recognizes text from scanned books. However, current OCR engines do not really work well. They are slow and inaccurate.
In fact, even without OCR, we still can extract text from the screen using Sikuli.
How?
Use the clipboard!
The following script tells you how to do that. At first, we use Sikuli to find all locations of the ZIP icon (line 7). On line 8, we sort the list of the matched locations by their y coordinate, it’s not necessary though. The key steps are line 10 and 11. We click on the space between the icon and the hyperlink next to it using the new target offset attribute of a pattern. (denoted by a small red cross in the ZIP icon on line 7. See the figure below for its setting window.)
And then we hold the mouse button and drag to the location 600-pixel right to the center of the icon. This step (line 2) actually simulates using a mouse to select a line of text with a dragDrop(). After the selection, we simply press Command+C (Ctrl-C on Windows) to copy the selected text into the system clipboard.
Copy (extract) text from a browser.
Aha! We got the precise text in the clipboard. 
The rest of the script are trivial. Sikuli 0.10 provides a new API Env.getClipboard() to get the content of the system clipboard. We can use this function to get the text of the hyperlinks and then use Python’s string function (startswith) to filter out the files we need.
I also made a tiny video that shows how this script looks like when it is running.
Sikuli is not only visual but musical. ram B has uncovered Sikuli’s musical potential with a creative script that plays a virtual piano. The script opens a browser and opens up an online virtual piano. Then it asks for an input to choose from a collection of two songs: Terminator’s theme and Godfather’s theme.
The whole script is shown below. You can also download the script to try it out on your computer. Enjoy!!
A while ago we posted several examples of automatic tweets on Twitter sent by Sikuli. Andrew Burton (a.k.a. jarod), from whom one of these tweets were originated, sent us a Twitter script he created that does much more than the simple example we showed last time. This script not only sends a tweet but also signs in one’s Twitter account. When the script is run, it prompts the user for signin id, password, and message. Then the script goes about looking for various buttons to click and text fields to type. In the process, it starts a web browser, types the address of Twitter, waits for the Twitter logo to appear, clicks on the sign-in button, types the login and password, clicks the message box, types the message, and finally clicks the update button to submit the tweet.
The whole script is shown below.
Many thanks to ram B for contributing another interesting script. The purpose of the script is to search for pictures on Picasa. The script first opens Picasa in Firefox browser. Then in this example it enters a keyword “cars” and clicks on the “search” button to submit the keyword. Unlike Flickr search, in Picasa, one has to click the “play” button even after clicking the slide show button. So this script will start the slide show increase the speed by 1 sec, hit the play button and will close the window after displaying around 10 pictures.
The full script can be seen below.
FFXI (i.e., Final Fantasy 11) is one of the most popular online RPG games, but notorious for its ridiculously long and tedious login process. firewing at the Bluegarttr created a Sikuli script to automate this process. By exporting the script to an executble (.skl) file and placing a shortcut on the desktop, all one needs to do to start playing FFXI is to double-click the shortcut. How easy!!
The full script annotated by the author is shown below.
Today’s showcase script is contributed by Ram B for tuning in to an online radio station on last.fm to listen to Beatles music. This script launches Firefox, types “www.last.fm” in the address bar, types “beatles” in the search box, clicks the “GO” button to confirm the search, and finally clicks on “Play The Beatles Radio” button to play. The entire script can be viewed below.
Sikuli is fun! Today’s showcase script is contributed by Garrett Bartley. He uses Sikuli to write a script to play Mafia Wars online. The script automates a simple yet important task in Mafia Wars: upgrading the casino!!
To display data, Sikuli currently supports only popup messages and debug messages, which can be very limiting. Garrett uses a very clever trick to get around this limitation. He uses Sikuli to open up another text editor and type the data in the editor to display.
It may not be clear to many Sikuli users that under the hood of Sikuli, there is a Java virtual machine providing a rich set of UI components such as tables and lists that can be called upon to display data. In the future release of Sikuli, we plan to introduce simple API calls to uses these components for output, while hiding the complexity of Java programming. Please stay tuned!
We thought one thing Sikuli can not automate is itself, since it hides itself every time when it executes a script. We are proven wrong by Koichi Tamura, who has demonstrated it is possible to run a command-line Sikuli script to click on the buttons of another Sikuli IDE in the foreground in order to run other Sikuli scripts. More interestingly, the process of loading Sikuli scripts into the IDE is automated by yet another automation tool. This tool is a visual programming tool Koichi is currently developing himself. Below is the example demonstrated by Koichi. Two Sikuli scripts are represented as two nodes connected by an arrow. One script opens a “hello world” popup, while the other opens a new tab in a web browser and loads his blog in the tab.
Watch the video in YouTube to see this example in action:
