calabash-android

Automated Functional testing for Android using cucumber

1,682

618

1,682

View on GitHub

Top Related Projects

appium

18,791

Cross-platform automation framework for all kinds of apps, built on top of the W3C WebDriver protocol

Quick Overview

Calabash-Android is an automated testing framework for Android applications. It allows developers and QA engineers to write and execute automated acceptance tests for Android apps using Cucumber, a behavior-driven development (BDD) tool. Calabash-Android provides a Ruby API for interacting with Android applications during testing.

Pros

Supports behavior-driven development (BDD) using Cucumber, making tests more readable and maintainable
Provides a rich set of predefined steps for common Android UI interactions
Allows for cross-platform test writing when used in conjunction with Calabash-iOS
Integrates well with continuous integration systems

Cons

Requires Ruby knowledge, which may be a barrier for some Android developers
Can be slower than native Android testing frameworks
May require additional setup and configuration compared to built-in Android testing tools
Maintenance and updates can be inconsistent, as it's an open-source project

Code Examples

Installing Calabash-Android gem:

gem install calabash-android

Generating a Cucumber project:

calabash-android gen

Writing a simple step definition:

Given(/^I see the "([^"]*)" button$/) do |button_text|
  wait_for_element_exists("android.widget.Button text:'#{button_text}'")
end

When(/^I tap the "([^"]*)" button$/) do |button_text|
  tap_when_element_exists("android.widget.Button text:'#{button_text}'")
end

Then(/^I should see the text "([^"]*)"$/) do |expected_text|
  wait_for_text(expected_text)
end

Getting Started

Install the Calabash-Android gem:
```
gem install calabash-android
```
Generate a Cucumber project in your Android app directory:
```
calabash-android gen
```

Create a feature file in the features directory (e.g., login.feature):

Feature: Login
  Scenario: Successful login
    Given I am on the login screen
    When I enter "user@example.com" as username
    And I enter "password123" as password
    And I tap the "Login" button
    Then I should see the text "Welcome"

Run the Calabash-Android console to interact with your app:
```
calabash-android console <path_to_your_apk>
```

Execute your tests:

calabash-android run <path_to_your_apk>

Competitor Comparisons

appium

18,791

Cross-platform automation framework for all kinds of apps, built on top of the W3C WebDriver protocol

Pros of Appium

Supports multiple platforms (iOS, Android, Windows) with a single API
Uses standard automation APIs provided by vendors, ensuring better stability
Allows tests to be written in various programming languages (e.g., Java, Python, Ruby)

Cons of Appium

Slower test execution compared to Calabash-Android
More complex setup and configuration process
Steeper learning curve for beginners

Code Comparison

Appium (Python):

from appium import webdriver

desired_caps = {
    'platformName': 'Android',
    'deviceName': 'Android Emulator',
    'app': '/path/to/app.apk'
}
driver = webdriver.Remote('http://localhost:4723/wd/hub', desired_caps)

Calabash-Android (Ruby):

require 'calabash-android/cucumber'

calabash_android_setup
Before do |scenario|
  start_test_server_in_background
end

Key Differences

Appium uses WebDriver protocol, while Calabash-Android uses its own custom protocol
Calabash-Android is specifically designed for Android, whereas Appium is cross-platform
Appium has a larger community and more frequent updates
Calabash-Android offers easier setup for Ruby developers but limits language choices

Both tools are valuable for mobile app testing, with Appium offering more flexibility and Calabash-Android providing a simpler approach for Android-specific testing.

robolectric

5,862

Android Unit Testing Framework

Pros of Robolectric

Faster test execution as tests run on JVM, not on emulator or device
Easier setup and integration with existing Java/Kotlin projects
Better support for unit testing and mocking Android components

Cons of Robolectric

Limited ability to test real device interactions and hardware features
Potential discrepancies between simulated environment and actual Android runtime
Steeper learning curve for developers new to Android testing

Code Comparison

Robolectric test example:

@RunWith(RobolectricTestRunner.class)
public class MyActivityTest {
    @Test
    public void clickingButton_shouldChangeText() {
        Activity activity = Robolectric.setupActivity(MyActivity.class);
        Button button = activity.findViewById(R.id.button);
        TextView textView = activity.findViewById(R.id.text_view);
        button.performClick();
        assertThat(textView.getText().toString()).isEqualTo("Button Clicked");
    }
}

Calabash-Android test example:

Feature: Login functionality
  Scenario: User can log in with valid credentials
    Given I am on the login screen
    When I enter "username" into input field with id "username_field"
    And I enter "password" into input field with id "password_field"
    And I press the "Login" button
    Then I should see "Welcome" on the screen

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

Welcome to Calabash for Android

After delivering support for the final releases of iOS 11 and Android 8 operating systems, Microsoft will discontinue our contributions to developing Calabash, the open-source mobile app testing tool. We hope that the community will continue to fully adopt and maintain it. As part of our transition on the development of Calabash, we've provided an overview of mobile app UI and end-to-end testing frameworks as a starting point for teams who are looking to re-evaluate their testing strategy. Please see our Mobile App Testing Frameworks Overview document.

Calabash is an automated testing technology for Android and iOS native and hybrid applications.

Calabash is a free open source project that is looking for a maintainer.

Documentation

The documentation is split into the following sections:

Calabash Android requires ruby >= 2.0 (latest stable release is preferred).

Ruby on MacOS

On MacOS, we recommend using a managed Ruby like rbenv or rvm).

Please do not install gems with sudo

For more information about ruby on MacOS, see these Wiki pages:

Upgrading to Calabash-android 0.5

Calabash-android 0.5 introduced new features and removed a lot of actions. If your test project is dependent on some of the removed actions, you will have to reimplement the actions using ruby wrappers, queries and gestures. This document describes all changes needed to migrate to calabash-android 0.5

Generate a Cucumber skeleton

To get started with calabash it might be a good idea to run calabash-android gen. It will create a Cucumber skeleton in the current folder like this:

features
|_support
| |_app_installation_hooks.rb
| |_app_life_cycle_hooks.rb
| |_env.rb
| |_hooks.rb
|_step_definitions
| |_calabash_steps.rb
|_my_first.feature

In this skeleton you find all the predefined steps that comes with calabash. Try to take a look my_first.feature and change it to fit your app.

Writing a test

The Cucumber features goes in the features library and should have the ".feature" extension.

You can start out by looking at features/my_first.feature. You can extend this feature or make your own using some of the predefined steps that comes with Calabash.

Running test

To run your test:

calabash-android run <apk>

Calabash-android will install an instrumentation along with your app when executing the app. We call this instrumentation for "test server". The "test server" has special permission that allows it to interact very closely with your app during test.

Every time you test a new binary or use an upgraded version of calabash a new test server will be build. The test server is an intrumentation that will run along with your app on the device to execute the test.

Using UIAutomator2

To benefit from UIAutomator2 test capabilities (like interacting with other apps or system elements), you must start the test server with: start_test_server_in_background(with_uiautomator: true)

Learn more about actions available though ui_automator2 in the Wiki

Screenshot location

Screenshots are placed in the current working directory by default. The location can be changed by setting the SCREENSHOT_PATH environment variable.

SCREENSHOT_PATH=/tmp/foo/ calabash-android run

would cause the first screenshot to appear at /tmp/foo/screenshot_0.png.

Predefined steps

The predefined steps are located in the features/step_definitions folder. A compiled list of predefined steps with comments is available here

Gestures

Learn about available gestures here Troubleshooting

Problems starting the tests

If your app instantaneously crashes right after being started, bear in mind that it needs the permission android.permission.INTERNET in order for calabash tests to run ( Issue #278 ). Add the following xml to your AndroidManifest.xml if you don't have it already:

<uses-permission android:name="android.permission.INTERNET" />

Problems clicking on buttons and text

If it seems that buttons/text aren't being clicked properly, you need to add the following xml to your AndroidManifest.xml:

<uses-sdk android:targetSdkVersion="SDK_VERSION" />

Where SDK_VERSION is the version of the Android SDK you are using. Version numbers can be found here

For example, Android 4.0 uses version 14, Android 4.0.3 uses version 15 and Android 4.1 uses version 16.

Top Related Projects

appium

18,791

Cross-platform automation framework for all kinds of apps, built on top of the W3C WebDriver protocol

robolectric

5,862

Android Unit Testing Framework

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot