Convert Figma logo to code with AI

calabash logocalabash-ios

Calabash for iOS

1,809
369
1,809
120

Top Related Projects

3,363

A home for issues that are common to multiple cucumber repositories

18,634

Cross-platform automation framework for all kinds of apps, built on top of the W3C WebDriver protocol

11,134

Gray box end-to-end testing and automation framework for mobile apps

Run your application at full speed while syncing your code for development, finally empowering you to utilize docker for development under OSX/Windows/*Linux

Quick Overview

Calabash-iOS is an automated testing framework for iOS applications. It allows developers and QA engineers to write and execute automated acceptance tests for iOS apps using Cucumber, a behavior-driven development (BDD) tool. Calabash-iOS integrates with the iOS simulator and physical devices to perform UI interactions and validate app behavior.

Pros

  • Supports both iOS simulator and physical devices for testing
  • Uses Cucumber for writing human-readable test scenarios
  • Provides a wide range of predefined steps for common UI interactions
  • Integrates well with continuous integration systems

Cons

  • Requires separate maintenance alongside the iOS app codebase
  • Learning curve for teams not familiar with Cucumber or Ruby
  • Can be slower than native UI testing frameworks like XCTest
  • May require frequent updates to keep up with iOS SDK changes

Code Examples

  1. Defining a simple step in Calabash-iOS:
Then(/^I should see the text "([^"]*)"$/) do |expected_text|
  wait_for_element_exists("* text:'#{expected_text}'")
end
  1. Tapping a button by its accessibility label:
When(/^I tap the "([^"]*)" button$/) do |button_label|
  tap_mark(button_label)
end
  1. Entering text into a text field:
When(/^I enter "([^"]*)" into the "([^"]*)" field$/) do |text, field_name|
  enter_text("textField marked:'#{field_name}'", text)
end

Getting Started

  1. Install Calabash-iOS:

    gem install calabash-cucumber
    
  2. Set up your Xcode project:

    calabash-ios setup
    
  3. Generate a Cucumber skeleton:

    calabash-ios gen
    
  4. Write your first feature file (e.g., features/my_first.feature):

    Feature: Login Screen
      Scenario: As a user I can log into the app
        Given I am on the Login Screen
        When I enter "user@example.com" into the "Email" field
        And I enter "password123" into the "Password" field
        And I tap the "Login" button
        Then I should see the text "Welcome"
    
  5. Run your tests:

    cucumber
    

Competitor Comparisons

3,363

A home for issues that are common to multiple cucumber repositories

Pros of Common

  • More active development with recent commits and releases
  • Broader scope, supporting multiple Cucumber implementations
  • Better documentation and community support

Cons of Common

  • Less focused on iOS-specific testing
  • May require additional setup for iOS projects
  • Potentially steeper learning curve for iOS developers

Code Comparison

Calabash-iOS example:

calabash_launcher = Calabash::Cucumber::Launcher.new
calabash_launcher.relaunch
calabash_launcher.calabash_notify(self)

Common example:

require 'cucumber/core'
require 'cucumber/core/gherkin/parser'

Cucumber::Core.run(feature_file, [MyFormatter.new])

Summary

Common is a more versatile and actively maintained project, suitable for various Cucumber implementations. However, Calabash-iOS offers more specialized iOS testing features. Common may require additional configuration for iOS projects but provides broader support and documentation. Developers should consider their specific needs and expertise when choosing between the two repositories.

18,634

Cross-platform automation framework for all kinds of apps, built on top of the W3C WebDriver protocol

Pros of Appium

  • Cross-platform support for iOS, Android, and Windows
  • Uses standard WebDriver protocol, allowing use of various programming languages
  • Supports both native and hybrid mobile applications

Cons of Appium

  • Slower test execution compared to Calabash
  • More complex setup and configuration process
  • Potential stability issues with certain device/OS combinations

Code Comparison

Calabash-iOS (Ruby):

Then /^I see the text "([^"]*)"$/ do |text|
  wait_for_element_exists("* text:'#{text}'")
end

Appium (Python):

from appium.webdriver.common.mobileby import MobileBy

def test_find_text(self):
    element = self.driver.find_element(MobileBy.ACCESSIBILITY_ID, "text_element")
    assert element.text == "Expected Text"

Key Differences

  • Calabash-iOS is specifically designed for iOS testing, while Appium supports multiple platforms
  • Calabash uses Cucumber for behavior-driven development, whereas Appium allows for various testing frameworks
  • Appium has a larger community and more frequent updates due to its cross-platform nature
  • Calabash-iOS offers tighter integration with iOS-specific features and APIs
11,134

Gray box end-to-end testing and automation framework for mobile apps

Pros of Detox

  • Faster test execution due to its gray-box approach and direct communication with the app
  • Better stability and less flakiness in tests, especially for complex UI interactions
  • Supports both iOS and Android platforms, providing a unified testing solution

Cons of Detox

  • Steeper learning curve, especially for developers new to mobile testing
  • Limited support for some native iOS controls and gestures
  • Requires more setup and configuration compared to Calabash-iOS

Code Comparison

Detox (JavaScript):

describe('Login Flow', () => {
  it('should login successfully', async () => {
    await element(by.id('username')).typeText('user@example.com');
    await element(by.id('password')).typeText('password123');
    await element(by.text('Login')).tap();
    await expect(element(by.text('Welcome'))).toBeVisible();
  });
});

Calabash-iOS (Ruby):

Given(/^I am on the login screen$/) do
  wait_for_element_exists("* marked:'Login'")
end

When(/^I enter valid credentials$/) do
  enter_text("textField marked:'Username'", "user@example.com")
  enter_text("textField marked:'Password'", "password123")
  touch("button marked:'Login'")
end

Then(/^I should see the welcome message$/) do
  wait_for_element_exists("* text:'Welcome'")
end

Run your application at full speed while syncing your code for development, finally empowering you to utilize docker for development under OSX/Windows/*Linux

Pros of docker-sync

  • Focuses on improving Docker performance on macOS and Windows
  • Provides real-time file synchronization between host and containers
  • Supports various sync strategies and customization options

Cons of docker-sync

  • Limited to Docker-specific use cases
  • Requires additional setup and configuration
  • May introduce complexity to development workflows

Code comparison

docker-sync configuration example:

version: "2"
syncs:
  app-sync:
    src: '.'
    sync_strategy: 'native_osx'
    sync_excludes: ['node_modules', 'tmp', 'log']

Calabash-iOS example (not directly comparable):

calabash_launcher = Calabash::Cucumber::Launcher.new
calabash_launcher.relaunch
calabash_launcher.calabash_notify(self)

Key differences

  • Purpose: docker-sync focuses on Docker performance, while Calabash-iOS is for iOS app testing
  • Language: docker-sync uses YAML for configuration, Calabash-iOS uses Ruby
  • Scope: docker-sync is platform-agnostic (for Docker), Calabash-iOS is iOS-specific
  • Integration: docker-sync works with Docker environments, Calabash-iOS integrates with iOS development tools

Conclusion

While both projects serve different purposes, docker-sync offers advantages for Docker-based development on macOS and Windows, whereas Calabash-iOS provides a robust framework for iOS app testing. The choice between them depends on the specific development needs and target platform.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

masterdevelopversioninglicensecontributing
Build StatusBuild StatusGitHub versionLicenseContributing

Calabash iOS

After delivering support for the final releases of iOS 11 and Android 8 operating systems, Microsoft will discontinue our contributions to developing Calabash, the open-source mobile app testing tool. We hope that the community will continue to fully adopt and maintain it. As part of our transition on the development of Calabash, we've provided an overview of mobile app UI and end-to-end testing frameworks as a starting point for teams who are looking to re-evaluate their testing strategy. Please see our Mobile App Testing Frameworks Overview document.

Calabash is an automated testing technology for Android and iOS native and hybrid applications.

Calabash is a free-to-use open source project that is looking for a maintainer.

Requirements

  • macOS Mojave or Catalina
  • Xcode 10.3+
  • ruby at least ruby 2.3.x

For the best Ruby experience we recommend that you use a managed Ruby like rbenv or rvm).

Please do not install gems with sudo

For more information about ruby on MacOS, see these Wiki pages:

IMPORTANT

Calabash uses private APIs to inspect your app's view hierarchy. Apps that include the Calabash iOS Server will be rejected if they are submitted to the AppStore. The tutorials below describe a number ways to add Calabash to your Xcode project that will ensure you do not accidently submit a binary that will be reject because it includes Calabash.

Getting Started

If you want to see Calabash iOS in action, head over to the Calabash iOS Smoke Test App and follow the instructions in the README. We use this app to document, demonstrate, and test Calabash iOS. You can use this app to explore Calabash and as an example for how to configure your Xcode project and Calabash workflow.

The examples below assume you are using a managed ruby or are working in the Calabash Sandbox:

$ calabash-sandbox
This terminal is now ready to use with Calabash.
To exit, type 'exit'.

Step 1: Link calabash.framework

To start using Calabash in your project, you need to link an Objective-C framework (calabash.framework) to your application. These instructions are compatible with apps that are written in Swift.

TutorialDescription
Debug ConfigUse linker flags in the Debug build config to load the calabash.framework
Calabash ConfigCreate a new Calabash Build Configuration
-cal TargetAdd a new app target to Xcode.

If you want to get started quickly, follow the Debug Config instructions. The Tutorial: How to add Calabash to Xcode wiki page discusses the merits of each approach and has instructions for using CocoaPods.

Step 2: Run Cucumber against an iOS Simulator

The Calabash iOS Example README has simple instructions for how to link the calabash.framwork, generate a features directory, run cucumber, and and open a Calabash console.

# In the directory where your .xcodeproj and Gemfile are
$ bundle exec calabash-ios gen

Build and run in Xcode, targeting an iOS Simulator. Calabash will try to detect the .app you just built.

$ bundle exec cucumber

If Calabash cannot find the .app you just built, it will raise an error. If this happens, you will to tell Calabash where it can find your .app.

By default, Xcode builds to a DerivedData directory:

~/Library/Developer/Xcode/DerivedData/<UDID>/Build/Products/Debug-iphonesimulator/<NAME>.app

Try to locate the .app and set the APP variable:

$ export APP="~/Library/Developer/Xcode/DerivedData/<UDID>/Build/Products/Debug-iphonesimulator/<NAME>.app"
$ bundle exec cucumber

We recommend using scripts and/or changing the location where Xcode stages build products. The sample projects use scripts to stage binaries to a ./Products, even when building from Xcode. You can use the Xcode > Preferences > Locations settings to do the same.

Where to go from here?

TopicDescription
Getting StartedA more in-depth tutorial using the LPSimpleExample.
Testing on Physical DevicesEverything you need to know about testing on physical devices.
API DocsThe Calabash iOS ruby API
iOS Smoke Test AppDemonstrates advanced features, setups, and workflows
iOS WebView Test AppDemonstrates how to interact with UIWebView and WKWebView
Getting HelpThe Calabash iOS Wiki

Links

License

Copyright (c) LessPainful APS. All rights reserved.
The use and distribution terms for this software are covered by the
Eclipse Public License 1.0 (http://opensource.org/licenses/eclipse-1.0.php)
which can be found in the file epl-v10.html at the root of this distribution.
By using this software in any fashion, you are agreeing to be bound by
the terms of this license. You must not remove this notice, or any other,
from this software.