bats

Bash Automated Testing System

7,121

518

7,121

137

View on GitHub View on NPM

Top Related Projects

inspec

2,846

InSpec: Auditing and Testing Framework

common

3,363

A home for issues that are common to multiple cucumber repositories

goss

5,563

Quick and Easy server testing/validation

Quick Overview

Bats (Bash Automated Testing System) is a testing framework for Bash scripts. It provides a simple way to write and run tests for shell scripts, allowing developers to ensure their Bash code works as expected. Bats uses a familiar, TAP-compliant output format and integrates well with existing CI/CD pipelines.

Pros

Easy to learn and use, especially for those already familiar with Bash
Lightweight and doesn't require additional dependencies
Integrates well with existing CI/CD tools and workflows
Supports both unit and integration testing for shell scripts

Cons

Limited to testing Bash scripts, not suitable for other languages
May require additional setup for complex testing scenarios
Less feature-rich compared to more comprehensive testing frameworks
Documentation could be more extensive and up-to-date

Code Examples

Basic test case:

@test "addition using bc" {
  result="$(echo 2+2 | bc)"
  [ "$result" -eq 4 ]
}

This example tests a simple addition operation using the bc command.

Testing a custom function:

#!/usr/bin/env bats

function add_numbers() {
  echo $(( $1 + $2 ))
}

@test "add_numbers function" {
  result=$(add_numbers 5 3)
  [ "$result" -eq 8 ]
}

This example tests a custom function that adds two numbers.

Using setup and teardown:

setup() {
  echo "Setting up test environment" >&3
  export TEMP_DIR=$(mktemp -d)
}

teardown() {
  echo "Cleaning up test environment" >&3
  rm -rf "$TEMP_DIR"
}

@test "create file in temp directory" {
  touch "$TEMP_DIR/testfile"
  [ -f "$TEMP_DIR/testfile" ]
}

This example demonstrates the use of setup and teardown functions to manage test environment.

Getting Started

To get started with Bats:

Install Bats:

git clone https://github.com/sstephenson/bats.git
cd bats
./install.sh /usr/local

Create a test file (e.g., test_example.bats):

#!/usr/bin/env bats

@test "example test" {
  result="$(echo 'Hello, World!')"
  [ "$result" = "Hello, World!" ]
}

Run the test:
```
bats test_example.bats
```

This will execute the test and display the results in TAP-compliant format.

Competitor Comparisons

bats-core

4,860

Bash Automated Testing System

Pros of bats-core

More actively maintained with frequent updates and bug fixes
Expanded test coverage and improved reliability
Enhanced compatibility with different operating systems and environments

Cons of bats-core

Potential compatibility issues with older bats scripts
Slightly steeper learning curve due to additional features

Code Comparison

bats:

#!/usr/bin/env bats

@test "addition using bc" {
  result="$(echo 2+2 | bc)"
  [ "$result" -eq 4 ]
}

bats-core:

#!/usr/bin/env bats

@test "addition using bc" {
  run bash -c "echo 2+2 | bc"
  assert_success
  assert_output "4"
}

The main difference in the code examples is that bats-core introduces the run command and assert_* functions, which provide more detailed output and easier assertion handling.

bats-core offers a more robust and actively maintained testing framework for shell scripts, with improved features and compatibility. However, users familiar with the original bats may need to adapt to some changes in syntax and functionality. Overall, bats-core is recommended for new projects and those seeking ongoing support and enhancements.

inspec

2,846

InSpec: Auditing and Testing Framework

Pros of InSpec

More comprehensive infrastructure testing framework
Supports multiple platforms and cloud providers
Provides a domain-specific language for writing tests

Cons of InSpec

Steeper learning curve due to more complex functionality
Requires more setup and configuration
Potentially overkill for simple shell script testing

Code Comparison

InSpec example:

describe file('/etc/passwd') do
  it { should exist }
  its('mode') { should cmp '0644' }
end

Bats example:

@test "Check /etc/passwd file" {
  [ -f "/etc/passwd" ]
  run stat -c "%a" /etc/passwd
  [ "$output" = "644" ]
}

Summary

InSpec is a more powerful and versatile infrastructure testing tool, suitable for complex environments and cloud deployments. It offers a rich set of resources and matchers but requires more investment in learning and setup.

Bats, on the other hand, is a simpler, lightweight testing framework specifically designed for Bash scripts. It's easier to get started with and ideal for basic shell script testing, but lacks the advanced features and cross-platform capabilities of InSpec.

Choose InSpec for comprehensive infrastructure testing across multiple platforms, or Bats for quick and easy Bash script testing in simpler environments.

common

3,363

A home for issues that are common to multiple cucumber repositories

Pros of Common

Supports multiple programming languages, offering broader applicability
Provides a more comprehensive testing framework with behavior-driven development (BDD) features
Offers better integration with continuous integration (CI) tools and reporting

Cons of Common

Steeper learning curve due to more complex syntax and concepts
Requires more setup and configuration compared to the simplicity of Bats
May be overkill for simple shell script testing scenarios

Code Comparison

Bats example:

@test "addition using bc" {
  result="$(echo 2+2 | bc)"
  [ "$result" -eq 4 ]
}

Common (Cucumber) example:

Feature: Calculator
  Scenario: Addition
    Given I have entered 2 into the calculator
    And I have entered 2 into the calculator
    When I press add
    Then the result should be 4 on the screen

The Bats example shows a straightforward test case for a simple addition operation, while the Common (Cucumber) example demonstrates a more verbose, human-readable format that describes the behavior of a calculator application. This illustrates the difference in approach between the two testing frameworks, with Bats focusing on simplicity and Common emphasizing readability and behavior-driven development.

goss

5,563

Quick and Easy server testing/validation

Pros of Goss

Designed specifically for server testing and validation
Supports multiple output formats (JSON, YAML, JUnit)
Can generate tests from the current system state

Cons of Goss

Limited to system testing, less flexible for general-purpose testing
Requires separate installation and setup
Less extensive documentation compared to Bats

Code Comparison

Bats test example:

@test "addition using bc" {
  result="$(echo 2+2 | bc)"
  [ "$result" -eq 4 ]
}

Goss test example:

file:
  /etc/passwd:
    exists: true
    mode: "0644"
    owner: root
    group: root
    filetype: file

Bats focuses on shell script testing with a syntax similar to traditional unit tests, while Goss uses YAML for system validation and infrastructure testing. Bats is more versatile for general scripting tasks, whereas Goss excels in server configuration validation.

Bats has a larger community and more extensive documentation, making it easier for newcomers to adopt. However, Goss offers more specialized features for system testing and can generate tests from the current system state, which can be advantageous for infrastructure validation.

Both tools have their strengths, and the choice between them depends on the specific testing requirements and the target environment.

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

Bats: Bash Automated Testing System

Bats is a TAP-compliant testing framework for Bash. It provides a simple way to verify that the UNIX programs you write behave as expected.

A Bats test file is a Bash script with special syntax for defining test cases. Under the hood, each test case is just a function with a description.

#!/usr/bin/env bats

@test "addition using bc" {
  result="$(echo 2+2 | bc)"
  [ "$result" -eq 4 ]
}

@test "addition using dc" {
  result="$(echo 2 2+p | dc)"
  [ "$result" -eq 4 ]
}

Bats is most useful when testing software written in Bash, but you can use it to test any UNIX program.

Test cases consist of standard shell commands. Bats makes use of Bash's errexit (set -e) option when running test cases. If every command in the test case exits with a 0 status code (success), the test passes. In this way, each line is an assertion of truth.

Running tests

To run your tests, invoke the bats interpreter with a path to a test file. The file's test cases are run sequentially and in isolation. If all the test cases pass, bats exits with a 0 status code. If there are any failures, bats exits with a 1 status code.

When you run Bats from a terminal, you'll see output as each test is performed, with a check-mark next to the test's name if it passes or an "X" if it fails.

$ bats addition.bats
 â addition using bc
 â addition using dc

2 tests, 0 failures

If Bats is not connected to a terminalâin other words, if you run it from a continuous integration system, or redirect its output to a fileâthe results are displayed in human-readable, machine-parsable TAP format.

You can force TAP output from a terminal by invoking Bats with the --tap option.

$ bats --tap addition.bats
1..2
ok 1 addition using bc
ok 2 addition using dc

Test suites

You can invoke the bats interpreter with multiple test file arguments, or with a path to a directory containing multiple .bats files. Bats will run each test file individually and aggregate the results. If any test case fails, bats exits with a 1 status code.

Writing tests

Each Bats test file is evaluated n+1 times, where n is the number of test cases in the file. The first run counts the number of test cases, then iterates over the test cases and executes each one in its own process.

For more details about how Bats evaluates test files, see Bats Evaluation Process on the wiki.

`run`: Test other commands

Many Bats tests need to run a command and then make assertions about its exit status and output. Bats includes a run helper that invokes its arguments as a command, saves the exit status and output into special global variables, and then returns with a 0 status code so you can continue to make assertions in your test case.

For example, let's say you're testing that the foo command, when passed a nonexistent filename, exits with a 1 status code and prints an error message.

@test "invoking foo with a nonexistent file prints an error" {
  run foo nonexistent_filename
  [ "$status" -eq 1 ]
  [ "$output" = "foo: no such file 'nonexistent_filename'" ]
}

The $status variable contains the status code of the command, and the $output variable contains the combined contents of the command's standard output and standard error streams.

A third special variable, the $lines array, is available for easily accessing individual lines of output. For example, if you want to test that invoking foo without any arguments prints usage information on the first line:

@test "invoking foo without arguments prints usage" {
  run foo
  [ "$status" -eq 1 ]
  [ "${lines[0]}" = "usage: foo <filename>" ]
}

`load`: Share common code

You may want to share common code across multiple test files. Bats includes a convenient load command for sourcing a Bash source file relative to the location of the current test file. For example, if you have a Bats test in test/foo.bats, the command

load test_helper

will source the script test/test_helper.bash in your test file. This can be useful for sharing functions to set up your environment or load fixtures.

`skip`: Easily skip tests

Tests can be skipped by using the skip command at the point in a test you wish to skip.

@test "A test I don't want to execute for now" {
  skip
  run foo
  [ "$status" -eq 0 ]
}

Optionally, you may include a reason for skipping:

@test "A test I don't want to execute for now" {
  skip "This command will return zero soon, but not now"
  run foo
  [ "$status" -eq 0 ]
}

Or you can skip conditionally:

@test "A test which should run" {
  if [ foo != bar ]; then
    skip "foo isn't bar"
  fi

  run foo
  [ "$status" -eq 0 ]
}

`setup` and `teardown`: Pre- and post-test hooks

You can define special setup and teardown functions, which run before and after each test case, respectively. Use these to load fixtures, set up your environment, and clean up when you're done.

Code outside of test cases

You can include code in your test file outside of @test functions. For example, this may be useful if you want to check for dependencies and fail immediately if they're not present. However, any output that you print in code outside of @test, setup or teardown functions must be redirected to stderr (>&2). Otherwise, the output may cause Bats to fail by polluting the TAP stream on stdout.

Special variables

There are several global variables you can use to introspect on Bats tests:

$BATS_TEST_FILENAME is the fully expanded path to the Bats test file.
$BATS_TEST_DIRNAME is the directory in which the Bats test file is located.
$BATS_TEST_NAMES is an array of function names for each test case.
$BATS_TEST_NAME is the name of the function containing the current test case.
$BATS_TEST_DESCRIPTION is the description of the current test case.
$BATS_TEST_NUMBER is the (1-based) index of the current test case in the test file.
$BATS_TMPDIR is the location to a directory that may be used to store temporary files.

Installing Bats from source

Check out a copy of the Bats repository. Then, either add the Bats bin directory to your $PATH, or run the provided install.sh command with the location to the prefix in which you want to install Bats. For example, to install Bats into /usr/local,

$ git clone https://github.com/sstephenson/bats.git
$ cd bats
$ ./install.sh /usr/local

Note that you may need to run install.sh with sudo if you do not have permission to write to the installation prefix.

Support

The Bats source code repository is hosted on GitHub. There you can file bugs on the issue tracker or submit tested pull requests for review.

For real-world examples from open-source projects using Bats, see Projects Using Bats on the wiki.

To learn how to set up your editor for Bats syntax highlighting, see Syntax Highlighting on the wiki.

Version history

0.4.0 (August 13, 2014)

Improved the display of failing test cases. Bats now shows the source code of failing test lines, along with full stack traces including function names, filenames, and line numbers.
Improved the display of the pretty-printed test summary line to include the number of skipped tests, if any.
Improved the speed of the preprocessor, dramatically shortening test and suite startup times.
Added support for absolute pathnames to the load helper.
Added support for single-line @test definitions.
Added bats(1) and bats(7) manual pages.
Modified the bats command to default to TAP output when the $CI variable is set, to better support environments such as Travis CI.

0.3.1 (October 28, 2013)

Fixed an incompatibility with the pretty formatter in certain environments such as tmux.
Fixed a bug where the pretty formatter would crash if the first line of a test file's output was invalid TAP.

0.3.0 (October 21, 2013)

Improved formatting for tests run from a terminal. Failing tests are now colored in red, and the total number of failing tests is displayed at the end of the test run. When Bats is not connected to a terminal (e.g. in CI runs), or when invoked with the --tap flag, output is displayed in standard TAP format.
Added the ability to skip tests using the skip command.
Added a message to failing test case output indicating the file and line number of the statement that caused the test to fail.
Added "ad-hoc" test suite support. You can now invoke bats with multiple filename or directory arguments to run all the specified tests in aggregate.
Added support for test files with Windows line endings.
Fixed regular expression warnings from certain versions of Bash.
Fixed a bug running tests containing lines that begin with -e.

0.2.0 (November 16, 2012)

Added test suite support. The bats command accepts a directory name containing multiple test files to be run in aggregate.
Added the ability to count the number of test cases in a file or suite by passing the -c flag to bats.
Preprocessed sources are cached between test case runs in the same file for better performance.

0.1.0 (December 30, 2011)

Initial public release.

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

Top Related Projects

Quick Overview

Pros

Cons

Code Examples

Getting Started

Competitor Comparisons

Pros of bats-core

Cons of bats-core

Code Comparison

Pros of InSpec

Cons of InSpec

Code Comparison

Summary

Pros of Common

Cons of Common

Code Comparison

Pros of Goss

Cons of Goss

Code Comparison

Convert designs to code with AI

README

Bats: Bash Automated Testing System

Running tests

Test suites

Writing tests

run: Test other commands

load: Share common code

skip: Easily skip tests

setup and teardown: Pre- and post-test hooks

Code outside of test cases

Special variables

Installing Bats from source

Support

Version history

Top Related Projects

Convert designs to code with AI

NPM DownloadsLast 30 Days

`run`: Test other commands

`load`: Share common code

`skip`: Easily skip tests

`setup` and `teardown`: Pre- and post-test hooks