Top Related Projects
ShellCheck, a static analysis tool for shell scripts
Bash Automated Testing System
The corrective bash syntax highlighter
A tool for writing better scripts
📖 A collection of pure POSIX sh alternatives to external processes.
A full-featured BDD unit testing framework for bash, ksh, zsh, dash and all POSIX shells
Quick Overview
mvdan/sh is a shell parser, formatter, and interpreter for POSIX shell and Bash. It provides a Go library to parse and execute shell scripts, as well as command-line tools for formatting and linting shell scripts. The project aims to improve shell script quality and maintainability.
Pros
- Comprehensive shell script parsing and execution capabilities
- Supports both POSIX shell and Bash syntax
- Provides formatting and linting tools for shell scripts
- Written in Go, offering good performance and cross-platform compatibility
Cons
- May not support all advanced Bash features or less common shell variants
- Learning curve for users unfamiliar with Go programming
- Potential discrepancies between mvdan/sh's behavior and actual shell implementations in edge cases
Code Examples
- Parsing a shell script:
import "mvdan.cc/sh/v3/syntax"
src := []byte("echo 'Hello, World!'")
prog, err := syntax.NewParser().Parse(bytes.NewReader(src), "")
if err != nil {
// Handle parsing error
}
- Formatting a shell script:
import "mvdan.cc/sh/v3/syntax"
src := []byte("echo 'Hello, World!'")
formatted, err := syntax.Format(src)
if err != nil {
// Handle formatting error
}
fmt.Println(string(formatted))
- Executing a shell command:
import "mvdan.cc/sh/v3/interp"
runner, err := interp.New(interp.StdIO(os.Stdin, os.Stdout, os.Stderr))
if err != nil {
// Handle runner creation error
}
err = runner.Run(context.Background(), "echo 'Hello, World!'")
if err != nil {
// Handle execution error
}
Getting Started
To use mvdan/sh in your Go project, first install it:
go get mvdan.cc/sh/v3
Then, import and use the desired packages in your Go code:
import (
"mvdan.cc/sh/v3/syntax"
"mvdan.cc/sh/v3/interp"
)
// Use syntax for parsing and formatting
// Use interp for executing shell commands
For command-line tools, install them using:
go install mvdan.cc/sh/v3/cmd/shfmt@latest
go install mvdan.cc/sh/v3/cmd/gosh@latest
Competitor Comparisons
ShellCheck, a static analysis tool for shell scripts
Pros of ShellCheck
- More comprehensive static analysis and error detection for shell scripts
- Extensive rule set covering a wide range of shell scripting issues and best practices
- Integration with many popular editors and CI/CD pipelines
Cons of ShellCheck
- Focused solely on shell script analysis, lacking formatting capabilities
- May produce false positives in some cases, requiring manual review
Code Comparison
ShellCheck example:
# ShellCheck can detect issues like this:
echo $foo
# SC2086: Double quote to prevent globbing and word splitting.
sh example:
# sh focuses on formatting and can rewrite scripts:
echo $foo
# Formatted by sh:
echo "$foo"
Key Differences
- ShellCheck is primarily a linter and static analysis tool, while sh is a shell parser, formatter, and interpreter
- sh offers formatting capabilities, which ShellCheck does not provide
- ShellCheck has a more extensive rule set for detecting shell scripting issues
Use Cases
- Use ShellCheck for comprehensive shell script analysis and error detection
- Use sh for parsing, formatting, and interpreting shell scripts, especially when working with multiple shell dialects
Bash Automated Testing System
Pros of bats-core
- Specifically designed for shell script testing, providing a robust framework for unit tests
- Supports test isolation and setup/teardown functions for more organized test suites
- Offers human-readable output and TAP-compliant results for easy integration with CI/CD pipelines
Cons of bats-core
- Limited to Bash scripting, not suitable for other shell languages
- Requires additional setup and learning curve compared to simpler shell script linting
- May introduce overhead for smaller projects or quick scripts
Code Comparison
bats-core example:
@test "addition using bc" {
result="$(echo 2+2 | bc)"
[ "$result" -eq 4 ]
}
mvdan/sh example:
#!/bin/sh
echo 2+2 | bc
Key Differences
- bats-core focuses on testing shell scripts, while mvdan/sh is primarily a shell parser, formatter, and interpreter
- mvdan/sh supports multiple shell dialects (sh, bash, mksh, bats), whereas bats-core is specific to Bash
- bats-core provides a structured testing framework, while mvdan/sh offers static analysis and formatting capabilities
The corrective bash syntax highlighter
Pros of shellharden
- Focuses specifically on hardening shell scripts for improved safety and reliability
- Provides automatic fixes for common shell scripting pitfalls
- Includes a detailed explanation of shell scripting best practices in its documentation
Cons of shellharden
- More limited in scope compared to sh's broader feature set
- Less actively maintained, with fewer recent updates
- Primarily targets bash, while sh supports multiple shell dialects
Code comparison
shellharden example:
shellharden --replace script.sh
sh example:
shfmt -w script.sh
Summary
shellharden is a specialized tool for improving shell script safety, offering automatic fixes and educational resources. sh, on the other hand, provides a more comprehensive set of features for shell script formatting, parsing, and interpretation across multiple shell dialects. While shellharden excels in its focused approach to script hardening, sh offers broader functionality and more active development. The choice between the two depends on whether you prioritize specific safety improvements or a more versatile shell scripting toolkit.
A tool for writing better scripts
Pros of zx
- Designed for writing shell scripts in JavaScript, offering a more familiar syntax for web developers
- Provides built-in functions for common shell operations, simplifying script writing
- Integrates well with Node.js ecosystem and npm packages
Cons of zx
- Requires Node.js runtime, which may not be available in all environments
- Larger overhead compared to pure shell scripts
- Limited portability across different operating systems
Code Comparison
zx:
#!/usr/bin/env zx
await $`echo "Hello, World!"`
let files = await glob('*.js')
await Promise.all(files.map(f => $`prettier ${f} --write`))
sh:
#!/bin/sh
echo "Hello, World!"
for file in *.js; do
prettier "$file" --write
done
Key Differences
- zx uses JavaScript syntax and async/await for shell operations
- sh follows traditional shell scripting syntax
- zx provides built-in functions like
glob
, while sh relies on shell commands - zx scripts are typically longer due to JavaScript verbosity, but may be more readable for JavaScript developers
- sh scripts are more portable and have lower overhead, but may be less intuitive for those unfamiliar with shell scripting
📖 A collection of pure POSIX sh alternatives to external processes.
Pros of pure-sh-bible
- Comprehensive collection of shell scripting techniques and best practices
- Focuses on POSIX-compliant, portable shell scripting
- Serves as an educational resource for learning shell scripting
Cons of pure-sh-bible
- Not a tool or library, but rather a documentation repository
- Lacks automated testing and validation of code snippets
- May require more effort to implement solutions compared to using a library
Code Comparison
pure-sh-bible example (string manipulation):
trim_string() {
: "${1#"${1%%[![:space:]]*}"}"
: "${_%"${_##*[![:space:]]}"}"
printf '%s\n' "$_"
}
sh example (string manipulation):
func TrimString(s string) string {
return strings.TrimSpace(s)
}
Key Differences
- pure-sh-bible is a documentation project, while sh is a Go library for shell parsing and formatting
- pure-sh-bible focuses on pure shell implementations, while sh provides programmatic tools for working with shell scripts
- sh offers automated testing and continuous integration, which pure-sh-bible lacks
- pure-sh-bible is more suitable for learning and reference, while sh is better for programmatic shell script manipulation
A full-featured BDD unit testing framework for bash, ksh, zsh, dash and all POSIX shells
Pros of shellspec
- Specifically designed for shell script testing, offering a more tailored experience
- Provides a BDD-style syntax for writing tests, which can be more intuitive for some developers
- Includes built-in mocking capabilities for commands and functions
Cons of shellspec
- Limited to shell script testing, whereas sh offers broader shell scripting utilities
- May have a steeper learning curve for those unfamiliar with BDD testing frameworks
- Smaller community and fewer contributors compared to sh
Code comparison
shellspec example:
Describe 'My script'
It 'should perform a specific action'
When call ./my_script.sh
The output should include 'Expected output'
The status should be success
End
End
sh example:
gofmt() {
mv_shell gofmt "$@" || return
# Simplify multi-line string comparisons
shfmt "$@" | diff -u "$@" -
}
Summary
shellspec is a specialized testing framework for shell scripts with a BDD approach, while sh is a more general-purpose shell formatting and parsing tool. shellspec offers a structured testing environment but is limited to testing, whereas sh provides broader functionality for shell script development and maintenance.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
sh
A shell parser, formatter, and interpreter. Supports POSIX Shell, Bash, and mksh. Requires Go 1.22 or later.
Quick start
To parse shell scripts, inspect them, and print them out, see the syntax examples.
For high-level operations like performing shell expansions on strings, see the shell examples.
shfmt
go install mvdan.cc/sh/v3/cmd/shfmt@latest
shfmt
formats shell programs. See canonical.sh for a
quick look at its default style. For example:
shfmt -l -w script.sh
For more information, see its manpage, which can be viewed directly as Markdown or rendered with scdoc.
Packages are available on Alpine, Arch, Debian, Docker, Fedora, FreeBSD, Homebrew, MacPorts, NixOS, OpenSUSE, Scoop, Snapcraft, Void and webi.
gosh
go install mvdan.cc/sh/v3/cmd/gosh@latest
Proof of concept shell that uses interp
. Note that it's not meant to replace a
POSIX shell at the moment, and its options are intentionally minimalistic.
Fuzzing
We use Go's native fuzzing support. For instance:
cd syntax
go test -run=- -fuzz=ParsePrint
Caveats
- When indexing Bash associative arrays, always use quotes. The static parser will otherwise have to assume that the index is an arithmetic expression.
$ echo '${array[spaced string]}' | shfmt
1:16: not a valid arithmetic operator: string
$ echo '${array[dash-string]}' | shfmt
${array[dash - string]}
$((
and((
ambiguity is not supported. Backtracking would complicate the parser and make streaming support viaio.Reader
impossible. The POSIX spec recommends to space the operands if$( (
is meant.
$ echo '$((foo); (bar))' | shfmt
1:1: reached ) without matching $(( with ))
- Some builtins like
export
andlet
are parsed as keywords. This allows statically building their syntax tree, as opposed to keeping the arguments as a slice of words. It is also required to supportdeclare foo=(bar)
. Note that this means expansions likedeclare {a,b}=c
are not supported.
JavaScript
A subset of the Go packages are available as an npm package called mvdan-sh. See the _js directory for more information.
Docker
All release tags are published via Docker, such as v3.5.1
.
The latest stable release is currently published as v3
,
and the latest development version as latest
.
The images only include shfmt
; -alpine
variants exist on Alpine Linux.
To build a Docker image, run:
docker build -t my:tag -f cmd/shfmt/Dockerfile .
To use a Docker image, run:
docker run --rm -u "$(id -u):$(id -g)" -v "$PWD:/mnt" -w /mnt my:tag <shfmt arguments>
Related projects
The following editor integrations wrap shfmt
:
- BashSupport-Pro - Bash plugin for JetBrains IDEs
- intellij-shellcript - Intellij Jetbrains
shell script
plugin - micro - Editor with a built-in plugin
- shell-format - VS Code plugin
- vscode-shfmt - VS Code plugin
- shfmt.el - Emacs package
- Sublime-Pretty-Shell - Sublime Text 3 plugin
- Trunk - Universal linter, available as a CLI, VS Code plugin, and GitHub action
- vim-shfmt - Vim plugin
Other noteworthy integrations include:
- modd - A developer tool that responds to filesystem changes
- prettier-plugin-sh - Prettier plugin using mvdan-sh
- sh-checker - A GitHub Action that performs static analysis for shell scripts
- mdformat-shfmt - mdformat plugin to format shell scripts embedded in Markdown with shfmt
- pre-commit-shfmt - pre-commit shfmt hook
Top Related Projects
ShellCheck, a static analysis tool for shell scripts
Bash Automated Testing System
The corrective bash syntax highlighter
A tool for writing better scripts
📖 A collection of pure POSIX sh alternatives to external processes.
A full-featured BDD unit testing framework for bash, ksh, zsh, dash and all POSIX shells
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot