Top Related Projects
Command-line JSON processor
yq is a portable command-line YAML, JSON, XML, CSV, TOML and properties processor
Select, put and delete data from JSON, TOML, YAML, XML and CSV files with a single tool. Supports conversion between formats and can be used as a Go package.
Pure Go implementation of jq
A simple Go package to Query over JSON/YAML/XML/CSV Data
Quick Overview
yq is a command-line YAML, XML, TOML, JSON, and CSV processor. It uses jq-like syntax but works with YAML, XML, TOML, and CSV files as well as JSON. yq allows you to slice, filter, and manipulate structured data with ease.
Pros
- Supports multiple data formats (YAML, XML, TOML, JSON, CSV)
- Uses familiar jq-like syntax for querying and manipulating data
- Can convert between different data formats
- Lightweight and easy to install
Cons
- Learning curve for those unfamiliar with jq syntax
- May have performance limitations with very large files
- Some advanced features of jq are not fully implemented
Code Examples
- Reading a specific field from a YAML file:
echo 'foo: bar' | yq '.foo'
# Output: bar
- Converting JSON to YAML:
echo '{"name": "John", "age": 30}' | yq -y '.'
# Output:
# name: John
# age: 30
- Filtering and transforming data:
echo '[{"name": "Alice", "age": 25}, {"name": "Bob", "age": 30}]' | yq '[.[] | select(.age > 28) | .name]'
# Output: ["Bob"]
- Working with XML:
echo '<root><item><name>Test</name><value>123</value></item></root>' | yq -p=xml '.root.item.name'
# Output: "Test"
Getting Started
To install yq, you can use pip:
pip install yq
For basic usage, you can pipe data into yq or use it with files:
echo '{"name": "John", "age": 30}' | yq '.name'
# Or
yq '.name' file.json
# For YAML files
yq '.users[0].name' file.yaml
# For XML files
yq -p=xml '.root.element' file.xml
For more advanced usage and options, refer to the project's documentation on GitHub.
Competitor Comparisons
Command-line JSON processor
Pros of jq
- Native command-line tool with faster execution
- More extensive and powerful JSON processing capabilities
- Larger community and ecosystem with better documentation
Cons of jq
- Steeper learning curve due to its unique syntax
- Limited to JSON processing only
Code Comparison
jq example:
echo '{"name": "John", "age": 30}' | jq '.name'
yq example:
echo '{"name": "John", "age": 30}' | yq '.name'
Summary
jq is a powerful, native JSON processor with a dedicated syntax, while yq is a Python-based tool that extends jq's functionality to YAML and XML. jq offers better performance and more advanced JSON manipulation features, but yq provides a more versatile approach for handling multiple data formats.
yq builds upon jq's syntax, making it easier for users familiar with jq to work with YAML and XML. However, jq's native implementation and focus on JSON processing result in faster execution times and a more extensive set of JSON-specific features.
Choose jq for pure JSON processing tasks and when performance is crucial. Opt for yq when working with multiple data formats or if you prefer a Python-based tool with a gentler learning curve.
yq is a portable command-line YAML, JSON, XML, CSV, TOML and properties processor
Pros of yq (mikefarah)
- Written in Go, offering better performance and easier installation
- More comprehensive YAML manipulation features, including in-place editing
- Supports multiple input formats (JSON, XML, TOML) and conversion between them
Cons of yq (mikefarah)
- Syntax differs from jq, requiring a learning curve for jq users
- Less integration with Python ecosystems compared to kislyuk/yq
Code Comparison
mikefarah/yq:
yq e '.foo.bar = "baz"' input.yaml
kislyuk/yq:
yq '.foo.bar = "baz"' input.yaml
Key Differences
- mikefarah/yq is a standalone tool, while kislyuk/yq is a Python wrapper for jq
- mikefarah/yq has a more YAML-centric approach, whereas kislyuk/yq leverages jq's JSON manipulation capabilities
- mikefarah/yq offers more advanced YAML-specific features, but kislyuk/yq provides better integration with Python projects
Use Cases
- mikefarah/yq: Ideal for complex YAML manipulation tasks and cross-format conversions
- kislyuk/yq: Better suited for Python-based projects and users familiar with jq syntax
Both tools have their strengths, and the choice depends on specific project requirements and user preferences.
Select, put and delete data from JSON, TOML, YAML, XML and CSV files with a single tool. Supports conversion between formats and can be used as a Go package.
Pros of dasel
- Supports multiple data formats (JSON, YAML, TOML, XML, CSV)
- Can perform in-place updates to files
- Offers a more extensive command-line interface with various subcommands
Cons of dasel
- Less focused on YAML processing compared to yq
- May have a steeper learning curve due to more complex syntax
- Slower performance for large datasets compared to yq
Code Comparison
dasel:
dasel put string -f data.yaml '.name' 'John Doe'
dasel select -f data.json '.users.[].name'
yq:
yq e '.name = "John Doe"' -i data.yaml
yq e '.users[].name' data.json
Both tools offer similar functionality for basic operations, but their syntax and approach differ. dasel uses subcommands and flags, while yq employs a more concise expression-based syntax. yq is primarily designed for YAML processing, whereas dasel aims to be a multi-format data manipulation tool. The choice between them depends on specific use cases and personal preferences.
Pure Go implementation of jq
Pros of gojq
- Written in Go, offering better performance and cross-platform compatibility
- Standalone binary with no dependencies, easier to deploy and distribute
- More closely follows jq syntax and behavior, providing better compatibility
Cons of gojq
- Lacks some advanced features present in yq, such as XML support
- May have a steeper learning curve for users not familiar with jq syntax
- Limited integration with other programming languages compared to yq
Code Comparison
gojq:
gojq.Run(".foo.bar")
yq:
yq.cli.main(["-y", ".foo.bar"])
Key Differences
- gojq is a Go implementation of jq, while yq is a Python wrapper around jq
- yq provides additional functionality for YAML and XML processing
- gojq focuses on jq-like JSON processing with better performance
- yq offers easier integration with Python projects and scripts
Use Cases
- Choose gojq for high-performance JSON processing in Go projects or as a standalone CLI tool
- Opt for yq when working with YAML, XML, or integrating with Python ecosystems
Community and Maintenance
- Both projects are actively maintained and have good community support
- gojq has a larger number of contributors and stars on GitHub
- yq has been around longer and has a more established user base
A simple Go package to Query over JSON/YAML/XML/CSV Data
Pros of gojsonq
- Written in Go, offering better performance for large JSON datasets
- Supports chaining multiple query operations for complex data manipulation
- Provides a wider range of query functions and operators
Cons of gojsonq
- Limited to JSON data only, while yq supports YAML, XML, and other formats
- Lacks command-line interface, primarily used as a library in Go applications
- Less extensive documentation and community support compared to yq
Code Comparison
gojsonq example:
result := gojsonq.New().File("data.json").
From("users").
Where("age", ">", 30).
Select("name", "email").
Get()
yq example:
yq '.users[] | select(.age > 30) | {name, email}' data.json
Both tools allow querying and manipulating JSON data, but gojsonq is designed for use within Go applications, while yq provides a command-line interface for various data formats. gojsonq offers more advanced querying capabilities, while yq excels in simplicity and multi-format support.
Convert 
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
yq: Command-line YAML/XML/TOML processor - jq wrapper for YAML, XML, TOML documents
Installation
::
pip install yq
Before using yq, you also have to install its dependency, jq. See the jq installation instructions <https://stedolan.github.io/jq/download/>_ for details and directions specific to your platform.
On macOS, yq is also available on Homebrew <https://brew.sh/>_: use brew install python-yq.
Synopsis
yq takes YAML input, converts it to JSON, and pipes it to jq <https://stedolan.github.io/jq/>_::
cat input.yml | yq .foo.bar
Like in jq, you can also specify input filename(s) as arguments::
yq .foo.bar input.yml
By default, no conversion of jq output is done. Use the --yaml-output/-y option to convert it back into YAML::
cat input.yml | yq -y .foo.bar
Mapping key order is preserved. By default, custom YAML tags <http://www.yaml.org/spec/1.2/spec.html#id2764295>_ and
styles <https://yaml.org/spec/current.html#id2509255>_ in the input are ignored. Use the --yaml-roundtrip/-Y
option to preserve YAML tags and styles by representing them as extra items in their enclosing mappings and sequences
while in JSON::
yq -Y .foo.bar input.yml
yq can be called as a module if needed. With -y/-Y, files can be edited in place like with sed -i::
python -m yq -Y --indentless --in-place '.["current-context"] = "staging-cluster"' ~/.kube/config
Use the --width/-w option to pass the line wrap width for string literals. Use
--explicit-start/--explicit-end to emit YAML start/end markers even when processing a single document. All other
command line arguments are forwarded to jq. yq forwards the exit code jq produced, unless there was an error
in YAML parsing, in which case the exit code is 1. See the jq manual <https://stedolan.github.io/jq/manual/>_ for more
details on jq features and options.
Because YAML treats JSON as a dialect of YAML, you can use yq to convert JSON to YAML: yq -y . < in.json > out.yml.
Preserving tags and styles using the -Y (--yaml-roundtrip) option
The ``-Y`` option helps preserve custom `string styles <https://yaml-multiline.info/>`_ and
`tags <https://camel.readthedocs.io/en/latest/yamlref.html#tags>`_ in your document. For example, consider the following
document (an `AWS CloudFormation <https://aws.amazon.com/cloudformation/>`_ template fragment)::
Resources:
ElasticLoadBalancer:
Type: 'AWS::ElasticLoadBalancing::LoadBalancer'
Properties:
AvailabilityZones: !GetAZs ''
Instances:
- !Ref Ec2Instance1
- !Ref Ec2Instance2
Description: >-
Load balancer for Big Important Service.
Good thing it's managed by this template.
Passing this document through ``yq -y .Resources.ElasticLoadBalancer`` will drop custom tags, such as ``!Ref``,
and styles, such as the `folded <https://yaml-multiline.info/>`_ style of the ``Description`` field::
Type: AWS::ElasticLoadBalancing::LoadBalancer
Properties:
AvailabilityZones: ''
Instances:
- Ec2Instance1
- Ec2Instance2
Description: 'Load balancer for Big Important Service.
Good thing it''s managed by this template.'
By contrast, passing it through ``yq -Y .Resources.ElasticLoadBalancer`` will preserve tags and styles::
Type: 'AWS::ElasticLoadBalancing::LoadBalancer'
Properties:
AvailabilityZones: !GetAZs ''
Instances:
- !Ref 'Ec2Instance1'
- !Ref 'Ec2Instance2'
Description: >-
Load balancer for Big Important Service.
Good thing it's managed by this template.
To accomplish this in ``-Y`` mode, yq carries extra metadata (mapping pairs and sequence values) in the JSON
representation of your document for any custom tags or styles that it finds. When converting the JSON back into YAML, it
parses this metadata, re-applies the tags and styles, and discards the extra pairs and values.
.. warning ::
The ``-Y`` option is incompatible with jq filters that do not expect the extra information injected into the document
to preserve the YAML formatting. For example, a jq filter that counts entries in the Instances array will come up with
4 entries instead of 2. A filter that expects all array entries to be mappings may break due to the presence of string
metadata keys. Check your jq filter for compatibility/semantic validity when using the ``-Y`` option.
yq does not support passing YAML comments into the JSON representation used by jq, or roundtripping such comments.
XML support
-----------
``yq`` also supports XML. The ``yq`` package installs an executable, ``xq``, which
`transcodes XML to JSON <https://www.xml.com/pub/a/2006/05/31/converting-between-xml-and-json.html>`_ using
`xmltodict <https://github.com/martinblech/xmltodict>`_ and pipes it to ``jq``. Roundtrip transcoding is available with
the ``xq --xml-output``/``xq -x`` option. Multiple XML documents can be passed in separate files/streams as
``xq a.xml b.xml``. Use ``--xml-item-depth`` to descend into large documents, streaming their contents without loading
the full doc into memory (for example, stream a `Wikipedia database dump <https://dumps.wikimedia.org>`_ with
``cat enwiki-*.xml.bz2 | bunzip2 | xq . --xml-item-depth=2``). Entity expansion and DTD resolution is disabled to avoid
XML parsing vulnerabilities. Use ``python -m yq.xq`` if you want to ensure a specific Python runtime.
TOML support
------------
``yq`` supports `TOML <https://toml.io/>`_ as well. The ``yq`` package installs an executable, ``tomlq``, which uses the
`tomlkit library <https://github.com/sdispater/tomlkit>`_ to transcode TOML to JSON, then pipes it to ``jq``. Roundtrip
transcoding is available with the ``tomlq --toml-output``/``tomlq -t`` option. Use ``python -m yq.tomlq`` if you want to
ensure a specific Python runtime.
.. admonition:: Compatibility note
This package's release series available on PyPI begins with version 2.0.0. Versions of ``yq`` prior to 2.0.0 are
distributed by https://github.com/abesto/yq and are not related to this package. No guarantees of compatibility are
made between abesto/yq and kislyuk/yq. This package follows the `Semantic Versioning 2.0.0 <http://semver.org/>`_
standard. To ensure proper operation, declare dependency version ranges according to SemVer.
Authors
-------
* Andrey Kislyuk
Links
-----
* `Project home page (GitHub) <https://github.com/kislyuk/yq>`_
* `Documentation <https://kislyuk.github.io/yq/>`_
* `Package distribution (PyPI) <https://pypi.python.org/pypi/yq>`_
* `Change log <https://github.com/kislyuk/yq/blob/master/Changes.rst>`_
* `jq <https://stedolan.github.io/jq/>`_ - the command-line JSON processor utility powering ``yq``
Bugs
~~~~
Please report bugs, issues, feature requests, etc. on `GitHub <https://github.com/kislyuk/yq/issues>`_.
License
-------
Licensed under the terms of the `Apache License, Version 2.0 <http://www.apache.org/licenses/LICENSE-2.0>`_.
.. image:: https://github.com/kislyuk/yq/workflows/Python%20package/badge.svg
:target: https://github.com/kislyuk/yq/actions
.. image:: https://codecov.io/github/kislyuk/yq/coverage.svg?branch=master
:target: https://codecov.io/github/kislyuk/yq?branch=master
.. image:: https://img.shields.io/pypi/v/yq.svg
:target: https://pypi.python.org/pypi/yq
.. image:: https://img.shields.io/pypi/l/yq.svg
:target: https://pypi.python.org/pypi/yq
Top Related Projects
Command-line JSON processor
yq is a portable command-line YAML, JSON, XML, CSV, TOML and properties processor
Select, put and delete data from JSON, TOML, YAML, XML and CSV files with a single tool. Supports conversion between formats and can be used as a Go package.
Pure Go implementation of jq
A simple Go package to Query over JSON/YAML/XML/CSV Data
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot