Top Related Projects
Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
Fast, easy and reliable testing for anything that runs in a browser.
JavaScript API for Chrome and Firefox
Cross-platform automation framework for all kinds of apps, built on top of the W3C WebDriver protocol
Generic automation framework for acceptance testing and RPA
Quick Overview
nut.js is a cross-platform native UI automation library for Node.js. It provides a simple API for simulating user interactions such as mouse movements, clicks, and keyboard input, as well as screen analysis capabilities. nut.js is designed to work on Windows, macOS, and Linux.
Pros
- Cross-platform compatibility (Windows, macOS, Linux)
- Easy-to-use API for automating user interactions
- Supports image recognition and screen analysis
- Active development and community support
Cons
- Performance may vary depending on the complexity of automation tasks
- Limited documentation for advanced use cases
- Potential compatibility issues with certain system configurations
- Requires additional setup for OpenCV functionality
Code Examples
- Moving the mouse and clicking:
const { mouse, left, right, up, down } = require("@nut-tree/nut-js");
(async () => {
await mouse.move(right(500));
await mouse.move(down(500));
await mouse.leftClick();
})();
- Typing text:
const { keyboard } = require("@nut-tree/nut-js");
(async () => {
await keyboard.type("Hello, nut.js!");
})();
- Finding and clicking on an image:
const { screen, mouse } = require("@nut-tree/nut-js");
(async () => {
const target = await screen.find("path/to/image.png");
await mouse.move(target);
await mouse.leftClick();
})();
Getting Started
- Install nut.js:
npm install @nut-tree/nut-js
- Create a new JavaScript file (e.g.,
example.js
) and add the following code:
const { mouse, screen, keyboard } = require("@nut-tree/nut-js");
(async () => {
await mouse.move([100, 100]);
await mouse.leftClick();
await keyboard.type("Hello, nut.js!");
const screenSize = await screen.width();
console.log(`Screen width: ${screenSize}`);
})();
- Run the script:
node example.js
This example moves the mouse, performs a left click, types text, and logs the screen width.
Competitor Comparisons
Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
Pros of Playwright
- Broader browser support (Chromium, Firefox, WebKit)
- More comprehensive API for web automation and testing
- Better documentation and community support
Cons of Playwright
- Focused primarily on web automation, less suitable for general desktop automation
- Steeper learning curve for non-web-specific tasks
- Larger package size and potentially higher resource usage
Code Comparison
Playwright:
const { chromium } = require('playwright');
(async () => {
const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({ path: 'screenshot.png' });
await browser.close();
})();
nut.js:
const { screen, mouse, keyboard } = require("@nut-tree/nut-js");
(async () => {
await mouse.move(screen.width / 2, screen.height / 2);
await mouse.leftClick();
await keyboard.type("Hello, World!");
})();
Playwright excels in web automation with its comprehensive API and multi-browser support. nut.js, on the other hand, is more versatile for desktop automation tasks, offering screen, mouse, and keyboard control. Playwright's code focuses on browser interactions, while nut.js provides lower-level system control. Choose Playwright for web-specific tasks and nut.js for general desktop automation.
Fast, easy and reliable testing for anything that runs in a browser.
Pros of Cypress
- More comprehensive end-to-end testing framework with built-in assertion library
- Extensive documentation and large community support
- Real-time browser testing with time-travel debugging
Cons of Cypress
- Limited to testing web applications only
- Runs only in-browser, limiting certain types of tests
- Steeper learning curve for complex scenarios
Code Comparison
Cypress example:
describe('Login', () => {
it('should log in successfully', () => {
cy.visit('/login')
cy.get('#username').type('user@example.com')
cy.get('#password').type('password123')
cy.get('button[type="submit"]').click()
cy.url().should('include', '/dashboard')
})
})
nut.js example:
import { keyboard, screen } from "@nut-tree/nut-js";
(async () => {
await keyboard.type("Hello, World!");
const screen = await screen.find("submit-button.png");
await screen.click();
})();
nut.js is more focused on native GUI automation across platforms, while Cypress specializes in web application testing. nut.js offers broader system-wide automation capabilities, including keyboard and mouse control, but has a smaller community and less extensive documentation compared to Cypress. The choice between the two depends on the specific testing needs and the type of application being tested.
JavaScript API for Chrome and Firefox
Pros of Puppeteer
- Robust browser automation with full control over Chrome/Chromium
- Extensive API for web scraping, testing, and PDF generation
- Large community and ecosystem with numerous plugins and extensions
Cons of Puppeteer
- Limited to browser-based automation, not suitable for desktop applications
- Heavier resource usage due to running a full browser instance
- Steeper learning curve for non-web developers
Code Comparison
Puppeteer (browser automation):
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({path: 'screenshot.png'});
await browser.close();
Nut.js (desktop automation):
const { mouse, screen } = require("@nut-tree/nut-js");
const target = await screen.find("button.png");
await mouse.move(target);
await mouse.click();
Key Differences
- Puppeteer focuses on web automation, while Nut.js is designed for desktop automation
- Nut.js provides cross-platform support for various operating systems
- Puppeteer offers more advanced web-specific features like network interception and JavaScript execution
Use Cases
Puppeteer:
- Web scraping
- Automated testing of web applications
- Generating PDFs from web pages
Nut.js:
- Desktop application testing
- GUI automation across different operating systems
- Image-based automation tasks
Cross-platform automation framework for all kinds of apps, built on top of the W3C WebDriver protocol
Pros of Appium
- Supports a wide range of mobile platforms (iOS, Android) and web browsers
- Extensive community support and documentation
- Integrates well with popular testing frameworks and CI/CD tools
Cons of Appium
- Steeper learning curve, especially for those new to mobile testing
- Can be slower in execution compared to native automation tools
- Requires separate setup and configuration for different platforms
Code Comparison
Appium (JavaScript):
const driver = await wdio.remote(opts);
await driver.setImplicitTimeout(5000);
await driver.$('~app-icon').click();
await driver.deleteSession();
nut.js (TypeScript):
const mouse = new Mouse();
const screen = new Screen();
await mouse.move(await screen.find("app-icon.png"));
await mouse.leftClick();
Key Differences
- Appium focuses on mobile and web automation, while nut.js is primarily for desktop automation
- Appium uses WebDriver protocol, nut.js uses native OS interactions
- nut.js has a simpler API for common desktop tasks like mouse movements and keyboard input
- Appium requires a server to be running, while nut.js can be used directly in Node.js applications
Both tools have their strengths, with Appium excelling in mobile testing and nut.js offering a straightforward approach to desktop automation.
Generic automation framework for acceptance testing and RPA
Pros of Robot Framework
- Extensive ecosystem with numerous built-in libraries and third-party extensions
- Keyword-driven approach allows for easy-to-read test cases, even for non-technical users
- Strong support for data-driven testing and test case reusability
Cons of Robot Framework
- Steeper learning curve, especially for those new to keyword-driven testing
- Less flexibility for complex programming tasks compared to pure Python solutions
- Can be slower in execution compared to lightweight automation tools
Code Comparison
Robot Framework:
*** Test Cases ***
Valid Login
Open Browser ${LOGIN URL} ${BROWSER}
Input Text username demo
Input Password password mode
Submit Form
Page Should Contain Welcome
[Teardown] Close Browser
nut.js:
const { keyboard, screen } = require("@nut-tree/nut-js");
(async () => {
await keyboard.type("demo");
await keyboard.type(Key.Tab);
await keyboard.type("mode");
await keyboard.type(Key.Enter);
await screen.waitFor("welcome.png");
})();
Both frameworks offer automation capabilities, but Robot Framework provides a more structured, keyword-driven approach suitable for larger test suites, while nut.js offers a more programmatic, JavaScript-based solution for desktop automation tasks.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
nut.js (Native UI Toolkit)
GitHub Actions | |
---|---|
Master | |
Develop |
Please visit
nutjs.dev
for detailed documentation and tutorials
Most importantly,
please read this
About
nut.js
is a cross-platform native UI automation / testing tool.
It allows for native UI interactions via keyboard and / or mouse, but additionally gives you the possibility to navigate the screen based on image matching.
The Price of Open Source
If you came here after I removed public packages from npm as announced in 'the blog post', please be aware that
- nut.js is still open source (you are right here, aren't you?)
- nut.js is still free to use, you'll just have to build it from sources
- nut.js is still maintained and developed further
- nut.js does not force anyone to pay anything, because you can even build every single plugin yourself. It's just interfaces to implement
nut.js
is developed with community in mind.
A huge "Thank you!" goes out to all sponsors who make open source a bit more sustainable!
Demo
Check out this demo video to get a first impression of what nut.js is capable of.
Tutorials
Please consult the project website at nutjs.dev for in-depth tutorials
API Docs
nut.js provides public API documentation auto-generated by TypeDoc.
Community
Feel free to join our Discord community
Modules
This list gives an overview on currently implemented and planned functionality. It's work in progress and will undergo constant modification.
Clipboard
- Copy text to clipboard
- Paste text from clipboard
Keyboard
- Support for standard US keyboard layout
- Support for multimedia keys
Mouse
- Support for mouse movement
- Support for mouse scroll
- Configurable movement speed
- Mouse drag
Window
- List all windows
- Retrieve active window
- Retrieve window title
- Retrieve window size and position
- Focus window
- Resize window
- Reposition window
- Minimize a window (*)
- Restore a window (*)
- Inspect GUI elements of a window (*)
- Search for specific GUI elements of a window (*)
Screen
- Retrieve RGBA color information on screen
- Highlighting screen regions
- Find a single or multiple occurrences of an image on screen (requires an additional provider package like e.g. nut-tree/template-matcher)
- Wait for an image to appear on screen (requires an additional provider package like e.g. nut-tree/template-matcher)
- Find a single or multiple occurrences of text on screen (*)
- Wait for a piece of text to appear on screen (*)
- Find a single or multiple windows on screen (*)
- Wait for a window to appear on screen (*)
- Hooks to trigger actions based on detected text, images or windows (*)
(*) Requires an additional provider package, visit nutjs.dev for more info
Integration
- Jest
- Electron
- Custom log integration
Sample
The following snippet shows a valid nut.js
example (using multiple addons):
"use strict";
const {
mouse,
screen,
singleWord,
sleep,
useConsoleLogger,
ConsoleLogLevel,
straightTo,
centerOf,
Button,
getActiveWindow,
} = require("@nut-tree/nut-js");
const {
preloadLanguages,
Language,
LanguageModelType,
configure,
} = require("@nut-tree/plugin-ocr");
configure({ languageModelType: LanguageModelType.BEST });
useConsoleLogger({ logLevel: ConsoleLogLevel.DEBUG });
screen.config.autoHighlight = true;
screen.config.ocrConfidence = 0.8;
function activeWindowRegion() {
return getActiveWindow().then((activeWindow) => activeWindow.region);
}
(async () => {
await preloadLanguages([Language.English], [LanguageModelType.BEST]);
await sleep(5000);
const result = await screen.find(singleWord("@nut-tree/nut-js"));
await mouse.move(straightTo(centerOf(result)));
await mouse.click(Button.LEFT);
await screen.waitFor(singleWord("Native"), 15000, 1000, {
providerData: { partialMatch: true },
});
const content = await screen.read({ searchRegion: activeWindowRegion() });
console.log(content);
})();
Installation
Prerequisites
This section lists runtime requirements for nut.js
on the respective target platform.
Windows
In case you're running Windows 10 N and want to use ImageFinder plugins, please make sure to have the Media Feature Pack installed.
macOS
On macOS, Xcode command line tools are required. You can install them by running
xcode-select --install
Permissions:
nut.js requires the executing application, e.g. your terminal, to be given both Accessibility
and Screen Recording
permissions.
Starting with release 2.3.0
, nut.js will check for and request these permissions automatically:
It will also give you a subtle hint in case permissions are lacking:
Accessibility: ##### WARNING! The application running this script is not a trusted process! Please visit https://github.com/nut-tree/nut.js#macos #####
- Screen
Recording:
##### WARNING! The application running this script is not allowed to capture screen content! Please visit https://github.com/nut-tree/nut.js#macos #####
Attention:
Prior to release 2.3.0
you'll have to grant these permissions manually.
Settings -> Security & Privacy -> Privacy tab -> Accessibility -> Add...
For example, if you want to execute your node script in e.g. iTerm2
, you'd have to add iTerm.app
to the list.
When running your script from a built-in terminal in e.g. VSCode
or IntelliJ
, you'd have to add the respective IDE.
Linux
Depending on your distribution, Linux setups may differ.
In general, nut.js
requires
- libXtst
Installation on *buntu
distributions:
sudo apt-get install libxtst-dev
Setups on other distributions might differ.
Attention:
At the moment nut.js only supports X11.
Wayland is NOT supported!
On e.g. Ubuntu you can switch to XWayland on your login screen as a workaround.
Install nut.js
Open Source
The core functionality of nut.js
is open source and available on GitHub.
To build nut.js from source you'll have to build native dependencies first.
- Start with @nut-tree/libnut-core
- A build pipeline can be found in the respective repository
- Update dependencies in
nut.js
to point to your local build oflibnut-core
- A build pipeline can be found in the respective repository
Pre-built packages
Pre-built packages are available for subscription plans.
Check out the pricing page for more information.
Once you subscribed to a plan, you'll receive a token which you can use to install the respective package, check out the registry access tutorial for reference.
With everything set up, running
npm i @nut-tree/nut-js
or
yarn add @nut-tree/nut-js
will install nut.js
and its required dependencies.
Snapshot releases
nut.js
also provides snapshot releases which allows to test upcoming features.
Running
npm i @nut-tree/nut-js@next
or
yarn add @nut-tree/nut-js@next
will install the most recent development release of nut.js
.
Attention: While snapshot releases are great to work with upcoming features before a new stable release, it is still a snapshot release. Please bear in mind that things might change and / or break on snapshot releases, so it is not recommended using them in production.
Top Related Projects
Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
Fast, easy and reliable testing for anything that runs in a browser.
JavaScript API for Chrome and Firefox
Cross-platform automation framework for all kinds of apps, built on top of the W3C WebDriver protocol
Generic automation framework for acceptance testing and RPA
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot