Top Related Projects
A programmer-oriented testing framework for Java.
Most popular Mocking framework for unit tests written in Java
TestNG testing framework
The Enterprise-ready testing and specification framework.
Cucumber for the JVM
AssertJ is a library providing easy to use rich typed assertions
Quick Overview
TestNG is a testing framework for Java inspired by JUnit and NUnit. It aims to cover a wider range of test categories including unit, functional, end-to-end, and integration testing, with powerful features for test configuration, data-driven testing, and parallel execution.
Pros
- Supports parallel execution of tests, improving test suite performance
- Offers flexible test configuration through annotations and XML files
- Provides built-in support for data-driven testing and parameterization
- Integrates well with various tools and IDEs
Cons
- Steeper learning curve compared to simpler frameworks like JUnit
- XML configuration can be verbose and complex for large test suites
- Less widespread adoption compared to JUnit in the Java ecosystem
- Some features may be overkill for simple testing scenarios
Code Examples
- Basic test method:
import org.testng.annotations.Test;
import static org.testng.Assert.assertEquals;
public class SimpleTest {
@Test
public void testAddition() {
assertEquals(2 + 2, 4);
}
}
- Parameterized test:
import org.testng.annotations.Parameters;
import org.testng.annotations.Test;
import static org.testng.Assert.assertEquals;
public class ParameterizedTest {
@Parameters({"a", "b", "expected"})
@Test
public void testAddition(int a, int b, int expected) {
assertEquals(a + b, expected);
}
}
- Data provider example:
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
import static org.testng.Assert.assertEquals;
public class DataProviderTest {
@DataProvider(name = "additionData")
public Object[][] additionData() {
return new Object[][] {{1, 1, 2}, {2, 3, 5}, {5, 5, 10}};
}
@Test(dataProvider = "additionData")
public void testAddition(int a, int b, int expected) {
assertEquals(a + b, expected);
}
}
Getting Started
- Add TestNG dependency to your project (Maven example):
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.7.1</version>
<scope>test</scope>
</dependency>
- Create a test class with TestNG annotations:
import org.testng.annotations.Test;
import static org.testng.Assert.assertTrue;
public class MyFirstTest {
@Test
public void testExample() {
assertTrue(true);
}
}
- Run the test using TestNG in your IDE or build tool.
Competitor Comparisons
A programmer-oriented testing framework for Java.
Pros of JUnit4
- Simpler and more straightforward API, making it easier for beginners to learn and use
- Better integration with IDEs and build tools due to its widespread adoption
- Extensive community support and a large ecosystem of extensions
Cons of JUnit4
- Limited support for parallel test execution compared to TestNG
- Less flexible test configuration options, especially for complex test suites
- Lack of built-in support for data-driven testing and test dependencies
Code Comparison
TestNG example:
@Test(groups = {"fast"})
public void testMethod1() {
// Test code
}
@Test(groups = {"slow"}, dependsOnMethods = {"testMethod1"})
public void testMethod2() {
// Test code
}
JUnit4 example:
@Test
public void testMethod1() {
// Test code
}
@Test
public void testMethod2() {
// Test code
}
The TestNG example demonstrates group-based test organization and method dependencies, which are not natively supported in JUnit4. JUnit4's simpler approach may be sufficient for basic testing needs, but TestNG offers more advanced features for complex test scenarios.
Most popular Mocking framework for unit tests written in Java
Pros of Mockito
- More focused on mocking and stubbing, providing a cleaner API for creating test doubles
- Easier to use and learn, with a more intuitive syntax for creating mocks
- Better integration with modern Java features and frameworks
Cons of Mockito
- Limited to mocking and stubbing, lacking TestNG's broader test management capabilities
- Doesn't provide built-in parallel execution or test grouping features
- May require additional libraries for more complex testing scenarios
Code Comparison
TestNG example:
@Test(groups = {"unit"})
public void testMethod() {
Assert.assertEquals(actualValue, expectedValue);
}
Mockito example:
@Test
public void testMethod() {
when(mockObject.someMethod()).thenReturn(expectedValue);
assertEquals(actualValue, mockObject.someMethod());
}
Summary
TestNG is a comprehensive testing framework with broader capabilities, while Mockito specializes in mocking and stubbing. TestNG offers more extensive test management features, including test grouping and parallel execution. Mockito, on the other hand, provides a more focused and user-friendly approach to creating test doubles. The choice between the two depends on the specific testing needs of the project, with many developers opting to use both in conjunction for a more robust testing strategy.
TestNG testing framework
Pros of TestNG
- More active development and maintenance
- Larger community and contributor base
- Better documentation and support resources
Cons of TestNG
- Potentially more complex setup and configuration
- May have more features than necessary for simpler projects
- Steeper learning curve for beginners
Code Comparison
TestNG:
@Test(groups = {"fast"})
public void testMethod() {
// Test logic here
}
TestNG>:
@Test
public void testMethod() {
// Test logic here
}
The main difference in the code examples is that TestNG offers more advanced features like test grouping, which is not present in the simplified TestNG> version. This showcases the additional functionality available in TestNG, but also highlights the potential complexity for simpler use cases.
Both repositories aim to provide a testing framework for Java, but TestNG offers a more comprehensive set of features and better ongoing support. TestNG> appears to be a simplified or alternative version, which may be beneficial for projects requiring a more straightforward testing approach. However, the limited information available about TestNG> makes it difficult to provide a more detailed comparison.
The Enterprise-ready testing and specification framework.
Pros of Spock
- More expressive and readable test syntax using Groovy
- Built-in mocking and stubbing capabilities
- Powerful data-driven testing features
Cons of Spock
- Steeper learning curve for developers unfamiliar with Groovy
- Smaller community and ecosystem compared to TestNG
Code Comparison
TestNG:
@Test
public void testAddition() {
int result = calculator.add(2, 3);
Assert.assertEquals(result, 5);
}
Spock:
def "adding two numbers"() {
expect:
calculator.add(2, 3) == 5
}
Key Differences
- Spock uses Groovy, while TestNG uses Java
- Spock offers a more descriptive and readable test structure
- TestNG has a larger user base and more extensive documentation
- Spock provides built-in mocking, whereas TestNG often requires additional libraries
Use Cases
- TestNG: Ideal for large-scale Java projects with complex test scenarios
- Spock: Well-suited for projects seeking more expressive and concise test code
Community and Support
- TestNG: Larger community, more third-party integrations
- Spock: Growing community, active development, but fewer resources available
Learning Curve
- TestNG: Easier for Java developers to adopt
- Spock: Requires learning Groovy, but offers a more intuitive testing approach
Cucumber for the JVM
Pros of Cucumber JVM
- Supports Behavior-Driven Development (BDD) with Gherkin syntax
- Enables non-technical stakeholders to understand and contribute to test scenarios
- Provides better test documentation and readability
Cons of Cucumber JVM
- Steeper learning curve for writing and maintaining step definitions
- Can be slower to execute compared to TestNG due to additional parsing of feature files
- Requires more setup and configuration for complex test scenarios
Code Comparison
TestNG:
@Test
public void testLogin() {
loginPage.enterUsername("user");
loginPage.enterPassword("pass");
loginPage.clickLogin();
Assert.assertTrue(homePage.isLoggedIn());
}
Cucumber JVM:
Scenario: User logs in successfully
Given I am on the login page
When I enter username "user" and password "pass"
And I click the login button
Then I should be logged in
@Given("I am on the login page")
public void iAmOnTheLoginPage() {
// Implementation
}
Both TestNG and Cucumber JVM are popular testing frameworks for Java, but they serve different purposes. TestNG is more focused on unit and integration testing, while Cucumber JVM excels in BDD and acceptance testing. The choice between them depends on the project requirements, team preferences, and the desired level of collaboration with non-technical stakeholders.
AssertJ is a library providing easy to use rich typed assertions
Pros of AssertJ
- Fluent and expressive assertion syntax, making tests more readable
- Rich set of assertions for various data types and collections
- Supports custom assertions for domain-specific objects
Cons of AssertJ
- Steeper learning curve for developers new to fluent assertion libraries
- Limited to assertion functionality, not a full testing framework like TestNG
Code Comparison
TestNG assertion:
Assert.assertEquals(actual, expected, "Error message");
Assert.assertTrue(condition, "Error message");
AssertJ assertion:
assertThat(actual).isEqualTo(expected).as("Error message");
assertThat(condition).isTrue().as("Error message");
Key Differences
- AssertJ focuses solely on assertions, while TestNG is a complete testing framework
- AssertJ provides a more expressive and chainable API for assertions
- TestNG offers broader testing capabilities, including test configuration, parallel execution, and reporting
Use Cases
- Use AssertJ when you need powerful, readable assertions in your tests
- Choose TestNG for a comprehensive testing framework with built-in test management features
Both libraries can be used together in a project, combining TestNG's testing framework capabilities with AssertJ's expressive assertions for improved test readability and maintainability.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Documentation available at TestNG's main web site. Visit TestNG Documentation's GitHub Repo to contribute to it.
Release Notes
Need help?
Before opening a new issue, did you ask your question on
If you posted on both sites, please provide the link to the other question to avoid duplicating the answer.
Are you sure it is a TestNG bug?
Before posting the issue, try to reproduce the issue in a shell window.
If the problem does not exist with the shell, first check if the issue exists on the bugtracker of the runner, and open an issue there first:
- Eclipse :Issues Page
- IntelliJ:Issues Page
- Maven :Issues Page
- Gradle :Issues Page
Which version are you using?
Always make sure your issue is happening on the latest TestNG version. Bug reports occurring on older versions will not be looked at quickly.
Have you considered sending a pull request instead of filing an issue?
The best way to report a bug is to provide the TestNG team with a full test case reproducing the issue.
Maybe you can write a runnable test case (check the src/test/
folder for examples) and propose it in a pull request
Don't worry if the CI fails because it is the expected behavior.
This pull request will be a perfect start to find the fix :)
How to create a pull request?
Refer our Contributing section for detailed set of steps.
We encourage pull requests that:
- Add new features to TestNG (or)
- Fix bugs in TestNG
If your pull request involves fixing SonarQube issues then we would suggest that you please discuss this with the TestNG-dev before you spend time working on it.
GPG Keys
Getting the keys
Download the keys as shown below:
gpg --keyserver keyserver.ubuntu.com --recv-keys 0F13D5631D6AF36D
gpg: key 0F13D5631D6AF36D: "Krishnan Mahadevan (krmahadevan-key) <krishnan.mahadevan1978@gmail.com>" not changed
gpg: Total number processed: 1
gpg: unchanged: 1
Trusting the keys
Trust the keys as shown below:
gpg --edit-key 0F13D5631D6AF36D
gpg (GnuPG) 2.4.4; Copyright (C) 2024 g10 Code GmbH
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Secret key is available.
sec rsa2048/0F13D5631D6AF36D
created: 2016-12-01 expires: never usage: SC
trust: full validity: unknown
ssb rsa2048/7295B61CC8DD9AE8
created: 2016-12-01 expires: never usage: E
[ unknown] (1). Krishnan Mahadevan (krmahadevan-key) <krishnan.mahadevan1978@gmail.com>
gpg> trust
sec rsa2048/0F13D5631D6AF36D
created: 2016-12-01 expires: never usage: SC
trust: full validity: unknown
ssb rsa2048/7295B61CC8DD9AE8
created: 2016-12-01 expires: never usage: E
[ unknown] (1). Krishnan Mahadevan (krmahadevan-key) <krishnan.mahadevan1978@gmail.com>
Please decide how far you trust this user to correctly verify other users' keys
(by looking at passports, checking fingerprints from different sources, etc.)
1 = I don't know or won't say
2 = I do NOT trust
3 = I trust marginally
4 = I trust fully
5 = I trust ultimately
m = back to the main menu
Your decision? 5
Do you really want to set this key to ultimate trust? (y/N) y
sec rsa2048/0F13D5631D6AF36D
created: 2016-12-01 expires: never usage: SC
trust: ultimate validity: unknown
ssb rsa2048/7295B61CC8DD9AE8
created: 2016-12-01 expires: never usage: E
[ unknown] (1). Krishnan Mahadevan (krmahadevan-key) <krishnan.mahadevan1978@gmail.com>
Please note that the shown key validity is not necessarily correct
unless you restart the program.
gpg> exit
Invalid command (try "help")
gpg> quit
Verifying the signature
- Download the
.asc
file fromhttps://repo1.maven.org/maven2/org/testng/testng/<versionGoesHere>
- Run the command
gpg --verify testng-<versionGoesHere>.jar.asc testng-<versionGoesHere>.jar
- You should see an output as below:
gpg: Signature made Tue Dec 26 15:06:16 2023 IST
gpg: using RSA key 0F13D5631D6AF36D
gpg: checking the trustdb
gpg: marginals needed: 3 completes needed: 1 trust model: pgp
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: Good signature from "Krishnan Mahadevan (krmahadevan-key) <krishnan.mahadevan1978@gmail.com>" [ultimate]
For more details regarding keys please refer:
Top Related Projects
A programmer-oriented testing framework for Java.
Most popular Mocking framework for unit tests written in Java
TestNG testing framework
The Enterprise-ready testing and specification framework.
Cucumber for the JVM
AssertJ is a library providing easy to use rich typed assertions
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot