Top Related Projects
PostgreSQL client for node.js.
A query builder for PostgreSQL, MySQL, CockroachDB, SQL Server, SQLite3 and Oracle, designed to be flexible, portable, and fun to use.
Feature-rich ORM for modern Node.js and TypeScript, it supports PostgreSQL (with JSON and JSONB support), MySQL, MariaDB, SQLite, MS SQL Server, Snowflake, Oracle DB (v6), DB2 and DB2 for IBM i.
ORM for TypeScript and JavaScript. Supports MySQL, PostgreSQL, MariaDB, SQLite, MS SQL Server, Oracle, SAP Hana, WebSQL databases. Works in NodeJS, Browser, Ionic, Cordova and Electron platforms.
Next-generation ORM for Node.js & TypeScript | PostgreSQL, MySQL, MariaDB, SQL Server, SQLite, MongoDB and CockroachDB
A javascript library to run SQLite on the web.
Quick Overview
pg-mem is an in-memory PostgreSQL database emulator for Node.js. It allows developers to run PostgreSQL queries entirely in memory, making it ideal for testing and development environments without the need for a real PostgreSQL server.
Pros
- Fast and lightweight, as it runs entirely in memory
- No need to set up a separate PostgreSQL server for testing
- Supports a wide range of PostgreSQL features and syntax
- Can be used with popular ORMs like TypeORM and Sequelize
Cons
- Not a complete replacement for a real PostgreSQL server in production
- May have some limitations or differences in behavior compared to a real PostgreSQL instance
- Performance may differ from a real PostgreSQL server for large datasets or complex queries
- Requires keeping up-to-date with PostgreSQL feature updates
Code Examples
- Creating a database and running a simple query:
const { newDb } = require('pg-mem');
const db = newDb();
const result = db.public.many("SELECT 'Hello, World!' as message");
console.log(result); // [{ message: 'Hello, World!' }]
- Using pg-mem with a connection string:
const { newDb } = require('pg-mem');
const db = newDb();
const { Client } = db.adapters.createPg();
const client = new Client('postgres://username:password@localhost:5432/mydb');
await client.connect();
const result = await client.query('SELECT NOW() as current_time');
console.log(result.rows[0].current_time);
- Working with tables and data:
const { newDb } = require('pg-mem');
const db = newDb();
db.public.none(`
CREATE TABLE users (
id SERIAL PRIMARY KEY,
name TEXT NOT NULL,
email TEXT UNIQUE NOT NULL
);
INSERT INTO users (name, email) VALUES
('Alice', 'alice@example.com'),
('Bob', 'bob@example.com');
`);
const users = db.public.many('SELECT * FROM users');
console.log(users);
Getting Started
To use pg-mem in your project, follow these steps:
-
Install pg-mem:
npm install pg-mem
-
Import and use pg-mem in your code:
const { newDb } = require('pg-mem'); const db = newDb(); // Use db to run queries or create tables
-
For TypeScript users, you can import types:
import { newDb } from 'pg-mem';
Now you can use pg-mem to create an in-memory PostgreSQL database for your tests or development environment.
Competitor Comparisons
PostgreSQL client for node.js.
Pros of node-postgres
- Production-ready and widely used in real-world applications
- Supports a wide range of PostgreSQL features and data types
- Offers both callback and Promise-based APIs for flexibility
Cons of node-postgres
- Requires a real PostgreSQL database for testing and development
- Higher setup complexity for local development environments
- May have performance overhead when connecting to remote databases
Code Comparison
node-postgres:
const { Client } = require('pg');
const client = new Client();
await client.connect();
const res = await client.query('SELECT * FROM users WHERE id = $1', [userId]);
await client.end();
pg-mem:
const { newDb } = require('pg-mem');
const db = newDb();
db.public.none(`CREATE TABLE users (id INT, name TEXT)`);
db.public.many(`SELECT * FROM users WHERE id = $1`, [userId]);
Key Differences
pg-mem is an in-memory PostgreSQL implementation, ideal for testing and development without a real database. It offers faster setup and execution for tests but may not support all PostgreSQL features. node-postgres, on the other hand, is a full-featured client for real PostgreSQL databases, suitable for production use but requiring more setup for local development and testing environments.
A query builder for PostgreSQL, MySQL, CockroachDB, SQL Server, SQLite3 and Oracle, designed to be flexible, portable, and fun to use.
Pros of Knex
- Supports multiple database systems (PostgreSQL, MySQL, SQLite, etc.)
- Provides a powerful query builder with chainable methods
- Offers migration and seeding tools for database schema management
Cons of Knex
- Requires a real database connection for testing
- Learning curve for complex queries and advanced features
- Overhead for simple database operations
Code Comparison
Knex query:
knex('users')
.where('age', '>', 18)
.orderBy('name')
.select('id', 'name')
pg-mem query:
db.public.users
.find(u => u.age > 18)
.sort(u => u.name)
.select('id', 'name')
Key Differences
- pg-mem is specifically designed for in-memory PostgreSQL database simulation
- Knex is a full-featured query builder and migration tool for multiple databases
- pg-mem focuses on testing and development without a real database connection
- Knex provides more extensive database management features
Use Cases
- Choose Knex for production-ready database interactions across various database systems
- Opt for pg-mem when rapid testing and development of PostgreSQL-specific code is needed without setting up a real database
Community and Ecosystem
- Knex has a larger community and more extensive documentation
- pg-mem is more specialized but growing in popularity for PostgreSQL testing scenarios
Feature-rich ORM for modern Node.js and TypeScript, it supports PostgreSQL (with JSON and JSONB support), MySQL, MariaDB, SQLite, MS SQL Server, Snowflake, Oracle DB (v6), DB2 and DB2 for IBM i.
Pros of Sequelize
- Supports multiple databases (PostgreSQL, MySQL, SQLite, and more)
- Provides a robust ORM with advanced features like associations and migrations
- Has a large, active community and extensive documentation
Cons of Sequelize
- Steeper learning curve due to its comprehensive feature set
- Can be slower for complex queries compared to raw SQL
- Requires more setup and configuration
Code Comparison
Sequelize:
const User = sequelize.define('User', {
username: DataTypes.STRING,
email: DataTypes.STRING
});
const users = await User.findAll();
pg-mem:
const db = newDb();
const users = await db.public.many(`SELECT * FROM users`);
Key Differences
- pg-mem is specifically designed for in-memory PostgreSQL database emulation, while Sequelize is a full-featured ORM for multiple databases
- Sequelize provides an abstraction layer with model definitions and associations, whereas pg-mem allows direct SQL queries
- pg-mem is primarily used for testing and development, while Sequelize is commonly used in production environments
Use Cases
- Choose Sequelize for complex applications requiring ORM features and database agnosticism
- Opt for pg-mem when you need a lightweight, in-memory PostgreSQL solution for testing or rapid prototyping
ORM for TypeScript and JavaScript. Supports MySQL, PostgreSQL, MariaDB, SQLite, MS SQL Server, Oracle, SAP Hana, WebSQL databases. Works in NodeJS, Browser, Ionic, Cordova and Electron platforms.
Pros of TypeORM
- Supports multiple databases (PostgreSQL, MySQL, SQLite, etc.)
- Provides a robust ORM with advanced features like migrations and relations
- Active community with frequent updates and extensive documentation
Cons of TypeORM
- Steeper learning curve due to its comprehensive feature set
- Can be overkill for simple projects or when only PostgreSQL support is needed
- Performance overhead compared to raw SQL queries
Code Comparison
TypeORM:
@Entity()
class User {
@PrimaryGeneratedColumn()
id: number;
@Column()
name: string;
}
pg-mem:
const db = newDb();
db.public.registerFunction({
name: 'my_function',
args: [DataType.text],
returns: DataType.integer,
implementation: (x) => x.length,
});
TypeORM focuses on defining entities and relationships, while pg-mem allows for more direct manipulation of the in-memory database, including custom function definitions. TypeORM provides a higher level of abstraction, whereas pg-mem offers more fine-grained control over the database structure and behavior.
Next-generation ORM for Node.js & TypeScript | PostgreSQL, MySQL, MariaDB, SQL Server, SQLite, MongoDB and CockroachDB
Pros of Prisma
- Robust ORM with support for multiple databases, not limited to PostgreSQL
- Provides a powerful query builder and type-safe database access
- Offers database migrations and schema management tools
Cons of Prisma
- Steeper learning curve due to its comprehensive feature set
- Requires more setup and configuration compared to pg-mem
- May introduce additional complexity for simple projects
Code Comparison
Prisma:
const user = await prisma.user.create({
data: { name: 'Alice', email: 'alice@example.com' },
})
const posts = await prisma.post.findMany({
where: { authorId: user.id },
})
pg-mem:
const db = newDb();
const user = db.public.users.insert({ name: 'Alice', email: 'alice@example.com' });
const posts = db.public.posts.find({ authorId: user.id });
Key Differences
- Prisma is a full-featured ORM, while pg-mem is primarily an in-memory PostgreSQL implementation
- Prisma supports multiple databases, pg-mem focuses on PostgreSQL
- pg-mem is lightweight and ideal for testing, while Prisma is better suited for production applications
- Prisma offers more advanced features like migrations and schema management
A javascript library to run SQLite on the web.
Pros of sql.js
- Supports SQLite, a widely used and well-documented database system
- Can work with existing SQLite databases, allowing for easier migration and integration
- Provides a familiar SQL interface for developers experienced with SQLite
Cons of sql.js
- Limited to SQLite functionality, which may not cover all PostgreSQL features
- Potentially slower performance for complex queries compared to pg-mem
- May require more memory usage due to loading the entire SQLite engine
Code Comparison
pg-mem example:
const { newDb } = require('pg-mem');
const db = newDb();
db.public.none("CREATE TABLE users (id INT, name TEXT)");
db.public.many("SELECT * FROM users");
sql.js example:
const SQL = require('sql.js');
const db = new SQL.Database();
db.run("CREATE TABLE users (id INT, name TEXT)");
db.exec("SELECT * FROM users");
Both libraries provide in-memory database functionality, but pg-mem focuses on PostgreSQL compatibility, while sql.js emulates SQLite. pg-mem may offer better performance and more PostgreSQL-specific features, making it suitable for projects requiring PostgreSQL compatibility. sql.js, on the other hand, provides a familiar SQLite environment and can work with existing SQLite databases, which could be advantageous for projects already using SQLite or requiring its specific features.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
pg-mem is an experimental in-memory emulation of a postgres database.
⤠It works both in Node or in the browser.
â this repo if you like this package, it helps to motivate me :)
ð See it in action with pg-mem playground
ð Usage
Using Node.js
As always, it starts with an:
npm i pg-mem --save
Then, assuming you're using something like webpack, if you're targeting a browser:
import { newDb } from "pg-mem";
const db = newDb();
db.public.many(/* put some sql here */);
Using Deno
Pretty straightforward :)
import { newDb } from "https://deno.land/x/pg_mem/mod.ts";
const db = newDb();
db.public.many(/* put some sql here */);
Only use the SQL syntax parser
⤠Head to the pgsql-ast-parser repo
â Disclaimer
The sql syntax parser is home-made. Which means that some features are not implemented, and will be considered as invalid syntaxes.
This lib is quite new, so forgive it if some obvious pg syntax is not supported !
... And open an issue if you feel like a feature should be implemented :)
Moreover, even if I wrote hundreds of tests, keep in mind that this implementation is a best effort to replicate PG. Keep an eye on your query results if you perform complex queries. Please file issues if some results seem incoherent with what should be returned.
Finally, I invite you to read the below section to have an idea of you can or cannot do.
ð Features
Rollback to a previous state
pg-mem
uses immutable data structures (here and here),
which means that you can have restore points for free!
This is super useful if you intend to use pg-mem
to mock your database for unit tests.
You could:
- Create your schema only once (which could be a heavy operation for a single unit test)
- Insert test data which will be shared by all test
- Create a restore point
- Run your tests with the same db instance, executing a
backup.restore()
before each test (which instantly resets db to the state it has after creating the restore point)
Usage:
const db = newDb();
db.public.none(`create table test(id text);
insert into test values ('value');`);
// create a restore point & mess with data
const backup = db.backup();
db.public.none(`update test set id='new value';`);
// restore it !
backup.restore();
db.public.many(`select * from test`); // => {test: 'value'}
Custom functions
You can declare custom functions like this:
db.public.registerFunction({
name: "say_hello",
args: [DataType.text],
returns: DataType.text,
implementation: (x) => "hello " + x,
});
And then use them like in SQL select say_hello('world')
.
Custom functions support overloading and variadic arguments.
â However, the value you return is not type checked. It MUST correspond to the datatype you provided as 'returns' (it won't fail if not, but could lead to weird bugs).
Custom types
Not all pg types are implemented in pg-mem. That said, most of the types are often equivalent to other types, with a format validation. pg-mem provides a way to register such types.
For instance, lets say you'd like to register the MACADDR type, which is basically a string, with a format constraint.
You can register it like this:
db.public.registerEquivalentType({
name: "macaddr",
// which type is it equivalent to (will be able to cast it from it)
equivalentTo: DataType.text,
isValid(val: string) {
// check that it will be this format
return isValidMacAddress(val);
},
});
Doing so, you'll be able to do things such as:
SELECT '08:00:2b:01:02:03:04:05'::macaddr; -- WORKS
SELECT 'invalid'::macaddr; -- will throw a conversion error
If you feel your implementation of a type matches the standard, and would like to include it in pg-mem for others to enjoy it, please consider filing a pull request ! (tip: see the INET type implementation as an example, and the pg_catalog index where supported types are registered)
Extensions
No native extension is implemented (pull requests are welcome), but you can define kind-of extensions like this:
db.registerExtension("my-ext", (schema) => {
// install your ext in 'schema'
// ex: schema.registerFunction(...)
});
Statements like create extension "my-ext"
will then be supported.
ð Libraries adapters
pg-mem provides handy shortcuts to create instances of popular libraries that will be bound to pg-mem instead of a real postgres db.
- pg-native
- node-postgres (pg)
- pg-promise (pgp)
- slonik
- typeorm
- knex
- kysely
- mikro-orm
ð¥ Inspection
Intercept queries
If you would like to hook your database, and return ad-hoc results, you can do so like this:
const db = newDb();
db.public.interceptQueries((sql) => {
if (sql === "select * from whatever") {
// intercept this statement, and return something custom:
return [{ something: 42 }];
}
// proceed to actual SQL execution for other requests.
return null;
});
Inspect a table
You can manually inspect a table content using the find()
method:
for (const item of db.public.getTable<TItem>("mytable").find(itemTemplate)) {
console.log(item);
}
Manually insert items
If you'd like to insert items manually into a table, you can do this like that:
db.public.getTable<TItem>('mytable').insert({ /* item to insert */ }))
Subscribe to events
You can subscribe to some events, like:
const db = newDb();
// called on each successful sql request
db.on("query", (sql) => {});
// called on each failed sql request
db.on("query-failed", (sql) => {});
// called on schema changes
db.on("schema-change", () => {});
// called when a CREATE EXTENSION schema is encountered.
db.on("create-extension", (ext) => {});
Experimental events
pg-mem
implements a basic support for indices.
These handlers are called when a request cannot be optimized using one of the created indices.
However, a real postgres instance will be much smarter to optimize its requests... so when pg-mem
says "this request does not use an index", dont take my word for it.
// called when a table is iterated entirely (ex: 'select * from data where notIndex=3' triggers it)
db.on('seq-scan', () => {});
// same, but on a specific table
db.getTable('myTable').on('seq-scan', () = {});
// will be called if pg-mem did not find any way to optimize a join
// (which leads to a O(n*m) lookup with the current implementation)
db.on('catastrophic-join-optimization', () => {});
ðââï¸ FAQ
- Why this instead of Docker ? TLDR : It's faster. Docker is overkill.
- What if I need an extension like uuid-ossp ? TLDR: You can mock those
- How to import my production schema in pg-mem ? TLDR: pg_dump with the right args
- Does pg-mem supports sql migrations ? TLDR: yes.
- Does pg-mem supports plpgsql/other scripts/"create functions"/"do statements" ? TLDR: kind of...
Detailed answers in the wiki
â ï¸ Current limitations
- Materialized views are implemented as views (meaning that they are always up-to-date, without needing them to refresh)
- Indices implementations are basic
- No support for timezones
- All number-like types are all handled as javascript numbers, meaning that types like
numeric(x,y)
could not behave as expected.
ð Development
Pull requests are welcome :)
Unit tests are ran using Bun, which you will have to install to run tests.
Run all tests
bun test
Debug a test
Using vscode:
- Add a
.only
on the test you'd like to debug - Just hit F5 (or execute via the debugger tab), which should launch your test with debugger attached
Top Related Projects
PostgreSQL client for node.js.
A query builder for PostgreSQL, MySQL, CockroachDB, SQL Server, SQLite3 and Oracle, designed to be flexible, portable, and fun to use.
Feature-rich ORM for modern Node.js and TypeScript, it supports PostgreSQL (with JSON and JSONB support), MySQL, MariaDB, SQLite, MS SQL Server, Snowflake, Oracle DB (v6), DB2 and DB2 for IBM i.
ORM for TypeScript and JavaScript. Supports MySQL, PostgreSQL, MariaDB, SQLite, MS SQL Server, Oracle, SAP Hana, WebSQL databases. Works in NodeJS, Browser, Ionic, Cordova and Electron platforms.
Next-generation ORM for Node.js & TypeScript | PostgreSQL, MySQL, MariaDB, SQL Server, SQLite, MongoDB and CockroachDB
A javascript library to run SQLite on the web.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot