Convert Figma logo to code with AI

oguimbal logopg-mem

An in memory postgres DB instance for your unit tests

1,955
95
1,955
165

Top Related Projects

PostgreSQL client for node.js.

19,158

A query builder for PostgreSQL, MySQL, CockroachDB, SQL Server, SQLite3 and Oracle, designed to be flexible, portable, and fun to use.

29,432

Feature-rich ORM for modern Node.js and TypeScript, it supports PostgreSQL (with JSON and JSONB support), MySQL, MariaDB, SQLite, MS SQL Server, Snowflake, Oracle DB (v6), DB2 and DB2 for IBM i.

34,121

ORM for TypeScript and JavaScript. Supports MySQL, PostgreSQL, MariaDB, SQLite, MS SQL Server, Oracle, SAP Hana, WebSQL databases. Works in NodeJS, Browser, Ionic, Cordova and Electron platforms.

38,831

Next-generation ORM for Node.js & TypeScript | PostgreSQL, MySQL, MariaDB, SQL Server, SQLite, MongoDB and CockroachDB

12,535

A javascript library to run SQLite on the web.

Quick Overview

pg-mem is an in-memory PostgreSQL database emulator for Node.js. It allows developers to run PostgreSQL queries entirely in memory, making it ideal for testing and development environments without the need for a real PostgreSQL server.

Pros

  • Fast and lightweight, as it runs entirely in memory
  • No need to set up a separate PostgreSQL server for testing
  • Supports a wide range of PostgreSQL features and syntax
  • Can be used with popular ORMs like TypeORM and Sequelize

Cons

  • Not a complete replacement for a real PostgreSQL server in production
  • May have some limitations or differences in behavior compared to a real PostgreSQL instance
  • Performance may differ from a real PostgreSQL server for large datasets or complex queries
  • Requires keeping up-to-date with PostgreSQL feature updates

Code Examples

  1. Creating a database and running a simple query:
const { newDb } = require('pg-mem');

const db = newDb();
const result = db.public.many("SELECT 'Hello, World!' as message");
console.log(result); // [{ message: 'Hello, World!' }]
  1. Using pg-mem with a connection string:
const { newDb } = require('pg-mem');

const db = newDb();
const { Client } = db.adapters.createPg();

const client = new Client('postgres://username:password@localhost:5432/mydb');
await client.connect();

const result = await client.query('SELECT NOW() as current_time');
console.log(result.rows[0].current_time);
  1. Working with tables and data:
const { newDb } = require('pg-mem');

const db = newDb();

db.public.none(`
  CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    name TEXT NOT NULL,
    email TEXT UNIQUE NOT NULL
  );

  INSERT INTO users (name, email) VALUES
    ('Alice', 'alice@example.com'),
    ('Bob', 'bob@example.com');
`);

const users = db.public.many('SELECT * FROM users');
console.log(users);

Getting Started

To use pg-mem in your project, follow these steps:

  1. Install pg-mem:

    npm install pg-mem
    
  2. Import and use pg-mem in your code:

    const { newDb } = require('pg-mem');
    
    const db = newDb();
    // Use db to run queries or create tables
    
  3. For TypeScript users, you can import types:

    import { newDb } from 'pg-mem';
    

Now you can use pg-mem to create an in-memory PostgreSQL database for your tests or development environment.

Competitor Comparisons

PostgreSQL client for node.js.

Pros of node-postgres

  • Production-ready and widely used in real-world applications
  • Supports a wide range of PostgreSQL features and data types
  • Offers both callback and Promise-based APIs for flexibility

Cons of node-postgres

  • Requires a real PostgreSQL database for testing and development
  • Higher setup complexity for local development environments
  • May have performance overhead when connecting to remote databases

Code Comparison

node-postgres:

const { Client } = require('pg');
const client = new Client();
await client.connect();
const res = await client.query('SELECT * FROM users WHERE id = $1', [userId]);
await client.end();

pg-mem:

const { newDb } = require('pg-mem');
const db = newDb();
db.public.none(`CREATE TABLE users (id INT, name TEXT)`);
db.public.many(`SELECT * FROM users WHERE id = $1`, [userId]);

Key Differences

pg-mem is an in-memory PostgreSQL implementation, ideal for testing and development without a real database. It offers faster setup and execution for tests but may not support all PostgreSQL features. node-postgres, on the other hand, is a full-featured client for real PostgreSQL databases, suitable for production use but requiring more setup for local development and testing environments.

19,158

A query builder for PostgreSQL, MySQL, CockroachDB, SQL Server, SQLite3 and Oracle, designed to be flexible, portable, and fun to use.

Pros of Knex

  • Supports multiple database systems (PostgreSQL, MySQL, SQLite, etc.)
  • Provides a powerful query builder with chainable methods
  • Offers migration and seeding tools for database schema management

Cons of Knex

  • Requires a real database connection for testing
  • Learning curve for complex queries and advanced features
  • Overhead for simple database operations

Code Comparison

Knex query:

knex('users')
  .where('age', '>', 18)
  .orderBy('name')
  .select('id', 'name')

pg-mem query:

db.public.users
  .find(u => u.age > 18)
  .sort(u => u.name)
  .select('id', 'name')

Key Differences

  • pg-mem is specifically designed for in-memory PostgreSQL database simulation
  • Knex is a full-featured query builder and migration tool for multiple databases
  • pg-mem focuses on testing and development without a real database connection
  • Knex provides more extensive database management features

Use Cases

  • Choose Knex for production-ready database interactions across various database systems
  • Opt for pg-mem when rapid testing and development of PostgreSQL-specific code is needed without setting up a real database

Community and Ecosystem

  • Knex has a larger community and more extensive documentation
  • pg-mem is more specialized but growing in popularity for PostgreSQL testing scenarios
29,432

Feature-rich ORM for modern Node.js and TypeScript, it supports PostgreSQL (with JSON and JSONB support), MySQL, MariaDB, SQLite, MS SQL Server, Snowflake, Oracle DB (v6), DB2 and DB2 for IBM i.

Pros of Sequelize

  • Supports multiple databases (PostgreSQL, MySQL, SQLite, and more)
  • Provides a robust ORM with advanced features like associations and migrations
  • Has a large, active community and extensive documentation

Cons of Sequelize

  • Steeper learning curve due to its comprehensive feature set
  • Can be slower for complex queries compared to raw SQL
  • Requires more setup and configuration

Code Comparison

Sequelize:

const User = sequelize.define('User', {
  username: DataTypes.STRING,
  email: DataTypes.STRING
});

const users = await User.findAll();

pg-mem:

const db = newDb();
const users = await db.public.many(`SELECT * FROM users`);

Key Differences

  • pg-mem is specifically designed for in-memory PostgreSQL database emulation, while Sequelize is a full-featured ORM for multiple databases
  • Sequelize provides an abstraction layer with model definitions and associations, whereas pg-mem allows direct SQL queries
  • pg-mem is primarily used for testing and development, while Sequelize is commonly used in production environments

Use Cases

  • Choose Sequelize for complex applications requiring ORM features and database agnosticism
  • Opt for pg-mem when you need a lightweight, in-memory PostgreSQL solution for testing or rapid prototyping
34,121

ORM for TypeScript and JavaScript. Supports MySQL, PostgreSQL, MariaDB, SQLite, MS SQL Server, Oracle, SAP Hana, WebSQL databases. Works in NodeJS, Browser, Ionic, Cordova and Electron platforms.

Pros of TypeORM

  • Supports multiple databases (PostgreSQL, MySQL, SQLite, etc.)
  • Provides a robust ORM with advanced features like migrations and relations
  • Active community with frequent updates and extensive documentation

Cons of TypeORM

  • Steeper learning curve due to its comprehensive feature set
  • Can be overkill for simple projects or when only PostgreSQL support is needed
  • Performance overhead compared to raw SQL queries

Code Comparison

TypeORM:

@Entity()
class User {
  @PrimaryGeneratedColumn()
  id: number;

  @Column()
  name: string;
}

pg-mem:

const db = newDb();
db.public.registerFunction({
  name: 'my_function',
  args: [DataType.text],
  returns: DataType.integer,
  implementation: (x) => x.length,
});

TypeORM focuses on defining entities and relationships, while pg-mem allows for more direct manipulation of the in-memory database, including custom function definitions. TypeORM provides a higher level of abstraction, whereas pg-mem offers more fine-grained control over the database structure and behavior.

38,831

Next-generation ORM for Node.js & TypeScript | PostgreSQL, MySQL, MariaDB, SQL Server, SQLite, MongoDB and CockroachDB

Pros of Prisma

  • Robust ORM with support for multiple databases, not limited to PostgreSQL
  • Provides a powerful query builder and type-safe database access
  • Offers database migrations and schema management tools

Cons of Prisma

  • Steeper learning curve due to its comprehensive feature set
  • Requires more setup and configuration compared to pg-mem
  • May introduce additional complexity for simple projects

Code Comparison

Prisma:

const user = await prisma.user.create({
  data: { name: 'Alice', email: 'alice@example.com' },
})
const posts = await prisma.post.findMany({
  where: { authorId: user.id },
})

pg-mem:

const db = newDb();
const user = db.public.users.insert({ name: 'Alice', email: 'alice@example.com' });
const posts = db.public.posts.find({ authorId: user.id });

Key Differences

  • Prisma is a full-featured ORM, while pg-mem is primarily an in-memory PostgreSQL implementation
  • Prisma supports multiple databases, pg-mem focuses on PostgreSQL
  • pg-mem is lightweight and ideal for testing, while Prisma is better suited for production applications
  • Prisma offers more advanced features like migrations and schema management
12,535

A javascript library to run SQLite on the web.

Pros of sql.js

  • Supports SQLite, a widely used and well-documented database system
  • Can work with existing SQLite databases, allowing for easier migration and integration
  • Provides a familiar SQL interface for developers experienced with SQLite

Cons of sql.js

  • Limited to SQLite functionality, which may not cover all PostgreSQL features
  • Potentially slower performance for complex queries compared to pg-mem
  • May require more memory usage due to loading the entire SQLite engine

Code Comparison

pg-mem example:

const { newDb } = require('pg-mem');
const db = newDb();
db.public.none("CREATE TABLE users (id INT, name TEXT)");
db.public.many("SELECT * FROM users");

sql.js example:

const SQL = require('sql.js');
const db = new SQL.Database();
db.run("CREATE TABLE users (id INT, name TEXT)");
db.exec("SELECT * FROM users");

Both libraries provide in-memory database functionality, but pg-mem focuses on PostgreSQL compatibility, while sql.js emulates SQLite. pg-mem may offer better performance and more PostgreSQL-specific features, making it suitable for projects requiring PostgreSQL compatibility. sql.js, on the other hand, provides a familiar SQLite environment and can work with existing SQLite databases, which could be advantageous for projects already using SQLite or requiring its specific features.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

pg-mem is an experimental in-memory emulation of a postgres database.

❤ It works both in Node or in the browser.

⭐ this repo if you like this package, it helps to motivate me :)

👉 See it in action with pg-mem playground

📐 Usage

Using Node.js

As always, it starts with an:

npm i pg-mem --save

Then, assuming you're using something like webpack, if you're targeting a browser:

import { newDb } from "pg-mem";

const db = newDb();
db.public.many(/* put some sql here */);

Using Deno

Pretty straightforward :)

import { newDb } from "https://deno.land/x/pg_mem/mod.ts";

const db = newDb();
db.public.many(/* put some sql here */);

Only use the SQL syntax parser

❤ Head to the pgsql-ast-parser repo

⚠ Disclaimer

The sql syntax parser is home-made. Which means that some features are not implemented, and will be considered as invalid syntaxes.

This lib is quite new, so forgive it if some obvious pg syntax is not supported !

... And open an issue if you feel like a feature should be implemented :)

Moreover, even if I wrote hundreds of tests, keep in mind that this implementation is a best effort to replicate PG. Keep an eye on your query results if you perform complex queries. Please file issues if some results seem incoherent with what should be returned.

Finally, I invite you to read the below section to have an idea of you can or cannot do.

🔍 Features

Rollback to a previous state

pg-mem uses immutable data structures (here and here), which means that you can have restore points for free!

This is super useful if you intend to use pg-mem to mock your database for unit tests.

You could:

  1. Create your schema only once (which could be a heavy operation for a single unit test)
  2. Insert test data which will be shared by all test
  3. Create a restore point
  4. Run your tests with the same db instance, executing a backup.restore() before each test (which instantly resets db to the state it has after creating the restore point)

Usage:

const db = newDb();
db.public.none(`create table test(id text);
                insert into test values ('value');`);
// create a restore point & mess with data
const backup = db.backup();
db.public.none(`update test set id='new value';`);
// restore it !
backup.restore();
db.public.many(`select * from test`); // => {test: 'value'}

Custom functions

You can declare custom functions like this:

db.public.registerFunction({
  name: "say_hello",
  args: [DataType.text],
  returns: DataType.text,
  implementation: (x) => "hello " + x,
});

And then use them like in SQL select say_hello('world').

Custom functions support overloading and variadic arguments.

⚠ However, the value you return is not type checked. It MUST correspond to the datatype you provided as 'returns' (it won't fail if not, but could lead to weird bugs).

Custom types

Not all pg types are implemented in pg-mem. That said, most of the types are often equivalent to other types, with a format validation. pg-mem provides a way to register such types.

For instance, lets say you'd like to register the MACADDR type, which is basically a string, with a format constraint.

You can register it like this:

db.public.registerEquivalentType({
  name: "macaddr",
  // which type is it equivalent to (will be able to cast it from it)
  equivalentTo: DataType.text,
  isValid(val: string) {
    // check that it will be this format
    return isValidMacAddress(val);
  },
});

Doing so, you'll be able to do things such as:

SELECT '08:00:2b:01:02:03:04:05'::macaddr; -- WORKS
SELECT 'invalid'::macaddr; -- will throw a conversion error

If you feel your implementation of a type matches the standard, and would like to include it in pg-mem for others to enjoy it, please consider filing a pull request ! (tip: see the INET type implementation as an example, and the pg_catalog index where supported types are registered)

Extensions

No native extension is implemented (pull requests are welcome), but you can define kind-of extensions like this:

db.registerExtension("my-ext", (schema) => {
  // install your ext in 'schema'
  // ex:  schema.registerFunction(...)
});

Statements like create extension "my-ext" will then be supported.

📃 Libraries adapters

pg-mem provides handy shortcuts to create instances of popular libraries that will be bound to pg-mem instead of a real postgres db.

  • pg-native
  • node-postgres (pg)
  • pg-promise (pgp)
  • slonik
  • typeorm
  • knex
  • kysely
  • mikro-orm

See the wiki for more details

💥 Inspection

Intercept queries

If you would like to hook your database, and return ad-hoc results, you can do so like this:

const db = newDb();

db.public.interceptQueries((sql) => {
  if (sql === "select * from whatever") {
    // intercept this statement, and return something custom:
    return [{ something: 42 }];
  }
  // proceed to actual SQL execution for other requests.
  return null;
});

Inspect a table

You can manually inspect a table content using the find() method:

for (const item of db.public.getTable<TItem>("mytable").find(itemTemplate)) {
  console.log(item);
}

Manually insert items

If you'd like to insert items manually into a table, you can do this like that:

db.public.getTable<TItem>('mytable').insert({ /* item to insert */ }))

Subscribe to events

You can subscribe to some events, like:

const db = newDb();

// called on each successful sql request
db.on("query", (sql) => {});
// called on each failed sql request
db.on("query-failed", (sql) => {});
// called on schema changes
db.on("schema-change", () => {});
// called when a CREATE EXTENSION schema is encountered.
db.on("create-extension", (ext) => {});

Experimental events

pg-mem implements a basic support for indices.

These handlers are called when a request cannot be optimized using one of the created indices.

However, a real postgres instance will be much smarter to optimize its requests... so when pg-mem says "this request does not use an index", dont take my word for it.

// called when a table is iterated entirely (ex: 'select * from data where notIndex=3' triggers it)
db.on('seq-scan', () => {});

// same, but on a specific table
db.getTable('myTable').on('seq-scan', () = {});

// will be called if pg-mem did not find any way to optimize a join
// (which leads to a O(n*m) lookup with the current implementation)
db.on('catastrophic-join-optimization', () => {});

🙋‍♂️ FAQ

Detailed answers in the wiki

⚠️ Current limitations

  • Materialized views are implemented as views (meaning that they are always up-to-date, without needing them to refresh)
  • Indices implementations are basic
  • No support for timezones
  • All number-like types are all handled as javascript numbers, meaning that types like numeric(x,y) could not behave as expected.

🐜 Development

Pull requests are welcome :)

Unit tests are ran using Bun, which you will have to install to run tests.

Run all tests

bun test

Debug a test

Using vscode:

  1. Add a .only on the test you'd like to debug
  2. Just hit F5 (or execute via the debugger tab), which should launch your test with debugger attached

NPM DownloadsLast 30 Days