Reiteach Aigean Logo
Reiteach Aigeanblog.reiteachaigean.com
Admin
HomeAI UpdatesCodingCloudApps & ToolsTech NewsArticles
DjangoCodingpostgresqldatabase12 min read·8 reads

Django Database Connection Pooling with PostgreSQL Guide

Learn Django database connection pooling with PostgreSQL: CONN_MAX_AGE, Django 5.1 native pooling, PgBouncer setup, pool sizing, Celery tips, and common pitfalls.

S
Sizan Mahmud
Published on November 17, 2026
Django Database Connection Pooling with PostgreSQL Guide

Sooner or later, a growing Django application meets this error:

FATAL: sorry, too many clients already

or its cousin:

FATAL: remaining connection slots are reserved for non-replication superuser connections

Nothing is wrong with your queries. You have simply run out of PostgreSQL connections. Every web worker, every Celery process, every cron script, and every admin tool wants its own connection, and PostgreSQL only allows so many.

Connection pooling is the standard solution. This guide explains why PostgreSQL connections are expensive, how Django handles connections by default, and the three main ways to pool them: persistent connections with CONN_MAX_AGE, Django's native pool (added in Django 5.1), and an external pooler such as PgBouncer. It also covers how to size a pool, what to watch out for with Celery, and the pitfalls that cause the most production incidents.

Why PostgreSQL Connections Are Expensive

Unlike some databases that use lightweight threads, PostgreSQL starts a separate operating system process for every client connection. Each backend process uses memory, and creating one takes time, because of the TCP handshake, authentication, and TLS negotiation if you use it.

This has two consequences:

  • Opening a connection per request is slow. The setup cost can be several milliseconds or more, which is a lot when the query itself takes one millisecond.
  • Having many open connections is costly too. The default max_connections is 100. Raising it to thousands increases memory use and can hurt performance, because many active backends compete for CPU, locks, and I/O.

The goal of pooling is simple: keep a small number of well-used connections open, and share them among many requests.

How Django Handles Connections by Default

Out of the box, Django opens a new database connection at the start of each request and closes it when the request ends. That is controlled by CONN_MAX_AGE, which defaults to 0.

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": "mydb",
        "USER": "myuser",
        "PASSWORD": "secret",
        "HOST": "127.0.0.1",
        "PORT": "5432",
        "CONN_MAX_AGE": 0,  # default: new connection per request
    }
}

This is safe and simple, but it pays the connection setup cost on every request. On a busy site, that adds latency and puts churn on PostgreSQL.

Option 1: Persistent Connections with CONN_MAX_AGE

The simplest improvement is to let each Django process keep its connection open between requests:

DATABASES = {
    "default": {
        # ...
        "CONN_MAX_AGE": 60,          # seconds; None means unlimited
        "CONN_HEALTH_CHECKS": True,  # verify the connection before reuse
    }
}

With this, each worker thread reuses its connection for up to 60 seconds. CONN_HEALTH_CHECKS makes Django check that a reused connection is still alive at the start of a request, which avoids errors after a database restart or a dropped connection.

What this is, and what it is not. This is connection reuse, not a true pool. Each thread or process holds its own connection, whether it is busy or idle. With 8 Gunicorn workers and 4 threads each, you can hold 32 connections open all the time. It does not share connections between workers.

Free tech newsletter

Free

Free to join — no card, no paywall. Weekly AI, cloud & tooling insights.

A warning about threads and ASGI. Persistent connections are tied to threads. Under ASGI, Django runs sync code in thread pools, and persistent connections can leak or accumulate. Django's documentation recommends CONN_MAX_AGE = 0 for ASGI deployments, and suggests using a real pool instead.

Persistent connections are a good first step for small and medium sites with a modest number of workers. They stop helping when the number of processes grows.

Option 2: Django's Native Connection Pool (Django 5.1+)

Django 5.1 added built-in connection pooling for PostgreSQL. It requires the psycopg (version 3) driver with its pool extra:

pip install "psycopg[binary,pool]"

Then enable it in OPTIONS:

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": "mydb",
        "USER": "myuser",
        "PASSWORD": "secret",
        "HOST": "127.0.0.1",
        "PORT": "5432",
        "CONN_MAX_AGE": 0,  # must be 0 when using the native pool
        "OPTIONS": {
            "pool": {
                "min_size": 2,
                "max_size": 10,
                "timeout": 10,
                "max_lifetime": 1800,
                "max_idle": 300,
            },
        },
    }
}

Setting "pool": True uses the driver's defaults, and a dictionary lets you pass options to psycopg's ConnectionPool. The ones that matter most:

  • min_size: connections kept open even when idle.
  • max_size: the most connections this pool will open.
  • timeout: how long a request waits for a free connection before failing.
  • max_lifetime: how long a connection lives before being replaced, which helps with load balancers and long-lived resources.
  • max_idle: how long an unused connection stays open before the pool shrinks.

Important points about the native pool:

  • It is per process. Each Gunicorn worker process creates its own pool. If you run 8 workers with max_size: 10, your application can open up to 80 connections. Size the pool with that multiplication in mind.
  • It replaces CONN_MAX_AGE. Django refuses to start if you combine the pool with persistent connections, so keep CONN_MAX_AGE at 0.
  • It uses psycopg 3 only. Projects still on psycopg2 need to migrate the driver first. That is usually a small change, but test your app because some behaviors differ.
  • It does not reduce connections across processes. Because each process has its own pool, it improves reuse inside a process but does not cap the total across your whole fleet. For that, you need an external pooler.

The native pool is a great fit for a single application server or a modest number of processes, with no extra infrastructure to run.

Option 3: An External Pooler Like PgBouncer

When you have many application processes, such as multiple web servers, Celery workers, and cron jobs, you need a pooler that sits between all of them and PostgreSQL. The most widely used one is PgBouncer.

Django web workers ─┐
Celery workers ─────┼──► PgBouncer ──► PostgreSQL (few real connections)
Cron / scripts ─────┘

Thousands of client connections can share a few dozen real PostgreSQL connections. This is how large Django deployments keep the database stable.

PgBouncer pool modes

The pool mode decides when a real server connection is handed back to the pool:

Mode Connection returned to the pool Notes
Session When the client disconnects Safest, but saves the least
Transaction When each transaction finishes Best sharing, the usual choice for web apps
Statement After every statement Breaks multi-statement transactions, not suitable for Django

Transaction pooling is what most Django teams use, because a connection is only held while a transaction is actually running.

A minimal pgbouncer.ini:

[databases]
mydb = host=127.0.0.1 port=5432 dbname=mydb

[pgbouncer]
listen_addr = 0.0.0.0
listen_port = 6432
auth_type = scram-sha-256
auth_file = /etc/pgbouncer/userlist.txt
pool_mode = transaction
max_client_conn = 1000
default_pool_size = 20
reserve_pool_size = 5
server_idle_timeout = 600

Here, up to 1,000 clients can connect to PgBouncer, but at most about 20 server connections per database and user pair are used against PostgreSQL.

Free tech newsletter

Free

Free to join — no card, no paywall. Weekly AI, cloud & tooling insights.

Django settings for transaction pooling

Point Django at PgBouncer's port, and turn off a feature that does not work in transaction mode:

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": "mydb",
        "USER": "myuser",
        "PASSWORD": "secret",
        "HOST": "127.0.0.1",
        "PORT": "6432",                      # PgBouncer, not PostgreSQL
        "CONN_MAX_AGE": 0,                   # or a modest value; see below
        "DISABLE_SERVER_SIDE_CURSORS": True, # required for transaction pooling
    }
}

Why disable server-side cursors? Django uses server-side cursors for QuerySet.iterator() on PostgreSQL. A cursor lives on one specific server connection, but in transaction mode your next statement may land on a different one. The result is errors or missing cursors. Setting DISABLE_SERVER_SIDE_CURSORS = True makes iterator() fetch results in chunks without them, which still keeps memory use low.

What about CONN_MAX_AGE? With PgBouncer, a client connection to the pooler is cheap, so CONN_MAX_AGE = 0 is perfectly fine. A modest persistent value also works, since the real server connections are managed by PgBouncer. Do not rely on long-lived Django connections for correctness.

Transaction Pooling Pitfalls

Transaction pooling is powerful but changes the rules. Anything that depends on session state can break, because consecutive transactions may run on different server connections.

  • Server-side cursors: disable them, as above.
  • Session-level advisory locks: pg_advisory_lock() held across transactions will not behave as expected. Use transaction-level locks (pg_advisory_xact_lock) instead.
  • LISTEN / NOTIFY: needs a stable session. Use a direct connection for listeners.
  • Session-level SET commands: a SET such as a custom search_path or statement_timeout may not stick. Use SET LOCAL inside a transaction, or configure the setting per role or database.
  • Prepared statements: historically a problem in transaction mode. Recent PgBouncer versions (1.21 and later) can support protocol-level prepared statements, but check your version and settings. With psycopg 3, you can also disable automatic server-side preparation by setting prepare_threshold to None in OPTIONS if you see "prepared statement does not exist" errors.
  • Temporary tables: they belong to a session, so they may vanish between transactions.

If you depend on any of these, either avoid them, use session pooling, or send those specific workloads through a direct connection that bypasses the pooler.

How to Size Your Pool

More connections does not mean more speed. PostgreSQL performs best when the number of active queries is close to what its CPU and disk can handle. A common rule of thumb for the number of active connections is a small multiple of the CPU core count, but the right number depends on your workload, so treat any formula as a starting point and measure.

Do the arithmetic across your whole system:

total possible app connections =
    (web processes × pool max_size)
  + (celery processes × connections each)
  + cron jobs, admin tools, migrations, monitoring

This total must stay below PostgreSQL's max_connections, with headroom for superuser access and maintenance. If it does not, either reduce pool sizes or put PgBouncer in front.

A few practical guidelines:

  • Start small. A pool of 5 to 20 connections per application server is often enough.
  • Watch for requests waiting on the pool (timeout errors). That is a sign of slow queries or too few connections, not always a reason to add more.
  • Slow queries hold connections longer, so fixing them often reduces the connections you need.
  • Keep max_connections modest on PostgreSQL, and let the pooler absorb client spikes.

Celery and Connection Pooling

Celery deserves special attention, because it multiplies your process count.

  • Each Celery prefork worker process opens its own database connection. A worker with --concurrency=16 can use 16 connections, and ten such workers use 160.
  • Long-running tasks hold connections for a long time, even while waiting on external APIs.
  • Django closes stale connections between requests, but Celery tasks do not go through the request cycle, so old or broken connections can linger. Django's close_old_connections() is called around Celery tasks by Django's Celery integration, but it is still worth checking how your setup behaves, especially with CONN_MAX_AGE set.

Practical advice: route Celery workers through PgBouncer too, keep concurrency reasonable, and avoid holding a transaction open while doing slow network calls. For bulk jobs, small chunks with short transactions use connections much more efficiently than one long task.

Monitoring Your Connections

Do not guess; look at what PostgreSQL is doing.

Free tech newsletter

Free

Free to join — no card, no paywall. Weekly AI, cloud & tooling insights.

-- How many connections, by state
SELECT state, count(*)
FROM pg_stat_activity
GROUP BY state;

-- Who is using them
SELECT usename, application_name, state, count(*)
FROM pg_stat_activity
GROUP BY usename, application_name, state
ORDER BY count(*) DESC;

-- The limit
SHOW max_connections;

Watch for a large number of connections in the idle in transaction state. These hold resources and locks while doing nothing, and are often caused by code that opens a transaction and then makes a slow external call. Setting idle_in_transaction_session_timeout in PostgreSQL helps catch them.

With PgBouncer, connect to its admin console and run SHOW POOLS; and SHOW STATS; to see active clients, waiting clients, and server connections.

Which Option Should You Choose?

Situation Recommended approach
Small app, few workers CONN_MAX_AGE with CONN_HEALTH_CHECKS
Single server, Django 5.1+, psycopg 3 Django's native pool
ASGI deployment Native pool or PgBouncer, not persistent connections
Many web servers and Celery workers PgBouncer in transaction mode
Managed cloud database Provider's pooler or proxy, if available
Need session features (LISTEN, advisory locks) Direct connection or session pooling for those workloads

You can also combine layers. For example, use PgBouncer for the fleet and keep Django's settings simple, instead of stacking multiple pools on top of each other, which makes behavior harder to reason about.

Related Reading: Keep Your Database Healthy Under Load

Connection pooling solves the "too many connections" problem, but a healthy database also needs efficient queries and well-behaved background jobs:

A Quick Checklist

Before going to production, confirm that:

  • You know your PostgreSQL max_connections and your total possible app connections
  • Each process type (web, Celery, cron) has a known connection budget
  • You use persistent connections, the native pool, or PgBouncer, and not several layers by accident
  • With PgBouncer in transaction mode, DISABLE_SERVER_SIDE_CURSORS = True is set
  • You do not depend on session state, such as session-level advisory locks or SET
  • CONN_MAX_AGE is 0 when using Django's native pool
  • Slow queries and long transactions are monitored
  • idle in transaction connections are tracked and timed out
  • You load-tested with realistic concurrency

Conclusion

Database connection pooling is not an optimization you add at the end. It is part of how a Django and PostgreSQL system survives growth. Start with persistent connections if your app is small, move to Django's native pool when you want real pooling inside a process, and put PgBouncer in front of PostgreSQL when many processes share the same database.

Whichever you choose, keep the total number of connections low, keep transactions short, and measure. A small, busy pool backed by fast queries will beat a large pool of idle connections almost every time.

Frequently Asked Questions

What is connection pooling in Django? It is the practice of keeping a limited set of database connections open and reusing them across requests, instead of opening and closing a new connection each time. Django supports persistent connections through CONN_MAX_AGE, a native pool for PostgreSQL since Django 5.1, and external poolers like PgBouncer.

Does Django have built-in connection pooling? Yes, starting with Django 5.1, for PostgreSQL only, using psycopg 3 with the pool extra. You enable it through the pool option inside OPTIONS. Earlier versions only offer persistent connections through CONN_MAX_AGE.

Is CONN_MAX_AGE the same as connection pooling? Not exactly. CONN_MAX_AGE keeps one connection open per thread or process for reuse, but those connections are not shared between workers. A real pool manages a set of connections that requests borrow and return.

Do I need PgBouncer if I use Django's native pool? Not necessarily. The native pool is per process, so it works well for a single server or a small number of processes. If many web servers and Celery workers all connect to one database, PgBouncer gives you a single place to cap total connections.

Why do I need DISABLE_SERVER_SIDE_CURSORS with PgBouncer? In transaction pooling mode, consecutive statements can run on different server connections, but a server-side cursor exists on only one. Disabling server-side cursors avoids errors when using QuerySet.iterator().

How many database connections should my Django app use? There is no universal number. Start with a small pool, add up connections across all web, Celery, and background processes, keep the total comfortably under max_connections, and adjust based on measured wait times and database load.

Share this article

Pass this technical dispatch along to fellow engineers

Did you enjoy this article?

Rate it so we know what content to bring next

Comments (0)