The non-blocking IO system was designed to solve three problems: 1. Deadlock that can occur when both sides of a connection are blocked writing because all buffers between are full. 2. The inability to use a write deadline with a TLS.Conn without killing the connection. 3. Efficiently check if a connection has been closed before writing. This reduces the cases where the application doesn't know if a query that does a INSERT/UPDATE/DELETE was actually sent to the server or not. However, the nbconn package is extraordinarily complex, has been a source of very tricky bugs, and has OS specific code paths. It also does not work at all with underlying net.Conn implementations that do not have platform specific non-blocking IO syscall support and do not properly implement deadlines. In particular, this is the case with golang.org/x/crypto/ssh. I believe the deadlock problem can be solved with a combination of a goroutine for CopyFrom like v4 used and a watchdog for regular queries that uses time.AfterFunc. The write deadline problem actually should be ignorable. We check for context cancellation before sending a query and the actual Write should be almost instant as long as the underlying connection is not blocked. (We should only have to wait until it is accepted by the OS, not until it is fully sent.) Efficiently checking if a connection has been closed is probably the hardest to solve without non-blocking reads. However, the existing code only solves part of the problem. It can detect a closed or broken connection the OS knows about, but it won't actually detect other types of broken connections such as a network interruption. This is currently implemented in CheckConn and called automatically when checking a connection out of the pool that has been idle for over one second. I think that changing CheckConn to a very short deadline read and changing the pool to do an actual Ping would be an acceptable solution. Remove nbconn and non-blocking code. This does not leave the system in an entirely working state. In particular, CopyFrom is broken, deadlocks can occur for extremely large queries or batches, and PgConn.CheckConn is now a `select 1` ping. These will be resolved in subsequent commits. |
||
---|---|---|
.github | ||
ci | ||
examples | ||
internal | ||
log/testingadapter | ||
pgconn | ||
pgproto3 | ||
pgtype | ||
pgxpool | ||
pgxtest | ||
stdlib | ||
testsetup | ||
tracelog | ||
.gitignore | ||
CHANGELOG.md | ||
CONTRIBUTING.md | ||
LICENSE | ||
README.md | ||
Rakefile | ||
batch.go | ||
batch_test.go | ||
bench_test.go | ||
conn.go | ||
conn_internal_test.go | ||
conn_test.go | ||
copy_from.go | ||
copy_from_test.go | ||
doc.go | ||
extended_query_builder.go | ||
go.mod | ||
go.sum | ||
helper_test.go | ||
large_objects.go | ||
large_objects_test.go | ||
named_args.go | ||
named_args_test.go | ||
pgbouncer_test.go | ||
pipeline_test.go | ||
query_test.go | ||
rows.go | ||
rows_test.go | ||
tracer.go | ||
tracer_test.go | ||
tx.go | ||
tx_test.go | ||
values.go | ||
values_test.go |
README.md
pgx - PostgreSQL Driver and Toolkit
pgx is a pure Go driver and toolkit for PostgreSQL.
The pgx driver is a low-level, high performance interface that exposes PostgreSQL-specific features such as LISTEN
/
NOTIFY
and COPY
. It also includes an adapter for the standard database/sql
interface.
The toolkit component is a related set of packages that implement PostgreSQL functionality such as parsing the wire protocol and type mapping between PostgreSQL and Go. These underlying packages can be used to implement alternative drivers, proxies, load balancers, logical replication clients, etc.
Example Usage
package main
import (
"context"
"fmt"
"os"
"github.com/jackc/pgx/v5"
)
func main() {
// urlExample := "postgres://username:password@localhost:5432/database_name"
conn, err := pgx.Connect(context.Background(), os.Getenv("DATABASE_URL"))
if err != nil {
fmt.Fprintf(os.Stderr, "Unable to connect to database: %v\n", err)
os.Exit(1)
}
defer conn.Close(context.Background())
var name string
var weight int64
err = conn.QueryRow(context.Background(), "select name, weight from widgets where id=$1", 42).Scan(&name, &weight)
if err != nil {
fmt.Fprintf(os.Stderr, "QueryRow failed: %v\n", err)
os.Exit(1)
}
fmt.Println(name, weight)
}
See the getting started guide for more information.
Features
- Support for approximately 70 different PostgreSQL types
- Automatic statement preparation and caching
- Batch queries
- Single-round trip query mode
- Full TLS connection control
- Binary format support for custom types (allows for much quicker encoding/decoding)
COPY
protocol support for faster bulk data loads- Tracing and logging support
- Connection pool with after-connect hook for arbitrary connection setup
LISTEN
/NOTIFY
- Conversion of PostgreSQL arrays to Go slice mappings for integers, floats, and strings
hstore
supportjson
andjsonb
support- Maps
inet
andcidr
PostgreSQL types tonetip.Addr
andnetip.Prefix
- Large object support
- NULL mapping to pointer to pointer
- Supports
database/sql.Scanner
anddatabase/sql/driver.Valuer
interfaces for custom types - Notice response handling
- Simulated nested transactions with savepoints
Choosing Between the pgx and database/sql Interfaces
The pgx interface is faster. Many PostgreSQL specific features such as LISTEN
/ NOTIFY
and COPY
are not available
through the database/sql
interface.
The pgx interface is recommended when:
- The application only targets PostgreSQL.
- No other libraries that require
database/sql
are in use.
It is also possible to use the database/sql
interface and convert a connection to the lower-level pgx interface as needed.
Testing
See CONTRIBUTING.md for setup instructions.
Supported Go and PostgreSQL Versions
pgx supports the same versions of Go and PostgreSQL that are supported by their respective teams. For Go that is the two most recent major releases and for PostgreSQL the major releases in the last 5 years. This means pgx supports Go 1.19 and higher and PostgreSQL 11 and higher. pgx also is tested against the latest version of CockroachDB.
Version Policy
pgx follows semantic versioning for the documented public API on stable releases. v5
is the latest stable major version.
PGX Family Libraries
github.com/jackc/pglogrepl
pglogrepl provides functionality to act as a client for PostgreSQL logical replication.
github.com/jackc/pgmock
pgmock offers the ability to create a server that mocks the PostgreSQL wire protocol. This is used internally to test pgx by purposely inducing unusual errors. pgproto3 and pgmock together provide most of the foundational tooling required to implement a PostgreSQL proxy or MitM (such as for a custom connection pooler).
github.com/jackc/tern
tern is a stand-alone SQL migration system.
github.com/jackc/pgerrcode
pgerrcode contains constants for the PostgreSQL error codes.
Adapters for 3rd Party Types
- github.com/jackc/pgx-gofrs-uuid
- github.com/jackc/pgx-shopspring-decimal
- github.com/vgarvardt/pgx-google-uuid
Adapters for 3rd Party Tracers
Adapters for 3rd Party Loggers
These adapters can be used with the tracelog package.
- github.com/jackc/pgx-go-kit-log
- github.com/jackc/pgx-log15
- github.com/jackc/pgx-logrus
- github.com/jackc/pgx-zap
- github.com/jackc/pgx-zerolog
- github.com/mcosta74/pgx-slog
3rd Party Libraries with PGX Support
github.com/pashagolub/pgxmock
pgxmock is a mock library implementing pgx interfaces. pgxmock has one and only purpose - to simulate pgx behavior in tests, without needing a real database connection.
github.com/georgysavva/scany
Library for scanning data from a database into Go structs and more.
github.com/vingarcia/ksql
A carefully designed SQL client for making using SQL easier, more productive, and less error-prone on Golang.
https://github.com/otan/gopgkrb5
Adds GSSAPI / Kerberos authentication support.
github.com/wcamarao/pmx
Explicit data mapping and scanning library for Go structs and slices.
github.com/stephenafamo/scan
Type safe and flexible package for scanning database data into Go types. Supports, structs, maps, slices and custom mapping functions.
https://github.com/z0ne-dev/mgx
Code first migration library for native pgx (no database/sql abstraction).