Commit Graph

486 Commits (7555c43033a6ea94c7fcc6ac5b5d37862cf932d6)

Author SHA1 Message Date
Jack Christensen e9770d6ff9 Add listen / notify implemented with pgconn
fixes #553
2019-08-24 10:49:10 -05:00
Jack Christensen b7b52ff079 Require conn and pool configs to be created by ParseConfig
This simplifies handling default values. Now there is no ambiguity
between a zero value and a default value. All default values are set by
ParseConfig and the user can modify them after the initial creation.

fixes #567
2019-08-24 09:53:38 -05:00
Jack Christensen f3c703a102 Remove Conn SetLogger and SetLogLevel
Simplifying interface
2019-08-17 15:34:29 -05:00
Jack Christensen 99c54fbec0 Tweak Conn comment 2019-08-17 13:46:22 -05:00
Jack Christensen 3028821487 Add context.Context to Logger interface
This allows custom logger adapters to add additional fields to log
messages. For example, a HTTP server may with to log the request ID.

fixes #428
2019-08-03 16:16:21 -05:00
Jack Christensen ab1edc79e0 Remove 0 bytes when sanitizing identifiers
Port of 95ea78048a from v3.
2019-08-03 09:51:03 -05:00
Jack Christensen 29f02807b0 Restore simple protocol support 2019-05-20 20:36:03 -05:00
Jack Christensen b0dac84d77 Use extended query builder everywhere 2019-05-17 13:59:41 -05:00
Jack Christensen 79f49ce300 Convert large objects to prepared statements
This allows removing semi-obsolete fastpath interface support. See
https://www.postgresql.org/docs/current/libpq-fastpath.html. This also
simplifies introducing context support.
2019-05-11 11:36:38 -05:00
Jack Christensen 583c8d3b25 Use pgproto3.FieldDescription instead of pgx version
This allows removing a malloc and memcpy.
2019-05-04 13:47:18 -05:00
Jack Christensen ea31df3b50 Extract extendedQueryBuilder 2019-05-04 12:32:21 -05:00
Jack Christensen 8b365ce37e Extract int16SlicePool 2019-05-04 11:01:09 -05:00
Jack Christensen dd571cf345 Remove another malloc from query path 2019-05-04 09:01:57 -05:00
Jack Christensen db68713809 Remove malloc from query path 2019-05-04 08:58:02 -05:00
Jack Christensen 85ddbfeeee Remove 2 mallocs from query path 2019-05-04 08:55:05 -05:00
Jack Christensen 243f9031b3 Remove extra prepare in stdlib 2019-04-27 15:45:30 -05:00
Jack Christensen 7b1272d254 Add SendBatch to pool 2019-04-25 15:07:35 -05:00
Jack Christensen 00d123a944 Prune methods that delegated to *pgconn.PgConn 2019-04-25 14:25:16 -05:00
Jack Christensen aed6b822d9 Split batch command and result 2019-04-24 16:39:06 -05:00
Jack Christensen c7d03eb555 Add RowsFromResultReader 2019-04-24 15:57:50 -05:00
Jack Christensen a19ca0638f Reorganize conn and rows 2019-04-24 14:16:52 -05:00
Jack Christensen 1b8f0016e9 Split pgtype into own repo 2019-04-20 19:20:00 -05:00
Jack Christensen cc3461e65d Use golang.org/x/xerrors 2019-04-20 17:43:44 -05:00
Jack Christensen efb333df6b Fix go modules
Wow. This is fun. Sure is easy to get modules wrong when upgrading a v2+
project.
2019-04-20 17:41:08 -05:00
Jack Christensen b7e56b003a Replace lastStmtSent with pgconn support 2019-04-20 17:12:20 -05:00
Jack Christensen 35a0f64876 Pass ctx to PgConn.Prepare 2019-04-20 12:06:11 -05:00
Jack Christensen de40fea100 Remove unused variables 2019-04-20 12:00:07 -05:00
Jack Christensen 66625e6489 Prepare takes context
Also remove PrepareEx. It's primary usage was for context. Supplying
parameter OIDs is unnecessary when you can type cast in the query SQL.
If it does become necessary or desirable to add options back it can be
added in a backwards compatible way by adding a varargs as last
argument.
2019-04-20 11:47:16 -05:00
Jack Christensen 95756b1d7f Deallocate takes context 2019-04-20 11:34:52 -05:00
Jack Christensen 427cfe1c16 Remove unnecessary mux 2019-04-19 16:23:06 -05:00
Jack Christensen 2978a7440a Use pgconn for more conn status 2019-04-19 16:18:47 -05:00
Jack Christensen ae73ca2da1 Update to more effecient pgconn 2019-04-19 15:55:45 -05:00
Jack Christensen 1b7ff4d774 Remove unused config field 2019-04-13 18:15:02 -05:00
Jack Christensen a6bdd8fd49 Hard code standard PostgreSQL types
Instead of needing to instrospect the database on connection preload the
standard OID / type map. Types from extensions (like hstore) and custom
types can be registered by the application developer. Otherwise, they
will be treated as strings.
2019-04-13 16:45:52 -05:00
Jack Christensen c53c9e6eb5 Remove simple protocol and one round trip query options
It is impossible to guarantee that the a query executed with the simple
protocol will behave the same as with the extended protocol. This is
because the normal pgx path relies on knowing the OID of query
parameters. Without this encoding a value can only be determined by the
value instead of the combination of value and PostgreSQL type. For
example, how should a []int32 be encoded? It might be encoded into a
PostgreSQL int4[] or json.

Removal also simplifies the core query path.

The primary reason for the simple protocol is for servers like PgBouncer
that may not be able to support normal prepared statements. After
further research it appears that issuing a "flush" instead "sync" after
preparing the unnamed statement would allow PgBouncer to work.

The one round trip mode can be better handled with prepared statements.

As a last resort, all original server functionality can still be accessed by
dropping down to PgConn.
2019-04-13 11:39:01 -05:00
Jack Christensen 5a374c467f Fix Exec prepared statement with 0 args 2019-04-13 08:37:15 -05:00
Jack Christensen ec0bae649a Fix Ping 2019-04-12 21:42:16 -05:00
Jack Christensen b78ecf26dc Fix encode empty value 2019-04-12 21:23:57 -05:00
Jack Christensen 3901f3ef88 Add Conn.PgConn escape hatch 2019-04-12 16:57:25 -05:00
Jack Christensen 3497f0e8f9 Expose ParseConfig 2019-04-12 16:56:54 -05:00
Jack Christensen 938ee9f434 Rows and Row are now interfaces 2019-04-11 17:58:16 -05:00
Jack Christensen 3e87a8b363 Conn.Close takes context 2019-04-10 14:56:14 -05:00
Jack Christensen 2f948c5249 Remove unused code 2019-04-10 14:33:35 -05:00
Jack Christensen acd15cf589 Remove some obsolete context handling 2019-04-10 14:25:06 -05:00
Jack Christensen 7718ee6207 Remove Ex versions of Query and QueryRow
Always require context and prepend options to arguments if necessary.
2019-04-10 12:12:22 -05:00
Jack Christensen b69179cebb Remove Conn.Listen and Conn.Unlisten
Use Conn.Exec instead to listen or unlisten.
2019-04-10 11:21:40 -05:00
Jack Christensen ec10fdde8b Core of new connection pool 2019-04-09 21:32:01 -05:00
Jack Christensen d619d8913e Remove cancel query logic 2019-04-05 11:04:57 -05:00
Jack Christensen 858d00788a Use extracted packages with Go modules 2019-04-05 10:59:47 -05:00
Jack Christensen d287776d97 Use LogLevel type consistently 2019-03-23 11:32:38 -05:00
Jack Christensen bd181764bf Remove ensureConnectionReadyForQuery 2019-02-02 13:00:31 -06:00
Jack Christensen 577bc57ba5 Remove sendSimpleQuery
In preparation for removing ready for query tracking.
2019-02-02 12:55:12 -06:00
Jack Christensen 1127f31cb3 Remove sendPreparedQuery and simplify sendSimpleQuery 2019-02-02 12:47:12 -06:00
Jack Christensen 52e8d0d679 Remove *Conn.sendQuery 2019-02-02 12:45:10 -06:00
Jack Christensen 1ccd6527f5 Remove rxMsg 2019-02-02 12:41:44 -06:00
Jack Christensen d3a2c1c107 Partial conversion of pgx to use pgconn 2019-01-26 16:46:30 -06:00
Jack Christensen c4050134cc Begin delegation of Prepare to pgconn 2019-01-12 12:19:12 -06:00
Jack Christensen 1257b89df7 Update main pgx package to compile with new pgconn 2019-01-05 19:00:41 -06:00
Jack Christensen e78fd95296 Fix switching CommandTag back to string 2019-01-05 18:53:07 -06:00
Jack Christensen 19a8df16b6 Move notice handling to pgconn 2019-01-02 13:15:26 -06:00
Jack Christensen 89c3d8af5d Remove ExecEx 2019-01-02 12:52:37 -06:00
Jack Christensen 12857ad05b Implement pgx.Conn.Exec in terms of pgconn.PgConn.Exec 2019-01-02 12:32:36 -06:00
Jack Christensen 7f9540438c pgx uses pgconn.CommandTag instead of own definition 2019-01-01 16:55:48 -06:00
Jack Christensen 356a6c43d2 Avoid allocating strings in common message types 2019-01-01 13:47:37 -06:00
Jack Christensen be7a7b44dc Access PID and SecretKey via method 2018-12-31 18:03:55 -06:00
Jack Christensen 1ff8024df9 Access underlying net.Conn via method
Also remove some dead code.
2018-12-31 18:00:08 -06:00
Jack Christensen af2fc36c27 Remove code moved to pgconn 2018-12-31 17:52:11 -06:00
Jack Christensen 378ccb8945 PG error type is *pgconn.PgError 2018-12-31 17:46:56 -06:00
Jack Christensen 103ab963eb pgx uses pgconn CancelRequest 2018-12-31 17:37:07 -06:00
Jack Christensen 084423ae69 Add context to potentially blocking methods 2018-12-31 17:17:11 -06:00
Jack Christensen b3c8a73dc7 Restructure connect process
- Moved lots of connection logic to pgconn from pgx
- Extracted pgpassfile package
2018-12-30 16:55:56 -06:00
Jack Christensen 9990e4894d Rename base package to pgconn 2018-12-28 17:09:56 -06:00
Jack Christensen 44de49ffa1 Rename runtime params to parameter status 2018-12-28 12:25:59 -06:00
Jack Christensen b63370e5d5 Rename base.Conn to base.PgConn
- pgx.Conn embeds base.PgConn privately
- Add pgx.Conn.ParameterStatus
2018-12-28 12:16:50 -06:00
Jack Christensen a9a178bc98 Merge branch 'master' into v4-experimental 2018-12-01 11:17:30 -06:00
Jack Christensen f25025a580
Merge pull request #486 from yobert/master
Fix two minor grammatical errors in source documentation
2018-12-01 10:38:57 -06:00
Joel Jensen 41b714110a Fix two minor grammatical errors in source documentation 2018-11-27 16:40:46 -07:00
James Hartig 6d336eccb1 Added LastStmtSent and use it to retry on errors if statement was not sent
Previously, a failed connection could be put back in a pool and when the
next query was attempted it would fail immediately trying to prepare the
query or reset the deadline. It wasn't clear if the Query or Exec call
could safely be retried since there was no way to know where it failed.

You can now call LastQuerySent and if it returns false then you're
guaranteed that the last call to Query(Ex)/Exec(Ex) didn't get far enough
to attempt to send the query. The call can be retried with a new
connection.

This is used in the stdlib to return a ErrBadConn if a network error
occurred and the statement was not attempted.

Fixes #427
2018-11-19 10:44:40 -05:00
Jack Christensen 65e69c5580 Initial base.Connect extraction 2018-11-12 18:08:46 -06:00
Jack Christensen 2e72ca854a First extraction of base.Conn 2018-11-10 18:48:43 -06:00
Jack Christensen 670e85136f Remove Conn.lastActivityTime
It was only read in one place and that was an optimization to get the
current time. Replaced that usage with time.Now()
2018-11-09 15:33:31 -06:00
Jack Christensen a0d2ce5a0e
Merge pull request #477 from fastest963/cancel
Added option to customize query cancellation and wait for ready
2018-11-08 19:30:15 -06:00
James Hartig 527f404bbc Added option to customize query cancellation and wait for ready
CockroachDB doesn't support 80877102 and doesn't plan to, so instead allow
the user to customize the cancellation with their own function. In our
function, we call CANCEL QUERY with the query_id based on the LocalAddr().

The WaitForReady method can be used by a pool to not put a connection back
in the pool until it is finished cancelled and ready for a new query.
2018-11-05 20:28:59 -05:00
Jack Christensen c10ee2b7bf
Merge pull request #473 from Mic92/unix-sockets
Add path query parameter to support unix sockets
2018-11-05 18:58:19 -06:00
Jörg Thalheim 3410ad9122
Add host query parameter to support unix sockets
Currently there is no way to specify unix sockets in the connection
url. This patch adds a `host` query parameter that allows to set the path.
2018-11-03 21:07:33 +00:00
Tejas Manohar e2010c858c tls: stop sending ssl_renegotiation_limit in startup message
This addresses https://github.com/jackc/pgx/issues/321 with the
fix @jackc proposed there. Redshift users that need to connect
w/ SSL currently fork the library to delete this parameter, e.g.

8e0028d742

And, that's annoying to keep up-to-date :)
2018-10-29 21:50:09 -07:00
Murat Kabilov d9f21d02a5 mind PreferSimpleProtocol parameter in the config Merge method 2018-07-24 16:03:26 +02:00
Jack Christensen 6556ef67cb Fix data race in domain handling 2018-04-07 14:26:26 -05:00
Jack Christensen db7df79e10 Add domain support
fixes #407
2018-04-07 14:04:16 -05:00
Jack Christensen a487caf45f Merge branch 'scheme-agnostic-uri' of https://github.com/sean-/pgx into sean--scheme-agnostic-uri 2018-02-03 11:43:46 -06:00
Jack Christensen f911070f68 Add new SSL config to envvar parsing 2018-02-03 11:20:42 -06:00
Sean Chittenden 6cfb436397
Parse URI-encoded connection strings as URIs.
CockroachDB is frequently referenced as: `cockroachdb://` as its
scheme.  Be scheme name agnostic.
2018-02-02 16:19:58 -08:00
Sean Chittenden 6a4303120f
Only read in TLS certs when the key and cert are present. 2018-02-02 08:37:23 -08:00
Sean Chittenden f0dc593c2f
Only initialize the CA if the path is not an empty string. 2018-02-02 08:24:41 -08:00
Sean Chittenden 8078930406
Add TLS arg parsing to ParseDSN().
Factor out the TLS cert handling and add it to `configTLS()` via a
`struct` argument.
2018-02-01 23:51:50 -08:00
Sean Chittenden d7f24b91f4
Make ParseURI() compatible with lib/pq's TLS keywords.
Add support for:

- `sslrootcert`
- `sslcert`
- `sslkey`

All three arguments, like thir `gitub.com/lib/pq` counterparts,
are filesystem paths.
2018-02-01 22:58:14 -08:00
Sean Chittenden 4506a3e359
Gratuitously rename `configSSL()` to `configTLS()`.
SSL is dead, Jim.
2018-02-01 15:07:59 -08:00
Jack Christensen 2c07b03087 Parse connect_timeout into Dial func
Instead of adding Timeout field which could conflict with custom Dial
func.
2018-01-13 18:02:13 -06:00
Jack Christensen 9281f057ae Merge branch 'timeout' of https://github.com/cyberdelia/pgx into cyberdelia-timeout 2018-01-13 17:31:22 -06:00
Jack Christensen bd76a96882 Add ConnConfig.PreferSimpleProtocol
Allows configuring on a connection basis to prefer the simple protocol /
disable implicit prepared statements.

refs #331
2018-01-13 12:05:33 -06:00
Timothée Peignier 1bec450326
Handle timeout parameters 2017-12-16 19:18:33 -08:00
James Lawrence 393e9f4fd3 allow overriding connection info 2017-12-16 18:02:21 -05:00
Felix Geisendörfer 4dec986171 Fix CrateDB support (regression in ab9a1af)
See https://github.com/jackc/pgx/issues/320#issuecomment-338457176
2017-11-09 13:53:58 +01:00
Jack Christensen ab9a1af65b Add support for array of enum
fixes #338
2017-10-17 20:31:11 -05:00
Felix Geisendörfer 52bec9bec5 Support CrateDB 2017-09-27 18:32:18 +02:00
Jack Christensen fd7b776540 Merge pull request #323 from kelseyfrancis/namespace-oids
Prefix types in namespaces other than pg_catalog or public
2017-09-14 13:59:51 -05:00
Kelsey Francis 953e08df99 Prefix types in namespaces other than pg_catalog or public
It's possible to define a type (e.g., an enum) with the same name in two
different schemas. When initializing data types after connecting, types
defined within schemas other than pg_catalog or public should be
qualified with their schema name to disambiguate them and ensure all
types with the same base name get added to the map of OID to type.

Prior to this commit, the last type scanned would "win", and all others
with the same name would be missing from the ConnInfo type maps, which
would subsequently cause any PREPARE involving columns of those missing
types to return the error "unknown oid".
2017-09-11 11:29:42 -07:00
Jack Christensen 9eb21591ee Update sslmode docs 2017-09-09 11:05:39 -05:00
Johan Brandhorst 30fa7cc37f Skip certificate verification for sslmode "require"
This more appropriately aligns the behaviour of the library with
that advertised by the postgres documentation.

According to the table on the official documentation page
https://www.postgresql.org/docs/current/static/libpq-ssl.html,
the "require" mode should be used when:

"I want my data to be encrypted, and I accept the overhead. I trust that the network will make sure I always connect to the server I want."

This maps reasonably well to a TLS config that skips certificate verification.
2017-09-08 20:44:10 +01:00
Kelsey Francis ea740fb316 Fix some invalid one round trip execs failing to return non-nil error
Prior to this commit, execEx() would write the one round trip exec to
the connection before first calling ensureConnectionReadyForQuery, which
ultimately caused any errors to be suppressed if the exec followed a
valid query, because the receive message processing would finish
successfully as soon as it received the ReadyForQuery that actually
belonged to the preceding query. So, the exec would never actually
receive the error message that it caused, leaving it to be incorrectly
received by the first subsequent query sent.
2017-08-28 18:51:35 -07:00
Jack Christensen f65776f084 Fix typo 2017-08-25 13:31:50 -05:00
Jack Christensen 4d0f8f041b Fix pid logging
fixes #300
2017-08-12 17:43:54 -05:00
Jack Christensen 53b4280456 Automatically register enum types
fixes #287
2017-07-04 11:38:50 -05:00
Jack Christensen 8f4178b3d3 Use github.com/pkg/errors 2017-06-04 21:30:03 -05:00
Jack Christensen 3ea41e6972 Remove unused global error 2017-06-04 21:22:34 -05:00
Jack Christensen fb90fb2729 Add notification response hook
refs #239
2017-06-04 21:18:26 -05:00
Jack Christensen 27ab289096 Use Go casing convention for OID 2017-06-03 11:53:49 -05:00
Jack Christensen fe0af9b357 Happy-path batch query mode 2017-06-02 14:54:17 -05:00
Jack Christensen dcf3ee2781 Fix sendPreparedQuery write error hang
If the Write call in sendPreparedQuery encountered a non-fatal error - which means it sent
no bytes. It still was marking the connection as not ready for query. That caused the next
call to hang.
2017-05-31 18:33:01 -05:00
Jack Christensen 90975ab5c2 Extract append message functions.
In general, pgproto3 types should be used. But these functions may be easier to
without incurring additional memory allocations.
2017-05-29 10:01:07 -05:00
Jack Christensen 4ee21a15de Use pgproto3 for startup message 2017-05-29 09:19:41 -05:00
Jack Christensen dd5de3e49e Add single round-trip mode for ExecEx 2017-05-29 09:11:52 -05:00
Jack Christensen 749fdfe7d5 Resolve race on conn.Close/die
Use sync.Mutex instead of atomic operations for clarity.
2017-05-21 19:35:37 -05:00
Jack Christensen 8a7165dd98 Add ctx to PrepareEx
Remove PrepareExContext
2017-05-20 18:03:59 -05:00
Jack Christensen 2df4b1406b Do not double call termContext in QueryEx
QueryEx was calling termContext and rows.fatal on err of sendPreparedQuery.
rows.fatal calls rows.Close which already calls termContext. This sequence of
calls was causing underlying io timeout errors to be returned instead of context
errors.

In addition, added fatalWriteErr helper method to allow recovery of write
timeout errors where no bytes were written.

This should solve flickering errors on Travis.
2017-05-20 10:58:44 -05:00
Jack Christensen e1397613fd Ping only makes sense with a context for timeout 2017-05-19 08:02:51 -05:00
Jack Christensen 8b6c32d13a Add ConnConfig.Merge 2017-05-06 15:20:40 -05:00
Jack Christensen 458dd24a9f Remove unneeded WriteBuf 2017-05-02 21:39:23 -05:00
Jack Christensen 280bce7078 Added log adapter for logrus
Also changed standard logger interface to take a map instead of varargs for
extra data.
2017-04-29 21:28:38 -05:00
Jack Christensen 855b735eae Add log adapters for testing and log15
Make LogLevel a type for Stringer interface.
2017-04-29 20:33:52 -05:00
Jack Christensen f04c58338b Introduce pgproto3 package
pgproto3 will wrap the message encoding and decoding for the PostgreSQL
frontend/backend protocol version 3.
2017-04-29 10:02:38 -05:00
Jack Christensen 7b1f461ec3 Add simple protocol suuport with (Query|Exec)Ex 2017-04-10 08:58:51 -05:00
James Lawrence c5d247830c enable sql.Open to support both DSN and URI based connection strings 2017-04-01 08:50:48 -05:00
Jack Christensen 7eae904eba Add int4range 2017-03-23 18:41:52 -05:00
Jack Christensen 9ab59a74a9 Remove oid constants from pgx 2017-03-18 14:59:34 -05:00
Jack Christensen a636ef31a4 Refactor encoding parameters for prepared statements 2017-03-18 14:23:04 -05:00
Jack Christensen ad2ce2ce3c Remove internalNativeGoTypeFormats 2017-03-18 14:02:55 -05:00
Jack Christensen 19c6689752 Add pgtype.Record and prerequisite restructuring
Because reading a record type requires the decoder to be able to look up oid
to type mapping and types such as hstore have types that are not fixed between
different PostgreSQL servers it was necessary to restructure the pgtype system
so all encoders and decodes take a *ConnInfo that includes oid/name/type
information.
2017-03-18 12:01:16 -05:00
j7b 94d56e8556 Support pgpass 2017-03-17 15:12:41 -05:00
Jack Christensen 5eb19bc66a Add *Conn.CopyFrom
This replaces *Conn.CopyTo. CopyTo was named incorrectly. In PostgreSQL
COPY FROM is the command that copies from the client to the server. In
addition, CopyTo does not accept a schema qualified table name. This
commit introduces the Identifier type which handles multi-part names and
correctly quotes/sanitizes them. The new CopyFrom method uses this
Identifier type.

Conn.CopyTo is deprecated.

refs #243 and #190
2017-03-17 08:25:49 -05:00
Jack Christensen ba5f97176a Move not null Oid to pgtype
In preparation to ConnInfo implementation.
2017-03-13 21:34:38 -05:00
Jack Christensen 26d57356f7 Remove old Scanner and Encoder system 2017-03-12 17:22:04 -05:00
Jack Christensen 7bb1f3677d Move hstore to pgtype
Also implement binary format
2017-03-12 17:06:06 -05:00
Jack Christensen 542eac08c6 Add json/jsonb to pgtype 2017-03-11 18:46:51 -06:00
Jack Christensen 743b98b298 Name PG types as words
Though this doesn't follow Go naming conventions exactly it makes names more
consistent with PostgreSQL and it is easier to read. For example, TIDOID becomes
TidOid. In addition this is one less breaking change in the move to V3.
2017-03-11 17:03:23 -06:00
Jack Christensen 6694e0e618 Move Tid to pgtype 2017-03-11 16:48:37 -06:00
Jack Christensen fa1c81fec4 Move ACLItem to pgtype 2017-03-11 16:13:05 -06:00
Jack Christensen 77c57c780d Add pgtype.ByteaArray
Also fix up quoting array elements for text arrays.
2017-03-11 13:32:32 -06:00
Jack Christensen fa36ad9196 Move "char" to pgtype 2017-03-07 19:39:57 -06:00
Jack Christensen 7b1dbd8558 Move Name to pgtype 2017-03-06 17:55:20 -06:00