Commit Graph

447 Commits (7fceb64deee5adddfcb40abacd271a74a596fca5)

Author SHA1 Message Date
Nick Farrell b770252a3b
Fix bug relating to reuse of types
When `LoadTypes` is being called, it does not include the
namespace-qualified types in its result. While these namespaces are
visible to `LoadTypes` itself, `RegisterTypes` will not recognise this
form of the types, only allowing them to be used if they are on the
schema path, and referred to without their namespace component.
2024-07-07 11:26:19 +10:00
Nick Farrell 47977703e1
Load types using a single SQL query
When loading even a single type into pgx's type map, multiple SQL
queries are performed in series. Over a slow link, this is not ideal.
Worse, if multiple types are being registered, this is repeated multiple
times.

This commit add LoadTypes, which can retrieve type
mapping information for multiple types in a single SQL call, including
recursive fetching of dependent types.
RegisterTypes performs the second stage of this operation.
2024-07-01 15:34:17 +10:00
Carlos Pérez-Aradros Herce 9f4a264f89 Fix interval encoding to allow 0s and avoid extra spaces
Fix a bugs introduced by 01d649b, also add some tests
2024-05-30 09:48:53 +02:00
Jack Christensen b4911f1da7
Merge pull request #2019 from jackc/fix-encode-driver-valuer-on-pointer
Fix encode driver.Valuer on pointer
2024-05-25 11:20:25 -05:00
Jack Christensen 9ca9203afb Move typed nil handling to Map.Encode from anynil
The new logic checks for any type of nil at the beginning of Encode and
then either treats it as NULL or calls the driver.Valuer method if
appropriate.

This should preserve the existing nil normalization while restoring the
ability to encode nil driver.Valuer values.
2024-05-18 22:39:28 -05:00
Jack Christensen 79cab4640f Only use anynil inside of pgtype 2024-05-18 21:06:23 -05:00
Jack Christensen 13beb380f5 Fix encode driver.Valuer on nil-able non-pointers
https://github.com/jackc/pgx/issues/1566
https://github.com/jackc/pgx/issues/1860
https://github.com/jackc/pgx/pull/2019#discussion_r1605806751
2024-05-18 17:17:46 -05:00
Mitar 732889728f Add support for custom JSON marshal and unmarshal.
The Codec interface is now implemented by *pgtype.JSONCodec
and *pgtype.JSONBCodec instead of pgtype.JSONCodec and
pgtype.JSONBCodec, respectively. This is technically a breaking
change, but it is extremely unlikely that anyone is depending on this,
and if there is downstream breakage it is trivial to fix.

Fixes #2005.
2024-05-18 08:02:09 -05:00
Jack Christensen 2a36a7032e Fix encode driver.Valuer on pointer
pgx v5 introduced nil normalization for typed nils. This means that
[]byte(nil) is normalized to nil at the edge of the encoding system.
This simplified encoding logic as nil could be encoded as NULL and type
specific handling was unneeded.

However, database/sql compatibility requires Value to be called on a
nil pointer that implements driver.Valuer. This was broken by
normalizing to nil.

This commit changes the normalization logic to not normalize pointers
that directly implement driver.Valuer to nil. It still normalizes
pointers that implement driver.Valuer through implicit derefence.

e.g.

type T struct{}

func (t *T) Value() (driver.Value, error) {
  return nil, nil
}

type S struct{}

func (s S) Value() (driver.Value, error) {
  return nil, nil
}

(*T)(nil) will not be normalized to nil but (*S)(nil) will be.

https://github.com/jackc/pgx/issues/1566
2024-05-18 07:41:10 -05:00
Oleksandr Redko c05cce7d41 Fix test asserts: reverse expected-actual 2024-05-14 20:07:10 -05:00
Jack Christensen 48cdd7bab0 Allow scanning time without time zone into string
https://github.com/jackc/pgx/issues/2002
2024-05-10 10:52:41 -05:00
Carlos Pérez-Aradros Herce 01d649b2bf Do not encode interval microseconds when they are 0
This make the encode match what postgres does
2024-05-09 17:29:13 -05:00
Jack Christensen 8649231bb3 Add ScanLocation to pgtype.TimestampCodec
If ScanLocation is set, the timestamps will be assumed to be in the
given location when scanning from the database.

The Codec interface is now implemented by *pgtype.TimestampCodec instead
of pgtype.TimestampCodec. This is technically a breaking change, but it
is extremely unlikely that anyone is depending on this, and if there is
downstream breakage it is trivial to fix.

https://github.com/jackc/pgx/issues/1195
https://github.com/jackc/pgx/issues/1945
2024-05-08 08:35:05 -05:00
Jack Christensen 33360ab479 Add ScanLocation to pgtype.TimestamptzCodec
If ScanLocation is set, it will be used to convert the time to the given
location when scanning from the database.

The Codec interface is now implemented by *pgtype.TimestamptzCodec
instead of pgtype.TimestamptzCodec. This is technically a breaking
change, but it is extremely unlikely that anyone is depending on this,
and if there is downstream breakage it is trivial to fix.

https://github.com/jackc/pgx/issues/1195
https://github.com/jackc/pgx/issues/1945
2024-05-08 08:35:05 -05:00
Oleksandr Redko 60a01d044a Fix typos in doc comments 2024-04-17 12:00:02 -05:00
Felix Röhrich fc416d237a make parsing stricter and add corresponding test 2024-04-16 12:18:06 -05:00
Carlos Pérez-Aradros Herce 78b22c3d2f fix tests 2024-03-20 18:21:11 -05:00
Carlos Pérez-Aradros Herce 221ad1b84c Add support for macaddr8 type
Postgres also has a `macaddr8` type, this PR adds support for it, using
the same codec as `macaddr`
2024-03-20 18:21:11 -05:00
Jack Christensen 88dfc22ae4 Fix simple protocol encoding of json.RawMessage
The underlying type of json.RawMessage is a []byte so to avoid it being
considered binary data we need to handle it specifically. This is done
by registerDefaultPgTypeVariants. In addition, handle json.RawMessage in
the JSONCodec PlanEncode to avoid it being mutated by json.Marshal.

https://github.com/jackc/pgx/issues/1763
2024-03-02 15:12:20 -06:00
David Kurman d149d3fe5c Fix panic in TryFindUnderlyingTypeScanPlan
Check if CanConvert before calling reflect.Value.Convert
2024-02-26 17:51:56 -06:00
Jack Christensen 85f15c4b3c Fix scan float4 into sql.Scanner
https://github.com/jackc/pgx/issues/1911
2024-02-23 18:18:03 -06:00
Jack Christensen 654dcab93e Fix: pgtype.Bits makes copy of data from read buffer
It was taking a reference. This would cause the data to be corrupted by
future reads.

fixes #1909
2024-02-23 17:40:11 -06:00
Jack Christensen 7b5fcac465 Add timetz and []timetz OID constants
https://github.com/jackc/pgx/issues/1883
2024-01-27 18:55:59 -06:00
Florent Viel 0fa533386c add ltree pgtype support 2024-01-26 09:06:13 -06:00
Kirill Malikov 517c654e2c feat: fast encodeUUID 2024-01-20 20:50:01 -06:00
Eshton Robateau 20bf953a17 pull out changes into new public function 2023-12-09 11:20:14 -06:00
Eshton Robateau 12582a0fd4 bitsize largest option is 64 2023-12-09 11:20:14 -06:00
Eshton Robateau 905f252667 uncomment tests 2023-12-09 11:20:14 -06:00
Eshton Robateau 9927e14bbf remove dead line 2023-12-09 11:20:14 -06:00
Eshton Robateau 95b2f85e60 support scientific notation big floats 2023-12-09 11:20:14 -06:00
Simon Paredes 4678e69599 fix error message to print the unexpected rune 2023-12-01 18:23:23 -06:00
Simon Paredes 89d699c2e8 wrap errors instead of just formatting them 2023-12-01 18:23:23 -06:00
Jacopo 7ebced92b5 Fix issue with order of json encoding #1805 2023-11-24 19:01:48 -06:00
Kirill Mironov d3fb6e00da implement json.Marshaler and json.Unmarshaler for Float4, Float8 2023-11-04 10:25:31 -05:00
Jack Christensen 45f807fdb4 Special case the underlying type of []byte
Underlying types were already tried. But []byte is not a normal
underlying type. It is a slice. But since is can be treated as a scalar
instead of an array / slice we need to special case it.

https://github.com/jackc/pgx/issues/1763
2023-10-12 20:52:49 -05:00
Ville Skyttä c6c50110db Spelling and grammar fixes 2023-10-07 09:26:23 -05:00
Ville Skyttä 24ed0e4257 Make use of strings.Cut 2023-10-04 20:41:55 +03:00
horpto f4533dc906 optimize parseNumericString 2023-07-25 19:25:23 -05:00
Jack Christensen 492283b90b zeronull.Timestamptz should use pgtype.Timestamptz
https://github.com/jackc/pgx/issues/1694
2023-07-22 08:35:32 -05:00
Jack Christensen d58fe2d53c Fix json scan of non-string pointer to pointer
https://github.com/jackc/pgx/issues/1691
2023-07-19 20:54:05 -05:00
Christoph Engelbert (noctarius) bd3e0d422c Fixes #1684 QCharArrayOID being defined with the wrong OID 2023-07-15 09:44:48 -05:00
Jack Christensen 05440f9d3f Drastically increase allowed test times for potato CI
The context timeouts for tests are designed to give a better error
message when something hangs rather than the test just timing out.
Unfortunately, the potato CI frequently has some test or another
randomly take a long time. While the increased times are somewhat less
than optimal on a real computer, hopefully this will solve the
flickering CI.
2023-07-11 21:16:08 -05:00
Jack Christensen e0c70201dc Skip json format test on CockroachDB 2023-07-11 20:51:22 -05:00
Jack Christensen 524f661136 Fix JSON encoding for pointer to structs implementing json.Marshaler
https://github.com/jackc/pgx/issues/1681
2023-07-11 20:28:36 -05:00
Dan McGee 507a9e9ad3 Remove some now unused pgtype code
Most of this is in conversion, and I assume it became unused with some
of the v5 changes and refactors to a codec-based approach.

There are likely a few more cleanups to be made, but these ones seemed
easy and safe to start with.
2023-07-10 20:23:42 -05:00
Dan McGee 0328d314ea Use bytes.Equal rather than bytes.Compare ==/!= 0
As recommended by go-staticcheck, but also might be a bit more efficient
for the compiler to implement, since we don't care about which slice of
bytes is greater than the other one.
2023-07-08 12:08:05 -05:00
Evan Jones dc94db6b3d pgtype.Hstore: add a round-trip test for binary and text codecs
This ensures the output of Encode can pass through Scan and produce
the same input. This found two two minor problems with the text
codec. These are not bugs: These situations do not happen when using
pgx with Postgres. However, I think it is worth fixing to ensure the
code is internally consistent.

The problems with the text codec are:

* It did not correctly distinguish between nil and empty. This is not
  a problem with Postgres, since NULL values are marked separately,
  but the binary codec distinguishes between them, so it seems like
  the text codec should as well.
* It did not output spaces between keys. Postgres produces output in
  this format, and the parser now only strictly parses the Postgres
  format. This is not a bug, but seems like a good idea.
2023-06-29 17:25:47 -05:00
Lev Zakharov 7c386112e3 fix concurrency bug in pgtype.defaultMap (#1650) 2023-06-18 08:23:56 -05:00
Evan Jones e5db6a0467 pgtype array: Fix encoding of vtab \v
Arrays with values that start or end with vtab ("\v") must be quoted.
Postgres's array parser skips leading and trailing whitespace with
the array_isspace() function, which is slightly different from the
scanner_isspace() function that was previously linked. Add a test
that reproduces this failure, and fix the definition of isSpace.

This also includes a change to use strings.EqualFold which should
really not matter, but does not require copying the string.
2023-06-17 17:15:58 -05:00
Evan Jones bc8b1ca320 remove the single backing string optimization
This is a bit slower than using this optimization, but ensures this
version does not change garbage collection behavior. This does still
using a single []string for all the *string value pointers because
that is what text parsing already does. This makes the two behave
similarly.

benchstat results of master versus this version:

                               │  orig.txt   │   new-binary-no-share-string.txt    │
                               │   sec/op    │   sec/op     vs base                │
HstoreScan/databasesql.Scan-10   82.11µ ± 1%   81.71µ ± 2%        ~ (p=0.280 n=10)
HstoreScan/text-10               83.30µ ± 1%   82.45µ ± 1%   -1.02% (p=0.000 n=10)
HstoreScan/binary-10             15.99µ ± 2%   10.12µ ± 1%  -36.67% (p=0.000 n=10)
geomean                          47.82µ        40.86µ       -14.56%

                               │   orig.txt   │   new-binary-no-share-string.txt    │
                               │     B/op     │     B/op      vs base               │
HstoreScan/databasesql.Scan-10   56.23Ki ± 0%   56.23Ki ± 0%       ~ (p=0.128 n=10)
HstoreScan/text-10               65.12Ki ± 0%   65.12Ki ± 0%       ~ (p=0.541 n=10)
HstoreScan/binary-10             21.09Ki ± 0%   19.87Ki ± 0%  -5.75% (p=0.000 n=10)
geomean                          42.58Ki        41.75Ki       -1.95%

                               │  orig.txt  │    new-binary-no-share-string.txt    │
                               │ allocs/op  │ allocs/op   vs base                  │
HstoreScan/databasesql.Scan-10   744.0 ± 0%   744.0 ± 0%        ~ (p=1.000 n=10) ¹
HstoreScan/text-10               743.0 ± 0%   743.0 ± 0%        ~ (p=1.000 n=10) ¹
HstoreScan/binary-10             464.0 ± 0%   316.0 ± 0%  -31.90% (p=0.000 n=10)
geomean                          635.4        559.0       -12.02%
¹ all samples are equal

benchstat results of the version with one string and this version:

                               │ new-binary-share-everything.txt │    new-binary-no-share-string.txt    │
                               │             sec/op              │    sec/op     vs base                │
HstoreScan/databasesql.Scan-10                       81.80µ ± 1%    81.71µ ± 2%        ~ (p=1.000 n=10)
HstoreScan/text-10                                   82.77µ ± 1%    82.45µ ± 1%        ~ (p=0.063 n=10)
HstoreScan/binary-10                                 7.330µ ± 2%   10.124µ ± 1%  +38.13% (p=0.000 n=10)
geomean                                              36.75µ         40.86µ       +11.18%

                               │ new-binary-share-everything.txt │   new-binary-no-share-string.txt    │
                               │              B/op               │     B/op      vs base               │
HstoreScan/databasesql.Scan-10                      56.23Ki ± 0%   56.23Ki ± 0%       ~ (p=0.232 n=10)
HstoreScan/text-10                                  65.12Ki ± 0%   65.12Ki ± 0%       ~ (p=0.218 n=10)
HstoreScan/binary-10                                20.73Ki ± 0%   19.87Ki ± 0%  -4.11% (p=0.000 n=10)
geomean                                             42.34Ki        41.75Ki       -1.39%

                               │ new-binary-share-everything.txt │     new-binary-no-share-string.txt     │
                               │            allocs/op            │  allocs/op   vs base                   │
HstoreScan/databasesql.Scan-10                        744.0 ± 0%    744.0 ± 0%         ~ (p=1.000 n=10) ¹
HstoreScan/text-10                                    743.0 ± 0%    743.0 ± 0%         ~ (p=1.000 n=10) ¹
HstoreScan/binary-10                                  41.00 ± 0%   316.00 ± 0%  +670.73% (p=0.000 n=10)
geomean                                               283.0         559.0        +97.53%
¹ all samples are equal
2023-06-16 15:31:37 -05:00