The test was relying on sending so big a message that the write blocked.
However, it appears that on Windows the TCP connections over localhost
have an very large or infinite sized buffer. Change the test to simply
set the deadline to the current time before triggering the write.
The first 5 fake non-blocking reads are limited to 1 byte. This should
ensure that there is a measurement of a read where bytes are already
waiting in Go or the OS's read buffer.
The reason for a high max wait time was to ensure that reads aren't
cancelled when there is data waiting for it in Go or the OS's receive
buffer. Unfortunately, there is no way to know ahead of time how long
this should take.
This new code uses 2x the fastest successful read time as the max read
time. This allows the code to adapt to whatever host it is running on.
https://github.com/jackc/pgx/issues/1481
Previously, a batch with 10 unique parameterized statements executed
100 times would entail 11 network round trips. 1 for each prepare /
describe and 1 for executing them all. Now pipeline mode is used to
prepare / describe all statements in a single network round trip. So it
would only take 2 round trips.
While this test always worked on my machine, it flickered in CI. And to
be fair the test can't guarantee the condition it is testing. Work
around this by trying many times before admitting failure.
This eliminates an edge case that can cause a deadlock and is a
prerequisite to cheaply testing connection liveness and to recoving a
connection after a timeout.
https://github.com/jackc/pgconn/issues/27
Squashed commit of the following:
commit 0d7b0dddea
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 25 13:15:05 2022 -0500
Add test for non-blocking IO preventing deadlock
commit 79d68d23d3
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 18 18:23:24 2022 -0500
Release CopyFrom buf when done
commit 95a43139c7
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 18 18:22:32 2022 -0500
Avoid allocations with non-blocking write
commit 6b63ceee07
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 18 17:46:49 2022 -0500
Simplify iobufpool usage
commit 60ecdda02e
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 18 11:51:59 2022 -0500
Add true non-blocking IO
commit 7dd26a34a1
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 4 20:28:23 2022 -0500
Fix block when reading more than buffered
commit afa702213f
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 4 20:10:23 2022 -0500
More TLS support
commit 51655bf8f4
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 4 17:46:00 2022 -0500
Steps toward TLS
commit 2b80beb1ed
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 4 13:06:29 2022 -0500
Litle more TLS support
commit 765b2c6e7b
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 4 12:29:30 2022 -0500
Add testing of TLS
commit 5b64432afb
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 4 09:48:19 2022 -0500
Introduce testVariants in prep for TLS
commit ecebd7b103
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 4 09:32:14 2022 -0500
Handle and test read of previously buffered data
commit 09c64d8cf3
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 4 09:04:48 2022 -0500
Rename nbbconn to nbconn
commit 73398bc67a
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 4 08:59:53 2022 -0500
Remove backup files
commit f1df39a29d
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 4 08:58:05 2022 -0500
Initial passing tests
commit ea3cdab234
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat Jun 4 08:38:57 2022 -0500
Fix connect timeout
commit ca22396789
Author: Jack Christensen <jack@jackchristensen.com>
Date: Thu Jun 2 19:32:55 2022 -0500
wip
commit 2e7b46d5d7
Author: Jack Christensen <jack@jackchristensen.com>
Date: Mon May 30 08:32:43 2022 -0500
Update comments
commit 7d04dc5caa
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat May 28 19:43:23 2022 -0500
Fix broken test
commit bf1edc77d7
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat May 28 19:40:33 2022 -0500
fixed putting wrong size bufs
commit 1f7a855b2e
Author: Jack Christensen <jack@jackchristensen.com>
Date: Sat May 28 18:13:47 2022 -0500
initial not quite working non-blocking conn
Use an internal buffer in pgproto3.Frontend and pgproto3.Backend instead
of directly writing to the underlying net.Conn. This will allow tracing
messages as well as simplify pipeline mode.
It is impossible to guarantee that the a query executed with the simple
protocol will behave the same as with the extended protocol. This is
because the normal pgx path relies on knowing the OID of query
parameters. Without this encoding a value can only be determined by the
value instead of the combination of value and PostgreSQL type. For
example, how should a []int32 be encoded? It might be encoded into a
PostgreSQL int4[] or json.
Removal also simplifies the core query path.
The primary reason for the simple protocol is for servers like PgBouncer
that may not be able to support normal prepared statements. After
further research it appears that issuing a "flush" instead "sync" after
preparing the unnamed statement would allow PgBouncer to work.
The one round trip mode can be better handled with prepared statements.
As a last resort, all original server functionality can still be accessed by
dropping down to PgConn.
Jack Christensen
Alexey Palazhchenko
Mark Chambers
Andrew Rusakow
Andrew Nicoll