…sors

Previously, we were treating request/response processors as forwarder methods,
and the original intention was to perform a connection migration inline with
the forwarding. However, this has proved to cause confusions and complications.
The new connection migration design uses a different approach by performing
the connection migration out-of-band with the forwarding processors. For this
to work, we would need to be able to suspend and resume those processors.
This commit implements support for that.

Additionally, we no longer wrap clientConn with a custom readTimeoutConn
wrapper as that seems to be problematic with idle connections. There's a
similar discussion here: cockroachdb#25585. Due to that, context cancellations from the
parent no longer close the connection if we are blocked on IO. We could
theoretically spin up a new goroutine in the forwarder to check for this, and
close the connections accordingly, but this case is rare, so we'll let the
caller handle this.

Release justification: sqlproxy-only change, and only used within
CockroachCloud.

Release note: None

…sors

Previously, we were treating request/response processors as forwarder methods,
and the original intention was to perform a connection migration inline with
the forwarding. However, this has proved to cause confusions and complications.
The new connection migration design uses a different approach by performing
the connection migration out-of-band with the forwarding processors. For this
to work, we would need to be able to suspend and resume those processors.
This commit implements support for that.

Additionally, we no longer wrap clientConn with a custom readTimeoutConn
wrapper as that seems to be problematic with idle connections. There's a
similar discussion here: cockroachdb#25585. Due to that, context cancellations from the
parent no longer close the connection if we are blocked on IO. We could
theoretically spin up a new goroutine in the forwarder to check for this, and
close the connections accordingly, but this case is rare, so we'll let the
caller handle this.

Release justification: sqlproxy-only change, and only used within
CockroachCloud.

Release note: None

…sors

Previously, we were treating request/response processors as forwarder methods,
and the original intention was to perform a connection migration inline with
the forwarding. However, this has proved to cause confusions and complications.
The new connection migration design uses a different approach by performing
the connection migration out-of-band with the forwarding processors. For this
to work, we would need to be able to suspend and resume those processors.
This commit implements support for that.

Additionally, we no longer wrap clientConn with a custom readTimeoutConn
wrapper as that seems to be problematic with idle connections. There's a
similar discussion here: cockroachdb#25585. Due to that, context cancellations from the
parent no longer close the connection if we are blocked on IO. We could
theoretically spin up a new goroutine in the forwarder to check for this, and
close the connections accordingly, but this case is rare, so we'll let the
caller handle this.

Release justification: sqlproxy-only change, and only used within
CockroachCloud.

Release note: None

124373: pgwire: remove readTimeoutConn in favor of a channel r=yuzefovich a=rafiss

Rather than using a connection that polls for the context being done every second, we now spin up an additional goroutine that blocks until the connection context is done, or the drain signal was received.

I wrote a simple benchmark of the idle connection loop to generate CPU profiles.

With the old readTimeoutConn:
<img width="861" alt="image" src="https://github.com/cockroachdb/cockroach/assets/1320573/9e0c88fc-eda7-4a51-8060-280b0d95a7a7">

With the new goroutine approach:
<img width="567" alt="image" src="https://github.com/cockroachdb/cockroach/assets/1320573/535ea962-ec03-46f7-a63b-382c65553553">

There's definitely less noise and overhead.

fixes #25585
Release note: None

Co-authored-by: Rafi Shamim <rafi@cockroachlabs.com>