[squid-users] Squid isn't using persistent server-side connections upon POST/PUT/DELETE

Good evening Squid experts,
We have a serious issue with persistent connections. We are testing a large REST-Application which heavily depends on caching and correct cache behaviour in particular concering the rather scarcely used HTTP-Methods like PUT.The scenario is the following:
Client <---> Squid Forward Proxy Cache in the same Network <---> Server
The Squid (Version 3.1.11 on Ubuntu Linux 11.04 running in VmWare 8) is configured as an explicit proxy, i.e. our client sends the full path-uri and host of the origin server over a tcp connection to the proxy on port 3128. Our client normally is the Restlet-Java-Framework, but we also used nodejs, telnet and curl to verify the behaviour, which we don't understand:
GET works as expected:
Client <-GET; persistent tcp connection-> Squid Forward Cache in the same Network <-GET; persistent tcp connection-> Server
POST and PUT don't:
Client <-POST/PUT; persistent tcp connection-> Squid Forward Cache in the same Network <-POST/PUT; NO! persistent tcp connection-> Server
The Squid finishes the TCP-Connection, although via Connection: keep-alive it explicitly announces not to do so. That is a very bad performance killer since the TCP slow start algorithm is - yes indeed - slow. Squid opens a new TCP connection for every POST and PUT for the server. With a single unexplicable exception:
Client <-PUT without Body (Content); persistent tcp connection-> Squid Forward Cache in the same Network <-PUT without body; persistent tcp connection-> Server
So only if the PUT doesn't contain an HTTP-Body, the squid<->server connection is persistent. The client<->squid connection on the other hand is persistent (we use both the non-standard Proxy-Connection: keep-alive and the correct hopy-by-hop Connection: keep-alive in our request).
Here is a detailed example of a successful GET:
1. Client sends to Squid:
GET http://192.168.14.194/ HTTP/1.1Cache-Control: no-cache, max-age=0Date: Tue, 11 Oct 2011 16:53:44 GMTContent-Length: 0Accept: application/x-java-serialized-object, application/x-java-serialized-object+xmlHost: 192.168.14.194User-Agent: Restlet-Framework/2.1rc1Connection: keep-aliveProxy-Connection: keep-alive
2. Squid sends to Server:
GET / HTTP/1.1Date: Tue, 11 Oct 2011 16:53:44 GMTContent-Length: 0Accept: application/x-java-serialized-object, application/x-java-serialized-object+xmlHost: 192.168.14.194User-Agent: Restlet-Framework/2.1rc1Via: 1.1 localhost (squid/3.1.11)X-Forwarded-For: 192.168.14.194Cache-Control: no-cache, max-age=0Connection: keep-alive
3. Server sends to Squid:
HTTP/1.1 200 OKDate: Tue, 11 Oct 2011 16:53:44 GMTServer: Restlet-Framework/2.1rc1Vary: Accept-Charset, Accept-Encoding, Accept-Language, AcceptConnection: keep-aliveContent-Length: 5Content-Type: text/plain; charset=UTF-8
blubb

4. Squid sends to Client:
HTTP/1.0 200 OKDate: Tue, 11 Oct 2011 16:53:44 GMTServer: Restlet-Framework/2.1rc1Vary: Accept-Charset, Accept-Encoding, Accept-Language, AcceptContent-Length: 5Content-Type: text/plain; charset=UTF-8X-Cache: MISS from localhostX-Cache-Lookup: MISS from localhost:3128Via: 1.0 localhost (squid/3.1.11)Connection: keep-alive
blubb
5. Both TCP-Connections Client<->Squid and Squid<->Server are still open and used for subsequent GET Requests.
And now an example of a failed persistent connection between squid and server:
1. Client sends to Squid:
POST http://192.168.14.194/ HTTP/1.1Cache-Control: no-cache, max-age=0Date: Tue, 11 Oct 2011 16:53:46 GMTContent-Length: 4Content-Type: text/plain; charset=UTF-8Accept: application/x-java-serialized-object, application/x-java-serialized-object+xmlHost: 192.168.14.194User-Agent: Restlet-Framework/2.1rc1Connection: keep-aliveProxy-Connection: keep-alive
test
2. Squid does Threeway-SYN-SYN/ACK-ACK-Handshake with Server and sends:
POST / HTTP/1.1Date: Tue, 11 Oct 2011 16:53:45 GMTContent-Length: 4Content-Type: text/plain; charset=UTF-8Accept: application/x-java-serialized-object, application/x-java-serialized-object+xmlHost: 192.168.14.194User-Agent: Restlet-Framework/2.1rc1Via: 1.1 localhost (squid/3.1.11)X-Forwarded-For: 192.168.14.194Cache-Control: no-cache, max-age=0Connection: keep-alive
test
3. Server replies over that connection:
HTTP/1.1 200 OKDate: Tue, 11 Oct 2011 16:53:45 GMTServer: Restlet-Framework/2.1rc1Vary: Accept-Charset, Accept-Encoding, Accept-Language, AcceptConnection: keep-aliveContent-Length: 4Content-Type: text/plain; charset=UTF-8
test
4. Wih FIN-ACK the Squid closes the connection to the server and replies to the client:
HTTP/1.0 200 OKDate: Tue, 11 Oct 2011 16:53:46 GMTServer: Restlet-Framework/2.1rc1Vary: Accept-Charset, Accept-Encoding, Accept-Language, AcceptContent-Length: 4Content-Type: text/plain; charset=UTF-8X-Cache: MISS from localhostX-Cache-Lookup: MISS from localhost:3128Via: 1.0 localhost (squid/3.1.11)Connection: keep-alive
test
5. For the subsequent requests, the client uses the same connection to the Squid, which is still open and the Squid opens a new connection to server to forward a PUT or POST. If the Squid isn't used at all, the client<->squid connection is persistent.
To us that behaviour seems inconsistent and faulty. RFC 2616 doesn't make any comments about closing a persistent connection if a POST/PUT/DELETE was send. Thus it's not reasonable at all. Is it a bug?
Btw, yesterday we already reported an unrelated TCP issue of Squid, which totally abandons a Connection if a Request having an If-Match Header results in a Cache Hit (http://bugs.squid-cache.org/show_bug.cgi?id=3379 ).
We are very thankful for any suggestions concerning our persistent connection problem, which keeps us up at night. We are trying to run a benchmark comparison and the TCP-Connection-kills totally ruin the performance.
Thanks in advance and bye,Flo & Felix, University of Hamburg
Received on Tue Oct 11 2011 - 18:25:13 MDT