Re: [squid-users] How to make squid ignore max-age?

On Mon, Apr 18, 2011 at 9:18 PM, Yang Zhang <yanghatespam_at_gmail.com> wrote:
> On Mon, Apr 18, 2011 at 9:08 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
>> On Mon, 18 Apr 2011 20:07:33 -0700, Yang Zhang wrote:
>>>
>>> We're using squid (3.0.STABLE19-1ubuntu0.1) as an application cache
>>> and we're trying to cache everything, for a long time, but max-age=0
>>> is throwing off squid.
>>
>> max-age=0 is called "reload" in HTTP terminology.
>>
>>>
>>> I have this refresh_pattern in squid.conf:
>>>
>>> $ grep ^refresh_pattern /etc/squid3/squid.conf
>>> refresh_pattern ^ftp:           1440    20%     10080
>>> refresh_pattern ^gopher:        1440    0%      1440
>>> refresh_pattern . 525600 100% 525600 override-expire ignore-private
>>> ignore-no-cache ignore-no-store
>>>
>>
>> To quote Frankenstein
>>  "Why? whyyyy?"
>>
>>
>>> I have the following response prepared to be sent from a dummy web server:
>>>
>>> $ cat response
>>> HTTP/1.1 200 OK
>>> Server: Apache-Coyote/1.1
>>> Expires: 0
>>> Pragma: no-cache
>>> Cache-Control: no-cache, must-revalidate, max-age=0
>>> Content-Type: text/html;charset=UTF-8
>>> Date: Mon, 18 Apr 2011 10:06:26 GMT
>>> Connection: close
>>
>> Those headers say two things:
>>  * that cached content MUST NOT be supplied to the client. EVER.
>>  * that any existing cached content also MUST be checked for updates on the
>> next request.
>>
>> Ignoring this requirement *from the server*. Again... why?
>>
>>
>> I can only imagine that you have fallen for the myth that "application
>> caches" must always provide HITs. This is false and following it will break
>> things for you in potentially nasty ways.
>>
>> A cachable application will not send max-age=0 or "Expires: 0", both of
>> which mean the object was obsolete the second it was produced.
>>
>> Squid operating as a reverse-proxy will cache the replies and keep updating
>> fresh copies as long as the server is available to present new ones. If the
>> server dies an 5xx start coming back, the stale versions will start to be
>> used. How long for and whether "must-revalidate" produces 4xx/5xx to the
>> client depends on particular squid versions.
>>  I recommend you grab the newest release you can for the best HTTP/1.1
>> caching behaviour. there are still things we are fixing but work is ongoing
>> so the newer the release the better things work. I maintain a PPA for Ubuntu
>> (https://launchpad.net/~yadi/+archive/ppa) with newer versions.
>>
>> <snip>
>>>
>>> This response prevents anything from being cached; all requests for
>>> http://localhost:8765 result in TCP_MISS.  I believe it's because of
>>> the max-age=0 - once I remove that, this request is cached fine.  Yet
>>> I had set override-expire in the refresh_pattern.
>>> (Note that I don't think it's a must-revalidate/ignore-revalidate (new
>>> in v3.1) problem - I can leave that in fine.)
>>
>> However "We're using squid (3.0.STABLE19-1ubuntu0.1)" ... not a v3.1
>> release.
>>
>> "ignore-revalidate" blocks the *client* from requesting a revalidation
>> (max-age=0). But still obeys the server controls.
>>
>>>
>>> Any hints on how to make squid ignore the max-age=0?  I've pored over
>>> the refresh_pattern documentation a bunch of times but still can't see
>>> what's wrong.  Thanks in advance for any help.
>>
>> If you must;
>>  * "ignore-reload" to ignore the max-age=0 from either client or server
>> IIRC.
>
> Just tried this out - it doesn't work.  The comments in the squid.conf
> also say that it's for max-age=0 from the client (no mention of
> server).
>
>>  * "reload-into-ims" to convert a client max-age=0 ("force a new copy or
>> die") into a soft IMS request ("new content or cached copy").
>
> The client is not issuing any max-age=0.  Tried adding this in anyway
> - didn't work.
>
>>
>> 'must-revalidate' will still have affect until you are using a 3.1 release
>> with the "ignore-must-revalidate". The result of revalidate should only be
>> 304 (TCP_IMS_*) or 200 (TCP_REFRESH_UPDATE), not a "TCP_MISS" since the
>> content being validated *is* in the cache.
>
> My point earlier is that the must-revalidate from the server doesn't
> seem be what's preventing the response from getting cached....

Also, FWIW, these -max-age=0 responses are making it *into* the cache
(verified by grepping /var/spool/squid3), for whatever reason (not
sure why Squid is deciding to keep them if only to later not use
them).

-- 
Yang Zhang
http://yz.mit.edu/