Re: [squid-users] Tuning for very expensive bandwidth links

On 02/04/11 11:46, Ed W wrote:
> Hi
>
>
>>> So the remote (client) side proxy would need an eCAP plugin that would
>>> modify the initial request to include an ETag. This would require some
>>> ability to interrogate what we have in cache and generate/request the
>>> ETag associated with what we have already - do you have a pointer to any
>>> API/code that I would need to look at to do this?
>>
>> I'm unsure sorry. Alex at The Measurement Factory has better info on
>> specific details of what the eCAP API can do.
>
> If I wanted to hack on Squid 3.2... Do you have a 60 second overview on
> the code points to examine with a view to basically:
>
> a) create an etag and insert the relevant header on any response content
> (although, perhaps done only in the case that an etag is not provided by
> upstream server)

StoreEntry would be the starting point. Currently everything goes
through it. In future it will be just cacheable stuff, but the bypassed
things will be useless adding a ETag for caching anyway.

Other than that my knowledge of the store system is patchy. Alex and
Henrik know a lot more about the inner cache workings than me.

>
> b) add an etag header to requests (without one) - ie we are looking at
> the case that client 2 requests content we have cached, but client 2
> doesn't know that, only local squid does.

http.cc does all the request relaying outward stuff. I believe the
if-modified-since requests Squid sends should have ETag in them (if one
is known either from client or from local cache copy). If you find
otherwise that is probably a bug worth fixing. Double-check with RFC
2616 though.

There is no way we can add ETag to requests clients send before looking
up the local cache. The local cache starts with URL and whatever results
that produces is them checked for Vary: match.
  ETag sent by the server might be worth adding as an implicit prefix to
the Vary: pieces. For matching against ETag sent by the client.
  BUT is of little use until multiple-variant caching is ported to 3.x
from 2.7.

>
> Just looking for a quick heads up on where to start investigating?

mentioned above.

>
>> IIRC we have Dimitry with The Measurement Factory assisting with HTTP
>> compliance fixes. I'm sure sponsorship towards a specific fix will be
>> welcomed.
>
> How do I get in contact with Dimitry?
>

Alex is his supervisor I think. rousskov at squid-cache.org.

> content might have been removed..?
>
> Seems that at least parts of this might need to be done internally to squid?
>
> Just to be clear, the point is that few web servers generate useful
> etags, and under the condition that bandwidth is the limiting constraint
> (plus a hierarchy of proxies), then it might be useful to generate (and
> later test) etags based on some consistent hash algorithm?
>

Yes. We came to that conclusion too.

You will find it a bit tricky (but mostly possible) to insert live, but
getting it into the cached items should be relatively easy.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.11
   Beta testers wanted for 3.2.0.5