Re: [squid-users] Re: ICP and HTCP and StoreID

On Thu, Feb 13, 2014 at 10:04 PM, Alex Rousskov
<rousskov_at_measurement-factory.com> wrote:
> On 02/13/2014 05:11 AM, Nikolai Gorchilov wrote:
>
>> I'd suggest first to review all possible StoreID use cases involving
>> cache peers before proceeding further.
>>
>> Let's define A as originating proxy and B - as the next hop proxy in
>> the request forwarding chain. UDP is alias for both ICP or HTCP query,
>> while TCP is synonym of the following HTTP request.
>>
>> Here are all valid usage scenarios I could think of:
>> 1. A & B use same StoreID rewiring logic
>> - No StoreID processing for incoming UDP on B is necessary
>> - UDP request uses StoreID
>> - TCP request uses URL
>> 2. A & B use different StoreID rewriting logic
>> - StoreID processing on incoming UDP on B
>> - UDP request uses URL
>> - TCP request uses URL
>> 3. A with StoreID enabled, B - disabled
>> - UDP request uses URL
>> - TCP request uses URL
>> 4. A with StoreIID disabled, B - enabled
>> - StoreID processing on incoming UDP on B
>> - UDP request uses URL
>> - TCP request uses URL
>>
>> In order to support all of the above we need the following two config options:
>> - configuration switch to enable or disable StoreID processing on incoming UDP
>> - cache_peer option to enable/disable querying the respective peer
>> using StoreID instead of URL
>
>
>> If you see any rifts in the above logic, please say.
>
> I question the value of supporting the implied "no StoreID processing"
> optimization above. AFAICT, if Squid always uses URLs for anything
> outside internal storage, everything would work correctly and all use
> cases will be supported well, without any additional options.

I believe this optimization covers the most common scenario when using
cache peers and StoreID at the same time. There's almost no practical
sense to have different cache peers using different StoreID logic.
They either use the same rewriter, or use no rewriter at all. Seems
common sense for me.

Maybe I'm wrong, but AFAIK Squid never uses "slow" processing methods
on incoming ICP/HTCP requests. Passing the incoming ICP/HTCP requested
URL via the StoreID will change this design principle.

> If somebody wants to extend ICP/HTCP to include StoreId in the request
> (as an optional additional field), they may do so, but that optional
> optimization does not change the overall design principle: StoreId for
> the internal storage; URL for everything else.

Let's put it another way: if we need correct UDP_HIT/UDP_MISS
responses between peers using StoreID we have to compromize on either
one of the following design priciples:
- "Squid always uses URLs for anything outside internal storage"
- "Squid never uses slow processing on UDP requests"

Please correct me if I'm wrong.

>>> Same logic applies to the URLs for all ICP and HTCP packets:
>>> a) queries sent to peers should send client HTTP request URL/URI (*not*
>>> StoreID value)
>>> b) queries received should convert to StoreID before looking up existence
>>> in the local cache.
>>>
>>> (a) is the bug. (b) is the known missing operation in current StoreID
>>> feature.
>>
>> IMHO, the real bug in the current implementation is using StoreID in
>> the TCP session.
>
> AFAICT, the "real bug" is just a side-effect of a true bug (a): The
> receiving Squid thinks it is getting a URL but it is being given a
> StoreID. However, the patch you posted below seems to be fixing
> something else so perhaps I am wrong.

My ugly patch (tries to) makes sure, the originating peer sends the
original URL instead of StoreID during the HTTP request to the
destination peer.

>> StoreID over UDP has real benefits in some scenarios,
>
> The only benefit is reduced computation time at the recipient, right? If
> somebody cares about that so much they can:
>
> 1) Add a StoreID field to ICP/HTCP requests as mentioned above.
> 2) Use an eCAP adapter instead of a helper to compute StoreIDs.

Actually don't mind much. In my case running same URL via StoreID
rewiriter - once on the incoming UDP, then incoming TCP request is not
a deal breaker.

What is important for me is to be able to properly answer incoming UDP
requests that require StoreID normalization (UDP_HIT/UDP_MISS), and
later, when the actual HTTP request comes, to be able to refresh the
object if refresh logic requires to do so.

Current implementation prevents the refresh.

Best,
Niki
Received on Fri Feb 14 2014 - 01:06:45 MST