Re: [squid-users] logging of headers after request modification

On 05/13/2013 04:58 AM, Martin Sperl wrote:

>> A correct implementation of that idea would require more work than it
>> may seem. From adaptation point of view, request_header_access vectoring
>> point is "post-cache REQMOD" while ICAP/eCAP and URL rewriter work at
>> "pre-cache REQMOD" point. We should not mix the two points.

> Instead of storing an additional set of headers and corresponding
> code: why not make these header_ modify and others an ecap service
> that can get added if needed? Then it would get modified in the
> request/response-modification phase and would get logged correctly...
> Also it would cleanup the code-paths I would assume...

It is already possible to apply an eCAP or ICAP service to request
headers, but only at the pre-cache REQMOD vectoring point. Squid does
not support post-cache REQMOD vectoring point. One could argue that Via
can be handled pre-cache, but (a) it adds unnecessary overhead to cache
hits and (b) not all request headers can be handled pre-cache.

We do see requests for post-cache REQMOD (and RESPMOD) support from time
to time. However, the demand is apparently not strong enough for the
feature to get implemented. FWIW, I am not aware of any popular proxy
that supports post-cache ICAP or eCAP adaptation today.

Cheers,

Alex.

> -----Original Message-----
> From: Alex Rousskov [mailto:rousskov_at_measurement-factory.com]
> Sent: Donnerstag, 09. Mai 2013 17:28
> To: squid-users_at_squid-cache.org
> Subject: Re: [squid-users] logging of headers after request modification
>
> On 05/08/2013 11:48 PM, Martin Sperl wrote:
>
>> Actually I found one that works with the "least" amount of trickery...
>>
>> Logging via "%{Via}<h"
>> And modifying the response headers the same way as the request headers.
>
> Well, if you do not care which Via you are logging (request or
> response), then <h will work. I thought you wanted to log the Via sent
> to the server, not the Via sent to the client. The last/Squid portion of
> that Via header will be the same, of course, but all the other entries
> (if any) will usually differ. If you only care about the last/Squid
> portion of Via, then logging response Via is a good solution.
>
>
>> Still in my interpretation the "header modification" via
>> request_header_access would fall into the "adaption and redirection"
>> phase - maybe an idea for a change to trunk?
>
> A correct implementation of that idea would require more work than it
> may seem. From adaptation point of view, request_header_access vectoring
> point is "post-cache REQMOD" while ICAP/eCAP and URL rewriter work at
> "pre-cache REQMOD" point. We should not mix the two points.
>
> We should add a new logformat code to log request headers sent by Squid,
> but it is not trivial because those headers are currently not preserved
> anywhere IIRC (and Squid may send multiple requests for a single client
> request). Quality patches or sponsorships welcome.
>
> For now, I polished [http::]>h and [http::]>ha logformat descriptions in
> trunk to emphasize their pre-cache scope.
>
>
> Cheers,
>
> Alex.
>
>
>> -----Original Message-----
>> From: Alex Rousskov [mailto:rousskov_at_measurement-factory.com]
>> Sent: Dienstag, 07. Mai 2013 18:22
>> To: squid-users_at_squid-cache.org
>> Subject: Re: [squid-users] logging of headers after request modification
>>
>> On 05/07/2013 09:49 AM, Martin Sperl wrote:
>>
>>> Is there any other configuration option (in squid 3.2) to log if an
>>> ACL matches in the access-log? (which is all I essentially need)
>>
>> I have listed all the options I could think of, but I missed the
>> "different logformat" options you mentioned. I think they boil down to:
>>
>> 4) Use multiple stdio/daemon loggers to append records to the same file.
>> This does not require development (assuming current stdio/daemon code
>> use O_APPEND when opening a log file). If you do not use NFS and manual
>> page for open(2) implications are correct, then there should be no
>> conflicts.
>>
>> 5) Use a single tcp logger address to log to the same file (requires
>> not-yet-accepted trunk patch to work well). Here is a sketch for such a
>> logger:
>>
>> nc -k -l 127.0.0.1 1234 > one-and-only.log
>>
>>
>> HTH,
>>
>> Alex.
>>
>>
>>> -----Original Message-----
>>> From: Alex Rousskov [mailto:rousskov_at_measurement-factory.com]
>>> Sent: Dienstag, 07. Mai 2013 17:37
>>> To: squid-users_at_squid-cache.org
>>> Subject: Re: [squid-users] logging of headers after request modification
>>>
>>> On 05/07/2013 08:03 AM, Martin Sperl wrote:
>>>
>>>> I have configured squid 3.2.7 logging with the following pattern to log:
>>>>
>>>> logformat xml ...<Via>%{Via}>ha</Via>...
>>>> access_log daemon:/var/logs/squid/access_log.xml xml all
>>>>
>>>> But I have the problem, that the <VIA> fields stay "empty" (actually "-")...
>>>>
>>>> So I wonder why and how I can change that, so that I get the Via header as well...
>>>
>>>> P.s: I am modifying the VIA header in my config like this under some circumstances:
>>>> request_header_access Via allow CUSTOM_ACL
>>>> request_header_access Via deny all
>>>> request_header_replace Via mypersonalvalue
>>>
>>>> The application sees the "expected" value in the Via header, so that
>>>> itself is not an issue - only logging the header is NOT working as
>>>> expected... (it actually only works IF
>>>
>>>
>>> [http::]>ha logs request headers received by Squid and adapted by
>>> ICAP/eCAP. It does not log request headers sent by Squid. There is
>>> currently no logformat code to log the latter.
>>>
>>> If you want to log outgoing Via, your options include:
>>>
>>> 1) add a logformat code to log outgoing request headers (requires
>>> development);
>>>
>>> 2) use "note" directive and %note logformat code to log mypersonalvalue
>>> when needed (requires Squid trunk?)
>>>
>>> 3) move Via manipulation to ICAP/eCAP while disabling Squid's Via
>>> generation (requires writing or adjusting an adaptation service).
>>>
>>
>> This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement,
>> you may review at http://www.amdocs.com/email_disclaimer.asp
>>
Received on Mon May 13 2013 - 14:20:20 MDT