Re: [squid-users] squid_session problem

Thank you for the abundance of information you have provided - I'm
going through it all now to get a better understanding of
squid_session and how it works, as well as the external_acl_type
directive.

On Mon, Jul 9, 2012 at 9:46 PM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
> On 10.07.2012 15:12, Jack Black wrote:
>
>> On Mon, Jul 9, 2012 at 7:48 PM, Amos Jeffries wrote:
>>>
>>> On 10.07.2012 13:18, Jack Black wrote:
>>>>
>>>>
>>>> Hi.
>>>>
>>>> Has anyone successfully used squid, and the squid_session helper in
>>>> order to force users of the proxy server to see a webpage (be
>>>> redirected to it) at the beginning of each session?
>>>
>>>
>>>
>>> Yes, many.
>>>
>>>
>>>>
>>>> After spending weeks trying to get this to work, I was finally
>>>> successful using squid version 3.1.10 on CentOS. Unfortunately, I need
>>>> it to work on Ubuntu Server 12.04 with squid version 3.1.19 instead,
>>>> and doing exactly the same thing as I did in CentOS, this fails to
>>>> work on the Ubuntu Server, and my /var/log/squid3/cache.log has a line
>>>> similar to:
>>>>
>>>>> externalAclLookup: 'session' queue overload (ch=0x....)
>>>
>>>
>>>
>>> HINT: "queue overload" - you are getting more requests per second than
>>> the
>>> helper can reply to. Even with TTL > 0.
>>>
>>> I'm a bit suspicious that with children=1 the queue is only 2 entries
>>> long
>>> (not good). Since it is based on number of helpers, and seems not to
>>> account
>>> for concurrency. The log message could be due to that, but that would not
>>> allow requests past the splash page, quite the opposite in fact.
>>
>>
>> How can you tell the queue is only 2 entries long?
>
>
> The queue is 2x n_children, or 2x1 in length, with 200x n_children slots of
> concurrency to use up before the queue starts filling. It should not matter
> if the helper is fast enough, but explains that queue overload message will
> appear if you have 202 lookups waiting for a response.
>
>
>
>> Am I missing
>> something? My main focus during the weeks I spent getting this to work
>> was getting squid to talk to Cisco using WCCP. I am still far from
>> understanding exactly how this helper works, and am no squid expert.
>
>
> The helper maintains a database of previously seen strings/sessions. In your
> case a database of previously seen IP addresses ("%SRC").
>
>
> When there is no existing entry in the session database, one is created
> immediately and ERR/no-match is returned. http_access results in a no-match
> and !session makes that a pass, so the deny is done and splash page is
> displayed.
> When a lookup is done the database is checked, any existing entry results
> in O/match and the DB record is updated to keep it another 900 seconds (-t
> 900). http_access results in a match and !session makes that a fail, so the
> deny is not done.
> Combining these, you can see that the first request gets the splash page,
> and following ones get the requested content.
>
> Using the %SRC, means any packets arriving from that IP can do the above and
> you have almost no control over whether an actual user or some blind
> automated software is getting the splash page.
> It is extremely easy for automated bits (I listed a few earlier) to end up
> with the splash p[age and users not even noticing the sessions. Which is
> what you described happening.
>
>
>
>> I
>> found the lines below for configuring squid_session in squid.conf
>> online (I didn't write them), and only slightly changed them to work
>> for me. The only instructions I can find online for how the
>> squid_session helper works is an exact copy of the man page for
>> squid_session, which only has one example and not much explanation for
>> what the different values mean on the line that starts with
>> "external_acl_type" (the man page has no mention of ttl, negative ttl,
>> children, etc...).
>
>
> Those are all options for the external_acl_type directive. Not the session
> helper itself. External ACL is an interface used by Squid to run custom
> scripts that do complex access control things.
> It is documented at
> http://www.squid-cache.org/Doc/config/external_acl_type/
>
>
> There is a bit more documentation about the session helper and how it does
> splash pages at
> http://wiki.squid-cache.org/ConfigExamples/Portal/Splash
>
>
>
>>
>> Oh and while my final goal is to get this working with WCCP, right now
>> I'm testing without it so as to keep things simple.
>>
>>>
>>>
>>>>
>>>> for every http request my client sends (so a lot of those lines). The
>>>> client is forwarded through the squid proxy directly to the page they
>>>> request every time, and the splash page is always ignored. Here are
>>>> the relevant lines from squid.conf:
>>>>
>>>>> external_acl_type session ttl=300 negative_ttl=0 children=1
>>>>> concurrency=200 %SRC /usr/lib/squid3/squid_session -t 900
>>>>> acl session external session
>>>>> http_access deny !session
>>>>> deny_info http://example.com session
>>>>
>>>>
>>>>
>>>> Does anyone know the problem? Am I doing something wrong?
>>>>
>>>> Tal
>>>
>>>
>>>
>>> Splash page is only at the beginning of a browsing "session". If their
>>> requests are less then 900 seconds apart the existing session is extended
>>> another 900 seconds from that time.
>>>
>>> * you are making a session based on any HTTP request made by that IP
>>> address, so *all* software operating on the client machine can trigger an
>>> HTTP request which extends the session or creates a new one.
>>
>>
>> I am aware of this, and having restarted squid many times in order to
>> reset the session counters, while surfing the web on the client at the
>> exact time squid starts, I have not seen the splash screen once (in
>> Ubuntu Server), making me fairly confident it's not working. When this
>> is done on CentOS and everything works, the splash screen is easy to
>> run into with exactly the same client.
>
>
>
> Ah. If you run the session helper without a disk-backed database it keeps
> everything in RAM. A simple reconfigure or log rotation is enough to restart
> the helper and wipe that RAM database clean.
>
> If you are using disk-backed, it will persist across even a full stop-start
> cycle of Squid. Use short -t values to make sessions expire quickly while
> testing that setup.
>
> If something like the automated tools are getting the splash page the only
> sign you will have of it is where the first line of access.log comes from.
> Session is created immediately and the only sign in Squid is the initial
> TCP_DENIED followed by acceptance.
>
>
> You might want to use active mode to ensure no session is created until
> after your special splash page URL has been requested by someone. Your
> access.log will then show clearly if anything is doing that since the
> http://example.com URL will show up whenever that happens.

I was very interested in active mode when I came across it here (same
page as you linked to):

http://wiki.squid-cache.org/ConfigExamples/Portal/Splash

but it seems a bit more complex than the simple configuration I'm
trying to get working currently. Once I get the simple configuration
working, I'll definitely be looking into active mode. As it is now - I
do not believe that updates and other background software are getting
the splash page instead of the browser, since the browser gets the
splash page perfectly fine with CentOS as the squid server. Don't get
me wrong - I'm not disputing that it's impossible, or even improbable
for other software to get the splash page - I'm just testing carefully
while keeping this in mind in order for this not to effect my tests.
The tip you gave about checking access.log for signs of this should
help exclude this as a possibility as well.

>
>
>
>>
>>> * it is common to have multiple software updaters active on the PC,
>>> which
>>> use HTTP to fetch their updates. Some of those may be running first and
>>> getting the splash page so the user never sees it.
>>
>>
>> I am aware, and that is why I test multiple times, restarting squid
>> each time, in order to see the splash page even once.
>>
>>> * it is common for browsers these days to do some form of CORS lookup
>>> before starting a page fetch. If your splash page is what comes back to
>>> that
>>> CORS request the naive security checks will fail and the website they
>>> wanted
>>> will appear without anything showing up.
>>
>>
>> I was not aware of this - maybe that is what's happening here - though
>> I'm not sure why it would trigger a session with Ubuntu, but not
>> CentOS as the squid proxy server. I'll have to do some reading on
>> this. Thanks for the hint.
>>
>>> Any one of those, or none of them could be the problem.
>>>
>>> Start with trying 3.1.19 on CentOS. Building it yourself from the Ubuntu
>>> source package would be a good test to see if it is a change in Squid
>>> over
>>> those 14 releases or something in the system.
>>>
>>> Amos
>>>
>>
>> I've tried compiling newer and older versions of squid on the Ubuntu
>> Server, and ran into the same problem. I will give it another try
>> though - maybe I missed something.
>>
>> Thanks again - always helps to have fresh eyes on a problem like this.
>>
>> Tal
>
>
>
>
> Amos
Received on Tue Jul 10 2012 - 14:45:40 MDT