RE: [squid-users] Squid and Squidguard using high disk IO

Hello Kaya,

May I recommend to try using qlproxy together with your Squid?
Qlproxy is an ICAP web filtering which may in your particular case do better as Squid Guard. At least you may give it a try to compare if the disk io goes down.

Best regards,
Raf

-----Original Message-----
From: Kaya Saman [mailto:kayasaman_at_gmail.com]
Sent: Saturday, November 09, 2013 4:58 PM
To: squid-users_at_squid-cache.org
Subject: [squid-users] Squid and Squidguard using high disk IO

Hi,

I'm wondering if anyone has any ideas on this one.....

Basically I have created a standard Squid proxy using Squid 3.3.8 built from OpenBSD ports - OS version is OpenBSD 5.4 Current.

Additionally from ports as well I have installed squidGuard 1.4p6.

The configuration seems ok as everything is working; the acls setup in squidGuard are redirecting to the proper "blocked" page when unwanted information is embedded in a site: eg. ads, p%rn.

Here is the rule list:

dest ads {
     domainlist blacklists/ads/domains
     urllist blacklists/ads/urls
}

dest adv {
     domainlist blacklists/adv/domains
     urllist blacklists/adv/urls
}

dest spyware {
     domainlist blacklists/spyware/domains
     urllist blacklists/spyware/urls
}

dest porn {
     domainlist blacklists/porn/domains
     urllist blacklists/porn/urls
     expressionlist blacklists/porn/expressions
     # Logged info is anonymized to protect users' privacy
     log anonymous dest/porn.log
}

acl {
     lan {
         # The built-in 'in-addr' destination group matches any IP address.
         pass !ads !adv !porn all
     }
     default {
         # Default deny to reject unknown clients
         pass none
         redirect http://127.0.0.1/blocked.html

     }
}

I removed the "spyware" option from the 'lan' acl as I'm trying to debug currently....

squidGuard is called by Squid using these lines in the squid.conf:

# Path to the redirector program
url_rewrite_program /usr/local/bin/squidGuard

# Number of redirector processes to spawn url_rewrite_children 500

# To prevent loops, don't send requests from localhost to the redirector
url_rewrite_access deny localhost

The issue I'm currently seeing is that the disk IO process is hammered???

The 'lan' clients are therefor unable to access the web through the proxy.

Running 'top' and 'ps' I can see that squidGuard has spawned many processes which seems to be causing the high IO usage.

The systems' hardware is quite powerful with 8GB RAM and a Xeon E5 CPU @3.6GHz, currently being tested with 3x lan machines.

What can I do to improve performance with this?

Is this line too high: url_rewrite_children 500

or simply have a misconfigured something?

I additionally have 'c-icap' running with squidclamav coupled to clamd
in case that is of importance - not using the squidGuard line in the
squidclamav.conf file!!!

Basically how can I get the IO usage down and get the system to work again?

- the logs don't indicate anything outside of 'starting squidGuard
process' many times.

Regards,

Kaya
Received on Sat Nov 09 2013 - 17:05:02 MST