Re: [squid-users] Need help to improve squid performance

On 2/21/06, Raj <sunfire2005@gmail.com> wrote:
> I need your suggestions to improve the performance of our proxy
> servers setup. we have 4 squid proxy servers running on Open BSD (HP
> Vectra 600, PIII 700). Below is squid -v output

What version of OpenBSD?
That's some pretty darn old hardware, with a slow CPU, slow IDE hard
drive, slow RAM, and only expandable to 384MB.

> /opt/squid/sbin/squid -v
> Squid Cache: Version 2.5.STABLE10
> configure options: --prefix=/opt/squid --enable-auth=ntlm,basic
> --enable-external-acl-helpers=wbinfo_group --localstatedir=/var/squid
> --enable-snmp
>
> We have two 1st tier proxies located on LAN and two 2nd tier proxies
> located in the DMZ. 1st Tier uses NTLM via the the Samba WINBIND
> process and 2nd tier with no authentication required.
>
> Server A & Server C - 1st tier proxies
> Server B & Server D - 2nd tier proxies
>
> We have been having performance issues for a long time. I have

Can you define "performance issues"?

One thing I found helped significantly in resolving performance
complaints was to implement a monitoring system to track network and
Internet utilization, Squid's own internal SNMP statistics, OS
statistics, and also single URL request processing time through each
proxy and also directly from a DMZ host that doesn't go through any
proxy.

I've fixed a lot of "the proxy is slow" problems by reconfiguring DNS
servers, upgrading the firmware on switches and routers, converting
from a DS3 to an OC3, fixing speed and duplex mismatches -- almost
never has the fix for a "the proxy is slow" complaint involved
actually touching OpenBSD or Squid or the proxy server hardware.

IOW, use instrumentation to isolate each "performance issue" and
address each issue individually.

> recommended them to buy Sun Fire V240 servers to replace these BSD
> boxes.

Actually, I'm not all that impressed with the price/performance of the
V240 series, IMHO the v20z gives more bang for the buck (and can run
OpenBSD with dual procs).

> But they want to increase memory (from 256 MB to 2GB) on the
> existing servers and see if that improves the performance.

Not a bad idea.

> Below is the current cache size on all 4 servers:
>
> cache_dir ufs /var/squid/cache 400 16 256
>
> On Server A & Server C we have the following config option on the
> cache_peer line:
>
> cache_peer ServerB parent 3128 3130 weight=15 no-query no-digest
> cache_peer ServerD parent 3128 3130 weight=10 no-query no-digest

What sort of firewall is between AC (inside) and BD (outside)? what
is the link throughput and latency?

> After I upgrade the memory I want to increase the cache_dir size to
> 2GB.

Squid will complain if you don't at least add sufficient cache_dir space
to match the new cache_mem size.

I'd suggest adding a new fast disk to each cache server, dedicated to cache_dir.

> Could someone suggest me if I need to change the cache_peer
> options to improve the performance.

It depends on the nature of the performance issues.
Enabling digests will give a better hit ratio on content which can be
served out of cache, without adding any real delay and only minimal
memory overhead.

Enabling ICP would give even higher cache hit ratios, but the extra
delay in processing requests (waiting for ICP responses) may be
unacceptable.

> I also have cache_mem value set to
> 64mb at the moment. Do I need to increase that after I upgrade the
> memory.

If you bring each server up to 2GB of RAM, you'll want to increase
cache_mem significantly, and remember to adjust /etc/login.conf to
permit the squid user to allocate enough memory and file descriptors.

OpenBSD isn't as aggressive as other OSes about using free memory to
cache frequently accessed disk pages, so I set cache_mem very high on
OpenBSD.

> Any other help to improve the performance would be really
> appreciated.

Faster/more memory/CPU/disk or even more servers in parallel could help.
Check that the first tier is configured for never_direct, neither
clients nor the internal proxy firewalls should be doing DNS lookups
or trying direct connections to Internet destinations.
You may be able to turn off request logging on the 2nd tier?

Kevin
Received on Tue Feb 21 2006 - 21:06:04 MST