Re: [squid-users] Need help to improve squid performance

On 2/22/06, Kevin <kkadow@gmail.com> wrote:
> On 2/21/06, Raj <sunfire2005@gmail.com> wrote:
> > I need your suggestions to improve the performance of our proxy
> > servers setup. we have 4 squid proxy servers running on Open BSD (HP
> > Vectra 600, PIII 700). Below is squid -v output
>
> What version of OpenBSD?

We are running OpenBSD version 3.6

> That's some pretty darn old hardware, with a slow CPU, slow IDE hard
> drive, slow RAM, and only expandable to 384MB.
>
>
> > /opt/squid/sbin/squid -v
> > Squid Cache: Version 2.5.STABLE10
> > configure options: --prefix=/opt/squid --enable-auth=ntlm,basic
> > --enable-external-acl-helpers=wbinfo_group --localstatedir=/var/squid
> > --enable-snmp
> >
> > We have two 1st tier proxies located on LAN and two 2nd tier proxies
> > located in the DMZ. 1st Tier uses NTLM via the the Samba WINBIND
> > process and 2nd tier with no authentication required.
> >
> > Server A & Server C - 1st tier proxies
> > Server B & Server D - 2nd tier proxies
> >
> > We have been having performance issues for a long time. I have
>
> Can you define "performance issues"?

If I access a website it takes 6 to 8 seconds to download the page. We
have a 10MB internet link and the link utilisation is only 50% on
average.

>
> One thing I found helped significantly in resolving performance
> complaints was to implement a monitoring system to track network and
> Internet utilization, Squid's own internal SNMP statistics, OS
> statistics, and also single URL request processing time through each
> proxy and also directly from a DMZ host that doesn't go through any
> proxy.
>
> I've fixed a lot of "the proxy is slow" problems by reconfiguring DNS
> servers, upgrading the firmware on switches and routers, converting
> from a DS3 to an OC3, fixing speed and duplex mismatches -- almost
> never has the fix for a "the proxy is slow" complaint involved
> actually touching OpenBSD or Squid or the proxy server hardware.
>
> IOW, use instrumentation to isolate each "performance issue" and
> address each issue individually.
>
>
> > recommended them to buy Sun Fire V240 servers to replace these BSD
> > boxes.
>
> Actually, I'm not all that impressed with the price/performance of the
> V240 series, IMHO the v20z gives more bang for the buck (and can run
> OpenBSD with dual procs).
>
>
> > But they want to increase memory (from 256 MB to 2GB) on the
> > existing servers and see if that improves the performance.
>
> Not a bad idea.
>
> > Below is the current cache size on all 4 servers:
> >
> > cache_dir ufs /var/squid/cache 400 16 256
> >
> > On Server A & Server C we have the following config option on the
> > cache_peer line:
> >
> > cache_peer ServerB parent 3128 3130 weight=15 no-query no-digest
> > cache_peer ServerD parent 3128 3130 weight=10 no-query no-digest
>
> What sort of firewall is between AC (inside) and BD (outside)? what
> is the link throughput and latency?

We have a Cisco Firewall between the 1st tier and 2nd tier proxies. I
am not sure about the model. But according to the network guys the
Internet link is 10mb and the utilisation is only 50%.

>
>
> > After I upgrade the memory I want to increase the cache_dir size to
> > 2GB.
>
> Squid will complain if you don't at least add sufficient cache_dir space
> to match the new cache_mem size.

If I increase the cache_mem size to 256MB what should be the cache_dir size?

>
> I'd suggest adding a new fast disk to each cache server, dedicated to cache_dir.
>
>
> > Could someone suggest me if I need to change the cache_peer
> > options to improve the performance.
>
> It depends on the nature of the performance issues.
> Enabling digests will give a better hit ratio on content which can be
> served out of cache, without adding any real delay and only minimal
> memory overhead.

Enabling digest means "removing no-digest" option on the cache_peer line right?

>
> Enabling ICP would give even higher cache hit ratios, but the extra
> delay in processing requests (waiting for ICP responses) may be
> unacceptable.

Again enabling ICP is "removing no-query" option right?

>
>
> > I also have cache_mem value set to
> > 64mb at the moment. Do I need to increase that after I upgrade the
> > memory.
>
> If you bring each server up to 2GB of RAM, you'll want to increase
> cache_mem significantly, and remember to adjust /etc/login.conf to
> permit the squid user to allocate enough memory and file descriptors.
>
> OpenBSD isn't as aggressive as other OSes about using free memory to
> cache frequently accessed disk pages, so I set cache_mem very high on
> OpenBSD.
>
>
> > Any other help to improve the performance would be really
> > appreciated.
>
> Faster/more memory/CPU/disk or even more servers in parallel could help.
> Check that the first tier is configured for never_direct, neither
> clients nor the internal proxy firewalls should be doing DNS lookups
> or trying direct connections to Internet destinations.
> You may be able to turn off request logging on the 2nd tier?
>
> Kevin
>
Received on Tue Feb 21 2006 - 21:40:58 MST