slow down with squid wccp linux

I'm having a little problem that I can't solve. Here's the
configuration:

Linux 2.2.12
RedHat 6.1
squid 2.3 devel 5
wccp
ip_wccp.c kernel module
echo "1023 32768" > ...
file descriptor limit 16k in types.h
file descriptors and max-inode done in /proc
squid has 12k file descriptors available.

In our 1st test configuration we had 15 req/s and squid would slow down
and begin to give timeouts to the clients after about 30 min to 1 hr of
caching. I fixed this by changing the local_ip_port_range back to "1024
4999" (linux default). Squid ran perfectly for 3 days straight.

Next we put the squid box on our main outgoing router. Requests jumped
up to 60-70/s and squid worked perfectly for about 30 minutes. After
this cpu usage slowly went down to 5% and timeouts started. Relating
the ip_port_range to squid working I tried "1024 16382". Squid worked
for 1 hour and then timed out.

I have included our squid configuration at the bottom of this message.
Statistics from the os include:

~1500 file descriptors in use
~1100 for a `netstat -an|wc -l`
CPU usage is about 70%-80% (I attribute this to the use of cache
digests)

Any ideas on the source of this behavior would be appreciated. Thanks
for any help.

                                --Dustin

squid configuration:
http_port 0.0.0.0:3128
icp_port 3130
tcp_outgoing_address 255.255.255.255
udp_incoming_address 0.0.0.0
udp_outgoing_address 255.255.255.255
cache_peer bo.cache.nlanr.net Sibling 3128 3130 no-query
icp_query_timeout 0
maximum_icp_query_timeout 2000
mcast_icp_query_timeout 2000
dead_peer_timeout 10 seconds
hierarchy_stoplist cgi-bin
hierarchy_stoplist ?
no_cache Deny QUERY
cache_mem 134217728 bytes
cache_swap_low 98
cache_swap_high 99
maximum_object_size 67108864 bytes
ipcache_size 10240
ipcache_low 98
ipcache_high 99
fqdncache_size 1024
cache_dir ufs /export/cache1/ 15600 48 256
cache_dir ufs /export/cache2/ 15600 48 256
cache_dir ufs /export/cache3/ 15600 48 256
cache_dir ufs /export/cache4/ 15600 48 256
cache_access_log /usr/local/squid/logs/access.log
cache_log /usr/local/squid/logs/cache.log
cache_store_log /usr/local/squid/logs/store.log
emulate_httpd_log off
mime_table /usr/local/squid/etc/mime.conf
log_mime_hdrs off
pid_filename /usr/local/squid/logs/squid.pid
debug_options ALL,1
log_fqdn off
client_netmask 255.255.0.0
ftp_user Squid@
ftp_list_width 32
cache_dns_program /usr/local/squid/bin/dnsserver
dns_children 32
dns_defnames off
unlinkd_program /usr/local/squid/bin/unlinkd
pinger_program /usr/local/squid/bin/pinger
redirect_children 5
redirect_rewrites_host_header on
authenticate_children 5
authenticate_ttl 3600
authenticate_ip_ttl 0
wais_relay_port 0
request_header_max_size 10240 bytes
request_body_max_size 1048576 bytes
reply_body_max_size 0 bytes
refresh_pattern ^ftp: 1440 20% 10080

refresh_pattern ^gopher: 1440 0% 1440

refresh_pattern . 0 20% 4320

reference_age 31557790 seconds
quick_abort_min 16 KB
quick_abort_max 16 KB
quick_abort_pct 95
negative_ttl 300 seconds
positive_dns_ttl 21600 seconds
negative_dns_ttl 300 seconds
range_offset_limit 0 bytes
connect_timeout 120 seconds
peer_connect_timeout 30 seconds
siteselect_timeout 4 seconds
read_timeout 900 seconds
request_timeout 30 seconds
client_lifetime 86400 seconds
half_closed_clients on
pconn_timeout 120 seconds
ident_timeout 10 seconds
shutdown_lifetime 30 seconds
acl QUERY urlpath_regex cgi-bin
acl QUERY urlpath_regex \?
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1
acl SSL_ports port 443
acl SSL_ports port 563
acl Safe_ports port 80
acl Safe_ports port 21
acl Safe_ports port 443
acl Safe_ports port 563
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 1025-65535
acl Safe_ports port 280
acl Safe_ports port 0
acl Safe_ports port 0
acl Safe_ports port 488
acl Safe_ports port 0
acl Safe_ports port 0
acl Safe_ports port 591
acl Safe_ports port 0
acl Safe_ports port 0
acl Safe_ports port 777
acl Safe_ports port 0
acl Safe_ports port 0
acl Safe_ports port 0
acl CONNECT method CONNECT
acl allowed_hosts src 0.0.0.0/0.0.0.0
acl local dstdomain .nmsu.edu
http_access Allow manager allowed_hosts
http_access Allow manager localhost
http_access Deny manager
http_access Deny !Safe_ports
http_access Deny CONNECT !SSL_ports
http_access Allow allowed_hosts
http_access Allow localhost
http_access Deny all
icp_access Allow allowed_hosts
icp_access Allow localhost
icp_access Deny all
miss_access Allow allowed_hosts
miss_access Allow localhost
miss_access Deny all
proxy_auth_realm Squid proxy-caching web server
ident_lookup_access Deny all
cache_mgr webcache@nmsu.edu
cache_effective_user squid
cache_effective_group squid
visible_hostname web-cache.nmsu.edu
announce_period 31536000 seconds
announce_host tracker.ircache.net
announce_port 3131
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
logfile_rotate 1
append_domain .nmsu.edu
tcp_recv_bufsize 0 bytes
err_html_text
memory_pools on
memory_pools_limit 0 bytes
forwarded_for on
log_icp_queries on
icp_hit_stale off
minimum_direct_hops 4
cachemgr_passwd disable shutdown
cachemgr_passwd none all
store_avg_object_size 13 KB
store_objects_per_bucket 50
client_db on
netdb_low 900
netdb_high 1000
netdb_ping_period 300 seconds
query_icmp on
test_reachability on
buffered_logs off
reload_into_ims off
always_direct Allow local
anonymize_headers
icon_directory /usr/local/squid/etc/icons
error_directory /usr/local/squid/etc/errors
minimum_retry_timeout 5 seconds
maximum_single_addr_tries 3
as_whois_server whois.ra.net
wccp_router 0.0.0.0
wccp_incoming_address 0.0.0.0
wccp_outgoing_address 255.255.255.255
incoming_icp_average 6
incoming_http_average 4
min_icp_poll_cnt 8
min_http_poll_cnt 8
max_open_disk_fds 0
offline_mode off
uri_whitespace allow
prefer_direct on
strip_query_terms on
redirector_bypass off
ignore_unknown_nameservers on
digest_generation on
digest_bits_per_entry 5
digest_rebuild_period 3600
digest_rewrite_period 3600
digest_swapout_chunk_size 4096
digest_rebuild_chunk_percentage 10
Received on Fri Nov 12 1999 - 17:45:16 MST