[squid-users] Increased service times using aufs vs diskd

I'm using squid as an accelerator and I switched my cache_dir from diskd
(used because server is SMP) to aufs because of various bugs in the
diskd code (761, 1500). However, when I make the switch (and clear the
cache_dir contents) the overall, hit, miss, and near miss service times
increase by almost 10 times. Using diskd, the 24 hour average for
overall, hit, and near miss is 4ms and near miss is 1ms. After the
switch, the times rocket up to 44ms, 43ms, 49ms, and 45ms for overall,
hit, miss, and near miss. I am wondering if this is just a function of
the squid process now handling disk requests or is an indication of
another problem (although ~40ms is probably not much of a problem).

Here's the details of the system:
OS: RHEL4
Squid: 2.5.stable14 with epoll patch
Build options: ./configure --enable-epoll --enable-snmp
--enable-removal-policies=heap,lru --enable-storeio=aufs,diskd,ufs
--with-pthreads --enable-cachemgr-hostname=localhost
--disable-ident-lookups --enable-truncate --enable-cache-digests
--enable-htcp
Config:

http_port 80
icp_port 0
acl NOCACHE urlpath_regex -i \.html \.xsl \.jsp \.xml \.flow
no_cache deny NOCACHE
cache_mem 16 MB
maximum_object_size 1024 KB
maximum_object_size_in_memory 32 KB
cache_dir aufs /var/spool/squid/01 25892 60 256
cache_dir aufs /var/spool/squid/02 25892 60 256
cache_store_log none
refresh_pattern . 10080 100% 43200 ignore-reload
override-lastmod
shutdown_lifetime 5 seconds
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl foo src xxx.xxx.xxx.xxx/255.255.255.255
acl bar src xxx.xxx.xxx.xxx/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl to_int dst xxx.xxx.xxx.xxx/24
acl to_int dst xxx.xxx.xxx.xxx/25
acl from_xx_xx src xxx.xxx.xxx.xxx/23
acl from_xx_xx src xxx.xxx.xxx.xxx/24
acl from_xx_xx src xxx.xxx.xxx.xxx/24
acl from_xx_xx src xxx.xxx.xxx.xxx/24
acl from_xx_xx src xxx.xxx.xxx.xxx/24
acl from_xx_xx src xxx.xxx.xxx.xxx/25
acl from_xx_xx src xxx.xxx.xxx.xxx/24
acl Safe_ports port 80 # http
acl PURGE method PURGE
acl snmppublic snmp_community XXXXX
http_access allow manager localhost
http_access allow manager qw-cx-mgmt
http_access deny manager
http_access deny !Safe_ports
http_access allow all to_int
http_access allow PURGE localhost
http_access allow PURGE foo
http_access allow PURGE bar
http_access deny PURGE all
http_access allow localhost
http_access deny all
http_reply_access allow all
httpd_accel_host host.domain.com
httpd_accel_port 80
logfile_rotate 10
log_icp_queries off
cachemgr_passwd XXXX all
snmp_port 3401
snmp_access allow snmppublic localhost
snmp_access allow snmppublic from_xx_xx
snmp_access deny all
coredump_dir /var/spool/squid

Cache log output on startup:

2006/06/22 16:32:24| Starting Squid Cache version 2.5.STABLE14 for
i686-redhat-l
inux-gnu...
2006/06/22 16:32:24| Process ID 20978
2006/06/22 16:32:24| With 32768 file descriptors available
2006/06/22 16:32:24| DNS Socket created at 0.0.0.0, port 32797, FD 6
2006/06/22 16:32:24| Adding nameserver 66.77.63.40 from /etc/resolv.conf
2006/06/22 16:32:24| Adding nameserver 66.77.63.41 from /etc/resolv.conf
2006/06/22 16:32:24| Unlinkd pipe opened on FD 11
2006/06/22 16:32:24| Swap maxSize 53026816 KB, estimated 4078985 objects
2006/06/22 16:32:24| Target number of buckets: 203949
2006/06/22 16:32:24| Using 262144 Store buckets
2006/06/22 16:32:24| Max Mem size: 16384 KB
2006/06/22 16:32:24| Max Swap size: 53026816 KB
2006/06/22 16:32:24| Local cache digest enabled; rebuild/rewrite every
3600/3600
 sec
2006/06/22 16:32:24| Store logging disabled
2006/06/22 16:32:24| Rebuilding storage in /var/spool/squid/01 (CLEAN)
2006/06/22 16:32:24| Rebuilding storage in /var/spool/squid/02 (CLEAN)
2006/06/22 16:32:24| Using Least Load store dir selection
2006/06/22 16:32:24| Set Current Directory to /var/spool/squid
2006/06/22 16:32:24| Loaded Icons.
2006/06/22 16:32:24| Accepting HTTP connections at 0.0.0.0, port 80, FD
16.
2006/06/22 16:32:24| Accepting HTCP messages on port 4827, FD 17.
2006/06/22 16:32:24| Accepting SNMP messages on port 3401, FD 18.
2006/06/22 16:32:24| WCCP Disabled.
2006/06/22 16:32:24| Ready to serve requests.
2006/06/22 16:32:24| Done reading /var/spool/squid/01 swaplog (382
entries)
2006/06/22 16:32:24| Done reading /var/spool/squid/02 swaplog (520
entries)
2006/06/22 16:32:24| Finished rebuilding storage from disk.
2006/06/22 16:32:24| 902 Entries scanned
2006/06/22 16:32:24| 0 Invalid entries.
2006/06/22 16:32:24| 0 With invalid flags.
2006/06/22 16:32:24| 902 Objects loaded.
2006/06/22 16:32:24| 0 Objects expired.
2006/06/22 16:32:24| 0 Objects cancelled.
2006/06/22 16:32:24| 0 Duplicate URLs purged.
2006/06/22 16:32:24| 0 Swapfile clashes avoided.
2006/06/22 16:32:24| Took 0.3 seconds (3110.0 objects/sec).
2006/06/22 16:32:24| Beginning Validation Procedure
2006/06/22 16:32:24| Completed Validation Procedure
2006/06/22 16:32:24| Validated 902 Entries
2006/06/22 16:32:24| store_swap_size = 12336k
2006/06/22 16:32:26| storeLateRelease: released 0 objects

Thanks,

-=Kevin=-
Received on Thu Jun 22 2006 - 14:27:07 MDT