Re: Squid crash bug in COSS

On Tue, Oct 24, 2006, Mark Bergsma wrote:
> Hi,
>
> We're seeing one or more Squid crash bugs since we started using COSS
> heavily.
>
> When I restarted a Squid 2.6.4 server today, it kept on crashing every
> few minutes, in a constant loop. Upon investigation this proved to be a
> bug in COSS, during rebuilding of its cache store.

I forget what stuff has been committed to COSS and when, but I'd suggest
running the latest squid-2.6 snapshot or waiting a few days until squid-2.6-stable5
is released. It'll be easier to trace bugs down when you're running something
recent.

That said, Steven did fix a few silly adrian mistakes in the rebuild code,
its entirely possible this is fixed in the current squid-2.6 code.

Adrian

>
> This is the backtrace given in cache.log by --enable-backtrace:
>
> (squid)[0x428805]
> (squid)[0x48dde1]
> (squid)[0x48b78a]
> (squid)[0x474c36]
> (squid)[0x427c66]
> (squid)[0x452352]
> /lib/libc.so.6(__libc_start_main+0xdb)[0x2aaaab1ca49b]
> (squid)[0x40417a]
>
> addr2line gives:
>
> root@sq13:~# addr2line -e /usr/sbin/squid
> 0x428805
> /home/mark/wikimedia/squid-2.6.4/src/debug.c:508
> 0x48dde1
> coss/store_dir_coss.c:1256
> 0x48b78a
> aufs/async_io.c:315
> 0x474c36
> /home/mark/wikimedia/squid-2.6.4/src/store_dir.c:510
> 0x427c66
> /home/mark/wikimedia/squid-2.6.4/src/comm_epoll.c:179
> 0x452352
> /home/mark/wikimedia/squid-2.6.4/src/main.c:860
>
> Clearing the cache dirs (block devices) with zeros seemed to fix it.
>
> I believe there are one or more other crash bugs in COSS, because ever
> since we started using it, sometimes suddenly a lot of Squids crash at
> around the same time. This doesn't happen very often, I'd say about once
> a week. Overall, COSS seems very performant and promising - we're very
> happy with it.
>
> --
> Mark Bergsma <mark@wikimedia.org>
> System & Network Administrator, Wikimedia Foundation
Received on Thu Oct 26 2006 - 08:10:28 MDT