Re: Multiple storeio types?

Henrik Nordstrom wrote:

> Joe Cooper wrote:
>
>
>>BTW-Henrik, I've been meaning to commend you and Adrian for the
>>aufs+storeio stuff. It overloads very gracefully, and without /too/
>>much of a hit to overall performance and hit ratio. I haven't been able
>>to crash the box because of load yet (and believe me I've tried). It's
>>definitely more relaxing to know that if I push it a little too hard it
>>won't fall down (2.2Stable5+async, while very zippy, had the annoying
>>habit of blowing up when pushed too hard).
>>
>
> Thanks. But I still think it can actually do considerably better.
>
> If you had given a little warning I could have spent some time on
> further tuning and optimizing aufs.. now there is not the time needed I
> am afraid. Requires at least a day of coding, and two days testing, and
> my testbench is not fully operational at the moment further complicating
> the issue.. (haven't been in deep Squid mode for quite a while now.. and
> last time I was it was mostly on other stuff than performance)

I didn't really have much warning myself and I was afraid I wasn't going
to make it to this one (Thanks Duane!). I just bought my plane ticket
last week, and have only had time for benchmarking this week (I finally
finished setting up the Polymix-4 bench this morning--this stuff gets
more complicated everytime!). Adrian hints that COSS will certainly be
finished in time for the /next/ cacheoff, and I saw that some folks are
volunteering to help out with the eventio branch...so, maybe next time
around (if there is a next time, the turnout isn't going to be very good
this time) we'll start nipping at the heels of the proprietary guys a bit.

Now that I've got the PM-4 benches set up, I'm seeing how much harder a
workload it is than previous cacheoff loads. I think we're only going
to handle about 130-140 reqs/sec even with so much RAM. Part of the
slowness I'm seeing is from CPU overload (the disks aren't working very
hard at the moment), so I'll pick up a 1GHz CPU when I get to
Denver/Boulder which will probably allow it to climb back up to 150-ish.
  Not great, but better than last time, and the box will be more
affordable than the previous one by a few hundred bucks, so
price/performance will be up some too.

But if you get inspired to code for speed over the weekend, I'll run the
changes on Tuesday or Wednesday! ;-) Actually, if Adrian gets COSS
into some semblance of stability under Linux, I'll run it once so only
one run is up for grabs. I can probably get a short test run each day
to insure things are at least slightly stable before trying anything
experimental (in addition to hitting it a little with my laptop before
even putting it up for trial by fire), but I'm only going to have three
days there so won't have too much time for crashes mid-run. Hmmm...it
just hit me...My office network is always online now. There's no reason
I can't run benches remotely on test boxes here while I'm at the
cacheoff in order to know if some changes are stable enough to run.
That's just /if/ you have a notion to poke at speed issues. No pressure!

As has been discussed, there is no 'band-aid' that will fix Squid's
performance in the short term, so realy we just need to keep working
gradually towards a deign that does what everyone wants. There's always
next year.

Thanks for all of your help and advice, Henrik.

-- 
Joe Cooper <joe@swelltech.com>
http://www.swelltech.com
Web Caching Appliances and Support