[squid-users] Re: Squid high bandwidth IO issue (ramdisk SSD)

On 02.08.2009, Adrian Chadd wrote:

> Well, from what I've read, SSDs don't necessarily provide very high
> random write throughput over time. You should do some further research
> into how they operate to understand what the issues may be.

Using Linux and SSD drives without build-in cache, the filesystem is the
major drag. The widely spread "hickups" vanish using ext4 or xfs. I did a
lot of testing in this direction, especially with SSD disks (not
squid-related testing, but...). My squid cache_dirs reside on
an SSD disk as well (without cache).

You can use Ted Tso's fsync-tester to measure overall latency related to
disk I/O and the elevator. Just put the system under heavy I/O load,
e.g. by writing/copying a very large file, and measure the response time.

/*
 * fsync-tester.c
 *
 * Written by Theodore Ts'o, 3/21/09.
 *
 * This file may be redistributed under the terms of the GNU Public
 * License, version 2.
 */

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <time.h>
#include <fcntl.h>
#include <string.h>

#define SIZE (32768*32)

static float timeval_subtract(struct timeval *tv1, struct timeval *tv2)
{
        return ((tv1->tv_sec - tv2->tv_sec) +
                ((float) (tv1->tv_usec - tv2->tv_usec)) / 1000000);
}

int main(int argc, char **argv)
{
        int fd;
        struct timeval tv, tv2;
        char buf[SIZE];

        fd = open("fsync-tester.tst-file", O_WRONLY|O_CREAT);
        if (fd < 0) {
                perror("open");
                exit(1);
        }
        memset(buf, 'a', SIZE);
        while (1) {
                pwrite(fd, buf, SIZE, 0);
                gettimeofday(&tv, NULL);
                fsync(fd);
                gettimeofday(&tv2, NULL);
                printf("fsync time: %5.4f\n", timeval_subtract(&tv2, &tv));
                sleep(1);
        }
}
        

I got the overall best results using XFS or ext4, together with the CFQ
elevator. The major showstoppers after formatting with these are noatime
and barrier support, and one should also consider to mount ext4 with
data=writeback. By using aufs, the asynchronous writes should also help a
lot.

liesel:~ # mount | grep squid
/dev/sdb1 on /var/cache/squid type xfs (rw,noatime,logbsize=256k,logbufs=2,nobarrier)

liesel:~ # hdparm -i /dev/sdb1

/dev/sdb1:

Model=OCZ SOLID_SSD , FwRev=02.10104,
SerialNo=MK0708520E8AA000B
Config={ Fixed } RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=63
BuffType=unknown, BuffSize=0kB, MaxMultSect=1, MultSect=?1?
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=62586880
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
AdvancedPM=no WriteCache=disabled
Drive conforms to: unknown: ATA/ATAPI-1,2,3,4,5,6,7
 
* signifies the current active mode
Received on Sun Aug 02 2009 - 11:27:58 MDT