Re: memory mapped store entries

Hiya,

> The process shouldn't be waiting on this, since the kernel would be
> doing this in the background (if at all; since we're the only one opening
> the file, the system could keep all pages in memory until we terminate
> and then write them back).

        I'm unclear as to how the kernel handles the flushing of memory
pages out to disk. If this is in progress will the kernel block the
current page access until that flush is complete? I thought it did,
since to do otherwise would require the kernel to copy the data and then
write that out. However mmap() states that it doesn't do this and just
writes from the same page copy in RAM as the process has access to. So
my point still stands that the main thread will stall while these kinds
of flushes happen. I suppose we could pre-empt this flushing by calling
fsync(), but then we're still stalling the main thread. If we push fsync()
into a sub-thread the main thread will still stall while the fsync() completes.

I'm happy to be corrected on the above point if you can prove otherwise.

> > or with it stalling
> >when the data needs to be paged in from disk to do a change.
>
> The process would stall if it had to be paged in from disk, yes. But,
> since we're the only one that have the file open, apart from the initial
> read when we read in the file the first time, there shouldn't really
> be any disk reads since everything can be kept in memory.

So you're saying that you want to mlock() all these pages into RAM so they
will never get paged out and cause delays?

> >I vote VERY strongly against any form of mmap() data access unless it is
> >wrapped within a thread to separate page fault activity away from the main
> >thread.
>
> Do you know of any kernels which do not deal with these mmap'd files
> intelligently as conjectured above?

I'm concerned with mmap() as a whole for any OS.

> I guess we'd have to maintain separate mmap/no-mmap cases anyway.
> BTW, on Linux (2.0.35), this particular mmap approach appears to be showing
> benefits only, no noticeable drawbacks so far.

Got some figures? How have you measured this? What about consistency in
the event of a crash? What about needing to extend the file which mmap()
can't do transparantly? (Sure we can just pre-allocate that one) Are we
happy to mlock() pages into RAM to ensure mmap()'s performance?

How about under extremely high loads where disks are busy and take up to
100ms to get a page in/page out and that 100ms means 10 requests that you
can't service while that happens?

I'm happy to admit that mmap() will provide some benefits under lowish loads.
My concern is always at the bleeding edge of performance and I'm ever happy
to sacrifice some low-end speed by 5-10% if it means high end speed is 10%
faster.

        Stew.

-- 
Stewart Forster (Snr. Development Engineer)
connect.com.au pty ltd, Level 9, 114 Albert Rd, Sth Melbourne, VIC 3205, Aust.
Email: slf@connect.com.au   Phone: +61 3 9251-3684   Fax: +61 3 9251-3666