Re: memory mapped store entries

On 27 Aug 98, at 10:38, Chris Wedgwood <chris@cybernet.co.nz> wrote:

> I was on the phone arguing with a `customer' when I typed this. I'd
> better clarify.
>
> On Thu, Aug 27, 1998 at 10:27:40AM +1200, Chris Wedgwood wrote:
> > Since dirty pages probably won't be flushed by the kernel until
> > they are paged out of explicitly msync'd - it should be more often,
> > much more often.
> >
> > We don't have to sync the whole file either, we can get away with
> > sync parts of it, and roll a window over the swap so that we sync
> > it bit by bit.
>
> Dirty pages won't normally be sync'd to disk by the kernel until they
> are paged out - and when this is happening, your squids performance
> will suck badly anyhow.

 Dirty pages have some max lifetime in any kernel. Kernel _must_ flush
 them in some timeframe, otherwise it is a _broken_ kernel.
 And this is a backgroound job that squid or any other procees
 will never notice. (unless kernel is poorly implemented)

> This means, the data has to be explicitly msync'd fairly often - but
> we don't need to sync the entire swap at any one time, we can do it
> bit by bit.

 No. You don't want to explicitly msync anything. msync will block the
 whole process for the duration of a pageout. You bust algoritms that
 try to accumulate changes to the same pages, nothing more. You can
 tune times that kernel keeps dirty unflushed pages in ram outside
 squid, and this belongs to configuration guidelines, not code.

> It also means, we can't always assume the swap is completely reliable
> and consistent with the store, it might be out of date, in which
> case, we have to rebuild part or all of it.

 Again I disagree. mmaped data could get inconsistent _only_ in a case
 that squid proccess is wiped out of existence just while it is in the
 proccess of updating any single store entry AND has not yet updated
 structure consistently. In any other case, the mmapped store data will
 be consistent, as kernel _must_ flush all dirty pages when it has to
 unmap all pages related to the process.

 So, you can kill -9 squid any time, and the mmaped data will be flushed
 with the state it was at the time of receiving signal.

 It is way much more reliable than catching signals and trying to write
 out swaplog as it is done currently.

 ----------------------------------------------------------------------
  Andres Kroonmaa mail: andre@online.ee
  Network Manager
  Organization: MicroLink Online Tel: 6308 909
  Tallinn, Sakala 19 Pho: +372 6308 909
  Estonia, EE0001 http://www.online.ee Fax: +372 6308 901
 ----------------------------------------------------------------------
Received on Tue Jul 29 2003 - 13:15:53 MDT