Re: dlmalloc

On Wed, 17 Dec 2003, Andres Kroonmaa wrote:

> Could you list some requirements for malloc implementation that needs
> to be met to be considered optimal and safe malloc for Squid?

For a start there would need to be a official standard in how one is to
replace the malloc implementation without risk for conflicts with the OS
provided C/C++ libraries or other libraries used by Squid, and this
standard would need to be well understood by autoconf to not pick up any
traces of the OS provided malloc implementation when you replace it with
another.

> But most interesting to me is in what ways has it bitten us? dlmalloc
> was written for C++ in the first place.. And linux glibc malloc is
> based on dlmalloc/ptmalloc afaik. Thats why hearing that it is unsafe
> is especially intriguing.

The most common error and risk is that dlmalloc does not implement all
entry points to the OS malloc implementation, and this causes a big mess
of things both in Squid and in any library used by Squid if any piece of
code ever tries to use one of the malloc functions/symbols not
implemented by dlmalloc. When this happens parts of both malloc
implementations gets linked into the program in uncompatible manners.

Another risk is dynamic linking of the OS libc if any of the symbols
exported by the OS provided malloc package is not a weak symbol. In such
case functions in the OS libc using such sybmol will continue to use the
OS malloc code even if dlmalloc provides another implementation of the
same.

> I have understood that we bundle dlmalloc because it fits well enough
> for Squid and can be used as alternate malloc on OS'es that would suffer
> if using native libs, so that instead of forcing users into an OS hunt
> we can switch over to dlmalloc transparently. Having one source bundled
> allows us better integrate expectations of both.

True, but it is also true that mempools is supposed to take most of the
heat away from the malloc implementation.

> dlmalloc is very tunable, at both compiletime and runtime. This has been
> ignored so far.

None of the developers has needed to care about dlmalloc tuning afaik.

Regards
Henrik
Received on Wed Dec 17 2003 - 01:57:26 MST