dead lock in forked child

[Salvatore Sanfilippo]

On Wed, Oct 10, 2012 at 2:53 AM, Mowry, Robert <Robert.Mowry at netapp.com> wrote:
> I can understand the motivation behind wanting this to work, but when I
> read the POSIX spec for threaded programs that call fork(), it seems
> pretty clear that malloc() and free() etc are not defined to be safe to
> call in the child process:

Hi Robert,

you are absolutely right, and I agree that avoiding to rely on this
kind of behaviour is safe for most applications.

However Redis is system software, and for it to work properly and at
the best of what current hardware can do, we actually rely on many
things that are common in modern unix-like operating systems and are
not specified by the POSIX standard (including the behaviour of the
virtual memory that must support copy on write).

So to write more complex and convoluted code to escape the
malloc+threads+fork problem is pointless in the case of Redis since
there are anyway bigger problems to make it a truly portable software.
As long as we are sure that the systems we support more closely
(Linux, Darwin, and *BSD basically) provide a version of malloc() that
is safe in this context, we try to be pragmatic and go forward with
it.

That said Redis is *almost* single threaded, but we use a few threads
to call system calls such as close(2) that may block our main thread
resulting into latency spikes. At the same time we need to fork() as
fork is used by Redis to dump the DB file on disk in the case of the
RDB persistence, or to rewrite the append only file (that is a
different persistence method). So we have a big advantage in this
setup that can not be easily simulated by other means... as long as it
works in a reliable way, we need to say at the limits (and over) of
POSIX...

However we ship Redis with a copy of jemalloc inside our source tree
that is used when building on Linux. As long as jemalloc is safe, we
are safe under Linux that is where 99.9% of deployments happen. So at
least we are not subject to random glibc changes and behaviours that
are outside of our control.

Cheers,
Salvatore

-- 
Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org

Beauty is more important in computing than anywhere else in technology
because software is so complicated. Beauty is the ultimate defence
against complexity.
       — David Gelernter