from A Fork in the Road, <https://www.microsoft.com/en-us/research/uploads/prod/2019/0...>

BTW, article title needs a (2016). It appears that the relevant Python bug has long since been closed, by avoiding linking with the system sqlite on macOS.

1. No analog to tcsetpgrp, so it's no good if job control is enabled

2. No analog to fchdir, meaning you have to synchronize with fchdir elsewhere in the progarm

3. Error codes do not convey enough information for good error messages (e.g. if a file doesn't exist, posix_spawn doesn't tell you which file)

4. Inconsistent behavior around dup2 fd redirections and CLO_EXEC.

5. Inconsistent behavior for shebangless scripts

These are basically deal-breakers so fish also supports a fork/exec path. However the performance benefits of posix_spawn are too real to ignore so fish uses posix_spawn when it can, and fork/exec when it must.

Go only exports os.ForkExec() -- there is no os.Fork() or os.Exec(), because the things you can do between the calls could break Go's threaded runtime. (Goroutines are implemented with OS threads.)

Some elaboration on that: https://lobste.rs/s/hj3np3/mvdan_sh_posix_shell_go#c_qszuer

That is, the space between fork and exec is where pipelines are implemented, but also entire subinterpreters/subshells. The shell actually uses copy-on-write usefully. (And yes I'm aware that there's a good argument that the shell is almost the ONLY program that needs fork() !)

----

A lot of people have asked me why not implement Oil in Go and various other languages, so I wrote this page:

https://github.com/oilshell/oil/wiki/FAQ:-Why-Not-Write-Oil-...

So the funny thing is that Python is a lower level language than Go for this particular problem. It doesn't do anything weird with regard to syscalls. I'm still looking for help on this (and donations to pay people other than me):

Oil Is Being Implemented "Middle Out" https://www.oilshell.org/blog/2022/03/middle-out.html

But to be fair, the only times I can recall using fork() without exec() were forking network servers, and that was mostly me learning about doing network stuff, and a forking server was the easiest to implement manually.

Oh yeah, and that one time I accidentally wrote a fork bomb trying to stress test a DNS server. At least I learned something from my mistake. ;-)

EDIT: To me, using fork() without exec() is kind of like operator overloading - there are cases where it absolutely is the right tool, but these aren't very numerous, so one should exercise caution. A lot.

  execvpehm(
    ..., 
    int *handles, size_t,
    void **pages, size_t,
    /* etc */
  );

"After a fork() in a multithreaded program, the child can safely call only async-signal-safe functions (see signal-safety(7)) until such time as it calls execve(2)."

So just use only async-singal-safe function https://man7.org/linux/man-pages/man7/signal-safety.7.html

I don't know why so many people still hit this issue when it already told you what you can do and not do in the document. I've done this sort of things without any issue.

Fork() without exec() is dangerous in large programs - https://smackernews.com/item/12302539 HN - Aug 2016 (101 comments)

https://smackernews.com/item/863871 HN (13 years? Yikes!)

Threads were bolted onto Unix in a hamfisted way, breaking more than just fork. For instance, threads broke relative paths, requiring "at" functions like openat to be invented, an ugly stop-gap measure. Threads were badly integrated with signal handling too, another example.

Blaming those existing mechanisms is purely an emotional argument, from the perspective of being infatuated with threads.

The design of threads (coming from various efforts that became POSIX threads) came from such an infatuation: the desire to get any kinds of threads working at any cost, while ignoring the global state that exists in a Unix process, and the need to make a lot of it thread-local, or at least optionally so.

A thread-local working directory or signal mask would have caused difficulties in hack thread implementations that used user space scheduling or M:N (M user space threads to N kernel tasks).

The situation we have today largely comes from the initial reluctance to accept the fact that each thread has to be an entity known to the kernel; the belief that user space threads are viable into the long-term future.

Only use fork in toy programs. The challenge is that successful toy programs grow into large ones, and large programs eventually use threads. It might be best just to not bother.

How do you create a new process and pipe it data in a fast fashion without using fork, exec or posix_spawn ?

Since you probably don't know what all the other threads in your process are up to, your only option is to attach a debugger to all of them, halt them all, and copy all their state into brand new threads in the child process.

Do it all correctly and you end up with a multi-threaded-fork.

You still need to fix up signal handlers, interrupted syscalls, various notification API's that no longer work, memory mapped temp files used for IPC, pipes and sockets, and a bunch of other things.

But a fork of a complex process is possible. It just isn't easy.

I also find that libraries that absolutely need to make their own threads are better off being their own process. Then you can use proper communication methods to pass data.

When I ran into this problem, I was just trying to run all of Bluecore's unit tests on my Mac laptop. We use nose's multiprocess mode, which uses Python's multiprocessing module to utilize multiple CPUs. Unfortunately, the tests hung, even though they passed on our Linux test server.

There will never be a time at which you can reliably expect any program developed on one system to "just work" on a different system. This person wasted a lot of time tracking down what was essentially a portability bug. Did they need this to be portable? Was this time well spent generating business value?

Pick one system for development through production, stick to it. There will be portability bugs hiding in your code, but you will never have to fix them. You will be upset for a minute that you can't use a different system, but you will get over it.