edit: Turns out it's related to the size of the return, not variability. Updated minimal example to reflect consistent size that still fails.

I have a script that's trying to analyse some images in parellel. For some reason, I get intermittent crashes if the func passed to the pool returns variable sized data. This is only if I try to exit early and call pool.terminate(), so clearly I'm missing some clean up.

Tried googling if that's intentional with multiprocessing pools but couldn't find anything.

import multiprocessing, time
from signal import signal, SIGINT
def pool_initializer():
    signal(SIGINT, lambda signum, frame: print(signum, frame))

def analyse(preview):
    time_start = time.perf_counter()

    totalLabels = 64
    label_ids = [[42 for x in range(2048)] for y in range(2048)]
    values = [[0,1,2,3,4] for x in range(totalLabels)]
    centroids = [[0,1,2,3,4] for x in range(totalLabels)]

    # Roughly equivelant to output of cv2.connectedComponentsWithStats()
    cc_stats = (totalLabels, label_ids, values, centroids)

    return (cc_stats, time.perf_counter() - time_start)

if __name__ == "__main__":
    with multiprocessing.Pool(processes=int(multiprocessing.cpu_count()/2), initializer=pool_initializer) as pool:
        for result, preview in zip(pool.imap(analyse, range(10)), range(10)):
            ## Not exiting early results in zero crashes regardless of which return is chosen
            if True:
                break
        
        ## Intermittently crashes in pool.terminate()
        pool.terminate()
        pool.join()

    print("Great success!")

Traceback (most recent call last):
  File "H:\wouldntyouliketoknowwaterboy\test.py", line 36, in <module>
    pool.terminate()
    ~~~~~~~~~~~~~~^^
  File "C:\Users\wouldntyouliketoknowwaterboy\AppData\Local\Programs\Python\Python313\Lib\multiprocessing\pool.py", line 657, in terminate        
    self._terminate()
    ~~~~~~~~~~~~~~~^^
  File "C:\Users\wouldntyouliketoknowwaterboy\AppData\Local\Programs\Python\Python313\Lib\multiprocessing\util.py", line 216, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "C:\Users\wouldntyouliketoknowwaterboy\AppData\Local\Programs\Python\Python313\Lib\multiprocessing\pool.py", line 703, in _terminate_pool  
    outqueue.put(None)                  # sentinel
    ~~~~~~~~~~~~^^^^^^
  File "C:\Users\wouldntyouliketoknowwaterboy\AppData\Local\Programs\Python\Python313\Lib\multiprocessing\queues.py", line 394, in put
    self._writer.send_bytes(obj)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "C:\Users\wouldntyouliketoknowwaterboy\AppData\Local\Programs\Python\Python313\Lib\multiprocessing\connection.py", line 200, in send_bytes 
    self._send_bytes(m[offset:offset + size])
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\wouldntyouliketoknowwaterboy\AppData\Local\Programs\Python\Python313\Lib\multiprocessing\connection.py", line 287, in _send_bytes
    raise ValueError("concurrent send_bytes() calls "
                     "are not supported")
ValueError: concurrent send_bytes() calls are not supported

I could just filter the output and call it a day but then I would be left ignorant. Can someone explain what is happening here?