← retour aux snippets

python: multiprocessing Pool.map

Paralléliser une fonction pure sur des éléments.

objectif

Paralléliser une fonction pure sur des éléments.

code minimal

from multiprocessing import Pool, cpu_count
def f(x): return x*x
with Pool(processes=min(2, cpu_count())) as p:
    out = p.map(f, [1,2,3])
print(out[0])

utilisation

from multiprocessing import Pool
def g(x): return x+1
with Pool(processes=2) as p:
    print(len(p.imap(g, range(5))))

variante(s) utile(s)

from multiprocessing import Pool
def h(x): return x
with Pool(processes=2) as p:
    out = list(p.starmap(h, [(1,), (2,)]))
print(out[-1])

notes

  • Éviter objets lourds; sérialisation via pickle coûteuse.