複数のパラメータを持つ ThreadPoolExecutor マップ メソッド

Oct 19, 2017

Python 3 の parallel.futures パッケージの ThreadPoolExecutor は、一連のデータ (パラメータ) を持つタスク (関数) を同時に実行するのに非常に便利です。この投稿では、実行中のタスクに複数のパラメータを渡す方法の例を示します。

Pass by same length iterables

from concurrent.futures import ThreadPoolExecutor
import threading

data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

prefix = "test_"

def add_prefix(prefix, i):
    print("%s: %s%s" % (threading.get_ident(), prefix, i))


if __name__ == "__main__":
    with ThreadPoolExecutor(10) as pool:
        pool.map(add_prefix, [prefix] * len(data), data)

Pass with wrapper lambda function

from concurrent.futures import ThreadPoolExecutor
import threading

data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

prefix = "test_"

def add_prefix(prefix, i):
    print("%s: %s%s" % (threading.get_ident(), prefix, i))


if __name__ == "__main__":
    with ThreadPoolExecutor(10) as pool:
        args = ((prefix, i) for i in data)
        pool.map(lambda p: add_prefix(*p), args)

Pass with repeat function from itertools

from concurrent.futures import ThreadPoolExecutor
import threading
from itertools import repeat

data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

prefix = "test_"

def add_prefix(prefix, i):
    print("%s: %s%s" % (threading.get_ident(), prefix, i))


if __name__ == "__main__":
    with ThreadPoolExecutor(10) as pool:
        pool.map(add_prefix, repeat(prefix), data)

参考文献

• concurrent.futures -- Launching parallel tasks
• Pass multiple parameters to concurrent.futures.Executor.map?

concurrent multithreading python