EVALUATION OF CUNCURRENCY METHODS IN PYTHON FOR PROCESSOR-INTENSIVE TASKS ON MULTICORE SYSTEMS

Abstract

Python’s popularity in scientific computing and security has raised interest in leveraging multicore processors for CPU-bound tasks such as cryptographic computations. This paper evaluates two concurrency approaches in Python – multithreading (using the threading module or ThreadPoolExecutor) and multiprocessing (using the multiprocessing module or ProcessPoolExecutor) – for CPU-bound cryptographic workloads on multicore systems. We focus on tasks like RSA encryption and decryption, which involve intensive computation. We discuss the impact of Python’s Global Interpreter Lock (GIL) on multithreading performance, and contrast it with multiprocessing, which sidesteps the GIL by using separate processes. A simulated experimental study is presented: threads and processes are compared in terms of execution time, speedup relative to single-thread execution, CPU utilization, and memory overhead on multicore hardware. The findings show that while multiprocessing produces near-linear speedups proportionate to the number of cores (e.g., nearly 4× speedup on 4 cores), threads offer absolutely no speedup for CPU-bound cryptographic workloads because of the GIL. Nevertheless, multiprocessing results in increased memory consumption and communication overhead between processes. We include a performance comparison table and chart to illustrate these findings. The study highlights that while Python threads are limited by the GIL, processes can fully utilize multiple cores for CPU-bound workloads at the cost of greater resource usage. We discuss over these trade-offs and how they affect cryptographic applications. We also point out potential developments that could affect concurrent strategy decisions, like attempts to reduce or get around the GIL in next Python versions. All things considered, this study presents a thorough assessment of Python concurrency techniques for CPU-intensive tasks and recommendations for improving multicore cryptographic computations