How to Run Your Python Code Concurrently Using Threads

Originally posted in makeuseof.

Threading significantly reduces the execution time of a program. Learn how to implement threading in Python.

Execution time is one of the common measures of the efficiency of a program. The faster the execution time the better the program. Threading is a technique that allows a program to perform multiple tasks or processes simultaneously.

You will learn how to use the Python built-in threading module and the concurrent.features module. Both of these modules offer simple ways to create and manage threads

Importance of Threading

Threading reduces the amount of time a program takes to complete a job. If the job contains multiple independent tasks, you can use threading to run the tasks concurrently, reducing the program’s wait time for one task to finish before moving on to the next.

For example, a program that downloads multiple image files from the internet. This program can utilize threading to download the files in parallel rather than one at a time. This eliminates the time the program would have to wait for the download process of one file to complete before moving on to the next one.

Initial Program Before Threading

The function in the following program represents a task. The task is to pause the execution of the program for one second. The program calls the function twice hence creating two tasks. It then calculates the time it took for the whole program to run and then displays it on the screen.

import time

start_time = time.perf_counter()

def pause():
    print('Sleeping 1 second...')
    time.sleep(1)
    print('Done Sleeping...')

pause()
pause()
finish_time = time.perf_counter()
print(f'Finished in {round(finish_time - start_time, 2)} second(s)')

The output shows the program took 2.01 seconds to execute. Each task took one second and the rest of the code took 0.01 seconds to execute.

You can use threading to concurrently execute both tasks. This will take both tasks one second to execute.

Implementing Threading Using the threading Module

To modify the initial code to implement threading, import the threading module. Create two threads, thread_1 and thread_2 using the Thread class. Call the start method on each thread to start its execution. Call the join method on each thread to wait for their execution to complete before the rest of the program executes.

import time
import threading
start_time = time.perf_counter()

def pause():
    print('Sleeping 1 second...')
    time.sleep(1)
    print('Done Sleeping...')

thread_1 = threading.Thread(target=pause)
thread_2 = threading.Thread(target=pause)

thread_1.start()
thread_2.start()

thread_1.join()
thread_2.join()

finish_time = time.perf_counter()
print(f'Finished in {round(finish_time - start_time, 2)} second(s)')

The program will run both threads concurrently. This will reduce the amount of time it takes to accomplish both tasks.

The output shows the time taken to run the same tasks is around a second. This is half the time the initial program took.

Implementing Threading Using the concurrent.futures Module

Python 3.2 saw the introduction of the concurrent.futures module. This module provides a high-level interface for executing asynchronous tasks using threads. It provides a simpler way of executing tasks in parallel.

To modify the initial program to use threading, import the concurrent.features module. Use the ThreadPoolExecutor class from the concurrent.futures module to create a pool of threads. Submit the pause function to the pool twice. The submit method returns a future object that represents the result of the function call.

Iterate over the futures and print their results using the result method.

import time
import concurrent.futures

start_time = time.perf_counter()

def pause():
    print('Sleeping 1 second...')
    time.sleep(1)
    return 'Done Sleeping...'

with concurrent.futures.ThreadPoolExecutor() as executor:
    results = [executor.submit(pause) for _ in range(2)]
    for f in concurrent.futures.as_completed(results):
        print(f.result())

finish_time = time.perf_counter()

print(f'Finished in {round(finish_time - start_time, 2)} second(s)')

The concurrent.features module takes care of starting and joining the threads for you. This makes your code cleaner.

The output is identical to that of the threading module. The threading module is useful for simple cases where you need to run a few threads in parallel. On the other hand, the concurrent.futures module is useful for more complex cases where you need to run many tasks concurrently.

Using Threading in a Real-World Scenario

Using threads to run the above program reduced the time by one second. In the real world, threads save more time. Create a program that downloads images from the internet. Start by creating a new virtual environment. Run the following command in the terminal to install the requests library:

pip install requests

The requests library will allow you to send HTTP requests. Import the requests library and the time library.

import requests
import time

Create a list of URLs of the images you would like to download. Let them be at least ten so that you can notice a significant difference when you implement threading.