How to Use Python Subprocess Module

In this article, you will learn how to use Python subprocess module.

Python Subprocess Module

The Python subprocess module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module provides a way to execute shell commands from within Python scripts and capture their output. Here is a simple example to illustrate how to use the subprocess module:

import subprocess

# Run a shell command and capture the output
output = subprocess.check_output("ls -l", shell=True)

# Print the output
print(output)

In this example, the check_output function is used to run the command ls -l, which lists the files in the current directory with detailed information. The shell parameter is set to True, which tells subprocess to execute the command through the shell. The output variable is assigned the output of the command.

You can also use the subprocess.Popen() method to spawn a new process, and then communicate with it by reading and writing to its input/output/error streams. Here is an example of how to use Popen:

import subprocess

# Spawn a new process
p = subprocess.Popen("echo hello world", stdout=subprocess.PIPE, shell=True)

# Read the output
output, err = p.communicate()

# Print the output
print(output.decode())

In this example, the Popen function is used to spawn a new process that simply echoes “hello world” to stdout. The stdout parameter is set to subprocess.PIPE, which redirects the process’s output to a pipe that can be read by the parent process. The communicate method is used to read the output from the pipe and capture any errors that occurred. The output is then printed to the console.

Here are some additional details about the subprocess module:

Executing Commands

There are several ways to execute a command using the subprocess module. In the example above, we used the check_output function to execute a command and capture its output. This function is a convenience function that runs a command and returns its output as a byte string. If the command returns a non-zero exit status, a CalledProcessError exception is raised.

If you need more control over the command execution, you can use the Popen function to spawn a new process. This function returns a Popen object that can be used to communicate with the process. Here is an example:

import subprocess

# Spawn a new process
p = subprocess.Popen(["ls", "-l"])

# Wait for the process to complete
p.wait()

# Check the exit code
if p.returncode != 0:
    print("Command failed")

In this example, the Popen function is used to spawn a new process that runs the ls -l command. The arguments to Popen are passed as a list, with the command as the first element and any arguments as subsequent elements. The wait method is used to wait for the process to complete. The returncode attribute of the Popen object is then checked to see if the command succeeded.

Communicating with Processes

Once you have a Popen object, you can communicate with the process using its stdin, stdout, and stderr streams. Here is an example that demonstrates how to execute a command and capture its output:

import subprocess

# Spawn a new process
p = subprocess.Popen(["ls", "-l"], stdout=subprocess.PIPE)

# Read the output
output, err = p.communicate()

# Print the output
print(output.decode())

In this example, the Popen function is used to spawn a new process that runs the ls -l command. The stdout parameter is set to subprocess.PIPE, which redirects the process’s output to a pipe that can be read by the parent process. The communicate method is used to read the output from the pipe and capture any errors that occurred. The output is then printed to the console.

Passing Arguments

If you need to pass arguments to a command, you can do so by passing them as separate elements in a list. For example, to pass a file name as an argument to the cat command, you can do this:

import subprocess

# Spawn a new process
p = subprocess.Popen(["cat", "myfile.txt"])

# Wait for the process to complete
p.wait()

# Check the exit code
if p.returncode != 0:
    print("Command failed")

In this example, the Popen function is used to spawn a new process that runs the cat command with the myfile.txt file as an argument.

Handling Errors

If a command returns a non-zero exit status, a CalledProcessError exception is raised. You can catch this exception and handle the error in your code. For example:

import subprocess

try:
    # Execute the command
    output = subprocess.check_output(["ls", "nonexistent_file"])
except subprocess.CalledProcessError as e:
    print("Command failed with error code", e.returncode)

In this example, the check_output function is used to run the ls nonexistent_file command, which will fail because the file does not exist. The CalledProcessError exception is caught and its returncode attribute is