Set Timeouts on Steps of an Excel Report Job

A report job that fails is easy to deal with: it raises, the alert goes out, someone looks at it. A job that hangs is worse. It never fails, so nothing alerts; it holds its lock, so the next scheduled run skips or stacks up behind it; and the first anyone hears of it is a manager asking where the morning report is. Hangs almost always come from waiting on something outside the script — an API that accepts the connection and never answers, an SMTP server that stops responding mid-conversation, a LibreOffice conversion stuck on a dialog nobody can see. Python waits forever by default in most of these places. This page puts a time limit on each kind of step, shows what happens when the limit is hit, and adds a final limit around the whole job, testing each one against something deliberately slow. It belongs to Error Handling and Logging in Excel Automation.

Prerequisites

pip install requests pandas openpyxl

The examples run a tiny local HTTP server that takes as long as it is told to before replying, so the timeouts can be demonstrated without depending on a real slow service.

Step 1 — HTTP calls: always pass a timeout

import time
import requests

def fetch(delay, timeout=(3.05, 2)):
    t0 = time.perf_counter()
    try:
        r = requests.get(f"http://127.0.0.1:8765/sales?delay={delay}", timeout=timeout)
        return f"ok {r.json()} in {time.perf_counter() - t0:.1f}s"
    except requests.Timeout as e:
        return f"{type(e).__name__} after {time.perf_counter() - t0:.1f}s"

print("fast server:", fetch(0.2))
print("slow server:", fetch(5))

Against the local test server, the fast request returns in a fraction of a second and the one that would take five seconds is abandoned after two with a ReadTimeout. requests has no default timeout: without the timeout argument a call to a server that accepts the connection but never replies waits indefinitely. The argument takes two numbers — how long to wait to connect and how long to wait between bytes of the response — and a value of a few seconds for connecting and somewhat longer for reading suits most reporting APIs. Note that the read timeout is not a limit on the total download time; it is the longest silence allowed. A server that trickles one byte every second will never trip it, which is why step 3 adds an overall limit as well. When the timeout fires, requests raises requests.Timeout, which the job can retry or report like any other error — see Retry a Failed Excel Report Job in Python.

Step 2 — SMTP and subprocesses

import smtplib, subprocess

# smtplib: the timeout applies to the connection and to every command after it
try:
    smtplib.SMTP("10.255.255.1", 25, timeout=2)          # an address that never answers
except (TimeoutError, OSError) as e:
    print("smtp:", type(e).__name__)

# subprocess: the child is killed when the timeout expires
t0 = time.perf_counter()
try:
    subprocess.run(["sleep", "30"], timeout=2, check=True)
except subprocess.TimeoutExpired as e:
    print(f"subprocess: TimeoutExpired after {time.perf_counter() - t0:.1f}s for {e.cmd}")

smtplib.SMTP takes a timeout that covers connecting and every later command on the connection, so a mail server that stalls halfway through a large attachment is caught too. Without it, the library falls back to the global socket default, which is no timeout at all. subprocess.run with timeout= waits that long and then kills the child process and raises TimeoutExpired. This matters most for converting workbooks to PDF with LibreOffice, which can sit forever on a recovery or macro-security dialog that nobody sees in a headless session; a generous limit of a minute or two per file turns that into an error the job can report. When LibreOffice is started with a separate profile directory per run, a killed conversion does not leave a lock behind for the next one, as explained in Convert Excel to PDF on Linux with LibreOffice.

Step 3 — A step with no timeout option: run it in a child process

import multiprocessing as mp

def slow_transform(path, seconds):
    import time, pandas as pd
    time.sleep(seconds)                          # stands in for a step that hangs
    pd.DataFrame({"a": [1, 2]}).to_excel(path, index=False)

def run_with_limit(target, args, limit):
    ctx = mp.get_context("spawn")
    p = ctx.Process(target=target, args=args)
    p.start()
    p.join(limit)
    if p.is_alive():
        p.terminate()
        p.join(5)
        if p.is_alive():
            p.kill()
        raise TimeoutError(f"{target.__name__} did not finish within {limit}s")
    if p.exitcode != 0:
        raise RuntimeError(f"{target.__name__} exited with code {p.exitcode}")

if __name__ == "__main__":
    run_with_limit(slow_transform, ("ok.xlsx", 0.5), limit=10)
    print("fast step finished")
    try:
        run_with_limit(slow_transform, ("stuck.xlsx", 60), limit=3)
    except TimeoutError as e:
        print("timeout:", e)

python step_limits.py

Some steps have no timeout setting at all: a long pandas operation, a third-party library that blocks inside C code, a database driver without a statement timeout. A thread does not help here, because Python has no way to stop a thread from outside — it keeps running after you stop waiting for it. A separate process can be stopped: run the step in a child process, wait for it with join(limit), and if it is still alive, terminate() it (and kill() it if it ignores that). The spawn start method gives the child a clean interpreter, which is the default on Windows and macOS and the safest choice everywhere. The cost is that the step's inputs and outputs have to cross the process boundary, so this pattern suits steps that read a file and write a file, like the one above. The if __name__ == "__main__": guard is required with spawn, because the child imports the main module; for the same reason the step function must be defined at module level in a file, not inside another function or typed into an interactive session. Run as a script, the fast step finishes normally and the stuck one is stopped after three seconds with a clear TimeoutError.

Step 4 — An overall limit for the whole job

The step limits handle the expected waits; a final limit around the whole job catches everything else — a step without a limit that someone adds next year, a server that trickles data slowly enough to stay under the read timeout. Put it in the scheduler, where it works even if the Python process itself is stuck:

# cron with coreutils timeout: SIGTERM after 30 minutes, SIGKILL 60 seconds later
0 6 * * 1-5  timeout --kill-after=60s 30m /opt/reports/.venv/bin/python /opt/reports/daily_sales.py

# systemd service: the same limit as a unit setting
# [Service]
# TimeoutStartSec=30min

On Windows, Task Scheduler's Stop the task if it runs longer than setting does the same. Choose the limit from the job's history — about twice its normal run time is a reasonable start — so it only fires when something is really wrong. A job killed this way exits with a non-zero status (124 for timeout), so the usual failure alert still goes out, and it releases its lock so the next run can start.

Choosing the numbers

A timeout that is too short creates failures where there were none, and one that is too long lets a hang eat into the time left before the deadline. The run history is the best guide: if fetching data from an API normally takes four seconds and has never taken more than fifteen, a read timeout of thirty seconds catches hangs without false alarms. Combine timeouts with a small number of retries for calls that are safe to repeat, and make sure the retries plus their timeouts still fit inside the job's overall limit — three retries of a two-minute timeout is six minutes, which a thirty-minute job can absorb but a five-minute one cannot.

Common pitfalls

• No timeout= on requests. The default is to wait forever.
• Treating the read timeout as a total limit. It is the longest silence; add an overall limit.
• Threads for timeouts. A thread cannot be stopped; use a child process for stuck steps.
• No limit in the scheduler. It is the only one that works when the process itself hangs.
• Retries that exceed the deadline. Retries × timeout must fit within the job limit.

Conclusion

Hangs are worse than failures because nothing reports them. Give every HTTP call a (connect, read) timeout, every SMTP connection a timeout=, and every subprocess a timeout= so LibreOffice and other tools are killed when stuck; run steps that cannot be interrupted in a child process that can be terminated; and put an overall limit on the job in cron, systemd or Task Scheduler.

Frequently asked questions

Does requests have a default timeout? No; without timeout= a request can wait forever. Pass a tuple such as (3.05, 30) for connect and read timeouts.

How do I stop a subprocess that hangs? Use subprocess.run(..., timeout=seconds); when the time is up, the child is killed and TimeoutExpired is raised.

How can I put a time limit on a Python function? Run it in a multiprocessing.Process, call join(limit), and terminate the process if it is still alive; threads cannot be stopped from outside.

How do I limit the total run time of a cron job? Wrap the command in timeout, for example timeout --kill-after=60s 30m python job.py, or set a time limit in systemd or Task Scheduler.

Related

• Up one level: Error Handling and Logging in Excel Automation — robust report jobs.
• Prevent Overlapping Runs of an Excel Report Job — locks between runs.
• Track Run History and Step Timings of a Report Job — choosing limits from history.
• Send an Alert When an Excel Report Job Fails — alerting when a limit fires.