Publishing Excel Reports to Cloud Storage

A report that nobody can find has not been delivered. Email attachments were the default answer for years, and they are a poor one: they duplicate the file into every mailbox, bounce off size limits, and leave five people holding four different versions. Publishing to shared storage fixes all of that — one canonical copy, a stable link, an access trail, and version history for free. This page covers the four destinations that account for nearly all real reporting jobs, and the transfer discipline that separates a reliable publish step from one that occasionally leaves a truncated file where the report should be. It is the delivery stage of Automating Reporting Workflows.

Publish, then link — not attach

The decision to make first is not which store, but whether to send the file at all. Attachments have four failure modes that a link does not: size limits reject the message, every recipient holds a private copy that immediately diverges, forwarding puts the file somewhere with no access control, and nobody can tell whose version is current.

Publishing inverts all of that. One copy exists, the link always resolves to it, and the store records who opened it. Access follows the organisation's existing permissions rather than the accident of who was on the distribution list, so somebody joining the team gets the report by being added to a group, and somebody leaving loses it the same way. Version history comes free in every store worth using, which turns "what did last month's figures say before the restatement?" from an archaeology exercise into a click. And because the file is no longer duplicated into a dozen mailboxes, the storage cost stops multiplying by the size of the audience.

The email becomes a short notification rather than a payload:

from email.message import EmailMessage
import smtplib

def notify(recipients, url, subject):
    msg = EmailMessage()
    msg["From"] = "reporting@example.com"
    msg["To"] = ", ".join(recipients)
    msg["Subject"] = subject
    msg.set_content(
        f"The August regional report is published.\n\n{url}\n\n"
        "This link always points at the current version."
    )
    with smtplib.SMTP("smtp.example.com", 587) as smtp:
        smtp.starttls()
        smtp.send_message(msg)

The attachment path still has its place — an external recipient with no access to your storage, or a regulator who wants a fixed copy — and that mechanics is covered in emailing Excel reports with smtplib. But for internal recurring reports, publish and link.

Name objects so both history and links work

The naming decision comes next, and getting it wrong is expensive to undo once links are circulating. Two requirements pull in opposite directions: history wants a new name every run, links want a name that never changes.

Satisfy both. Write a dated object as the record, then update a stable pointer that links and dashboards reference:

from datetime import date

def report_keys(name, on=None, prefix="reports"):
    """Return the dated key and the stable latest key for a report."""
    on = on or date.today()
    dated = f"{prefix}/{on:%Y/%m}/{name}-{on:%Y-%m}.xlsx"
    latest = f"{prefix}/{name}-latest.xlsx"
    return dated, latest

print(report_keys("regional"))
# ('reports/2026/08/regional-2026-08.xlsx', 'reports/regional-latest.xlsx')

Use ISO dates in the name — 2026-08 sorts correctly in every listing, where Aug-2026 does not. Avoid spaces and non-ASCII characters, which survive most stores but produce awkward URLs and break the occasional downstream tool.

The prefix structure deserves the same care as the filename. Grouping by year and month, as above, keeps a listing usable after a few years of monthly runs, and on object storage it also gives you a natural boundary for lifecycle rules — expire or archive everything under an old year prefix in one policy rather than matching on names. Where several report types share a bucket, put the report name before the date rather than after it, so all of one report's history sits together instead of interleaving with everything else that ran the same month.

Upload atomically

The failure that costs the most credibility is a reader opening a report mid-upload and finding a truncated file. It looks like a corrupt report, and it happens whenever a large file is written directly to its final name over a slow link.

The fix is universal: write somewhere temporary, then move it into place as a single operation.

import boto3

s3 = boto3.client("s3")

def publish_atomic(local_path, bucket, key):
    """Upload to a staging key, then copy into place so readers never see a partial file."""
    staging = f"{key}.uploading"

    with open(local_path, "rb") as fh:
        s3.upload_fileobj(fh, bucket, staging)

    s3.copy_object(
        Bucket=bucket,
        Key=key,
        CopySource={"Bucket": bucket, "Key": staging},
        MetadataDirective="COPY",
    )
    s3.delete_object(Bucket=bucket, Key=staging)
    return f"s3://{bucket}/{key}"

On a filesystem or network share the same idea uses a rename, which is atomic within a volume:

from pathlib import Path
import shutil

def publish_to_share(local_path, share_dir, name):
    dest = Path(share_dir) / name
    staging = dest.with_suffix(dest.suffix + ".part")

    shutil.copyfile(local_path, staging)
    staging.replace(dest)          # atomic within the same filesystem
    return dest

Path.replace is the important call — it overwrites atomically, where copying directly to dest leaves a growing partial file visible for the whole transfer. Note the constraint: the staging file must sit on the same volume as the destination, or the rename degrades into a copy and the guarantee is lost.

Verify the bytes arrived

An HTTP 200 means the request was accepted. It does not mean the file is intact. For anything that matters, compare a checksum:

import hashlib

def file_md5(path, chunk=1 << 20):
    digest = hashlib.md5()
    with open(path, "rb") as fh:
        for block in iter(lambda: fh.read(chunk), b""):
            digest.update(block)
    return digest.hexdigest()

def verify_s3(local_path, bucket, key):
    """Confirm the object in S3 matches the local file."""
    head = s3.head_object(Bucket=bucket, Key=key)
    remote_size = head["ContentLength"]
    local_size = Path(local_path).stat().st_size

    if remote_size != local_size:
        raise RuntimeError(
            f"size mismatch: local {local_size} vs remote {remote_size}"
        )

    etag = head["ETag"].strip('"')
    if "-" not in etag and etag != file_md5(local_path):
        raise RuntimeError("checksum mismatch — the upload is corrupt")
    return True

The "-" not in etag guard matters: for multipart uploads S3's ETag is a digest of digests with a part-count suffix, not the file's MD5, so comparing it directly gives a false failure on large objects. Fall back to size for those, or use the store's own checksum feature.

At minimum, check the size. It is one call and it catches the truncation case, which is by far the most common corruption in practice.

Authenticate as a machine, not a person

Every one of these destinations supports both a personal login and a machine identity, and scheduled jobs should always use the latter. A job authenticating as a person breaks when they change their password, hits interactive multi-factor prompts, and stops entirely when they leave.

Read secrets from the environment or a secrets manager, never from source:

import os

def required(name):
    value = os.environ.get(name)
    if not value:
        raise RuntimeError(f"{name} is not set — the publish step cannot run")
    return value

client_id = required("GRAPH_CLIENT_ID")
client_secret = required("GRAPH_CLIENT_SECRET")

Checking at start-up rather than at the point of use means the job fails immediately with a clear message, instead of thirty minutes later after building a report it cannot deliver — the same fail-fast principle applied throughout error handling and logging in Excel automation.

Retry the transient, not the permanent

Network transfers fail. Most failures are worth retrying and some are guaranteed not to be, and treating them the same way either gives up too early or hammers a service that will never say yes.

import time
import random
from botocore.exceptions import ClientError

RETRYABLE = {"500", "502", "503", "504", "SlowDown", "RequestTimeout"}

def upload_with_retry(local_path, bucket, key, attempts=5):
    for attempt in range(1, attempts + 1):
        try:
            return publish_atomic(local_path, bucket, key)
        except ClientError as exc:
            code = exc.response["Error"]["Code"]
            if code not in RETRYABLE or attempt == attempts:
                raise
            # Exponential backoff with jitter, so parallel jobs do not sync up.
            delay = min(2 ** attempt, 30) + random.uniform(0, 1)
            print(f"attempt {attempt} failed ({code}); retrying in {delay:.1f}s")
            time.sleep(delay)

Never retry an authentication or permission failure — 403 will be 403 again, and a retry loop only delays the alert. The jitter matters more than it looks: without it, several jobs that failed together retry together, which is exactly the pattern that keeps a struggling service down.

Verify the report before you publish it

Publishing is the point of no return. Once a link is circulated, a wrong number has been read, and pulling it back costs far more than the check that would have caught it. So the publish step should be a gate, not just a transfer.

from pathlib import Path
import pandas as pd

def ready_to_publish(path, expected_sheets, min_rows=1, period=None):
    """Refuse to publish a workbook that fails a basic sanity check."""
    path = Path(path)
    problems = []

    if not path.exists() or path.stat().st_size == 0:
        return [f"{path} is missing or empty"]

    try:
        sheets = pd.read_excel(path, sheet_name=None)
    except Exception as exc:
        return [f"{path} does not open: {exc}"]

    missing = set(expected_sheets) - set(sheets)
    if missing:
        problems.append(f"missing sheets: {', '.join(sorted(missing))}")

    for name, frame in sheets.items():
        if len(frame) < min_rows:
            problems.append(f"sheet {name!r} has only {len(frame)} rows")

    if period and "date" in sheets.get(expected_sheets[0], pd.DataFrame()):
        dates = pd.to_datetime(sheets[expected_sheets[0]]["date"],
                               errors="coerce")
        if dates.max() < pd.Timestamp(period):
            problems.append(
                f"newest row is {dates.max():%Y-%m-%d}, before {period}"
            )

    return problems

issues = ready_to_publish("report.xlsx", ["Summary", "Detail"], min_rows=1)
if issues:
    raise RuntimeError("not published:\n  " + "\n  ".join(issues))

The freshness check is the one that earns its keep most often. A pipeline whose upstream extract quietly failed produces a workbook that is structurally perfect and contains last month's numbers — the only signal is that the newest row is older than it should be. The wider set of pre-send checks is in validating an Excel report before sending it.

Leaving the previous version in place on failure is deliberate. A stale report that readers recognise as last month's is far less damaging than a fresh-looking one with wrong numbers, and it buys time to fix the cause.

Choosing a destination

The best destination is usually the one your readers already have open.

That last row is worth expanding. When a report is both read by people and consumed by machines, publish two artefacts from one run: the formatted .xlsx for readers, and a plain columnar file for the pipeline. Making the pipeline parse a styled workbook is a recurring source of breakage, because a formatting change nobody thought was significant moves a header row.

def publish_both(df, bucket, name):
    """One run, two artefacts: a formatted workbook and a machine-readable copy."""
    dated, latest = report_keys(name)

    df.to_excel("report.xlsx", index=False, engine="xlsxwriter")
    df.to_parquet("report.parquet", index=False)

    publish_atomic("report.xlsx", bucket, dated)
    publish_atomic("report.xlsx", bucket, latest)
    publish_atomic("report.parquet", bucket, dated.replace(".xlsx", ".parquet"))
    return dated, latest

Record what was published

The publish step is also where a report's audit trail is created, and it costs almost nothing to write one. A small manifest alongside each run answers the questions that come up weeks later: which run produced this file, what did it contain, and did anything look unusual at the time?

import hashlib
import json
import platform
from datetime import datetime, timezone
from pathlib import Path

def manifest(local_path, destination, rows, period):
    """A small JSON record describing one publication."""
    data = Path(local_path).read_bytes()
    return {
        "published_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
        "destination": destination,
        "bytes": len(data),
        "sha256": hashlib.sha256(data).hexdigest(),
        "rows": rows,
        "period": period,
        "host": platform.node(),
    }

record = manifest("report.xlsx", "s3://acme-reports/reports/regional-latest.xlsx",
                  rows=1_482, period="2026-08")
Path("report.manifest.json").write_text(json.dumps(record, indent=2))

Publish the manifest next to the report. Because it carries a checksum, it also settles the "is this the file you sent me?" question definitively — somebody holding a copy can hash it and compare. And because it records the row count, a sudden change between months is visible in the manifests alone, without opening a single workbook.

That row count is worth a comparison rather than just a record. Run-over-run volume checks catch a whole class of upstream failure that structural validation misses:

import json
from pathlib import Path

def check_against_last(record, history_dir="manifests", tolerance=0.35):
    """Warn when this run's row count departs sharply from the previous one."""
    history = sorted(Path(history_dir).glob("*.json"))
    if not history:
        return None

    previous = json.loads(history[-1].read_text())
    before, now = previous.get("rows", 0), record["rows"]
    if not before:
        return None

    change = (now - before) / before
    if abs(change) > tolerance:
        return (
            f"row count moved {change:+.0%} ({before:,} to {now:,}) — "
            "check the upstream extract before circulating"
        )
    return None

warning = check_against_last(record)
if warning:
    print("WARNING:", warning)

Whether that warning blocks the publish or merely annotates it is a judgement call. For a report where a real business change could plausibly move volumes by half, a hard block would fire constantly and be ignored within a month; for a stable operational feed, blocking is right. Pick deliberately rather than defaulting, and log the decision either way — the logging patterns in logging Python Excel script output to a file apply directly, and a publish step that leaves no trace is the hardest kind of pipeline to debug six months on.

Key takeaways

• Publish and link rather than attach. One canonical copy, a stable URL, and an access trail.
• Write a dated object and a stable latest key. History and links have different needs; satisfy both.
• Upload to a staging name and move it into place. Readers never see a partially written report.
• Verify the transfer. A 200 response is not proof; compare size, and a checksum where you can.
• Authenticate as a machine. Roles, app registrations and service accounts survive people leaving.
• Retry the transient only. Exponential backoff with jitter, and never on a permission error.
• Ship a machine-readable copy alongside the workbook when a pipeline consumes the same data.

Frequently asked questions

Should I email a report or publish it to storage? Publish it and email the link. Attachments duplicate the file into every mailbox, hit size limits, and leave people arguing over which copy is current. A link always resolves to the latest version and leaves an access trail.

How do I stop readers seeing a half-written file? Upload to a temporary key or name, then rename or copy it into place. Most stores make that final step atomic, so a reader either sees the previous version or the complete new one, never a partial upload.

What is the best way to authenticate a scheduled job? A machine identity rather than a person's account — an IAM role for S3, an app registration with client credentials for Microsoft Graph, a service account for Google Drive. Personal credentials break when the person changes their password or leaves.

Should reports overwrite the same name or be dated? Both. Write a dated object for the history and update a stable latest pointer that dashboards and links can rely on. Overwriting alone destroys the audit trail; dating alone means every link goes stale.

How do I know the upload actually worked? Compare a checksum of the local file with the one the store reports, or at minimum re-read the object's size. A successful HTTP status only means the request was accepted, not that the bytes are intact.

Related

• Up to the parent: Automating Reporting Workflows — the pipeline this delivery stage completes.
• Upload an Excel Report to Amazon S3 with boto3 — object storage end to end.
• Upload an Excel Report to SharePoint with Python — Microsoft Graph, tokens and large uploads.
• Save Excel Reports to Google Drive with Python — the Drive API with a service account.
• Write Excel Files to a Network Share from Python — the on-premise option, done safely.
• Emailing Excel Reports with smtplib — the notification that carries the link.
• Scheduling Python Excel Scripts with Cron — running the publish step unattended.