Python Excel AutomationPractical guides for developers
How-to
Automating Reporting WorkflowsDeep dive

Orchestrate Excel Reports with Dagster

Run an Excel reporting pipeline as Dagster software-defined assets: read the source workbook, clean it and write the report as separate assets, pass the output folder in as a resource, partition by month for backfills, validate with asset checks, schedule the job, and test it all in-process with materialize.
Scheduling & orchestration

A monthly Excel report usually starts as one script: read the export, clean it, write the workbook, email it. When it grows to several reports sharing the same cleaned data, needs re-running for a past month, and should stop before sending if the numbers look wrong, cron and a script stop being enough. Dagster approaches the problem through the data each step produces rather than the tasks. You declare assets — the source table, the cleaned table, the report workbook — and the functions that compute them, and Dagster works out the order, records every run, re-runs any month on demand, and shows the lineage in its web UI. This page models an Excel report as Dagster assets, adds monthly partitions, validation checks and a schedule, and tests the whole pipeline in-process. It belongs to Scheduling Python Excel Scripts with Cron.

Prerequisites

Bash
pip install dagster dagster-webserver pandas openpyxl

The source system drops one export per month into inbox/ — sales_2026-08.xlsx and sales_2026-09.xlsx, with the usual problems: a region typed as "north ", and a missing revenue in August. The examples were run with Dagster 1.13; dagster-webserver provides the UI and is only needed for Step 6.

Step 1 — Describe the report as assets

Each asset is a function decorated with @asset. Its parameters name the assets it depends on, so the dependency graph comes from the function signatures:

Python
import dagster as dg
import pandas as pd

monthly = dg.MonthlyPartitionsDefinition(start_date="2026-08-01")

class ReportPaths(dg.ConfigurableResource):
    inbox: str = "inbox"
    outbox: str = "outbox"

@dg.asset(partitions_def=monthly, group_name="sales_report")
def raw_sales(context: dg.AssetExecutionContext, paths: ReportPaths) -> pd.DataFrame:
    """The monthly export as delivered."""
    month = context.partition_key[:7]
    df = pd.read_excel(f"{paths.inbox}/sales_{month}.xlsx")
    context.add_output_metadata({"rows": len(df), "source": f"sales_{month}.xlsx"})
    return df

@dg.asset(partitions_def=monthly, group_name="sales_report")
def clean_sales(raw_sales: pd.DataFrame) -> pd.DataFrame:
    """Trimmed, title-cased regions; missing revenue treated as zero."""
    return raw_sales.assign(Region=raw_sales["Region"].str.strip().str.title(),
                            Revenue=raw_sales["Revenue"].fillna(0.0))

@dg.asset(partitions_def=monthly, group_name="sales_report")
def sales_report_xlsx(context: dg.AssetExecutionContext, clean_sales: pd.DataFrame, paths: ReportPaths) -> str:
    """The workbook sent to finance."""
    from pathlib import Path
    month = context.partition_key[:7]
    Path(paths.outbox).mkdir(exist_ok=True)
    out = f"{paths.outbox}/sales-report_{month}.xlsx"
    summary = clean_sales.groupby("Region", as_index=False)["Revenue"].sum()
    with pd.ExcelWriter(out) as xw:
        summary.to_excel(xw, sheet_name="Summary", index=False)
        clean_sales.to_excel(xw, sheet_name="Detail", index=False)
    context.add_output_metadata({"report_file": out, "total": float(summary["Revenue"].sum())})
    return out

print([a.key.to_user_string() for a in (raw_sales, clean_sales, sales_report_xlsx)])

clean_sales(raw_sales) depends on raw_sales simply by naming it as a parameter, and sales_report_xlsx depends on clean_sales. Each asset's return value is stored between steps by Dagster's I/O manager (pickled to local disk by default), so a failed report step can be retried without re-reading the source. add_output_metadata attaches facts — row counts, the output path, the total — that appear in the UI next to every materialisation, which is often the quickest way to answer "what did last month's run produce?". The paths parameter is a resource: configuration injected at run time, covered in Step 3.

The report as three assets raw_sales reads the month's export, clean_sales trims and fixes it, and sales_report_xlsx writes the workbook. Each asset is partitioned by month and records metadata such as row count and total. raw_sales read inbox/sales_YYYY-MM.xlsx clean_sales trim, title-case, fill sales_report_xlsx Summary + Detail sheets

Step 2 — Validate before anyone sees the report

An asset check runs after an asset is materialised and records a pass or fail with details. Checks are the place for the rules a person would otherwise eyeball — no blank regions, revenue not negative, every expected region present:

Python
EXPECTED = {"North", "South", "East"}

@dg.asset_check(asset=clean_sales, blocking=True, partitions_def=monthly)
def regions_complete(clean_sales: pd.DataFrame) -> dg.AssetCheckResult:
    found = set(clean_sales["Region"])
    missing = sorted(EXPECTED - found)
    return dg.AssetCheckResult(passed=not missing, metadata={"missing": ", ".join(missing) or "none"})

@dg.asset_check(asset=clean_sales, blocking=True, partitions_def=monthly)
def no_negative_revenue(clean_sales: pd.DataFrame) -> dg.AssetCheckResult:
    bad = int((clean_sales["Revenue"] < 0).sum())
    return dg.AssetCheckResult(passed=bad == 0, metadata={"negative_rows": bad})

print([c.check_key.name for c in (regions_complete, no_negative_revenue)])

partitions_def=monthly makes each check run per month, against the same partition as the asset it checks — without it, a check loads every partition of the asset at once, and fails as soon as one month has not been materialised yet. Partitioned checks were marked as a preview feature in the Dagster version used here, so expect the API to settle in later releases. blocking=True means that if the check fails, assets downstream of clean_sales — the report workbook — are not materialised in that run. A broken export therefore stops before a wrong report reaches finance, and the failure, with its metadata, is visible in the UI and in alerts. Non-blocking checks still record the problem but let the run continue, which suits warnings such as an unusually low total. The same rules as plain code are discussed in Validate an Excel Report Before Sending It.

Blocking and non-blocking checks A blocking check that fails stops downstream assets, so the report workbook is not written in that run. A non-blocking check records the failure and its metadata but lets the run continue, which suits warnings. blocking=True failure stops downstream assets no report written that run for rules that must hold blocking=False failure recorded with metadata run continues for warnings, e.g. low totals

Step 3 — Definitions, a job and a schedule

Definitions gathers everything Dagster should load: assets, checks, resources, jobs and schedules. A job selects the assets to run, and a schedule built from a partitioned job runs the latest complete month:

Python
report_job = dg.define_asset_job("monthly_sales_report", selection=dg.AssetSelection.groups("sales_report"))
report_schedule = dg.build_schedule_from_partitioned_job(report_job, hour_of_day=6, minute_of_hour=30)

defs = dg.Definitions(
    assets=[raw_sales, clean_sales, sales_report_xlsx],
    asset_checks=[regions_complete, no_negative_revenue],
    jobs=[report_job],
    schedules=[report_schedule],
    resources={"paths": ReportPaths(inbox="inbox", outbox="outbox")},
)
schedule = defs.get_schedule_def(report_schedule.name)
print(schedule.name, "|", schedule.cron_schedule, "|", schedule.execution_timezone or "UTC")

The job's partitions come from the assets it selects, and Dagster resolves the schedule when the definitions are loaded — which is why the cron expression is read from defs rather than from the schedule object. For a monthly partition, build_schedule_from_partitioned_job produces a schedule that runs at 06:30 on the first day of each month and materialises the month that has just ended — on 1 October it runs the September partition. Set execution_timezone on the schedule if 06:30 should mean local time rather than UTC; see Schedule Reports Correctly Across Time Zones and DST. The resource is configured once here; in production its values can come from environment variables with dg.EnvVar("REPORT_OUTBOX"), so the same code runs against test and live folders.

Step 4 — Test the pipeline in-process

dg.materialize() runs assets in the current Python process without a Dagster instance or UI, which makes it ideal for tests. Materialise the September partition and inspect the results:

Python
result = dg.materialize(
    [raw_sales, clean_sales, sales_report_xlsx, regions_complete, no_negative_revenue],
    partition_key="2026-09-01",
    resources={"paths": ReportPaths()},
)
print("success:", result.success)
print("checks:", {e.check_name: e.passed for e in result.get_asset_check_evaluations()})
summary = pd.read_excel(result.output_for_node("sales_report_xlsx"), sheet_name="Summary")
print(summary.to_dict("records"))

The run succeeds, both checks pass, and the Summary sheet shows North with 495 — the stray "north " row was trimmed and merged into North by clean_sales. output_for_node returns an asset's value from the run, here the path of the written workbook. A pytest test can make the same calls against a small fixture workbook and assert on the summary; see Test Excel Output with pytest.

Step 5 — Re-run a past month

Partitions make backfills a first-class operation. Re-running August is the same call with a different partition key — in the UI it is a Backfill over a range of months:

Python
august = dg.materialize(
    [raw_sales, clean_sales, sales_report_xlsx, regions_complete, no_negative_revenue],
    partition_key="2026-08-01",
    resources={"paths": ReportPaths()},
)
meta = august.asset_materializations_for_node("sales_report_xlsx")[0].metadata
print(meta["report_file"].value, "| total:", meta["total"].value)
print(sorted(p.name for p in __import__("pathlib").Path("outbox").glob("*.xlsx")))

August's report is written to its own file with a total of 790 — East's missing revenue counted as zero by the cleaning rule. Each month is an independent partition, so re-running August never touches September's report. A blank revenue being treated as zero is exactly the kind of decision worth recording in the asset's docstring, which Dagster shows in the UI. Backfilling without an orchestrator is covered in Backfill Excel Reports for Past Dates.

Each month is its own partition A grid of assets against monthly partitions. For August and September, raw_sales, clean_sales and the report workbook are all materialised, and both checks passed. Re-running one month leaves the other untouched. 2026-08 2026-09 raw_sales ✓ 3 rows ✓ 4 rows clean_sales ✓ checks pass ✓ checks pass sales_report_xlsx ✓ total 790 ✓ total 1,158

Step 6 — Run it for real

Put the definitions in a module such as reports/definitions.py, then start the local UI and daemon:

Bash
dagster dev -m reports.definitions

The UI shows the asset graph, every materialisation with its metadata and check results, and lets you launch runs and backfills by month. The schedule only fires while the Dagster daemon is running — dagster dev starts one for local use; in production run the webserver and daemon as services, or use Dagster's hosted offering. Compared with Orchestrate Excel Reports with Prefect, which models tasks and flows, Dagster's asset model fits best when several reports share intermediate tables and you care about which version of each table a report was built from.

Common pitfalls

  • Returning huge DataFrames between assets. The default I/O manager pickles them to disk; for big data, write files and pass paths.
  • Non-blocking checks on critical rules. Use blocking=True so a failed check stops the report.
  • Hard-coded folders. Use a resource, configured per environment.
  • Expecting schedules without the daemon. Schedules run only while the daemon is up.
  • Forgetting the partition key in file names. Each month should write its own file.

Conclusion

In Dagster, an Excel report becomes a chain of assets — source, cleaned data, workbook — whose dependencies come from function parameters. Monthly partitions give each period its own runs and make backfills trivial, blocking asset checks stop a bad report before it is written, a resource carries the folder configuration, and a schedule built from the partitioned job runs the latest month automatically. materialize() runs the whole pipeline in-process, so it can be tested like any other code.

Frequently asked questions

How do I run an Excel report with Dagster? Define assets with the asset decorator for each step — reading the workbook, cleaning, writing the report — collect them in Definitions with a job and schedule, and run dagster dev to start the UI and daemon.

How do I re-run a Dagster report for a previous month? Partition the assets with MonthlyPartitionsDefinition and materialise the earlier partition, either with a backfill in the UI or with materialize and partition_key in code.

How do I stop a Dagster pipeline when the data is wrong? Add asset checks with blocking set to True. If such a check fails, downstream assets are not materialised in that run.

How do I test Dagster assets? Call dagster's materialize function with the assets, a partition key and test resources. It runs in-process and returns a result with success, check evaluations and asset outputs.

More on scheduling & orchestration

More on scheduling & orchestration →