[{"data":1,"prerenderedAt":2310},["ShallowReactive",2],{"doc:\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-duplicate-column-names-when-reading-excel":3,"surround:\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-duplicate-column-names-when-reading-excel":2274,"ancestors:\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-duplicate-column-names-when-reading-excel":2283,"children:\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-duplicate-column-names-when-reading-excel":2288,"related:\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-duplicate-column-names-when-reading-excel":2289},{"id":4,"title":5,"body":6,"dateModified":2239,"datePublished":2239,"description":2240,"extension":2241,"faq":2242,"keyword":2252,"kind":2253,"meta":2254,"navigation":311,"path":2263,"seo":2264,"slug":2267,"stem":2268,"topics":2269,"type":2272,"__hash__":2273},"docs\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-duplicate-column-names-when-reading-excel\u002Findex.md","Fix Duplicate Column Names When Reading Excel with pandas",{"type":7,"value":8,"toc":2227},"minimark",[9,13,24,41,46,72,83,276,280,283,365,390,398,584,605,609,612,896,913,917,920,1222,1237,1275,1287,1430,1434,1440,1787,1800,1941,1948,1952,1955,2075,2079,2134,2138,2152,2156,2162,2172,2178,2187,2191,2223],[10,11,12],"p",{},"A sheet has two columns headed \"Sales\", and pandas quietly renames the second one:",[14,15,21],"pre",{"className":16,"code":18,"language":19,"meta":20},[17],"language-text","['Region', 'Sales', 'Sales.1', 'Sales ', 'sales', 'Unnamed: 5', 'Region.1']\n","text","",[22,23,18],"code",{"__ignoreMap":20},[10,25,26,27,30,31,34,35,40],{},"Nothing fails yet. The trouble comes later: code that selects ",[22,28,29],{},"df[\"Sales\"]"," gets only the first column, a lookup table built from the sheet loses rows, or a DataFrame assembled with duplicate names makes assignments fail with ",[22,32,33],{},"ValueError: Cannot set a DataFrame with multiple columns to the single column Total",". This page shows how duplicates arise when reading Excel, how to find all of them including near-duplicates, and how to give every column a name that says what it is. It is part of ",[36,37,39],"a",{"href":38},"\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002F","Troubleshooting Common Python Excel Errors",".",[42,43,45],"h2",{"id":44},"prerequisites","Prerequisites",[14,47,51],{"className":48,"code":49,"language":50,"meta":20,"style":20},"language-bash shiki shiki-themes github-light github-dark-high-contrast","pip install pandas openpyxl\n","bash",[22,52,53],{"__ignoreMap":20},[54,55,58,62,66,69],"span",{"class":56,"line":57},"line",1,[54,59,61],{"class":60},"sMTad","pip",[54,63,65],{"class":64},"srMev"," install",[54,67,68],{"class":64}," pandas",[54,70,71],{"class":64}," openpyxl\n",[10,73,74,75,78,79,82],{},"Two files. ",[22,76,77],{},"sales_by_year.xlsx"," has a two-row header: years in merged cells on the first row, and \"Sales\" and \"Units\" repeated under each year on the second, plus a region code column also headed \"Region\". ",[22,80,81],{},"typed_by_hand.xlsx"," is the kind of sheet people maintain by hand: \"Sales\", \"Sales \" with a trailing space and \"sales\" in lower case, and a column with no header at all.",[84,85,93,94,93,98,93,102,93,110,93,119,93,123,93,129,93,133,93,137,93,141,93,145,93,149,93,153,93,156,93,160,93,163,93,167,93,171,93,175,93,177,93,180,93,185,93,188,93,192,93,194,93,197,93,201,93,206,93,210,93,214,93,218,93,222,93,226,93,228,93,230,93,232,93,234,93,236,93,238,93,241,93,245,93,247,93,251,93,253,93,256,93,258,93,261,93,263,93,266,93,268,93,271,93,273],"svg",{"viewBox":86,"role":87,"ariaLabelledBy":88,"xmlns":91,"style":92},"172 0 416 140","img",[89,90],"dc-header-t","dc-header-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;max-width:416px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif","\n  ",[95,96,97],"title",{"id":89},"A two-row header that produces duplicates",[99,100,101],"desc",{"id":90},"The top of the sales sheet. The first row holds the years 2025 and 2026 in merged cells; the second row repeats Sales and Units under each year and has Region twice. Read with only the second row as the header, Sales, Units and Region each appear twice.",[103,104],"rect",{"x":105,"y":106,"width":107,"height":108,"fill":109},"172","0","416","140","#ffffff",[103,111],{"x":112,"y":113,"width":114,"height":115,"rx":106,"fill":116,"stroke":117,"style":118},"188.1","16.0","32.0","24.0","#e7ebef","var(--line,#cdd5e6)","stroke-width:1px",[103,120],{"x":121,"y":113,"width":122,"height":115,"rx":106,"fill":116,"stroke":117,"style":118},"220.1","63.2",[19,124,128],{"x":125,"y":126,"style":127},"251.7","33.0","font-size:11px;font-weight:600;fill:var(--muted,#5b6780);text-anchor:middle","A",[103,130],{"x":131,"y":113,"width":132,"height":115,"rx":106,"fill":116,"stroke":117,"style":118},"283.3","59.0",[19,134,136],{"x":135,"y":126,"style":127},"312.8","B",[103,138],{"x":139,"y":113,"width":140,"height":115,"rx":106,"fill":116,"stroke":117,"style":118},"342.3","53.5",[19,142,144],{"x":143,"y":126,"style":127},"369.0","C",[103,146],{"x":147,"y":113,"width":148,"height":115,"rx":106,"fill":116,"stroke":117,"style":118},"395.8","59.5",[19,150,152],{"x":151,"y":126,"style":127},"425.5","D",[103,154],{"x":155,"y":113,"width":140,"height":115,"rx":106,"fill":116,"stroke":117,"style":118},"455.3",[19,157,159],{"x":158,"y":126,"style":127},"482.0","E",[103,161],{"x":162,"y":113,"width":122,"height":115,"rx":106,"fill":116,"stroke":117,"style":118},"508.7",[19,164,166],{"x":165,"y":126,"style":127},"540.3","F",[103,168],{"x":112,"y":169,"width":114,"height":170,"rx":106,"fill":116,"stroke":117,"style":118},"40.0","28.0",[19,172,174],{"x":173,"y":132,"style":127},"204.1","1",[103,176],{"x":121,"y":169,"width":122,"height":170,"rx":106,"fill":109,"stroke":117,"style":118},[103,178],{"x":131,"y":169,"width":179,"height":170,"rx":106,"fill":109,"stroke":117,"style":118},"112.5",[19,181,184],{"x":182,"y":132,"style":183},"339.5","font-size:11px;font-weight:700;fill:var(--text,#172033);text-anchor:middle","2025",[103,186],{"x":147,"y":169,"width":187,"height":170,"rx":106,"fill":109,"stroke":117,"style":118},"113.0",[19,189,191],{"x":190,"y":132,"style":183},"452.2","2026",[103,193],{"x":162,"y":169,"width":122,"height":170,"rx":106,"fill":109,"stroke":117,"style":118},[103,195],{"x":112,"y":196,"width":114,"height":170,"rx":106,"fill":116,"stroke":117,"style":118},"68.0",[19,198,200],{"x":173,"y":199,"style":127},"87.0","2",[103,202],{"x":121,"y":196,"width":122,"height":170,"rx":106,"fill":203,"stroke":204,"style":205},"#fce9e9","var(--danger,#dc2626)","stroke-width:1.5px",[19,207,209],{"x":125,"y":199,"style":208},"font-size:11px;font-weight:700;fill:var(--accent-ink,#be185d);text-anchor:middle","Region",[103,211],{"x":131,"y":196,"width":132,"height":170,"rx":106,"fill":212,"stroke":213,"style":205},"#fdefd8","var(--gold,#b4740a)",[19,215,217],{"x":135,"y":199,"style":216},"font-size:11px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","Sales",[103,219],{"x":139,"y":196,"width":140,"height":170,"rx":106,"fill":220,"stroke":221,"style":205},"#d9f4f1","var(--teal,#0f9488)",[19,223,225],{"x":143,"y":199,"style":224},"font-size:11px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","Units",[103,227],{"x":147,"y":196,"width":148,"height":170,"rx":106,"fill":212,"stroke":213,"style":205},[19,229,217],{"x":151,"y":199,"style":216},[103,231],{"x":155,"y":196,"width":140,"height":170,"rx":106,"fill":220,"stroke":221,"style":205},[19,233,225],{"x":158,"y":199,"style":224},[103,235],{"x":162,"y":196,"width":122,"height":170,"rx":106,"fill":203,"stroke":204,"style":205},[19,237,209],{"x":165,"y":199,"style":208},[103,239],{"x":112,"y":240,"width":114,"height":170,"rx":106,"fill":116,"stroke":117,"style":118},"96.0",[19,242,244],{"x":173,"y":243,"style":127},"115.0","3",[103,246],{"x":121,"y":240,"width":122,"height":170,"rx":106,"fill":109,"stroke":117,"style":118},[19,248,250],{"x":125,"y":243,"style":249},"font-size:11px;font-weight:400;fill:var(--text,#172033);text-anchor:middle","North",[103,252],{"x":131,"y":240,"width":132,"height":170,"rx":106,"fill":109,"stroke":117,"style":118},[19,254,255],{"x":135,"y":243,"style":249},"12500",[103,257],{"x":139,"y":240,"width":140,"height":170,"rx":106,"fill":109,"stroke":117,"style":118},[19,259,260],{"x":143,"y":243,"style":249},"410",[103,262],{"x":147,"y":240,"width":148,"height":170,"rx":106,"fill":109,"stroke":117,"style":118},[19,264,265],{"x":151,"y":243,"style":249},"13900",[103,267],{"x":155,"y":240,"width":140,"height":170,"rx":106,"fill":109,"stroke":117,"style":118},[19,269,270],{"x":158,"y":243,"style":249},"452",[103,272],{"x":162,"y":240,"width":122,"height":170,"rx":106,"fill":109,"stroke":117,"style":118},[19,274,275],{"x":165,"y":243,"style":249},"N",[42,277,279],{"id":278},"step-1-see-what-pandas-did-with-the-headers","Step 1 — See what pandas did with the headers",[10,281,282],{},"Read the sheet with the second row as the header and look at the column names:",[14,284,288],{"className":285,"code":286,"language":287,"meta":20,"style":20},"language-python shiki shiki-themes github-light github-dark-high-contrast","import pandas as pd\n\ndf = pd.read_excel(\"sales_by_year.xlsx\", header=1)\nprint(df.columns.tolist())\nprint(df.head(2).to_string())\n","python",[22,289,290,306,313,343,352],{"__ignoreMap":20},[54,291,292,296,300,303],{"class":56,"line":57},[54,293,295],{"class":294},"s-kum","import",[54,297,299],{"class":298},"skGVy"," pandas ",[54,301,302],{"class":294},"as",[54,304,305],{"class":298}," pd\n",[54,307,309],{"class":56,"line":308},2,[54,310,312],{"emptyLinePlaceholder":311},true,"\n",[54,314,316,319,322,325,328,331,335,337,340],{"class":56,"line":315},3,[54,317,318],{"class":298},"df ",[54,320,321],{"class":294},"=",[54,323,324],{"class":298}," pd.read_excel(",[54,326,327],{"class":64},"\"sales_by_year.xlsx\"",[54,329,330],{"class":298},", ",[54,332,334],{"class":333},"sa561","header",[54,336,321],{"class":294},[54,338,174],{"class":339},"sP0c6",[54,341,342],{"class":298},")\n",[54,344,346,349],{"class":56,"line":345},4,[54,347,348],{"class":339},"print",[54,350,351],{"class":298},"(df.columns.tolist())\n",[54,353,355,357,360,362],{"class":56,"line":354},5,[54,356,348],{"class":339},[54,358,359],{"class":298},"(df.head(",[54,361,200],{"class":339},[54,363,364],{"class":298},").to_string())\n",[10,366,367,368,371,372,330,375,378,379,381,382,385,386,389],{},"pandas never returns duplicate names from ",[22,369,370],{},"read_excel","; it appends ",[22,373,374],{},".1",[22,376,377],{},".2"," to repeats. So ",[22,380,217],{}," is 2025 and ",[22,383,384],{},"Sales.1"," is 2026, which nobody reading the code will guess, and ",[22,387,388],{},"Region.1"," is actually a region code. The names are unique but meaningless.",[10,391,392,393,397],{},"The dangerous version is when duplicate names are created ",[394,395,396],"em",{},"after"," reading, for example by assigning a header row yourself, which does not deduplicate:",[14,399,401],{"className":285,"code":400,"language":287,"meta":20,"style":20},"raw = pd.read_excel(\"sales_by_year.xlsx\", header=None)\nmanual = raw.iloc[2:].copy()\nmanual.columns = raw.iloc[1].tolist()\nprint(manual.columns.tolist())\nprint(\"df['Sales'] is a\", type(manual[\"Sales\"]).__name__)\ntry:\n    manual[\"Total\"] = manual[\"Sales\"] * 2\nexcept ValueError as e:\n    print(\"ValueError:\", e)\nrecords = manual.to_dict(\"records\")\nprint(records[0])\n",[22,402,403,425,440,454,461,490,499,526,541,555,571],{"__ignoreMap":20},[54,404,405,408,410,412,414,416,418,420,423],{"class":56,"line":57},[54,406,407],{"class":298},"raw ",[54,409,321],{"class":294},[54,411,324],{"class":298},[54,413,327],{"class":64},[54,415,330],{"class":298},[54,417,334],{"class":333},[54,419,321],{"class":294},[54,421,422],{"class":339},"None",[54,424,342],{"class":298},[54,426,427,430,432,435,437],{"class":56,"line":308},[54,428,429],{"class":298},"manual ",[54,431,321],{"class":294},[54,433,434],{"class":298}," raw.iloc[",[54,436,200],{"class":339},[54,438,439],{"class":298},":].copy()\n",[54,441,442,445,447,449,451],{"class":56,"line":315},[54,443,444],{"class":298},"manual.columns ",[54,446,321],{"class":294},[54,448,434],{"class":298},[54,450,174],{"class":339},[54,452,453],{"class":298},"].tolist()\n",[54,455,456,458],{"class":56,"line":345},[54,457,348],{"class":339},[54,459,460],{"class":298},"(manual.columns.tolist())\n",[54,462,463,465,468,471,473,476,479,482,485,488],{"class":56,"line":354},[54,464,348],{"class":339},[54,466,467],{"class":298},"(",[54,469,470],{"class":64},"\"df['Sales'] is a\"",[54,472,330],{"class":298},[54,474,475],{"class":339},"type",[54,477,478],{"class":298},"(manual[",[54,480,481],{"class":64},"\"Sales\"",[54,483,484],{"class":298},"]).",[54,486,487],{"class":339},"__name__",[54,489,342],{"class":298},[54,491,493,496],{"class":56,"line":492},6,[54,494,495],{"class":294},"try",[54,497,498],{"class":298},":\n",[54,500,502,505,508,511,513,516,518,520,523],{"class":56,"line":501},7,[54,503,504],{"class":298},"    manual[",[54,506,507],{"class":64},"\"Total\"",[54,509,510],{"class":298},"] ",[54,512,321],{"class":294},[54,514,515],{"class":298}," manual[",[54,517,481],{"class":64},[54,519,510],{"class":298},[54,521,522],{"class":294},"*",[54,524,525],{"class":339}," 2\n",[54,527,529,532,535,538],{"class":56,"line":528},8,[54,530,531],{"class":294},"except",[54,533,534],{"class":339}," ValueError",[54,536,537],{"class":294}," as",[54,539,540],{"class":298}," e:\n",[54,542,544,547,549,552],{"class":56,"line":543},9,[54,545,546],{"class":339},"    print",[54,548,467],{"class":298},[54,550,551],{"class":64},"\"ValueError:\"",[54,553,554],{"class":298},", e)\n",[54,556,558,561,563,566,569],{"class":56,"line":557},10,[54,559,560],{"class":298},"records ",[54,562,321],{"class":294},[54,564,565],{"class":298}," manual.to_dict(",[54,567,568],{"class":64},"\"records\"",[54,570,342],{"class":298},[54,572,574,576,579,581],{"class":56,"line":573},11,[54,575,348],{"class":339},[54,577,578],{"class":298},"(records[",[54,580,106],{"class":339},[54,582,583],{"class":298},"])\n",[10,585,586,587,590,591,594,595,598,599,601,602,604],{},"Selecting a duplicated name returns a DataFrame instead of a Series, so arithmetic and assignment fail. Worse, ",[22,588,589],{},"to_dict(\"records\")"," only warns that \"columns are not unique\" and keeps the ",[394,592,593],{},"last"," value per name: the first record says ",[22,596,597],{},"Region: 'N'",", the code, having silently dropped ",[22,600,250],{},", and its ",[22,603,217],{}," is the 2026 figure with 2025 gone.",[42,606,608],{"id":607},"step-2-find-duplicates-and-near-duplicates","Step 2 — Find duplicates and near-duplicates",[10,610,611],{},"Exact duplicates are easy to spot; the ones that cause most confusion differ only in case or whitespace, so pandas treats them as different columns. Normalise the raw headers and group them:",[14,613,615],{"className":285,"code":614,"language":287,"meta":20,"style":20},"from openpyxl import load_workbook\n\ndef header_report(path, row=1):\n    ws = load_workbook(path, read_only=True).active\n    headers = next(ws.iter_rows(min_row=row, max_row=row, values_only=True))\n    report = pd.DataFrame({\"position\": range(len(headers)), \"raw\": headers})\n    report[\"key\"] = [\" \".join(str(h).split()).casefold() if h is not None and str(h).strip() else \"\u003Cblank>\"\n                     for h in headers]\n    return report[report.duplicated(\"key\", keep=False) | (report[\"key\"] == \"\u003Cblank>\")]\n\nprint(header_report(\"typed_by_hand.xlsx\").to_string(index=False))\nprint(pd.read_excel(\"typed_by_hand.xlsx\").columns.tolist())\n",[22,616,617,630,634,653,674,712,745,802,815,857,861,883],{"__ignoreMap":20},[54,618,619,622,625,627],{"class":56,"line":57},[54,620,621],{"class":294},"from",[54,623,624],{"class":298}," openpyxl ",[54,626,295],{"class":294},[54,628,629],{"class":298}," load_workbook\n",[54,631,632],{"class":56,"line":308},[54,633,312],{"emptyLinePlaceholder":311},[54,635,636,639,643,646,648,650],{"class":56,"line":315},[54,637,638],{"class":294},"def",[54,640,642],{"class":641},"s_Opv"," header_report",[54,644,645],{"class":298},"(path, row",[54,647,321],{"class":294},[54,649,174],{"class":339},[54,651,652],{"class":298},"):\n",[54,654,655,658,660,663,666,668,671],{"class":56,"line":345},[54,656,657],{"class":298},"    ws ",[54,659,321],{"class":294},[54,661,662],{"class":298}," load_workbook(path, ",[54,664,665],{"class":333},"read_only",[54,667,321],{"class":294},[54,669,670],{"class":339},"True",[54,672,673],{"class":298},").active\n",[54,675,676,679,681,684,687,690,692,695,698,700,702,705,707,709],{"class":56,"line":354},[54,677,678],{"class":298},"    headers ",[54,680,321],{"class":294},[54,682,683],{"class":339}," next",[54,685,686],{"class":298},"(ws.iter_rows(",[54,688,689],{"class":333},"min_row",[54,691,321],{"class":294},[54,693,694],{"class":298},"row, ",[54,696,697],{"class":333},"max_row",[54,699,321],{"class":294},[54,701,694],{"class":298},[54,703,704],{"class":333},"values_only",[54,706,321],{"class":294},[54,708,670],{"class":339},[54,710,711],{"class":298},"))\n",[54,713,714,717,719,722,725,728,731,733,736,739,742],{"class":56,"line":492},[54,715,716],{"class":298},"    report ",[54,718,321],{"class":294},[54,720,721],{"class":298}," pd.DataFrame({",[54,723,724],{"class":64},"\"position\"",[54,726,727],{"class":298},": ",[54,729,730],{"class":339},"range",[54,732,467],{"class":298},[54,734,735],{"class":339},"len",[54,737,738],{"class":298},"(headers)), ",[54,740,741],{"class":64},"\"raw\"",[54,743,744],{"class":298},": headers})\n",[54,746,747,750,753,755,757,760,763,766,769,772,775,778,781,784,787,790,793,796,799],{"class":56,"line":501},[54,748,749],{"class":298},"    report[",[54,751,752],{"class":64},"\"key\"",[54,754,510],{"class":298},[54,756,321],{"class":294},[54,758,759],{"class":298}," [",[54,761,762],{"class":64},"\" \"",[54,764,765],{"class":298},".join(",[54,767,768],{"class":339},"str",[54,770,771],{"class":298},"(h).split()).casefold() ",[54,773,774],{"class":294},"if",[54,776,777],{"class":298}," h ",[54,779,780],{"class":294},"is",[54,782,783],{"class":294}," not",[54,785,786],{"class":339}," None",[54,788,789],{"class":294}," and",[54,791,792],{"class":339}," str",[54,794,795],{"class":298},"(h).strip() ",[54,797,798],{"class":294},"else",[54,800,801],{"class":64}," \"\u003Cblank>\"\n",[54,803,804,807,809,812],{"class":56,"line":528},[54,805,806],{"class":294},"                     for",[54,808,777],{"class":298},[54,810,811],{"class":294},"in",[54,813,814],{"class":298}," headers]\n",[54,816,817,820,823,825,827,830,832,835,838,841,844,846,848,851,854],{"class":56,"line":543},[54,818,819],{"class":294},"    return",[54,821,822],{"class":298}," report[report.duplicated(",[54,824,752],{"class":64},[54,826,330],{"class":298},[54,828,829],{"class":333},"keep",[54,831,321],{"class":294},[54,833,834],{"class":339},"False",[54,836,837],{"class":298},") ",[54,839,840],{"class":294},"|",[54,842,843],{"class":298}," (report[",[54,845,752],{"class":64},[54,847,510],{"class":298},[54,849,850],{"class":294},"==",[54,852,853],{"class":64}," \"\u003Cblank>\"",[54,855,856],{"class":298},")]\n",[54,858,859],{"class":56,"line":557},[54,860,312],{"emptyLinePlaceholder":311},[54,862,863,865,868,871,874,877,879,881],{"class":56,"line":573},[54,864,348],{"class":339},[54,866,867],{"class":298},"(header_report(",[54,869,870],{"class":64},"\"typed_by_hand.xlsx\"",[54,872,873],{"class":298},").to_string(",[54,875,876],{"class":333},"index",[54,878,321],{"class":294},[54,880,834],{"class":339},[54,882,711],{"class":298},[54,884,886,888,891,893],{"class":56,"line":885},12,[54,887,348],{"class":339},[54,889,890],{"class":298},"(pd.read_excel(",[54,892,870],{"class":64},[54,894,895],{"class":298},").columns.tolist())\n",[10,897,898,899,330,901,904,905,908,909,912],{},"pandas read that sheet without complaint and produced ",[22,900,217],{},[22,902,903],{},"Sales "," and ",[22,906,907],{},"sales"," as three separate columns, plus ",[22,910,911],{},"Unnamed: 4"," for the blank header. The report shows they collide once whitespace and case are ignored, and that position 4 has no header at all.",[42,914,916],{"id":915},"step-3-build-names-from-a-two-row-header","Step 3 — Build names from a two-row header",[10,918,919],{},"When the duplicates are real, different columns that share a label under a group heading, the group heading is the missing part of the name. The years sit in merged cells, and a merged range stores its value only in its first cell, so take the merged ranges from openpyxl and spread each year across the columns it covers. Columns outside a merged range get no year:",[14,921,923],{"className":285,"code":922,"language":287,"meta":20,"style":20},"def two_row_names(path):\n    ws = load_workbook(path).active\n    top = [c.value for c in ws[1]]\n    for rng in ws.merged_cells.ranges:\n        if rng.min_row == 1:\n            for col in range(rng.min_col, rng.max_col + 1):\n                top[col - 1] = ws.cell(row=1, column=rng.min_col).value\n    bottom = [c.value for c in ws[2]]\n    return [f\"{b} {t}\" if t not in (None, \"\") else b for t, b in zip(top, bottom)]\n\nnames = two_row_names(\"sales_by_year.xlsx\")\ntwo = pd.read_excel(\"sales_by_year.xlsx\", header=None, skiprows=2, names=names)\nprint(names)\nprint(two.columns.tolist())\n",[22,924,925,935,944,970,983,998,1021,1055,1076,1150,1154,1168,1206,1214],{"__ignoreMap":20},[54,926,927,929,932],{"class":56,"line":57},[54,928,638],{"class":294},[54,930,931],{"class":641}," two_row_names",[54,933,934],{"class":298},"(path):\n",[54,936,937,939,941],{"class":56,"line":308},[54,938,657],{"class":298},[54,940,321],{"class":294},[54,942,943],{"class":298}," load_workbook(path).active\n",[54,945,946,949,951,954,957,960,962,965,967],{"class":56,"line":315},[54,947,948],{"class":298},"    top ",[54,950,321],{"class":294},[54,952,953],{"class":298}," [c.value ",[54,955,956],{"class":294},"for",[54,958,959],{"class":298}," c ",[54,961,811],{"class":294},[54,963,964],{"class":298}," ws[",[54,966,174],{"class":339},[54,968,969],{"class":298},"]]\n",[54,971,972,975,978,980],{"class":56,"line":345},[54,973,974],{"class":294},"    for",[54,976,977],{"class":298}," rng ",[54,979,811],{"class":294},[54,981,982],{"class":298}," ws.merged_cells.ranges:\n",[54,984,985,988,991,993,996],{"class":56,"line":354},[54,986,987],{"class":294},"        if",[54,989,990],{"class":298}," rng.min_row ",[54,992,850],{"class":294},[54,994,995],{"class":339}," 1",[54,997,498],{"class":298},[54,999,1000,1003,1006,1008,1011,1014,1017,1019],{"class":56,"line":492},[54,1001,1002],{"class":294},"            for",[54,1004,1005],{"class":298}," col ",[54,1007,811],{"class":294},[54,1009,1010],{"class":339}," range",[54,1012,1013],{"class":298},"(rng.min_col, rng.max_col ",[54,1015,1016],{"class":294},"+",[54,1018,995],{"class":339},[54,1020,652],{"class":298},[54,1022,1023,1026,1029,1031,1033,1035,1038,1041,1043,1045,1047,1050,1052],{"class":56,"line":501},[54,1024,1025],{"class":298},"                top[col ",[54,1027,1028],{"class":294},"-",[54,1030,995],{"class":339},[54,1032,510],{"class":298},[54,1034,321],{"class":294},[54,1036,1037],{"class":298}," ws.cell(",[54,1039,1040],{"class":333},"row",[54,1042,321],{"class":294},[54,1044,174],{"class":339},[54,1046,330],{"class":298},[54,1048,1049],{"class":333},"column",[54,1051,321],{"class":294},[54,1053,1054],{"class":298},"rng.min_col).value\n",[54,1056,1057,1060,1062,1064,1066,1068,1070,1072,1074],{"class":56,"line":528},[54,1058,1059],{"class":298},"    bottom ",[54,1061,321],{"class":294},[54,1063,953],{"class":298},[54,1065,956],{"class":294},[54,1067,959],{"class":298},[54,1069,811],{"class":294},[54,1071,964],{"class":298},[54,1073,200],{"class":339},[54,1075,969],{"class":298},[54,1077,1078,1080,1082,1085,1088,1092,1095,1098,1101,1104,1106,1108,1111,1114,1117,1120,1123,1125,1127,1130,1132,1134,1137,1139,1142,1144,1147],{"class":56,"line":543},[54,1079,819],{"class":294},[54,1081,759],{"class":298},[54,1083,1084],{"class":294},"f",[54,1086,1087],{"class":64},"\"",[54,1089,1091],{"class":1090},"sSjpA","{",[54,1093,1094],{"class":298},"b",[54,1096,1097],{"class":1090},"}",[54,1099,1100],{"class":1090}," {",[54,1102,1103],{"class":298},"t",[54,1105,1097],{"class":1090},[54,1107,1087],{"class":64},[54,1109,1110],{"class":294}," if",[54,1112,1113],{"class":298}," t ",[54,1115,1116],{"class":294},"not",[54,1118,1119],{"class":294}," in",[54,1121,1122],{"class":298}," (",[54,1124,422],{"class":339},[54,1126,330],{"class":298},[54,1128,1129],{"class":64},"\"\"",[54,1131,837],{"class":298},[54,1133,798],{"class":294},[54,1135,1136],{"class":298}," b ",[54,1138,956],{"class":294},[54,1140,1141],{"class":298}," t, b ",[54,1143,811],{"class":294},[54,1145,1146],{"class":339}," zip",[54,1148,1149],{"class":298},"(top, bottom)]\n",[54,1151,1152],{"class":56,"line":557},[54,1153,312],{"emptyLinePlaceholder":311},[54,1155,1156,1159,1161,1164,1166],{"class":56,"line":573},[54,1157,1158],{"class":298},"names ",[54,1160,321],{"class":294},[54,1162,1163],{"class":298}," two_row_names(",[54,1165,327],{"class":64},[54,1167,342],{"class":298},[54,1169,1170,1173,1175,1177,1179,1181,1183,1185,1187,1189,1192,1194,1196,1198,1201,1203],{"class":56,"line":885},[54,1171,1172],{"class":298},"two ",[54,1174,321],{"class":294},[54,1176,324],{"class":298},[54,1178,327],{"class":64},[54,1180,330],{"class":298},[54,1182,334],{"class":333},[54,1184,321],{"class":294},[54,1186,422],{"class":339},[54,1188,330],{"class":298},[54,1190,1191],{"class":333},"skiprows",[54,1193,321],{"class":294},[54,1195,200],{"class":339},[54,1197,330],{"class":298},[54,1199,1200],{"class":333},"names",[54,1202,321],{"class":294},[54,1204,1205],{"class":298},"names)\n",[54,1207,1209,1211],{"class":56,"line":1208},13,[54,1210,348],{"class":339},[54,1212,1213],{"class":298},"(names)\n",[54,1215,1217,1219],{"class":56,"line":1216},14,[54,1218,348],{"class":339},[54,1220,1221],{"class":298},"(two.columns.tolist())\n",[10,1223,1224,1225,1227,1228,1230,1231,1233,1234,1236],{},"The sales and units columns now say which year they belong to. The two region columns are still both called ",[22,1226,209],{},", because no heading distinguishes them, and passing duplicate ",[22,1229,1200],{}," to ",[22,1232,370],{}," gets the same treatment as a duplicate header: pandas renames the second to ",[22,1235,388],{}," without comment. That column needs a decision, which is what Step 4 is about; here it is a region code, so name it that:",[14,1238,1240],{"className":285,"code":1239,"language":287,"meta":20,"style":20},"two = two.rename(columns={\"Region.1\": \"Region code\"})\nprint(two.columns.tolist())\n",[22,1241,1242,1269],{"__ignoreMap":20},[54,1243,1244,1246,1248,1251,1254,1256,1258,1261,1263,1266],{"class":56,"line":57},[54,1245,1172],{"class":298},[54,1247,321],{"class":294},[54,1249,1250],{"class":298}," two.rename(",[54,1252,1253],{"class":333},"columns",[54,1255,321],{"class":294},[54,1257,1091],{"class":298},[54,1259,1260],{"class":64},"\"Region.1\"",[54,1262,727],{"class":298},[54,1264,1265],{"class":64},"\"Region code\"",[54,1267,1268],{"class":298},"})\n",[54,1270,1271,1273],{"class":56,"line":308},[54,1272,348],{"class":339},[54,1274,1221],{"class":298},[10,1276,1277,1278,1281,1282,1286],{},"Forward-filling the top row with pandas instead, a common recipe, would have labelled the region code column ",[22,1279,1280],{},"Region 2026",", because an empty cell after a merged range looks exactly like the empty cells inside one. ",[36,1283,1285],{"href":1284},"\u002Fadvanced-data-transformation-and-cleaning\u002Fextracting-data-from-messy-excel-sheets\u002Fread-excel-with-multi-row-headers-in-pandas\u002F","Read Excel with Multi-Row Headers in pandas"," covers deeper hierarchies.",[84,1288,93,1294,93,1297,93,1300,93,1305,93,1311,93,1315,93,1319,93,1322,93,1325,93,1328,93,1331,93,1334,93,1336,93,1338,93,1340,93,1342,93,1344,93,1346,93,1348,93,1350,93,1354,93,1357,93,1361,93,1364,93,1368,93,1371,93,1375,93,1378,93,1380,93,1383,93,1385,93,1388,93,1390,93,1393,93,1395,93,1398,93,1403,93,1406,93,1411,93,1416,93,1424],{"viewBox":1289,"role":87,"ariaLabelledBy":1290,"xmlns":91,"style":1293},"4 0 752 112",[1291,1292],"dc-names-t","dc-names-d","width:100%;max-width:752px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif",[95,1295,1296],{"id":1291},"Header rows joined into unique names",[99,1298,1299],{"id":1292},"On the left the second header row with duplicate names. On the right the joined names, Sales 2025, Units 2025, Sales 2026 and Units 2026, after the merged year row is forward-filled.",[103,1301],{"x":1302,"y":106,"width":1303,"height":1304,"fill":109},"4","752","112",[103,1306],{"x":1307,"y":169,"width":1308,"height":170,"rx":106,"fill":1309,"stroke":1310,"style":205},"20.0","54.0","#f0f4ff","var(--brand,#5b5cf0)",[19,1312,136],{"x":1313,"y":132,"style":1314},"47.0","font-size:11px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle",[103,1316],{"x":1317,"y":169,"width":1318,"height":170,"rx":106,"fill":1309,"stroke":1310,"style":205},"74.0","52.8",[19,1320,144],{"x":1321,"y":132,"style":1314},"100.4",[103,1323],{"x":1324,"y":169,"width":1308,"height":170,"rx":106,"fill":1309,"stroke":1310,"style":205},"126.8",[19,1326,152],{"x":1327,"y":132,"style":1314},"153.8",[103,1329],{"x":1330,"y":169,"width":1318,"height":170,"rx":106,"fill":1309,"stroke":1310,"style":205},"180.8",[19,1332,159],{"x":1333,"y":132,"style":1314},"207.2",[103,1335],{"x":1307,"y":196,"width":1308,"height":170,"rx":106,"fill":212,"stroke":213,"style":205},[19,1337,217],{"x":1313,"y":199,"style":216},[103,1339],{"x":1317,"y":196,"width":1318,"height":170,"rx":106,"fill":109,"stroke":117,"style":118},[19,1341,225],{"x":1321,"y":199,"style":249},[103,1343],{"x":1324,"y":196,"width":1308,"height":170,"rx":106,"fill":212,"stroke":213,"style":205},[19,1345,217],{"x":1327,"y":199,"style":216},[103,1347],{"x":1330,"y":196,"width":1318,"height":170,"rx":106,"fill":109,"stroke":117,"style":118},[19,1349,225],{"x":1333,"y":199,"style":249},[103,1351],{"x":1352,"y":169,"width":1353,"height":170,"rx":106,"fill":1309,"stroke":1310,"style":205},"398.5","85.8",[19,1355,136],{"x":1356,"y":132,"style":1314},"441.4",[103,1358],{"x":1359,"y":169,"width":1360,"height":170,"rx":106,"fill":1309,"stroke":1310,"style":205},"484.4","84.6",[19,1362,144],{"x":1363,"y":132,"style":1314},"526.7",[103,1365],{"x":1366,"y":169,"width":1367,"height":170,"rx":106,"fill":1309,"stroke":1310,"style":205},"568.9","86.2",[19,1369,152],{"x":1370,"y":132,"style":1314},"612.0",[103,1372],{"x":1373,"y":169,"width":1374,"height":170,"rx":106,"fill":1309,"stroke":1310,"style":205},"655.1","84.9",[19,1376,159],{"x":1377,"y":132,"style":1314},"697.6",[103,1379],{"x":1352,"y":196,"width":1353,"height":170,"rx":106,"fill":220,"stroke":221,"style":205},[19,1381,1382],{"x":1356,"y":199,"style":224},"Sales 2025",[103,1384],{"x":1359,"y":196,"width":1360,"height":170,"rx":106,"fill":220,"stroke":221,"style":205},[19,1386,1387],{"x":1363,"y":199,"style":224},"Units 2025",[103,1389],{"x":1366,"y":196,"width":1367,"height":170,"rx":106,"fill":220,"stroke":221,"style":205},[19,1391,1392],{"x":1370,"y":199,"style":224},"Sales 2026",[103,1394],{"x":1373,"y":196,"width":1374,"height":170,"rx":106,"fill":220,"stroke":221,"style":205},[19,1396,1397],{"x":1377,"y":199,"style":224},"Units 2026",[19,1399,1402],{"x":1324,"y":1400,"style":1401},"29.0","font-size:12.5px;font-weight:700;fill:var(--muted,#5b6780);text-anchor:middle","before",[19,1404,396],{"x":1405,"y":1400,"style":1401},"569.3",[56,1407],{"x1":1408,"y1":196,"x2":1409,"y2":196,"stroke":1310,"style":1410},"243.6","384.5","stroke-width:2px",[1412,1413],"polygon",{"points":1414,"fill":1415},"388.5,68.0 379.5,73.0 379.5,63.0","#5b5cf0",[103,1417],{"x":1418,"y":1419,"width":1420,"height":1421,"rx":1422,"fill":109,"stroke":117,"style":1423},"266.1","35.0","100.0","22.0","11","stroke-width:1.2px",[19,1425,1429],{"x":1426,"y":1427,"style":1428},"316.0","50.0","font-size:11.5px;font-weight:400;fill:var(--brand-strong,#4338ca);text-anchor:middle;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace","join levels",[42,1431,1433],{"id":1432},"step-4-rename-by-position-when-the-header-cannot-be-trusted","Step 4 — Rename by position when the header cannot be trusted",[10,1435,1436,1437,1439],{},"Sheets typed by hand need a decision about what each column is, and the decision belongs in the code, not in pandas' ",[22,1438,374],{}," suffixes. Map positions to names explicitly and check the raw headers are the ones you expect, so a column inserted next month stops the script instead of shifting every name:",[14,1441,1443],{"className":285,"code":1442,"language":287,"meta":20,"style":20},"EXPECTED = {0: (\"region\", \"Region\"), 1: (\"sales\", \"Sales Q1\"), 2: (\"sales\", \"Sales Q2\"),\n            3: (\"sales\", \"Sales Q3\"), 4: (\"\u003Cblank>\", \"Flag\"), 5: (\"notes\", \"Notes\")}\n\nraw_headers = pd.read_excel(\"typed_by_hand.xlsx\", header=None, nrows=1).iloc[0].tolist()\nkeys = [\" \".join(str(h).split()).casefold() if pd.notna(h) else \"\u003Cblank>\" for h in raw_headers]\nmismatch = {i: k for i, k in enumerate(keys) if EXPECTED.get(i, (None,))[0] != k}\nassert not mismatch and len(keys) == len(EXPECTED), f\"unexpected headers: {mismatch}\"\n\nclean = pd.read_excel(\"typed_by_hand.xlsx\", header=0)\nclean.columns = [EXPECTED[i][1] for i in range(len(EXPECTED))]\nprint(clean.to_string(index=False))\n",[22,1444,1445,1499,1545,1549,1584,1620,1666,1709,1713,1734,1772],{"__ignoreMap":20},[54,1446,1447,1450,1453,1455,1457,1460,1463,1465,1468,1471,1473,1475,1478,1480,1483,1485,1487,1489,1491,1493,1496],{"class":56,"line":57},[54,1448,1449],{"class":339},"EXPECTED",[54,1451,1452],{"class":294}," =",[54,1454,1100],{"class":298},[54,1456,106],{"class":339},[54,1458,1459],{"class":298},": (",[54,1461,1462],{"class":64},"\"region\"",[54,1464,330],{"class":298},[54,1466,1467],{"class":64},"\"Region\"",[54,1469,1470],{"class":298},"), ",[54,1472,174],{"class":339},[54,1474,1459],{"class":298},[54,1476,1477],{"class":64},"\"sales\"",[54,1479,330],{"class":298},[54,1481,1482],{"class":64},"\"Sales Q1\"",[54,1484,1470],{"class":298},[54,1486,200],{"class":339},[54,1488,1459],{"class":298},[54,1490,1477],{"class":64},[54,1492,330],{"class":298},[54,1494,1495],{"class":64},"\"Sales Q2\"",[54,1497,1498],{"class":298},"),\n",[54,1500,1501,1504,1506,1508,1510,1513,1515,1517,1519,1522,1524,1527,1529,1532,1534,1537,1539,1542],{"class":56,"line":308},[54,1502,1503],{"class":339},"            3",[54,1505,1459],{"class":298},[54,1507,1477],{"class":64},[54,1509,330],{"class":298},[54,1511,1512],{"class":64},"\"Sales Q3\"",[54,1514,1470],{"class":298},[54,1516,1302],{"class":339},[54,1518,1459],{"class":298},[54,1520,1521],{"class":64},"\"\u003Cblank>\"",[54,1523,330],{"class":298},[54,1525,1526],{"class":64},"\"Flag\"",[54,1528,1470],{"class":298},[54,1530,1531],{"class":339},"5",[54,1533,1459],{"class":298},[54,1535,1536],{"class":64},"\"notes\"",[54,1538,330],{"class":298},[54,1540,1541],{"class":64},"\"Notes\"",[54,1543,1544],{"class":298},")}\n",[54,1546,1547],{"class":56,"line":315},[54,1548,312],{"emptyLinePlaceholder":311},[54,1550,1551,1554,1556,1558,1560,1562,1564,1566,1568,1570,1573,1575,1577,1580,1582],{"class":56,"line":345},[54,1552,1553],{"class":298},"raw_headers ",[54,1555,321],{"class":294},[54,1557,324],{"class":298},[54,1559,870],{"class":64},[54,1561,330],{"class":298},[54,1563,334],{"class":333},[54,1565,321],{"class":294},[54,1567,422],{"class":339},[54,1569,330],{"class":298},[54,1571,1572],{"class":333},"nrows",[54,1574,321],{"class":294},[54,1576,174],{"class":339},[54,1578,1579],{"class":298},").iloc[",[54,1581,106],{"class":339},[54,1583,453],{"class":298},[54,1585,1586,1589,1591,1593,1595,1597,1599,1601,1603,1606,1608,1610,1613,1615,1617],{"class":56,"line":354},[54,1587,1588],{"class":298},"keys ",[54,1590,321],{"class":294},[54,1592,759],{"class":298},[54,1594,762],{"class":64},[54,1596,765],{"class":298},[54,1598,768],{"class":339},[54,1600,771],{"class":298},[54,1602,774],{"class":294},[54,1604,1605],{"class":298}," pd.notna(h) ",[54,1607,798],{"class":294},[54,1609,853],{"class":64},[54,1611,1612],{"class":294}," for",[54,1614,777],{"class":298},[54,1616,811],{"class":294},[54,1618,1619],{"class":298}," raw_headers]\n",[54,1621,1622,1625,1627,1630,1632,1635,1637,1640,1643,1645,1648,1651,1653,1656,1658,1660,1663],{"class":56,"line":492},[54,1623,1624],{"class":298},"mismatch ",[54,1626,321],{"class":294},[54,1628,1629],{"class":298}," {i: k ",[54,1631,956],{"class":294},[54,1633,1634],{"class":298}," i, k ",[54,1636,811],{"class":294},[54,1638,1639],{"class":339}," enumerate",[54,1641,1642],{"class":298},"(keys) ",[54,1644,774],{"class":294},[54,1646,1647],{"class":339}," EXPECTED",[54,1649,1650],{"class":298},".get(i, (",[54,1652,422],{"class":339},[54,1654,1655],{"class":298},",))[",[54,1657,106],{"class":339},[54,1659,510],{"class":298},[54,1661,1662],{"class":294},"!=",[54,1664,1665],{"class":298}," k}\n",[54,1667,1668,1671,1673,1676,1679,1682,1684,1686,1688,1690,1692,1694,1696,1699,1701,1704,1706],{"class":56,"line":501},[54,1669,1670],{"class":294},"assert",[54,1672,783],{"class":294},[54,1674,1675],{"class":298}," mismatch ",[54,1677,1678],{"class":294},"and",[54,1680,1681],{"class":339}," len",[54,1683,1642],{"class":298},[54,1685,850],{"class":294},[54,1687,1681],{"class":339},[54,1689,467],{"class":298},[54,1691,1449],{"class":339},[54,1693,1470],{"class":298},[54,1695,1084],{"class":294},[54,1697,1698],{"class":64},"\"unexpected headers: ",[54,1700,1091],{"class":1090},[54,1702,1703],{"class":298},"mismatch",[54,1705,1097],{"class":1090},[54,1707,1708],{"class":64},"\"\n",[54,1710,1711],{"class":56,"line":528},[54,1712,312],{"emptyLinePlaceholder":311},[54,1714,1715,1718,1720,1722,1724,1726,1728,1730,1732],{"class":56,"line":543},[54,1716,1717],{"class":298},"clean ",[54,1719,321],{"class":294},[54,1721,324],{"class":298},[54,1723,870],{"class":64},[54,1725,330],{"class":298},[54,1727,334],{"class":333},[54,1729,321],{"class":294},[54,1731,106],{"class":339},[54,1733,342],{"class":298},[54,1735,1736,1739,1741,1743,1745,1748,1750,1752,1754,1757,1759,1761,1763,1765,1767,1769],{"class":56,"line":557},[54,1737,1738],{"class":298},"clean.columns ",[54,1740,321],{"class":294},[54,1742,759],{"class":298},[54,1744,1449],{"class":339},[54,1746,1747],{"class":298},"[i][",[54,1749,174],{"class":339},[54,1751,510],{"class":298},[54,1753,956],{"class":294},[54,1755,1756],{"class":298}," i ",[54,1758,811],{"class":294},[54,1760,1010],{"class":339},[54,1762,467],{"class":298},[54,1764,735],{"class":339},[54,1766,467],{"class":298},[54,1768,1449],{"class":339},[54,1770,1771],{"class":298},"))]\n",[54,1773,1774,1776,1779,1781,1783,1785],{"class":56,"line":573},[54,1775,348],{"class":339},[54,1777,1778],{"class":298},"(clean.to_string(",[54,1780,876],{"class":333},[54,1782,321],{"class":294},[54,1784,834],{"class":339},[54,1786,711],{"class":298},[10,1788,1789,1790,1792,1793,1796,1797,1799],{},"The names in ",[22,1791,1449],{}," cannot be derived from the sheet; they come from asking whoever maintains it what each column means. Here the three \"Sales\" columns turned out to be three quarters typed by different people, and the unlabelled column a review flag. Writing that knowledge into the mapping is the real fix: the next person reading the code sees ",[22,1794,1795],{},"Sales Q2"," instead of guessing what ",[22,1798,903],{}," with a trailing space was supposed to be. Keep the expected raw headers next to the new names, as here, so the check fails the day someone relabels a column.",[84,1801,93,1806,93,1809,93,1812,93,1815,93,1821,93,1832,93,1837,93,1841,93,1845,93,1848,93,1851,93,1858,93,1863,93,1868,93,1874,93,1878,93,1883,93,1885,93,1888,93,1892,93,1895,93,1898,93,1905,93,1909,93,1912,93,1914,93,1917,93,1921,93,1924,93,1927,93,1934,93,1938],{"viewBox":1802,"role":87,"ariaLabelledBy":1803,"xmlns":91,"style":1293},"4 0 752 272",[1804,1805],"dc-decide-t","dc-decide-d",[95,1807,1808],{"id":1804},"What to do with a repeated header",[99,1810,1811],{"id":1805},"A decision diagram. If a group heading above distinguishes the columns, join the header rows. If the columns are different things typed with the same label, rename them by position with a checked mapping. If they are genuine copies of the same data, compare them and drop one.",[103,1813],{"x":1302,"y":106,"width":1303,"height":1814,"fill":109},"272",[103,1816],{"x":1817,"y":113,"width":1818,"height":1819,"rx":1820,"fill":1309,"stroke":1310,"style":1410},"240.0","280.0","75.5","12",[1822,1823,1825],"g",{"transform":1824},"translate(368.0,29.0) scale(1.000)",[1826,1827],"path",{"d":1828,"fill":1829,"stroke":1830,"style":1831},"M11 4a7 7 0 1 0 .01 0z M16 16l5 5","none","var(--brand-strong,#4338ca)","stroke-width:1.80px;stroke-linecap:round;stroke-linejoin:round",[19,1833,1836],{"x":1834,"y":1819,"style":1835},"380.0","font-size:13.5px;font-weight:700;fill:var(--brand-strong,#4338ca);text-anchor:middle","Why is the header repeated?",[56,1838],{"x1":1834,"y1":1839,"x2":1834,"y2":1840,"stroke":1310,"style":1410},"91.5","111.5",[56,1842],{"x1":1843,"y1":1840,"x2":1844,"y2":1840,"stroke":1310,"style":1410},"134.7","625.3",[56,1846],{"x1":1843,"y1":1840,"x2":1843,"y2":1847,"stroke":1310,"style":1410},"157.5",[1412,1849],{"points":1850,"fill":1415},"134.7,161.5 129.7,152.5 139.7,152.5",[103,1852],{"x":1853,"y":1854,"width":1855,"height":1856,"rx":1857,"fill":109,"stroke":117,"style":1423},"56.1","121.5","157.2","21.0","10",[19,1859,1862],{"x":1843,"y":1860,"style":1861},"136.0","font-size:11.5px;font-weight:700;fill:var(--muted,#5b6780);text-anchor:middle","a heading above differs",[103,1864],{"x":1307,"y":1865,"width":1866,"height":1867,"rx":1820,"fill":1309,"stroke":1310,"style":1410},"163.5","229.3","93.0",[1822,1869,1871],{"transform":1870},"translate(122.7,176.5) scale(1.000)",[1826,1872],{"d":1873,"fill":1829,"stroke":1830,"style":1831},"M12 3l9 5-9 5-9-5z M3 13l9 5 9-5 M3 17l9 5 9-5",[19,1875,1877],{"x":1843,"y":1876,"style":1835},"223.0","Join header rows",[19,1879,1882],{"x":1843,"y":1880,"style":1881},"242.5","font-size:11.5px;font-weight:400;fill:var(--text,#172033);text-anchor:middle","Step 3",[56,1884],{"x1":1834,"y1":1840,"x2":1834,"y2":1847,"stroke":1310,"style":1410},[1412,1886],{"points":1887,"fill":1415},"380.0,161.5 375.0,152.5 385.0,152.5",[103,1889],{"x":1890,"y":1854,"width":1891,"height":1856,"rx":1857,"fill":109,"stroke":117,"style":1423},"330.7","98.7",[19,1893,1894],{"x":1834,"y":1860,"style":1861},"different data",[103,1896],{"x":1897,"y":1865,"width":1866,"height":1867,"rx":1820,"fill":220,"stroke":221,"style":1410},"265.3",[1822,1899,1901],{"transform":1900},"translate(368.0,176.5) scale(1.000)",[1826,1902],{"d":1903,"fill":1829,"stroke":1904,"style":1831},"M3 5h18v14H3z M3 10h18 M3 15h18 M9 5v14","var(--teal-ink,#0b6157)",[19,1906,1908],{"x":1834,"y":1876,"style":1907},"font-size:13.5px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","Rename by position",[19,1910,1911],{"x":1834,"y":1880,"style":1881},"Step 4",[56,1913],{"x1":1844,"y1":1840,"x2":1844,"y2":1847,"stroke":1310,"style":1410},[1412,1915],{"points":1916,"fill":1415},"625.3,161.5 620.3,152.5 630.3,152.5",[103,1918],{"x":1919,"y":1854,"width":1920,"height":1856,"rx":1857,"fill":109,"stroke":117,"style":1423},"569.9","110.9",[19,1922,1923],{"x":1844,"y":1860,"style":1861},"identical copies",[103,1925],{"x":1926,"y":1865,"width":1866,"height":1867,"rx":1820,"fill":212,"stroke":213,"style":1410},"510.7",[1822,1928,1930],{"transform":1929},"translate(613.3,176.5) scale(1.000)",[1826,1931],{"d":1932,"fill":1829,"stroke":1933,"style":1831},"M5 12.5l4.5 4.5L19 7","var(--gold-ink,#7a4e06)",[19,1935,1937],{"x":1844,"y":1876,"style":1936},"font-size:13.5px;font-weight:700;fill:var(--gold-ink,#7a4e06);text-anchor:middle","Compare and drop",[19,1939,1940],{"x":1844,"y":1880,"style":1881},"equals() first",[10,1942,1943,1944,1947],{},"The third branch is the only one where deleting a column is right, and only after checking: two columns called \"Sales\" that hold the same numbers are a copy-paste leftover, but two that differ in one row are two different things with a typo in the header. ",[22,1945,1946],{},"df[\"Sales\"].equals(df[\"Sales.1\"])"," answers it before anything is dropped.",[42,1949,1951],{"id":1950},"step-5-guard-the-rest-of-the-pipeline","Step 5 — Guard the rest of the pipeline",[10,1953,1954],{},"Whatever the source, assert uniqueness at the boundary where data enters your code; it is one line, and it turns every later symptom into a clear message at the right place:",[14,1956,1958],{"className":285,"code":1957,"language":287,"meta":20,"style":20},"def require_unique_columns(frame, source):\n    dupes = frame.columns[frame.columns.duplicated()].tolist()\n    if dupes:\n        raise ValueError(f\"{source}: duplicate column names {dupes}\")\n    return frame\n\nrequire_unique_columns(two, \"sales_by_year.xlsx\")\ntry:\n    require_unique_columns(manual, \"manual header\")\nexcept ValueError as e:\n    print(e)\n",[22,1959,1960,1970,1980,1988,2022,2029,2033,2042,2048,2058,2068],{"__ignoreMap":20},[54,1961,1962,1964,1967],{"class":56,"line":57},[54,1963,638],{"class":294},[54,1965,1966],{"class":641}," require_unique_columns",[54,1968,1969],{"class":298},"(frame, source):\n",[54,1971,1972,1975,1977],{"class":56,"line":308},[54,1973,1974],{"class":298},"    dupes ",[54,1976,321],{"class":294},[54,1978,1979],{"class":298}," frame.columns[frame.columns.duplicated()].tolist()\n",[54,1981,1982,1985],{"class":56,"line":315},[54,1983,1984],{"class":294},"    if",[54,1986,1987],{"class":298}," dupes:\n",[54,1989,1990,1993,1995,1997,1999,2001,2003,2006,2008,2011,2013,2016,2018,2020],{"class":56,"line":345},[54,1991,1992],{"class":294},"        raise",[54,1994,534],{"class":339},[54,1996,467],{"class":298},[54,1998,1084],{"class":294},[54,2000,1087],{"class":64},[54,2002,1091],{"class":1090},[54,2004,2005],{"class":298},"source",[54,2007,1097],{"class":1090},[54,2009,2010],{"class":64},": duplicate column names ",[54,2012,1091],{"class":1090},[54,2014,2015],{"class":298},"dupes",[54,2017,1097],{"class":1090},[54,2019,1087],{"class":64},[54,2021,342],{"class":298},[54,2023,2024,2026],{"class":56,"line":354},[54,2025,819],{"class":294},[54,2027,2028],{"class":298}," frame\n",[54,2030,2031],{"class":56,"line":492},[54,2032,312],{"emptyLinePlaceholder":311},[54,2034,2035,2038,2040],{"class":56,"line":501},[54,2036,2037],{"class":298},"require_unique_columns(two, ",[54,2039,327],{"class":64},[54,2041,342],{"class":298},[54,2043,2044,2046],{"class":56,"line":528},[54,2045,495],{"class":294},[54,2047,498],{"class":298},[54,2049,2050,2053,2056],{"class":56,"line":543},[54,2051,2052],{"class":298},"    require_unique_columns(manual, ",[54,2054,2055],{"class":64},"\"manual header\"",[54,2057,342],{"class":298},[54,2059,2060,2062,2064,2066],{"class":56,"line":557},[54,2061,531],{"class":294},[54,2063,534],{"class":339},[54,2065,537],{"class":294},[54,2067,540],{"class":298},[54,2069,2070,2072],{"class":56,"line":573},[54,2071,546],{"class":339},[54,2073,2074],{"class":298},"(e)\n",[42,2076,2078],{"id":2077},"common-pitfalls","Common pitfalls",[2080,2081,2082,2095,2105,2114,2120],"ul",{},[2083,2084,2085,2092,2093,40],"li",{},[2086,2087,2088,2089,2091],"strong",{},"Trusting ",[22,2090,374],{}," suffixes."," They depend on column order; a new column shifts which one is ",[22,2094,374],{},[2083,2096,2097,2104],{},[2086,2098,2099,2100,2103],{},"Setting ",[22,2101,2102],{},"df.columns"," from a header row."," No deduplication happens, and duplicate names break selection and assignment.",[2083,2106,2107,2113],{},[2086,2108,2109,2112],{},[22,2110,2111],{},"to_dict"," with duplicates."," It warns and keeps one value per name; data disappears.",[2083,2115,2116,2119],{},[2086,2117,2118],{},"Invisible differences."," Trailing spaces and case make near-duplicates look distinct; normalise before comparing.",[2083,2121,2122,2125,2126,2129,2130,40],{},[2086,2123,2124],{},"Blank headers."," ",[22,2127,2128],{},"Unnamed: n"," names are positional too; give them real names. See ",[36,2131,2133],{"href":2132},"\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-unnamed-0-columns-when-reading-excel\u002F","Fix \"Unnamed: 0\" Columns When Reading Excel with pandas",[42,2135,2137],{"id":2136},"conclusion","Conclusion",[10,2139,2140,2141,2143,2144,330,2146,2148,2149,2151],{},"pandas' ",[22,2142,370],{}," makes repeated headers unique with ",[22,2145,374],{},[22,2147,377],{}," suffixes, which hides the problem rather than solving it, and manual header assignment creates true duplicates that break selection, assignment and ",[22,2150,2111],{},". Report duplicates and near-duplicates from the raw header row, build meaningful names from multi-row headers or an explicit position mapping checked against the expected headers, and assert column uniqueness where data enters the pipeline.",[42,2153,2155],{"id":2154},"frequently-asked-questions","Frequently asked questions",[10,2157,2158,2161],{},[2086,2159,2160],{},"Why does pandas add .1 to my column names?","\nThe sheet has two columns with the same header. read_excel keeps both and appends .1, .2 to the repeats so every name is unique.",[10,2163,2164,2167,2168,2171],{},[2086,2165,2166],{},"How do I read an Excel sheet with a two-row header?","\nPass header=",[54,2169,2170],{},"0, 1"," to read_excel to get a MultiIndex, forward-fill the top level across merged cells, and join the two levels into one name per column.",[10,2173,2174,2177],{},[2086,2175,2176],{},"How do I find columns whose names differ only by spaces or case?","\nNormalise each header by collapsing whitespace and applying casefold, then look for duplicates in the normalised names.",[10,2179,2180,2186],{},[2086,2181,2182,2183,2185],{},"Why does df",[54,2184,481],{}," return a DataFrame instead of a Series?","\nThe DataFrame has more than one column named Sales. Rename the columns so each name is unique.",[42,2188,2190],{"id":2189},"related","Related",[2080,2192,2193,2199,2204,2209,2216],{},[2083,2194,2195,2196,2198],{},"Up one level: ",[36,2197,39],{"href":38}," — the other common errors.",[2083,2200,2201,2203],{},[36,2202,2133],{"href":2132}," — the blank-header case.",[2083,2205,2206,2208],{},[36,2207,1285],{"href":1284}," — deeper header hierarchies.",[2083,2210,2211,2215],{},[36,2212,2214],{"href":2213},"\u002Fgetting-started-with-python-excel-automation\u002Freading-excel-files-with-pandas\u002Flist-sheet-names-and-column-headers-without-loading-data\u002F","List Sheet Names and Column Headers Without Loading Data"," — checking headers cheaply.",[2083,2217,2218,2222],{},[36,2219,2221],{"href":2220},"\u002Fadvanced-data-transformation-and-cleaning\u002Fvalidating-excel-data-with-python\u002Fprofile-an-excel-dataset-with-pandas\u002F","Profile an Excel Dataset with pandas"," — a first look at a new file.",[2224,2225,2226],"style",{},"html pre.shiki code .sMTad, html code.shiki .sMTad{--shiki-default:#6F42C1;--shiki-dark:#FFB757}html pre.shiki code .srMev, html code.shiki .srMev{--shiki-default:#032F62;--shiki-dark:#ADDCFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s-kum, html code.shiki .s-kum{--shiki-default:#D73A49;--shiki-dark:#FF9492}html pre.shiki code .skGVy, html code.shiki .skGVy{--shiki-default:#24292E;--shiki-dark:#F0F3F6}html pre.shiki code .sa561, html code.shiki .sa561{--shiki-default:#E36209;--shiki-dark:#FFB757}html pre.shiki code .sP0c6, html code.shiki .sP0c6{--shiki-default:#005CC5;--shiki-dark:#91CBFF}html pre.shiki code .s_Opv, html code.shiki .s_Opv{--shiki-default:#6F42C1;--shiki-dark:#DBB7FF}html pre.shiki code .sSjpA, html code.shiki .sSjpA{--shiki-default:#005CC5;--shiki-dark:#FF9492}",{"title":20,"searchDepth":308,"depth":308,"links":2228},[2229,2230,2231,2232,2233,2234,2235,2236,2237,2238],{"id":44,"depth":308,"text":45},{"id":278,"depth":308,"text":279},{"id":607,"depth":308,"text":608},{"id":915,"depth":308,"text":916},{"id":1432,"depth":308,"text":1433},{"id":1950,"depth":308,"text":1951},{"id":2077,"depth":308,"text":2078},{"id":2136,"depth":308,"text":2137},{"id":2154,"depth":308,"text":2155},{"id":2189,"depth":308,"text":2190},"2026-09-29","When a sheet has two columns with the same header, pandas renames the second to \"Sales.1\", and near-duplicates such as \"Sales \" and \"sales\" slip through as separate columns. Find every duplicate, decide what each column really is, give them unique names from a two-row header or an explicit mapping, and stop surprises such as to_dict silently dropping data.","md",[2243,2245,2247,2249],{"q":2160,"a":2244},"The sheet has two columns with the same header. read_excel keeps both and appends .1, .2 to the repeats so every name is unique.",{"q":2166,"a":2246},"Pass header=[0, 1] to read_excel to get a MultiIndex, forward-fill the top level across merged cells, and join the two levels into one name per column.",{"q":2176,"a":2248},"Normalise each header by collapsing whitespace and applying casefold, then look for duplicates in the normalised names.",{"q":2250,"a":2251},"Why does df[\"Sales\"] return a DataFrame instead of a Series?","The DataFrame has more than one column named Sales. Rename the columns so each name is unique.","fix duplicate column names when reading excel",null,{"breadcrumb":2255},[2256,2259,2262],{"name":2257,"item":2258},"Home","\u002F",{"name":2260,"item":2261},"Getting Started with Python Excel Automation","\u002Fgetting-started-with-python-excel-automation\u002F",{"name":39,"item":38},"\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-duplicate-column-names-when-reading-excel",{"title":2265,"description":2266},"Fix Duplicate Column Names in pandas read_excel (Sales.1)","Why pandas read_excel creates columns like Sales.1 and Unnamed: 5, how duplicate headers break assignment, to_dict and to_parquet, and how to rename them from two-row headers or a mapping and validate expected columns.","fix-duplicate-column-names-when-reading-excel","getting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-duplicate-column-names-when-reading-excel\u002Findex",[2270,2271],"troubleshooting","reading-excel","how-to","Z0UQ5xnh9xIiXv7XU1InKizw18NMzoNyzy3AlYRoxYI",[2275,2279],{"title":2276,"path":2277,"stem":2278,"children":-1},"Fix the \"Data Validation extension is not supported\" Warning in openpyxl","\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-data-validation-extension-is-not-supported-warning","getting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-data-validation-extension-is-not-supported-warning\u002Findex",{"title":2280,"path":2281,"stem":2282,"children":-1},"Fix \"Excel does not support datetimes with timezones\" in pandas","\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-excel-does-not-support-timezones-in-datetimes","getting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors\u002Ffix-excel-does-not-support-timezones-in-datetimes\u002Findex",[2284,2286],{"path":2285,"title":2260},"\u002Fgetting-started-with-python-excel-automation",{"path":2287,"title":39},"\u002Fgetting-started-with-python-excel-automation\u002Ftroubleshooting-common-python-excel-errors",[],[2290,2294,2297,2300,2303,2306],{"path":2291,"title":2292,"kind":2293},"\u002Fautomating-reporting-workflows\u002Ferror-handling-and-logging-in-excel-automation\u002Fhandle-permission-denied-when-writing-excel-in-python","Handle Permission Denied When Writing Excel in Python","fix",{"path":2295,"title":2296,"kind":2272},"\u002Fgetting-started-with-python-excel-automation\u002Fusing-openpyxl-for-excel-file-manipulation\u002Fwhat-openpyxl-loses-when-saving-an-existing-workbook","What openpyxl Loses When You Save an Existing Workbook",{"path":2298,"title":2299,"kind":2293},"\u002Fadvanced-data-transformation-and-cleaning\u002Fworking-with-dates-and-times-in-excel-data\u002Ffix-excel-serial-numbers-showing-instead-of-dates","Fix Excel Serial Numbers Showing Instead of Dates",{"path":2301,"title":2302,"kind":2272},"\u002Fadvanced-data-transformation-and-cleaning\u002Fworking-with-dates-and-times-in-excel-data\u002Fhandle-daylight-saving-time-in-excel-timestamps","Handle Daylight Saving Time in Excel Timestamps",{"path":2304,"title":2305,"kind":2272},"\u002Fautomating-reporting-workflows\u002Fautomating-excel-online-with-microsoft-graph\u002Fhandle-microsoft-graph-throttling-for-excel","Handle Microsoft Graph Throttling in Excel Automation",{"path":2307,"title":2308,"kind":2309},"\u002Fautomating-reporting-workflows\u002Ferror-handling-and-logging-in-excel-automation","Error Handling and Logging in Excel Automation","guide",1790972065562]