[{"data":1,"prerenderedAt":2429},["ShallowReactive",2],{"doc:\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Finterpolate-missing-numeric-values-in-excel-data":3,"surround:\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Finterpolate-missing-numeric-values-in-excel-data":2421},{"id":4,"title":5,"body":6,"dateModified":2396,"datePublished":2396,"description":2397,"extension":2398,"faq":2399,"meta":2412,"navigation":222,"path":2413,"seo":2414,"slug":2417,"stem":2418,"type":2419,"__hash__":2420},"docs\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Finterpolate-missing-numeric-values-in-excel-data\u002Findex.md","Interpolate Missing Numeric Values in Excel Data",{"type":7,"value":8,"toc":2383},"minimark",[9,19,142,147,178,181,375,379,387,588,600,604,607,627,630,743,793,802,813,922,925,929,932,1021,1188,1194,1198,1201,1427,1430,1843,1850,1854,1927,1936,2035,2038,2042,2164,2168,2173,2180,2240,2256,2267,2270,2274,2286,2290,2296,2307,2319,2332,2338,2342,2379],[10,11,12,13,18],"p",{},"Interpolation fills a gap by looking at its neighbours, and that makes it either the right tool or a way of inventing data — depending entirely on whether the rows are ordered along an axis where \"between\" means something. A missing daily temperature between Monday and Wednesday can reasonably be estimated. A missing region name between North and South cannot. This guide covers the methods pandas offers, the arguments that stop it fabricating values past the end of a series, and the flagging that lets a reader tell an estimate from a measurement. It extends ",[14,15,17],"a",{"href":16},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002F","Handling Missing Data in Excel Reports",".",[20,21,30,31,30,35,30,39,30,46,30,53,30,61,30,64,30,71,30,75,30,82,30,85,30,92,30,96,30,99,30,102,30,106,30,110,30,114,30,118,30,122,30,128,30,131,30,135,30,138],"svg",{"viewBox":22,"role":23,"ariaLabel":24,"ariaLabelledBy":25,"xmlns":28,"style":29},"0 0 800 254","img","A daily series with three gaps, showing linear interpolation joining the observations either side and the trailing gap left unfilled.",[26,27],"interp-t","interp-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;max-width:800px;height:auto;display:block;margin:1.5rem auto;font-family:Inter,ui-sans-serif,system-ui,sans-serif","\n  ",[32,33,34],"title",{"id":26},"What interpolation fills, and what it should leave alone",[36,37,38],"desc",{"id":27},"A line chart of a daily series. Solid points are observed values. Two interior gaps are bridged by interpolated points sitting on the straight line between their neighbours, which is a defensible estimate. A trailing gap after the last observation is left empty, because there is no later value to interpolate towards and extending the line would be extrapolation rather than interpolation.",[40,41],"rect",{"x":42,"y":42,"width":43,"height":44,"fill":45},"0","800","254","#ffffff",[47,48,52],"text",{"x":49,"y":50,"style":51},"400","26","font-size:12px;font-weight:700;fill:var(--muted,#5b6780);text-anchor:middle","daily readings with gaps",[54,55],"line",{"x1":56,"y1":57,"x2":58,"y2":57,"stroke":59,"style":60},"60","196","756","var(--line,#cdd5e6)","stroke-width:1.5px",[54,62],{"x1":56,"y1":57,"x2":56,"y2":63,"stroke":59,"style":60},"46",[65,66],"path",{"d":67,"fill":68,"stroke":69,"style":70},"M92 156 L 156 132 L 220 118 L 284 104 L 348 92 L 412 118 L 476 96 L 540 78 L 604 66","none","var(--brand,#5b5cf0)","stroke-width:2.5px",[65,72],{"d":73,"fill":68,"stroke":59,"style":74},"M604 66 L 668 60","stroke-width:2px;stroke-dasharray:5 5",[76,77],"circle",{"cx":78,"cy":79,"r":80,"fill":81},"92","156","6","#5b5cf0",[76,83],{"cx":79,"cy":84,"r":80,"fill":81},"132",[76,86],{"cx":87,"cy":88,"r":80,"fill":89,"stroke":90,"style":91},"220","118","var(--teal,#14b8a6)","#0f9488","stroke-width:2px",[76,93],{"cx":94,"cy":95,"r":80,"fill":89,"stroke":90,"style":91},"284","104",[76,97],{"cx":98,"cy":78,"r":80,"fill":81},"348",[76,100],{"cx":101,"cy":88,"r":80,"fill":81},"412",[76,103],{"cx":104,"cy":105,"r":80,"fill":89,"stroke":90,"style":91},"476","96",[76,107],{"cx":108,"cy":109,"r":80,"fill":81},"540","78",[76,111],{"cx":112,"cy":113,"r":80,"fill":81},"604","66",[76,115],{"cx":116,"cy":56,"r":80,"fill":45,"stroke":117,"style":91},"668","var(--accent,#f43f8f)",[76,119],{"cx":57,"cy":120,"r":121,"fill":81},"222","5",[47,123,127],{"x":124,"y":125,"style":126},"212","226","font-size:11px;fill:var(--muted,#5b6780)","observed",[76,129],{"cx":130,"cy":120,"r":121,"fill":89,"stroke":90,"style":91},"336",[47,132,134],{"x":133,"y":125,"style":126},"352","interpolated — between two real values",[76,136],{"cx":137,"cy":120,"r":121,"fill":45,"stroke":117,"style":91},"640",[47,139,141],{"x":140,"y":125,"style":126},"656","left empty",[143,144,146],"h2",{"id":145},"prerequisites","Prerequisites",[148,149,154],"pre",{"className":150,"code":151,"language":152,"meta":153,"style":153},"language-bash shiki shiki-themes github-light github-dark-high-contrast","pip install pandas openpyxl xlsxwriter\n","bash","",[155,156,157],"code",{"__ignoreMap":153},[158,159,161,165,169,172,175],"span",{"class":54,"line":160},1,[158,162,164],{"class":163},"sMTad","pip",[158,166,168],{"class":167},"srMev"," install",[158,170,171],{"class":167}," pandas",[158,173,174],{"class":167}," openpyxl",[158,176,177],{"class":167}," xlsxwriter\n",[10,179,180],{},"An irregular daily series with interior and trailing gaps:",[148,182,186],{"className":183,"code":184,"language":185,"meta":153,"style":153},"language-python shiki shiki-themes github-light github-dark-high-contrast","import numpy as np\nimport pandas as pd\n\nreadings = pd.DataFrame({\n    \"date\": pd.to_datetime([\n        \"2026-08-01\", \"2026-08-02\", \"2026-08-03\", \"2026-08-04\",\n        \"2026-08-05\", \"2026-08-12\", \"2026-08-13\", \"2026-08-14\",\n    ]),\n    \"region\": [\"North\"] * 4 + [\"South\"] * 4,\n    \"temperature\": [18.2, np.nan, np.nan, 21.6, 15.4, np.nan, 17.9, np.nan],\n})\n","python",[155,187,188,204,217,224,236,245,270,293,299,338,369],{"__ignoreMap":153},[158,189,190,194,198,201],{"class":54,"line":160},[158,191,193],{"class":192},"s-kum","import",[158,195,197],{"class":196},"skGVy"," numpy ",[158,199,200],{"class":192},"as",[158,202,203],{"class":196}," np\n",[158,205,207,209,212,214],{"class":54,"line":206},2,[158,208,193],{"class":192},[158,210,211],{"class":196}," pandas ",[158,213,200],{"class":192},[158,215,216],{"class":196}," pd\n",[158,218,220],{"class":54,"line":219},3,[158,221,223],{"emptyLinePlaceholder":222},true,"\n",[158,225,227,230,233],{"class":54,"line":226},4,[158,228,229],{"class":196},"readings ",[158,231,232],{"class":192},"=",[158,234,235],{"class":196}," pd.DataFrame({\n",[158,237,239,242],{"class":54,"line":238},5,[158,240,241],{"class":167},"    \"date\"",[158,243,244],{"class":196},": pd.to_datetime([\n",[158,246,248,251,254,257,259,262,264,267],{"class":54,"line":247},6,[158,249,250],{"class":167},"        \"2026-08-01\"",[158,252,253],{"class":196},", ",[158,255,256],{"class":167},"\"2026-08-02\"",[158,258,253],{"class":196},[158,260,261],{"class":167},"\"2026-08-03\"",[158,263,253],{"class":196},[158,265,266],{"class":167},"\"2026-08-04\"",[158,268,269],{"class":196},",\n",[158,271,273,276,278,281,283,286,288,291],{"class":54,"line":272},7,[158,274,275],{"class":167},"        \"2026-08-05\"",[158,277,253],{"class":196},[158,279,280],{"class":167},"\"2026-08-12\"",[158,282,253],{"class":196},[158,284,285],{"class":167},"\"2026-08-13\"",[158,287,253],{"class":196},[158,289,290],{"class":167},"\"2026-08-14\"",[158,292,269],{"class":196},[158,294,296],{"class":54,"line":295},8,[158,297,298],{"class":196},"    ]),\n",[158,300,302,305,308,311,314,317,321,324,327,330,332,334,336],{"class":54,"line":301},9,[158,303,304],{"class":167},"    \"region\"",[158,306,307],{"class":196},": [",[158,309,310],{"class":167},"\"North\"",[158,312,313],{"class":196},"] ",[158,315,316],{"class":192},"*",[158,318,320],{"class":319},"sP0c6"," 4",[158,322,323],{"class":192}," +",[158,325,326],{"class":196}," [",[158,328,329],{"class":167},"\"South\"",[158,331,313],{"class":196},[158,333,316],{"class":192},[158,335,320],{"class":319},[158,337,269],{"class":196},[158,339,341,344,346,349,352,355,357,360,363,366],{"class":54,"line":340},10,[158,342,343],{"class":167},"    \"temperature\"",[158,345,307],{"class":196},[158,347,348],{"class":319},"18.2",[158,350,351],{"class":196},", np.nan, np.nan, ",[158,353,354],{"class":319},"21.6",[158,356,253],{"class":196},[158,358,359],{"class":319},"15.4",[158,361,362],{"class":196},", np.nan, ",[158,364,365],{"class":319},"17.9",[158,367,368],{"class":196},", np.nan],\n",[158,370,372],{"class":54,"line":371},11,[158,373,374],{"class":196},"})\n",[143,376,378],{"id":377},"step-1-check-that-interpolation-is-even-valid","Step 1 — Check that interpolation is even valid",[10,380,381,382,386],{},"Ask one question first: ",[383,384,385],"strong",{},"is there a meaningful order to the rows?"," If sorting the frame differently would change the filled values, interpolation is only defensible when that order is real.",[148,388,390],{"className":183,"code":389,"language":185,"meta":153,"style":153},"import pandas as pd\n\ndef interpolation_is_sensible(df, order_column):\n    \"\"\"Cheap sanity checks before interpolating.\"\"\"\n    problems = []\n    if order_column not in df.columns:\n        problems.append(f\"no ordering column named {order_column!r}\")\n        return problems\n\n    ordered = df[order_column]\n    if not ordered.is_monotonic_increasing:\n        problems.append(f\"{order_column} is not sorted — sort before interpolating\")\n    if ordered.duplicated().any():\n        problems.append(f\"{order_column} has duplicates — the order is ambiguous\")\n    return problems\n\nprint(interpolation_is_sensible(readings, \"date\"))\n",[155,391,392,402,406,418,423,433,450,480,488,492,502,512,532,540,560,568,573],{"__ignoreMap":153},[158,393,394,396,398,400],{"class":54,"line":160},[158,395,193],{"class":192},[158,397,211],{"class":196},[158,399,200],{"class":192},[158,401,216],{"class":196},[158,403,404],{"class":54,"line":206},[158,405,223],{"emptyLinePlaceholder":222},[158,407,408,411,415],{"class":54,"line":219},[158,409,410],{"class":192},"def",[158,412,414],{"class":413},"s_Opv"," interpolation_is_sensible",[158,416,417],{"class":196},"(df, order_column):\n",[158,419,420],{"class":54,"line":226},[158,421,422],{"class":167},"    \"\"\"Cheap sanity checks before interpolating.\"\"\"\n",[158,424,425,428,430],{"class":54,"line":238},[158,426,427],{"class":196},"    problems ",[158,429,232],{"class":192},[158,431,432],{"class":196}," []\n",[158,434,435,438,441,444,447],{"class":54,"line":247},[158,436,437],{"class":192},"    if",[158,439,440],{"class":196}," order_column ",[158,442,443],{"class":192},"not",[158,445,446],{"class":192}," in",[158,448,449],{"class":196}," df.columns:\n",[158,451,452,455,458,461,465,468,471,474,477],{"class":54,"line":272},[158,453,454],{"class":196},"        problems.append(",[158,456,457],{"class":192},"f",[158,459,460],{"class":167},"\"no ordering column named ",[158,462,464],{"class":463},"sSjpA","{",[158,466,467],{"class":196},"order_column",[158,469,470],{"class":192},"!r",[158,472,473],{"class":463},"}",[158,475,476],{"class":167},"\"",[158,478,479],{"class":196},")\n",[158,481,482,485],{"class":54,"line":295},[158,483,484],{"class":192},"        return",[158,486,487],{"class":196}," problems\n",[158,489,490],{"class":54,"line":301},[158,491,223],{"emptyLinePlaceholder":222},[158,493,494,497,499],{"class":54,"line":340},[158,495,496],{"class":196},"    ordered ",[158,498,232],{"class":192},[158,500,501],{"class":196}," df[order_column]\n",[158,503,504,506,509],{"class":54,"line":371},[158,505,437],{"class":192},[158,507,508],{"class":192}," not",[158,510,511],{"class":196}," ordered.is_monotonic_increasing:\n",[158,513,515,517,519,521,523,525,527,530],{"class":54,"line":514},12,[158,516,454],{"class":196},[158,518,457],{"class":192},[158,520,476],{"class":167},[158,522,464],{"class":463},[158,524,467],{"class":196},[158,526,473],{"class":463},[158,528,529],{"class":167}," is not sorted — sort before interpolating\"",[158,531,479],{"class":196},[158,533,535,537],{"class":54,"line":534},13,[158,536,437],{"class":192},[158,538,539],{"class":196}," ordered.duplicated().any():\n",[158,541,543,545,547,549,551,553,555,558],{"class":54,"line":542},14,[158,544,454],{"class":196},[158,546,457],{"class":192},[158,548,476],{"class":167},[158,550,464],{"class":463},[158,552,467],{"class":196},[158,554,473],{"class":463},[158,556,557],{"class":167}," has duplicates — the order is ambiguous\"",[158,559,479],{"class":196},[158,561,563,566],{"class":54,"line":562},15,[158,564,565],{"class":192},"    return",[158,567,487],{"class":196},[158,569,571],{"class":54,"line":570},16,[158,572,223],{"emptyLinePlaceholder":222},[158,574,576,579,582,585],{"class":54,"line":575},17,[158,577,578],{"class":319},"print",[158,580,581],{"class":196},"(interpolation_is_sensible(readings, ",[158,583,584],{"class":167},"\"date\"",[158,586,587],{"class":196},"))\n",[10,589,590,591,594,595,599],{},"Two situations rule it out entirely. A gap that means ",[383,592,593],{},"zero rather than unknown"," — no sales because the shop was closed — should be filled with zero, not estimated from the days either side. And a table keyed by category rather than by a continuous axis has no meaningful \"between\", so interpolating a missing revenue from the alphabetically neighbouring region is nonsense dressed as an estimate. When either applies, the fill strategies in ",[14,596,598],{"href":597},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Ffill-missing-values-in-excel-with-pandas-fillna\u002F","filling missing values with pandas fillna"," are the right tool instead.",[143,601,603],{"id":602},"step-2-interpolate-with-the-arguments-that-matter","Step 2 — Interpolate, with the arguments that matter",[10,605,606],{},"The bare call does more than most people want:",[148,608,610],{"className":183,"code":609,"language":185,"meta":153,"style":153},"readings[\"temperature\"].interpolate()      # also extends past the last value\n",[155,611,612],{"__ignoreMap":153},[158,613,614,617,620,623],{"class":54,"line":160},[158,615,616],{"class":196},"readings[",[158,618,619],{"class":167},"\"temperature\"",[158,621,622],{"class":196},"].interpolate()      ",[158,624,626],{"class":625},"s-wDw","# also extends past the last value\n",[10,628,629],{},"Three arguments make it behave:",[148,631,633],{"className":183,"code":632,"language":185,"meta":153,"style":153},"import pandas as pd\n\nseries = readings.set_index(\"date\")[\"temperature\"]\n\nfilled = series.interpolate(\n    method=\"time\",            # respect the actual date spacing\n    limit=3,                  # bridge at most three consecutive gaps\n    limit_area=\"inside\",      # only between two real observations\n)\nprint(filled)\n",[155,634,635,645,649,669,673,683,700,716,732,736],{"__ignoreMap":153},[158,636,637,639,641,643],{"class":54,"line":160},[158,638,193],{"class":192},[158,640,211],{"class":196},[158,642,200],{"class":192},[158,644,216],{"class":196},[158,646,647],{"class":54,"line":206},[158,648,223],{"emptyLinePlaceholder":222},[158,650,651,654,656,659,661,664,666],{"class":54,"line":219},[158,652,653],{"class":196},"series ",[158,655,232],{"class":192},[158,657,658],{"class":196}," readings.set_index(",[158,660,584],{"class":167},[158,662,663],{"class":196},")[",[158,665,619],{"class":167},[158,667,668],{"class":196},"]\n",[158,670,671],{"class":54,"line":226},[158,672,223],{"emptyLinePlaceholder":222},[158,674,675,678,680],{"class":54,"line":238},[158,676,677],{"class":196},"filled ",[158,679,232],{"class":192},[158,681,682],{"class":196}," series.interpolate(\n",[158,684,685,689,691,694,697],{"class":54,"line":247},[158,686,688],{"class":687},"sa561","    method",[158,690,232],{"class":192},[158,692,693],{"class":167},"\"time\"",[158,695,696],{"class":196},",            ",[158,698,699],{"class":625},"# respect the actual date spacing\n",[158,701,702,705,707,710,713],{"class":54,"line":272},[158,703,704],{"class":687},"    limit",[158,706,232],{"class":192},[158,708,709],{"class":319},"3",[158,711,712],{"class":196},",                  ",[158,714,715],{"class":625},"# bridge at most three consecutive gaps\n",[158,717,718,721,723,726,729],{"class":54,"line":295},[158,719,720],{"class":687},"    limit_area",[158,722,232],{"class":192},[158,724,725],{"class":167},"\"inside\"",[158,727,728],{"class":196},",      ",[158,730,731],{"class":625},"# only between two real observations\n",[158,733,734],{"class":54,"line":301},[158,735,479],{"class":196},[158,737,738,740],{"class":54,"line":340},[158,739,578],{"class":319},[158,741,742],{"class":196},"(filled)\n",[744,745,746,759],"table",{},[747,748,749],"thead",{},[750,751,752,756],"tr",{},[753,754,755],"th",{},"Argument",[753,757,758],{},"Why it matters",[760,761,762,773,783],"tbody",{},[750,763,764,770],{},[765,766,767],"td",{},[155,768,769],{},"method=\"time\"",[765,771,772],{},"Treats a seven-day gap as seven days, not one row",[750,774,775,780],{},[765,776,777],{},[155,778,779],{},"limit=3",[765,781,782],{},"Refuses to bridge a gap longer than you trust",[750,784,785,790],{},[765,786,787],{},[155,788,789],{},"limit_area=\"inside\"",[765,791,792],{},"Never extrapolates past the first or last observation",[10,794,795,797,798,801],{},[155,796,789],{}," is the one to set by default. Without it, ",[155,799,800],{},"interpolate"," happily extends the last observed value forward — which is extrapolation, and it produces a report where the final rows look like measurements and are not.",[10,803,804,805,807,808,812],{},"The difference ",[155,806,769],{}," makes is easy to underestimate. With linear interpolation the missing value between 5 August and 12 August lands halfway in ",[809,810,811],"em",{},"row"," terms; with time interpolation it lands proportionally along the seven-day gap:",[148,814,816],{"className":183,"code":815,"language":185,"meta":153,"style":153},"import pandas as pd\n\nfor method in (\"linear\", \"time\"):\n    result = series.interpolate(method=method, limit_area=\"inside\")\n    print(f\"{method:\u003C8} {result.round(2).tolist()}\")\n",[155,817,818,828,832,856,883],{"__ignoreMap":153},[158,819,820,822,824,826],{"class":54,"line":160},[158,821,193],{"class":192},[158,823,211],{"class":196},[158,825,200],{"class":192},[158,827,216],{"class":196},[158,829,830],{"class":54,"line":206},[158,831,223],{"emptyLinePlaceholder":222},[158,833,834,837,840,843,846,849,851,853],{"class":54,"line":219},[158,835,836],{"class":192},"for",[158,838,839],{"class":196}," method ",[158,841,842],{"class":192},"in",[158,844,845],{"class":196}," (",[158,847,848],{"class":167},"\"linear\"",[158,850,253],{"class":196},[158,852,693],{"class":167},[158,854,855],{"class":196},"):\n",[158,857,858,861,863,866,869,871,874,877,879,881],{"class":54,"line":226},[158,859,860],{"class":196},"    result ",[158,862,232],{"class":192},[158,864,865],{"class":196}," series.interpolate(",[158,867,868],{"class":687},"method",[158,870,232],{"class":192},[158,872,873],{"class":196},"method, ",[158,875,876],{"class":687},"limit_area",[158,878,232],{"class":192},[158,880,725],{"class":167},[158,882,479],{"class":196},[158,884,885,888,891,893,895,897,899,902,904,907,910,913,916,918,920],{"class":54,"line":238},[158,886,887],{"class":319},"    print",[158,889,890],{"class":196},"(",[158,892,457],{"class":192},[158,894,476],{"class":167},[158,896,464],{"class":463},[158,898,868],{"class":196},[158,900,901],{"class":192},":\u003C8",[158,903,473],{"class":463},[158,905,906],{"class":463}," {",[158,908,909],{"class":196},"result.round(",[158,911,912],{"class":319},"2",[158,914,915],{"class":196},").tolist()",[158,917,473],{"class":463},[158,919,476],{"class":167},[158,921,479],{"class":196},[10,923,924],{},"On an evenly spaced series they agree. On any irregular one — which is every real operational series, because of weekends and outages — they do not.",[143,926,928],{"id":927},"step-3-interpolate-within-groups","Step 3 — Interpolate within groups",[10,930,931],{},"Interpolating a stacked frame across group boundaries is the most common way to get nonsense: the last North reading and the first South reading are unrelated, and a straight line between them is meaningless.",[20,933,30,939,30,942,30,945,30,948,30,955,30,961,30,967,30,972,30,975,30,980,30,983,30,987,30,991,30,996,30,1001,30,1004,30,1007,30,1009,30,1013,30,1015,30,1018],{"viewBox":934,"role":23,"ariaLabel":935,"ariaLabelledBy":936,"xmlns":28,"style":29},"0 0 800 234","Interpolating a stacked frame without grouping bridges the boundary between two regions, while grouping first keeps each region's series independent.",[937,938],"grp2-t","grp2-d",[32,940,941],{"id":937},"Why a stacked frame must be grouped before interpolating",[36,943,944],{"id":938},"Two treatments of a frame holding North rows followed by South rows. Interpolating the whole column draws a line from the last North observation to the first South observation, filling the boundary gap with a value derived from two unrelated series. Grouping by region first treats each block independently, so the boundary is never bridged and a gap at the start of the South block correctly stays empty.",[40,946],{"x":42,"y":42,"width":43,"height":947,"fill":45},"234",[40,949],{"x":950,"y":951,"width":952,"height":953,"rx":950,"fill":954,"stroke":117,"style":91},"14","24","368","192","#fee8f2",[47,956,960],{"x":957,"y":958,"style":959},"198","52","font-size:12.5px;font-weight:700;fill:var(--accent-ink,#be185d);text-anchor:middle","interpolate the whole column",[40,962],{"x":63,"y":963,"width":964,"height":965,"rx":80,"fill":966,"stroke":69},"68","304","34","#ebebfd",[47,968,971],{"x":957,"y":969,"style":970},"90","font-size:10.5px;fill:var(--text,#172033);text-anchor:middle","North · last observed 21.6",[40,973],{"x":63,"y":974,"width":964,"height":965,"rx":80,"fill":954,"stroke":117,"style":91},"108",[47,976,979],{"x":957,"y":977,"style":978},"130","font-size:10.5px;font-weight:700;fill:var(--accent-ink,#be185d);text-anchor:middle","gap filled from ACROSS the boundary",[40,981],{"x":63,"y":982,"width":964,"height":965,"rx":80,"fill":966,"stroke":69},"148",[47,984,986],{"x":957,"y":985,"style":970},"170","South · first observed 15.4",[47,988,990],{"x":957,"y":989,"style":978},"204","two unrelated series joined by a line",[40,992],{"x":993,"y":951,"width":952,"height":953,"rx":950,"fill":994,"stroke":995,"style":91},"418","#d9f4f1","var(--teal,#0f9488)",[47,997,1000],{"x":998,"y":958,"style":999},"602","font-size:12.5px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","groupby(\"region\").interpolate()",[40,1002],{"x":1003,"y":963,"width":964,"height":965,"rx":80,"fill":994,"stroke":995},"450",[47,1005,1006],{"x":998,"y":969,"style":970},"North · filled from North only",[40,1008],{"x":1003,"y":974,"width":964,"height":965,"rx":80,"fill":45,"stroke":995,"style":91},[47,1010,1012],{"x":998,"y":977,"style":1011},"font-size:10.5px;font-weight:700;fill:var(--teal-ink,#0b6157);text-anchor:middle","boundary never bridged",[40,1014],{"x":1003,"y":982,"width":964,"height":965,"rx":80,"fill":994,"stroke":995},[47,1016,1017],{"x":998,"y":985,"style":970},"South · filled from South only",[47,1019,1020],{"x":998,"y":989,"style":1011},"each series stands on its own",[148,1022,1024],{"className":183,"code":1023,"language":185,"meta":153,"style":153},"import pandas as pd\n\ndef interpolate_by_group(df, value, group, order, **kwargs):\n    \"\"\"Interpolate a column independently within each group.\"\"\"\n    out = df.sort_values([group, order]).copy()\n\n    out[value] = (\n        out.set_index(order)\n           .groupby(group)[value]\n           .transform(lambda s: s.interpolate(\n               method=\"time\", limit_area=\"inside\", **kwargs))\n           .to_numpy()\n    )\n    return out\n\nfilled = interpolate_by_group(readings, \"temperature\", \"region\", \"date\", limit=3)\nprint(filled)\n",[155,1025,1026,1036,1040,1056,1061,1071,1075,1085,1090,1095,1106,1130,1135,1140,1147,1151,1182],{"__ignoreMap":153},[158,1027,1028,1030,1032,1034],{"class":54,"line":160},[158,1029,193],{"class":192},[158,1031,211],{"class":196},[158,1033,200],{"class":192},[158,1035,216],{"class":196},[158,1037,1038],{"class":54,"line":206},[158,1039,223],{"emptyLinePlaceholder":222},[158,1041,1042,1044,1047,1050,1053],{"class":54,"line":219},[158,1043,410],{"class":192},[158,1045,1046],{"class":413}," interpolate_by_group",[158,1048,1049],{"class":196},"(df, value, group, order, ",[158,1051,1052],{"class":192},"**",[158,1054,1055],{"class":196},"kwargs):\n",[158,1057,1058],{"class":54,"line":226},[158,1059,1060],{"class":167},"    \"\"\"Interpolate a column independently within each group.\"\"\"\n",[158,1062,1063,1066,1068],{"class":54,"line":238},[158,1064,1065],{"class":196},"    out ",[158,1067,232],{"class":192},[158,1069,1070],{"class":196}," df.sort_values([group, order]).copy()\n",[158,1072,1073],{"class":54,"line":247},[158,1074,223],{"emptyLinePlaceholder":222},[158,1076,1077,1080,1082],{"class":54,"line":272},[158,1078,1079],{"class":196},"    out[value] ",[158,1081,232],{"class":192},[158,1083,1084],{"class":196}," (\n",[158,1086,1087],{"class":54,"line":295},[158,1088,1089],{"class":196},"        out.set_index(order)\n",[158,1091,1092],{"class":54,"line":301},[158,1093,1094],{"class":196},"           .groupby(group)[value]\n",[158,1096,1097,1100,1103],{"class":54,"line":340},[158,1098,1099],{"class":196},"           .transform(",[158,1101,1102],{"class":192},"lambda",[158,1104,1105],{"class":196}," s: s.interpolate(\n",[158,1107,1108,1111,1113,1115,1117,1119,1121,1123,1125,1127],{"class":54,"line":371},[158,1109,1110],{"class":687},"               method",[158,1112,232],{"class":192},[158,1114,693],{"class":167},[158,1116,253],{"class":196},[158,1118,876],{"class":687},[158,1120,232],{"class":192},[158,1122,725],{"class":167},[158,1124,253],{"class":196},[158,1126,1052],{"class":192},[158,1128,1129],{"class":196},"kwargs))\n",[158,1131,1132],{"class":54,"line":514},[158,1133,1134],{"class":196},"           .to_numpy()\n",[158,1136,1137],{"class":54,"line":534},[158,1138,1139],{"class":196},"    )\n",[158,1141,1142,1144],{"class":54,"line":542},[158,1143,565],{"class":192},[158,1145,1146],{"class":196}," out\n",[158,1148,1149],{"class":54,"line":562},[158,1150,223],{"emptyLinePlaceholder":222},[158,1152,1153,1155,1157,1160,1162,1164,1167,1169,1171,1173,1176,1178,1180],{"class":54,"line":570},[158,1154,677],{"class":196},[158,1156,232],{"class":192},[158,1158,1159],{"class":196}," interpolate_by_group(readings, ",[158,1161,619],{"class":167},[158,1163,253],{"class":196},[158,1165,1166],{"class":167},"\"region\"",[158,1168,253],{"class":196},[158,1170,584],{"class":167},[158,1172,253],{"class":196},[158,1174,1175],{"class":687},"limit",[158,1177,232],{"class":192},[158,1179,709],{"class":319},[158,1181,479],{"class":196},[158,1183,1184,1186],{"class":54,"line":575},[158,1185,578],{"class":319},[158,1187,742],{"class":196},[10,1189,1190,1191,1193],{},"Sorting inside the function is deliberate. ",[155,1192,800],{}," walks the frame in its current row order, so an unsorted frame produces values that depend on how the rows happened to arrive — which is a bug that only appears when the source ordering changes.",[143,1195,1197],{"id":1196},"step-4-flag-what-was-estimated","Step 4 — Flag what was estimated",[10,1199,1200],{},"An interpolated value in a spreadsheet is indistinguishable from a measured one, and that is a problem the moment somebody acts on it. Capture the mask before filling:",[148,1202,1204],{"className":183,"code":1203,"language":185,"meta":153,"style":153},"import pandas as pd\n\ndef interpolate_and_flag(df, value, order, **kwargs):\n    \"\"\"Interpolate, recording which cells were estimated.\"\"\"\n    out = df.sort_values(order).copy()\n    was_missing = out[value].isna()\n\n    out[value] = (\n        out.set_index(order)[value]\n           .interpolate(method=\"time\", limit_area=\"inside\", **kwargs)\n           .to_numpy()\n    )\n\n    out[f\"{value}_estimated\"] = was_missing & out[value].notna()\n    out[f\"{value}_still_missing\"] = out[value].isna()\n    return out\n\nresult = interpolate_and_flag(readings, \"temperature\", \"date\", limit=3)\nprint(result[[\"date\", \"temperature\", \"temperature_estimated\"]])\n",[155,1205,1206,1216,1220,1234,1239,1248,1258,1262,1270,1275,1301,1305,1309,1313,1345,1368,1374,1378,1405],{"__ignoreMap":153},[158,1207,1208,1210,1212,1214],{"class":54,"line":160},[158,1209,193],{"class":192},[158,1211,211],{"class":196},[158,1213,200],{"class":192},[158,1215,216],{"class":196},[158,1217,1218],{"class":54,"line":206},[158,1219,223],{"emptyLinePlaceholder":222},[158,1221,1222,1224,1227,1230,1232],{"class":54,"line":219},[158,1223,410],{"class":192},[158,1225,1226],{"class":413}," interpolate_and_flag",[158,1228,1229],{"class":196},"(df, value, order, ",[158,1231,1052],{"class":192},[158,1233,1055],{"class":196},[158,1235,1236],{"class":54,"line":226},[158,1237,1238],{"class":167},"    \"\"\"Interpolate, recording which cells were estimated.\"\"\"\n",[158,1240,1241,1243,1245],{"class":54,"line":238},[158,1242,1065],{"class":196},[158,1244,232],{"class":192},[158,1246,1247],{"class":196}," df.sort_values(order).copy()\n",[158,1249,1250,1253,1255],{"class":54,"line":247},[158,1251,1252],{"class":196},"    was_missing ",[158,1254,232],{"class":192},[158,1256,1257],{"class":196}," out[value].isna()\n",[158,1259,1260],{"class":54,"line":272},[158,1261,223],{"emptyLinePlaceholder":222},[158,1263,1264,1266,1268],{"class":54,"line":295},[158,1265,1079],{"class":196},[158,1267,232],{"class":192},[158,1269,1084],{"class":196},[158,1271,1272],{"class":54,"line":301},[158,1273,1274],{"class":196},"        out.set_index(order)[value]\n",[158,1276,1277,1280,1282,1284,1286,1288,1290,1292,1294,1296,1298],{"class":54,"line":340},[158,1278,1279],{"class":196},"           .interpolate(",[158,1281,868],{"class":687},[158,1283,232],{"class":192},[158,1285,693],{"class":167},[158,1287,253],{"class":196},[158,1289,876],{"class":687},[158,1291,232],{"class":192},[158,1293,725],{"class":167},[158,1295,253],{"class":196},[158,1297,1052],{"class":192},[158,1299,1300],{"class":196},"kwargs)\n",[158,1302,1303],{"class":54,"line":371},[158,1304,1134],{"class":196},[158,1306,1307],{"class":54,"line":514},[158,1308,1139],{"class":196},[158,1310,1311],{"class":54,"line":534},[158,1312,223],{"emptyLinePlaceholder":222},[158,1314,1315,1318,1320,1322,1324,1327,1329,1332,1334,1336,1339,1342],{"class":54,"line":542},[158,1316,1317],{"class":196},"    out[",[158,1319,457],{"class":192},[158,1321,476],{"class":167},[158,1323,464],{"class":463},[158,1325,1326],{"class":196},"value",[158,1328,473],{"class":463},[158,1330,1331],{"class":167},"_estimated\"",[158,1333,313],{"class":196},[158,1335,232],{"class":192},[158,1337,1338],{"class":196}," was_missing ",[158,1340,1341],{"class":192},"&",[158,1343,1344],{"class":196}," out[value].notna()\n",[158,1346,1347,1349,1351,1353,1355,1357,1359,1362,1364,1366],{"class":54,"line":562},[158,1348,1317],{"class":196},[158,1350,457],{"class":192},[158,1352,476],{"class":167},[158,1354,464],{"class":463},[158,1356,1326],{"class":196},[158,1358,473],{"class":463},[158,1360,1361],{"class":167},"_still_missing\"",[158,1363,313],{"class":196},[158,1365,232],{"class":192},[158,1367,1257],{"class":196},[158,1369,1370,1372],{"class":54,"line":570},[158,1371,565],{"class":192},[158,1373,1146],{"class":196},[158,1375,1376],{"class":54,"line":575},[158,1377,223],{"emptyLinePlaceholder":222},[158,1379,1381,1384,1386,1389,1391,1393,1395,1397,1399,1401,1403],{"class":54,"line":1380},18,[158,1382,1383],{"class":196},"result ",[158,1385,232],{"class":192},[158,1387,1388],{"class":196}," interpolate_and_flag(readings, ",[158,1390,619],{"class":167},[158,1392,253],{"class":196},[158,1394,584],{"class":167},[158,1396,253],{"class":196},[158,1398,1175],{"class":687},[158,1400,232],{"class":192},[158,1402,709],{"class":319},[158,1404,479],{"class":196},[158,1406,1408,1410,1413,1415,1417,1419,1421,1424],{"class":54,"line":1407},19,[158,1409,578],{"class":319},[158,1411,1412],{"class":196},"(result[[",[158,1414,584],{"class":167},[158,1416,253],{"class":196},[158,1418,619],{"class":167},[158,1420,253],{"class":196},[158,1422,1423],{"class":167},"\"temperature_estimated\"",[158,1425,1426],{"class":196},"]])\n",[10,1428,1429],{},"Then show it in the workbook, so the flag is visible rather than buried in a column nobody reads:",[148,1431,1433],{"className":183,"code":1432,"language":185,"meta":153,"style":153},"import pandas as pd\n\ndef write_with_estimates(df, path, value=\"temperature\", sheet_name=\"Readings\"):\n    \"\"\"Write the series, tinting the cells that were interpolated.\"\"\"\n    flag = f\"{value}_estimated\"\n    visible = df.drop(columns=[c for c in df.columns if c.endswith(\"_still_missing\")])\n\n    with pd.ExcelWriter(path, engine=\"xlsxwriter\",\n                        date_format=\"yyyy-mm-dd\") as writer:\n        visible.to_excel(writer, sheet_name=sheet_name, index=False)\n        book, sheet = writer.book, writer.sheets[sheet_name]\n\n        estimated = book.add_format({\n            \"bg_color\": \"#FDEFD8\", \"italic\": True, \"num_format\": \"0.0\",\n        })\n        column = list(visible.columns).index(value)\n\n        for offset, is_estimate in enumerate(df[flag], start=1):\n            if is_estimate:\n                sheet.write_number(offset, column,\n                                   float(df[value].iloc[offset - 1]), estimated)\n\n        sheet.write(len(visible) + 2, 0,\n                    \"Shaded, italic values were interpolated from neighbouring \"\n                    \"observations and are estimates, not measurements.\")\n        sheet.set_column(\"A:A\", 13)\n        sheet.set_column(\"B:D\", 16)\n        sheet.freeze_panes(1, 0)\n\nwrite_with_estimates(result, \"readings.xlsx\")\n",[155,1434,1435,1445,1449,1473,1478,1499,1539,1543,1561,1579,1602,1612,1616,1626,1659,1664,1677,1681,1707,1715,1721,1739,1744,1768,1774,1782,1798,1813,1827,1832],{"__ignoreMap":153},[158,1436,1437,1439,1441,1443],{"class":54,"line":160},[158,1438,193],{"class":192},[158,1440,211],{"class":196},[158,1442,200],{"class":192},[158,1444,216],{"class":196},[158,1446,1447],{"class":54,"line":206},[158,1448,223],{"emptyLinePlaceholder":222},[158,1450,1451,1453,1456,1459,1461,1463,1466,1468,1471],{"class":54,"line":219},[158,1452,410],{"class":192},[158,1454,1455],{"class":413}," write_with_estimates",[158,1457,1458],{"class":196},"(df, path, value",[158,1460,232],{"class":192},[158,1462,619],{"class":167},[158,1464,1465],{"class":196},", sheet_name",[158,1467,232],{"class":192},[158,1469,1470],{"class":167},"\"Readings\"",[158,1472,855],{"class":196},[158,1474,1475],{"class":54,"line":226},[158,1476,1477],{"class":167},"    \"\"\"Write the series, tinting the cells that were interpolated.\"\"\"\n",[158,1479,1480,1483,1485,1488,1490,1492,1494,1496],{"class":54,"line":238},[158,1481,1482],{"class":196},"    flag ",[158,1484,232],{"class":192},[158,1486,1487],{"class":192}," f",[158,1489,476],{"class":167},[158,1491,464],{"class":463},[158,1493,1326],{"class":196},[158,1495,473],{"class":463},[158,1497,1498],{"class":167},"_estimated\"\n",[158,1500,1501,1504,1506,1509,1512,1514,1517,1519,1522,1524,1527,1530,1533,1536],{"class":54,"line":247},[158,1502,1503],{"class":196},"    visible ",[158,1505,232],{"class":192},[158,1507,1508],{"class":196}," df.drop(",[158,1510,1511],{"class":687},"columns",[158,1513,232],{"class":192},[158,1515,1516],{"class":196},"[c ",[158,1518,836],{"class":192},[158,1520,1521],{"class":196}," c ",[158,1523,842],{"class":192},[158,1525,1526],{"class":196}," df.columns ",[158,1528,1529],{"class":192},"if",[158,1531,1532],{"class":196}," c.endswith(",[158,1534,1535],{"class":167},"\"_still_missing\"",[158,1537,1538],{"class":196},")])\n",[158,1540,1541],{"class":54,"line":272},[158,1542,223],{"emptyLinePlaceholder":222},[158,1544,1545,1548,1551,1554,1556,1559],{"class":54,"line":295},[158,1546,1547],{"class":192},"    with",[158,1549,1550],{"class":196}," pd.ExcelWriter(path, ",[158,1552,1553],{"class":687},"engine",[158,1555,232],{"class":192},[158,1557,1558],{"class":167},"\"xlsxwriter\"",[158,1560,269],{"class":196},[158,1562,1563,1566,1568,1571,1574,1576],{"class":54,"line":301},[158,1564,1565],{"class":687},"                        date_format",[158,1567,232],{"class":192},[158,1569,1570],{"class":167},"\"yyyy-mm-dd\"",[158,1572,1573],{"class":196},") ",[158,1575,200],{"class":192},[158,1577,1578],{"class":196}," writer:\n",[158,1580,1581,1584,1587,1589,1592,1595,1597,1600],{"class":54,"line":340},[158,1582,1583],{"class":196},"        visible.to_excel(writer, ",[158,1585,1586],{"class":687},"sheet_name",[158,1588,232],{"class":192},[158,1590,1591],{"class":196},"sheet_name, ",[158,1593,1594],{"class":687},"index",[158,1596,232],{"class":192},[158,1598,1599],{"class":319},"False",[158,1601,479],{"class":196},[158,1603,1604,1607,1609],{"class":54,"line":371},[158,1605,1606],{"class":196},"        book, sheet ",[158,1608,232],{"class":192},[158,1610,1611],{"class":196}," writer.book, writer.sheets[sheet_name]\n",[158,1613,1614],{"class":54,"line":514},[158,1615,223],{"emptyLinePlaceholder":222},[158,1617,1618,1621,1623],{"class":54,"line":534},[158,1619,1620],{"class":196},"        estimated ",[158,1622,232],{"class":192},[158,1624,1625],{"class":196}," book.add_format({\n",[158,1627,1628,1631,1634,1637,1639,1642,1644,1647,1649,1652,1654,1657],{"class":54,"line":542},[158,1629,1630],{"class":167},"            \"bg_color\"",[158,1632,1633],{"class":196},": ",[158,1635,1636],{"class":167},"\"#FDEFD8\"",[158,1638,253],{"class":196},[158,1640,1641],{"class":167},"\"italic\"",[158,1643,1633],{"class":196},[158,1645,1646],{"class":319},"True",[158,1648,253],{"class":196},[158,1650,1651],{"class":167},"\"num_format\"",[158,1653,1633],{"class":196},[158,1655,1656],{"class":167},"\"0.0\"",[158,1658,269],{"class":196},[158,1660,1661],{"class":54,"line":562},[158,1662,1663],{"class":196},"        })\n",[158,1665,1666,1669,1671,1674],{"class":54,"line":570},[158,1667,1668],{"class":196},"        column ",[158,1670,232],{"class":192},[158,1672,1673],{"class":319}," list",[158,1675,1676],{"class":196},"(visible.columns).index(value)\n",[158,1678,1679],{"class":54,"line":575},[158,1680,223],{"emptyLinePlaceholder":222},[158,1682,1683,1686,1689,1691,1694,1697,1700,1702,1705],{"class":54,"line":1380},[158,1684,1685],{"class":192},"        for",[158,1687,1688],{"class":196}," offset, is_estimate ",[158,1690,842],{"class":192},[158,1692,1693],{"class":319}," enumerate",[158,1695,1696],{"class":196},"(df[flag], ",[158,1698,1699],{"class":687},"start",[158,1701,232],{"class":192},[158,1703,1704],{"class":319},"1",[158,1706,855],{"class":196},[158,1708,1709,1712],{"class":54,"line":1407},[158,1710,1711],{"class":192},"            if",[158,1713,1714],{"class":196}," is_estimate:\n",[158,1716,1718],{"class":54,"line":1717},20,[158,1719,1720],{"class":196},"                sheet.write_number(offset, column,\n",[158,1722,1724,1727,1730,1733,1736],{"class":54,"line":1723},21,[158,1725,1726],{"class":319},"                                   float",[158,1728,1729],{"class":196},"(df[value].iloc[offset ",[158,1731,1732],{"class":192},"-",[158,1734,1735],{"class":319}," 1",[158,1737,1738],{"class":196},"]), estimated)\n",[158,1740,1742],{"class":54,"line":1741},22,[158,1743,223],{"emptyLinePlaceholder":222},[158,1745,1747,1750,1753,1756,1759,1762,1764,1766],{"class":54,"line":1746},23,[158,1748,1749],{"class":196},"        sheet.write(",[158,1751,1752],{"class":319},"len",[158,1754,1755],{"class":196},"(visible) ",[158,1757,1758],{"class":192},"+",[158,1760,1761],{"class":319}," 2",[158,1763,253],{"class":196},[158,1765,42],{"class":319},[158,1767,269],{"class":196},[158,1769,1771],{"class":54,"line":1770},24,[158,1772,1773],{"class":167},"                    \"Shaded, italic values were interpolated from neighbouring \"\n",[158,1775,1777,1780],{"class":54,"line":1776},25,[158,1778,1779],{"class":167},"                    \"observations and are estimates, not measurements.\"",[158,1781,479],{"class":196},[158,1783,1785,1788,1791,1793,1796],{"class":54,"line":1784},26,[158,1786,1787],{"class":196},"        sheet.set_column(",[158,1789,1790],{"class":167},"\"A:A\"",[158,1792,253],{"class":196},[158,1794,1795],{"class":319},"13",[158,1797,479],{"class":196},[158,1799,1801,1803,1806,1808,1811],{"class":54,"line":1800},27,[158,1802,1787],{"class":196},[158,1804,1805],{"class":167},"\"B:D\"",[158,1807,253],{"class":196},[158,1809,1810],{"class":319},"16",[158,1812,479],{"class":196},[158,1814,1816,1819,1821,1823,1825],{"class":54,"line":1815},28,[158,1817,1818],{"class":196},"        sheet.freeze_panes(",[158,1820,1704],{"class":319},[158,1822,253],{"class":196},[158,1824,42],{"class":319},[158,1826,479],{"class":196},[158,1828,1830],{"class":54,"line":1829},29,[158,1831,223],{"emptyLinePlaceholder":222},[158,1833,1835,1838,1841],{"class":54,"line":1834},30,[158,1836,1837],{"class":196},"write_with_estimates(result, ",[158,1839,1840],{"class":167},"\"readings.xlsx\"",[158,1842,479],{"class":196},[10,1844,1845,1846,18],{},"A tinted cell plus one sentence of explanation is the whole intervention, and it is what stops an estimate being quoted back as a fact three meetings later. The conditional-formatting alternative is covered in ",[14,1847,1849],{"href":1848},"\u002Fadvanced-data-transformation-and-cleaning\u002Fapplying-conditional-formatting-with-openpyxl\u002Fhighlight-cells-above-a-threshold-with-openpyxl\u002F","highlighting cells above a threshold with openpyxl",[143,1851,1853],{"id":1852},"step-5-choose-a-method","Step 5 — Choose a method",[20,1855,30,1860,30,1863,30,1866,30,1868,30,1873,30,1876,30,1880,30,1884,30,1887,30,1890,30,1894,30,1898,30,1901,30,1903,30,1908,30,1912,30,1915,30,1920,30,1923],{"viewBox":934,"role":23,"ariaLabel":1856,"ariaLabelledBy":1857,"xmlns":28,"style":29},"Linear interpolation stays inside the observed range while a high-order spline overshoots between sparse points, producing filled values that never occurred.",[1858,1859],"meth-t","meth-d",[32,1861,1862],{"id":1858},"Why linear is the conservative choice",[36,1864,1865],{"id":1859},"Four sparse observations plotted twice. The linear path joins them with straight segments, so every filled value lies between its two neighbours and inside the observed range. The spline path curves smoothly through the same points but overshoots between them, dipping below the lowest observation and rising above the highest. Those overshoot values look like plausible measurements and never occurred.",[40,1867],{"x":42,"y":42,"width":43,"height":947,"fill":45},[54,1869],{"x1":1870,"y1":1871,"x2":1872,"y2":1871,"stroke":59,"style":60},"70","180","740",[54,1874],{"x1":1870,"y1":1871,"x2":1870,"y2":1875,"stroke":59,"style":60},"36",[65,1877],{"d":1878,"fill":68,"stroke":995,"style":1879},"M140 140 L 320 68 L 500 128 L 680 56","stroke-width:3px",[65,1881],{"d":1882,"fill":68,"stroke":117,"style":1883},"M140 140 C 200 200, 260 34, 320 68 C 380 100, 440 176, 500 128 C 560 84, 620 8, 680 56","stroke-width:3px;stroke-dasharray:6 4",[76,1885],{"cx":1886,"cy":1886,"r":80,"fill":81},"140",[76,1888],{"cx":1889,"cy":963,"r":80,"fill":81},"320",[76,1891],{"cx":1892,"cy":1893,"r":80,"fill":81},"500","128",[76,1895],{"cx":1896,"cy":1897,"r":80,"fill":81},"680","56",[54,1899],{"x1":1870,"y1":1897,"x2":1872,"y2":1897,"stroke":59,"style":1900},"stroke-width:1px;stroke-dasharray:3 4",[54,1902],{"x1":1870,"y1":1886,"x2":1872,"y2":1886,"stroke":59,"style":1900},[47,1904,1907],{"x":1905,"y":56,"style":1906},"62","font-size:10px;fill:var(--muted,#5b6780);text-anchor:end","max seen",[47,1909,1911],{"x":1905,"y":1910,"style":1906},"144","min seen",[40,1913],{"x":1871,"y":1914,"width":950,"height":950,"rx":709,"fill":90},"200",[47,1916,1919],{"x":1917,"y":1918,"style":126},"202","211","linear — never leaves the observed range",[40,1921],{"x":1892,"y":1914,"width":950,"height":950,"rx":709,"fill":1922},"#f43f8f",[47,1924,1926],{"x":1925,"y":1918,"style":126},"522","spline — overshoots above and below",[10,1928,1929,1932,1933,1935],{},[155,1930,1931],{},"method=\"linear\""," and ",[155,1934,769],{}," cover most reporting needs. The others are worth knowing about but rarely worth reaching for:",[744,1937,1938,1951],{},[747,1939,1940],{},[750,1941,1942,1945,1948],{},[753,1943,1944],{},"Method",[753,1946,1947],{},"Behaviour",[753,1949,1950],{},"Reasonable for",[760,1952,1953,1966,1979,1992,2009,2022],{},[750,1954,1955,1960,1963],{},[765,1956,1957],{},[155,1958,1959],{},"linear",[765,1961,1962],{},"Straight line by row position",[765,1964,1965],{},"Evenly spaced rows",[750,1967,1968,1973,1976],{},[765,1969,1970],{},[155,1971,1972],{},"time",[765,1974,1975],{},"Straight line by datetime distance",[765,1977,1978],{},"Any dated series",[750,1980,1981,1986,1989],{},[765,1982,1983],{},[155,1984,1985],{},"nearest",[765,1987,1988],{},"Copies the closer neighbour",[765,1990,1991],{},"Step-like values such as a rate",[750,1993,1994,2003,2006],{},[765,1995,1996,1999,2000],{},[155,1997,1998],{},"pad"," \u002F ",[155,2001,2002],{},"ffill",[765,2004,2005],{},"Carries the last value forward",[765,2007,2008],{},"A value that holds until changed",[750,2010,2011,2016,2019],{},[765,2012,2013],{},[155,2014,2015],{},"polynomial",[765,2017,2018],{},"Fits a curve of given order",[765,2020,2021],{},"Smooth physical measurements",[750,2023,2024,2029,2032],{},[765,2025,2026],{},[155,2027,2028],{},"spline",[765,2030,2031],{},"Smooth piecewise curve",[765,2033,2034],{},"Genuinely smooth signals",[10,2036,2037],{},"Higher-order methods are where interpolation stops being conservative. A cubic fit through sparse business data can overshoot dramatically between points, producing a \"filled\" value well outside the observed range — plausible-looking, and wrong. For reporting, linear and time are the honest defaults.",[143,2039,2041],{"id":2040},"common-pitfalls-and-fixes","Common pitfalls and fixes",[744,2043,2044,2057],{},[747,2045,2046],{},[750,2047,2048,2051,2054],{},[753,2049,2050],{},"Symptom",[753,2052,2053],{},"Cause",[753,2055,2056],{},"Fix",[760,2058,2059,2071,2088,2104,2115,2126,2137,2153],{},[750,2060,2061,2064,2067],{},[765,2062,2063],{},"Values appear after the last observation",[765,2065,2066],{},"Default extrapolates forward",[765,2068,2069,18],{},[155,2070,789],{},[750,2072,2073,2076,2082],{},[765,2074,2075],{},"A six-month gap filled with a straight line",[765,2077,2078,2079,2081],{},"No ",[155,2080,1175],{}," set",[765,2083,2084,2085,2087],{},"Set ",[155,2086,1175],{}," to the largest gap you trust.",[750,2089,2090,2093,2098],{},[765,2091,2092],{},"Irregular gaps filled evenly",[765,2094,2095,2097],{},[155,2096,1931],{}," on a dated series",[765,2099,2100,2101,2103],{},"Use ",[155,2102,769],{}," with a datetime index.",[750,2105,2106,2109,2112],{},[765,2107,2108],{},"Values differ between runs",[765,2110,2111],{},"Frame not sorted before interpolating",[765,2113,2114],{},"Sort by the ordering column first.",[750,2116,2117,2120,2123],{},[765,2118,2119],{},"Nonsense at group boundaries",[765,2121,2122],{},"Interpolated across the whole column",[765,2124,2125],{},"Group first, then interpolate.",[750,2127,2128,2131,2134],{},[765,2129,2130],{},"Filled values outside the observed range",[765,2132,2133],{},"High-order polynomial or spline",[765,2135,2136],{},"Use linear or time.",[750,2138,2139,2147,2150],{},[765,2140,2141,2144,2145],{},[155,2142,2143],{},"TypeError"," on ",[155,2146,769],{},[765,2148,2149],{},"Index is not datetime",[765,2151,2152],{},"Set the date column as the index.",[750,2154,2155,2158,2161],{},[765,2156,2157],{},"Readers quote estimates as measurements",[765,2159,2160],{},"Nothing marks them",[765,2162,2163],{},"Flag and tint the interpolated cells.",[143,2165,2167],{"id":2166},"performance-and-scale-notes","Performance and scale notes",[10,2169,2170,2172],{},[155,2171,800],{}," is vectorised and fast on a single series. The expensive shape is the group-wise version, which runs once per group in Python.",[10,2174,2175,2176,2179],{},"For many groups, that overhead dominates. Two mitigations. ",[383,2177,2178],{},"Filter out groups with nothing to fill"," before the loop — most groups in a typical frame are complete:",[148,2181,2183],{"className":183,"code":2182,"language":185,"meta":153,"style":153},"import pandas as pd\n\nhas_gaps = readings.groupby(\"region\")[\"temperature\"].transform(\n    lambda s: s.isna().any()\n)\nsubset = readings.loc[has_gaps]\n",[155,2184,2185,2195,2199,2218,2226,2230],{"__ignoreMap":153},[158,2186,2187,2189,2191,2193],{"class":54,"line":160},[158,2188,193],{"class":192},[158,2190,211],{"class":196},[158,2192,200],{"class":192},[158,2194,216],{"class":196},[158,2196,2197],{"class":54,"line":206},[158,2198,223],{"emptyLinePlaceholder":222},[158,2200,2201,2204,2206,2209,2211,2213,2215],{"class":54,"line":219},[158,2202,2203],{"class":196},"has_gaps ",[158,2205,232],{"class":192},[158,2207,2208],{"class":196}," readings.groupby(",[158,2210,1166],{"class":167},[158,2212,663],{"class":196},[158,2214,619],{"class":167},[158,2216,2217],{"class":196},"].transform(\n",[158,2219,2220,2223],{"class":54,"line":226},[158,2221,2222],{"class":192},"    lambda",[158,2224,2225],{"class":196}," s: s.isna().any()\n",[158,2227,2228],{"class":54,"line":238},[158,2229,479],{"class":196},[158,2231,2232,2235,2237],{"class":54,"line":247},[158,2233,2234],{"class":196},"subset ",[158,2236,232],{"class":192},[158,2238,2239],{"class":196}," readings.loc[has_gaps]\n",[10,2241,2242,2252,2253,2255],{},[383,2243,2244,2245,2248,2249,18],{},"Prefer ",[155,2246,2247],{},"transform"," over ",[155,2250,2251],{},"apply"," ",[155,2254,2247],{}," returns an aligned result without constructing an intermediate frame per group, which on thousands of groups is a substantial saving.",[10,2257,2258,2259,2261,2262,2266],{},"There is also a correctness point that matters more than either at scale. Interpolating chunk by chunk does ",[383,2260,443],{}," work: a gap that straddles a chunk boundary has neighbours in two different chunks, so the fill either fails or uses the wrong values. If a file is too large to hold, interpolate per group after partitioning by the group key — never by arbitrary row ranges — so every series stays whole. The chunked reading approach in ",[14,2263,2265],{"href":2264},"\u002Fadvanced-data-transformation-and-cleaning\u002Fworking-with-large-excel-files-in-python\u002Fread-large-excel-file-in-chunks-with-pandas\u002F","reading large Excel files in chunks"," still applies to getting the data in; the interpolation just has to happen after the rows for a given series are together.",[10,2268,2269],{},"Finally, interpolate once at the point the series is assembled, not repeatedly downstream. Each call rewrites the column, and a value interpolated from already-interpolated neighbours compounds the estimate without anything recording that it did.",[143,2271,2273],{"id":2272},"conclusion","Conclusion",[10,2275,2276,2277,2279,2280,2282,2283,2285],{},"Interpolation is only valid when the rows sit on a meaningful axis, so check that first and reach for a plain fill when they do not. Use ",[155,2278,769],{}," for anything dated, set ",[155,2281,1175],{}," so an enormous gap is never bridged by a straight line, and always pass ",[155,2284,789],{}," so the series is never extended past its last real observation. Group before interpolating a stacked frame, sort before either, and record the mask so every estimated cell can be tinted and labelled in the output. A reader who can see which numbers were estimated will trust the rest more, not less.",[143,2287,2289],{"id":2288},"frequently-asked-questions","Frequently asked questions",[10,2291,2292,2295],{},[383,2293,2294],{},"When is interpolation the wrong choice?","\nWhenever the rows are not ordered along a meaningful axis, or when a gap means the value was genuinely zero rather than unrecorded. Interpolating a category-keyed table invents a value from unrelated neighbours; interpolating a sales series across a closed month invents revenue.",[10,2297,2298,2306],{},[383,2299,2300,2301,1932,2303,2305],{},"What is the difference between ",[155,2302,1931],{},[155,2304,769],{},"?","\nLinear treats rows as evenly spaced regardless of their index, so a one-day gap and a one-year gap are filled identically. Time uses the actual datetime index spacing, which is what you want for any irregular series.",[10,2308,2309,2315,2316,2318],{},[383,2310,2311,2312,2314],{},"Why did ",[155,2313,800],{}," fill values at the end of my series?","\nBy default it extends forward past the last observation. Pass ",[155,2317,789],{}," to fill only gaps that sit between two real values, which is almost always the correct behaviour.",[10,2320,2321,2324,2325,2327,2328,2331],{},[383,2322,2323],{},"How do I stop it bridging an enormous gap?","\nSet ",[155,2326,1175],{}," to the largest number of consecutive gaps you are willing to fill. Anything longer stays ",[155,2329,2330],{},"NaN",", which keeps the honest gap visible rather than drawing a straight line across half a year.",[10,2333,2334,2337],{},[383,2335,2336],{},"Should I mark which values were interpolated?","\nYes. Record the mask before filling and write it into the workbook as a flag column or a cell fill, so a reader can tell an estimate from a measurement.",[143,2339,2341],{"id":2340},"related","Related",[2343,2344,2345,2352,2359,2365,2372],"ul",{},[2346,2347,2348,2349,2351],"li",{},"Up to the parent: ",[14,2350,17],{"href":16}," — the wider set of strategies.",[2346,2353,2354,2358],{},[14,2355,2357],{"href":2356},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Ffind-and-report-missing-values-in-an-excel-file\u002F","Find and Report Missing Values in an Excel File"," — the audit that should precede any fill.",[2346,2360,2361,2364],{},[14,2362,2363],{"href":597},"Fill Missing Values in Excel with pandas fillna"," — the right tool when rows have no meaningful order.",[2346,2366,2367,2371],{},[14,2368,2370],{"href":2369},"\u002Fadvanced-data-transformation-and-cleaning\u002Fworking-with-dates-and-times-in-excel-data\u002Fgroup-excel-rows-by-month-and-quarter-with-pandas\u002F","Group Excel Rows by Month and Quarter with pandas"," — filling the empty periods a summary would otherwise skip.",[2346,2373,2374,2378],{},[14,2375,2377],{"href":2376},"\u002Fformatting-and-charting-excel-reports-with-python\u002Fcreating-charts-in-excel-with-openpyxl\u002Fadd-line-chart-to-excel-report-with-python\u002F","Add a Line Chart to an Excel Report with Python"," — plotting a series where the gaps matter.",[2380,2381,2382],"style",{},"html pre.shiki code .sMTad, html code.shiki .sMTad{--shiki-default:#6F42C1;--shiki-dark:#FFB757}html pre.shiki code .srMev, html code.shiki .srMev{--shiki-default:#032F62;--shiki-dark:#ADDCFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s-kum, html code.shiki .s-kum{--shiki-default:#D73A49;--shiki-dark:#FF9492}html pre.shiki code .skGVy, html code.shiki .skGVy{--shiki-default:#24292E;--shiki-dark:#F0F3F6}html pre.shiki code .sP0c6, html code.shiki .sP0c6{--shiki-default:#005CC5;--shiki-dark:#91CBFF}html pre.shiki code .s_Opv, html code.shiki .s_Opv{--shiki-default:#6F42C1;--shiki-dark:#DBB7FF}html pre.shiki code .sSjpA, html code.shiki .sSjpA{--shiki-default:#005CC5;--shiki-dark:#FF9492}html pre.shiki code .s-wDw, html code.shiki .s-wDw{--shiki-default:#6A737D;--shiki-dark:#BDC4CC}html pre.shiki code .sa561, html code.shiki .sa561{--shiki-default:#E36209;--shiki-dark:#FFB757}",{"title":153,"searchDepth":206,"depth":206,"links":2384},[2385,2386,2387,2388,2389,2390,2391,2392,2393,2394,2395],{"id":145,"depth":206,"text":146},{"id":377,"depth":206,"text":378},{"id":602,"depth":206,"text":603},{"id":927,"depth":206,"text":928},{"id":1196,"depth":206,"text":1197},{"id":1852,"depth":206,"text":1853},{"id":2040,"depth":206,"text":2041},{"id":2166,"depth":206,"text":2167},{"id":2272,"depth":206,"text":2273},{"id":2288,"depth":206,"text":2289},{"id":2340,"depth":206,"text":2341},"2026-08-15","Fill gaps in an ordered series without inventing nonsense — linear and time interpolation in pandas, limits and directions, group-wise fills, and flagging what was estimated.","md",[2400,2402,2405,2408,2410],{"q":2294,"a":2401},"Whenever the rows are not ordered along a meaningful axis, or when a gap means the value was genuinely zero rather than unrecorded. Interpolating a category-keyed table invents a value from unrelated neighbours; interpolating a sales series across a closed month invents revenue.",{"q":2403,"a":2404},"What is the difference between method=\"linear\" and method=\"time\"?","Linear treats rows as evenly spaced regardless of their index, so a one-day gap and a one-year gap are filled identically. Time uses the actual datetime index spacing, which is what you want for any irregular series.",{"q":2406,"a":2407},"Why did interpolate fill values at the end of my series?","By default it extends forward past the last observation. Pass limit_area=\"inside\" to fill only gaps that sit between two real values, which is almost always the correct behaviour.",{"q":2323,"a":2409},"Set limit to the largest number of consecutive gaps you are willing to fill. Anything longer stays NaT or NaN, which keeps the honest gap visible rather than drawing a straight line across half a year.",{"q":2336,"a":2411},"Yes. Record the mask before filling and write it into the workbook as a flag column or a cell fill, so a reader can tell an estimate from a measurement.",{},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Finterpolate-missing-numeric-values-in-excel-data",{"title":2415,"description":2416},"Interpolate Missing Values in Excel Data with pandas","When interpolation is valid and when it is not: DataFrame.interpolate methods, limit and limit_direction, time-aware gaps, per-group interpolation, and marking estimated cells.","interpolate-missing-numeric-values-in-excel-data","advanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Finterpolate-missing-numeric-values-in-excel-data\u002Findex","how-to","4MH89GF4tz9XPH8MFO-fT8ZCuxKYdLt_RvdawkbDvzI",[2422,2425],{"title":2357,"path":2423,"stem":2424,"children":-1},"\u002Fadvanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Ffind-and-report-missing-values-in-an-excel-file","advanced-data-transformation-and-cleaning\u002Fhandling-missing-data-in-excel-reports\u002Ffind-and-report-missing-values-in-an-excel-file\u002Findex",{"title":2426,"path":2427,"stem":2428,"children":-1},"Merging and Joining Excel DataFrames with Pandas","\u002Fadvanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes","advanced-data-transformation-and-cleaning\u002Fmerging-and-joining-excel-dataframes\u002Findex",1786800028562]