TL;DR: At Recall Space Agents, we solve complex supply chain operational problems with agentic systems powered by frontier models. Our learning loops work both ways: experts guide agents at key decision points, and agents uncover things experts may have missed. As those lessons feed back into the system, agents take on more repetitive work and experts focus on decisions that need their judgment. This example shows why that exchange matters.
I asked Claude Opus 5 in Claude Code, with maximum reasoning effort, to build a simple predictor of heating load. It produced a script with preprocessing, cross-validation, a baseline and two models. It ran successfully, and the results looked good.
For the prediction task I asked for, it worked. But the linear model had a textbook statistical issue, the kind you would flag in a junior analyst's code review. Its coefficients were not uniquely determined: different sets of numbers could give the same predictions. If I tried to use those coefficients to explain what drove heating load, I could end up telling very different stories about the same data.
When I asked it to fix the encoding, it added a check and found a second problem I had not spotted.
An expert's correction should help the agent handle a similar case next time, while the agent's findings can help the expert see a problem differently. In our supply chain work, we want those lessons to carry forward, so routine cases need less human involvement over time.
The example below is deliberately small, and everything you need to try the task is described here. Download the public dataset from Kaggle, open Claude Code and give it the same prompt. The Try it yourself section includes the data setup I used and what to look for in the result.
What the agent built
My prompt was:
use "uv" as package manager and use sklearn to build a basic predictor of "Heating Load", put it on a single scriptI used the Energy Efficiency Dataset: 768 simulated building configurations with eight features describing their geometry, orientation and glazing. The target was heating load.
In the local copy, two features contain text labels. Orientation can be North, East, South or West. Glazing distribution has six categories, including NoGlazing and Uniform. The README tells the reader to one-hot encode both columns.
The agent followed those instructions and compared linear regression and a random forest against a baseline that always predicts the training mean:
model CV RMSE RMSE MAE R2
----------------------------------------------------------
baseline (mean) 10.047 10.238 9.272 -0.006
linear regression 2.827 2.872 2.059 0.921
random forest 0.535 0.533 0.368 0.997The random forest won. The issue I noticed was in the preprocessing shared by the models:
preprocess = ColumnTransformer(
[
("num", StandardScaler(), NUMERIC),
("cat", OneHotEncoder(handle_unknown="ignore"), NOMINAL),
]
)One-hot encoding gives each category its own yes or no column. For orientation, that means four columns, exactly one of which is 1 in each row. Together, they always sum to 1.
The linear regression also has an intercept, represented by a column of ones. That creates a redundancy: you can add the same amount to all four orientation coefficients, subtract it from the intercept, and leave every prediction unchanged. The glazing categories introduce the same problem.
This is the dummy-variable trap. One way to remove it is to drop one category from each group and interpret the remaining coefficients relative to that reference category.
Keeping all the categories does not stop this model from predicting. The problem appears when you try to interpret its individual coefficients. The original script did not print those coefficients; it reported feature importances for the winning random forest. My concern was about using the linear model to explain the result.
The first fix was not enough
I asked the agent to make a separate version:
Okay, I see that the current implementation falls into the dummy-variable trap when using linear regression. Please do not modify predict_heating_load.py. Instead, create a new script that reproduces the same functionality but avoids the dummy-variable trap.The agent changed the encoding. It also checked the rank of the resulting design matrix: whether the columns supplied independent information, including the intercept.
Dropping the reference categories removed two redundant columns. There was still one left:
design matrix columns rank
----------------------------------------------------
original 17 14
drop one level per categorical feature 15 14
also remove surface area 14 14The remaining dependency was in the building geometry:
X2 == X3 + 2 * X4
# surface area == wall area + 2 * roof areaThat relationship holds exactly across all 768 rows. For example, one building has a wall area of 294 and a roof area of 110.25. Its surface area is 514.50: the walls, roof and an equally sized floor. Including all three columns gives the linear model another way to express the same information twice.
I had noticed the encoding issue. I had not checked for this relationship. Once the agent had a rank check, it found the dependency and removed surface area from the linear model's inputs.
The two versions then gave effectively identical linear regression predictions. In a local comparison, the largest difference on the test set was about 8.5e-14, well below any meaningful precision here. Yet the coefficients changed substantially:
Feature Original coefficient After both fixes
---------------------------------------------------
Roof area -3.94 -7.64
Wall area 0.77 -1.02These are coefficients for standardized inputs, not changes per square metre. Wall area changed sign even though the predictions stayed the same.
It is tempting to read that as opposite advice: more wall area raises heating load in one version and lowers it in the other. But neither coefficient, by itself, supports that conclusion. Removing a redundant column changes what the remaining coefficients describe. It does not turn a predictive model into evidence about what would happen if we redesigned a building.
The prediction scores could not tell me whether the coefficients were uniquely determined. A rank check could, and it exposed a problem that another accuracy check would have missed.
What I needed to review
The agent had tested how well the models predicted heating load. That was appropriate for my request. Once I started looking at the linear model as an explanation, I was asking more of it, and the checks needed to reflect that. I needed enough statistics to spot that gap.
This comes up outside teaching datasets. ERP data often contains totals alongside their components: gross weight, net weight and packaging weight, for example, or total lead time alongside transit, handling and inspection time. Those fields look separate in a table, but some carry information already present in the others.
If someone uses a model's coefficients to decide where to hold inventory or which source of delay to address, a good prediction score will not settle whether that decision is justified. You have to check the assumptions behind the explanation as well.
A script can run, produce sensible numbers and satisfy the original request while being unsuitable for the next thing someone wants to do with it.
Consider an agent combining shipments to reduce freight costs. The savings might be calculated correctly, but waiting for the combined load could leave a production line short of parts or miss a customer delivery window. Before changing the plan, the agent needs to check inventory coverage and delivery commitments. Someone who understands the operation has to define those checks and decide which tradeoffs need review.
What we take from this
After I flagged the encoding issue, the agent added a diagnostic and found another problem on its own. The next time I ask for interpretable linear coefficients, that check can be part of the task from the start.
Technical knowledge helps us test whether a result is sound. Supply chain experience helps us judge whether acting on it makes operational sense. Both need to shape the instructions and checks an agent works with, so familiar mistakes are caught automatically and unresolved decisions reach someone who understands their consequences.
Try it yourself
Download the Energy Efficiency Dataset from Kaggle, extract it into a fresh folder and open that folder in Claude Code.
There is one setup detail to preserve: my copy used text labels for the two categorical columns. If your download uses numeric codes, replace them using these mappings. You can ask Claude Code to do this preparation.
Code to label
X6 (Orientation): 2 to North, 3 to East, 4 to South, 5 to West.
X8 (Glazing Area Distribution): 0 to NoGlazing, 1 to Uniform, 2 to North, 3 to East, 4 to South, 5 to West.
Add a README.md in that folder with the same modelling context my agent had:
Y1 is Heating Load. X6 and X8 are nominal: they hold string labels, not numbers, and carry no ordering. One-hot encode them before modelling.Then start a fresh Claude Code session in the same folder and give it my original prompt:
use "uv" as package manager and use sklearn to build a basic predictor of "Heating Load", put it on a single scriptLet it finish before mentioning the dummy-variable trap. If it builds an ordinary linear regression, check whether it keeps every category alongside an intercept, and whether it includes surface area, wall area and roof area together. Also look for a rank check: did it test those assumptions on its own?
If it misses the encoding issue, try the follow-up prompt from earlier in this post, adjusting the script filename if needed. See whether it only changes the encoder or also finds the dependency in the building geometry. Compare the prediction scores before and after.
Your run may produce different code or catch the issue immediately. This is an invitation to test what the agent checks for itself; the mistake described here happened in my session, and is not guaranteed in every run.
Explore more articles on AI, procurement, and supply chain execution on the Recall Space blog and subscribe to our newsletter to receive future insights.
.png)
