Logged behavior changes what can be estimated; pessimism changes the selected policy; a separate holdout reveals the result; every run is deterministic and reproducible.
Menu
Decision Lab 02 · Learning from logged choices
More data cannot reveal what you never offered.
Train an assortment policy on a fixed synthetic customer log. Compare repetitive incumbent data with designed exploration, then test the learned decision in a fresh hidden-world holdout.
Offline learningFixed seedSynthetic MNL world
M2W · Model and simulate → learn and decide → test toward transfer
- 01Logoffered set + final choice
- 02Estimateitem versus no purchase
- 03Protectlower confidence values
- 04Optimizeenumerate the small shelf
- 05Testfresh fixed holdout
Live learning experiment
Change the log. Keep the customer fixed.
Both strategies face the same hidden preference model, prices, capacity, training seed, and 10,000-visitor holdout. Only historical exposure changes.
Assortments in this logbalanced rotation
Pro + Plus25%Pro + Entry25%Plus + Everyday25%Everyday + Entry25%
Capacity 2LCB tuning δ = 0.10Holdout 10,000
What the learner can see
Coverage and uncertainty
ItemOfferedItem vs exitEstimate v̂Lower v−Hidden v*
Pro2502080.490.330.50
Plus2501210.040.000.05
Everyday2502362.061.492.00
Entry2501520.500.310.50
Rank breaking compares each offered item with no purchase. The hidden truth is revealed here only for evaluation; it is never passed to the learner.
Data-volume test
What more rows change
oracle value
204/4 seen
404/4 seen
1004/4 seen
2504/4 seen
1,0004/4 seen
4,0004/4 seen
Exploration fixes coverage immediately; additional rows then tighten pessimistic estimates. A single seeded curve is illustrative, not an average-case guarantee.
Separate evaluation
Train here. Judge elsewhere.
The policy is frozen before the shared 10,000-visitor synthetic holdout is opened. Oracle regret uses exact expectation under the disclosed hidden model.
PolicyAssortmentHoldout valueExact regretRole
Current shelfPro + Plus$40.57$7.12Incumbent
Learned candidatePro + Everyday$48.03$0.00Eligible for evaluation
Hidden oraclePro + Everyday$48.03$0.00Evaluator only
The decision lesson: sample size controls uncertainty; exposure controls identifiability. Repeating the incumbent policy cannot teach the learner how an unseen item performs.
Oracle pair in training 0 timesResearch foundation
Coverage—not clairvoyance—is the hinge.
The learner implements a small, transparent version of the rank-breaking and pessimistic lower-confidence recipe studied by Yuxuan Han, Han Zhong, Miao Lu, José Blanchet, and Zhengyuan Zhou for offline assortment learning under multinomial-logit choice.
The exploratory log covers every item but deliberately never displays the hidden oracle pair. That distinction makes the central idea testable here: useful item coverage can be weaker than observing the complete optimal assortment in historical data. This is a second synthetic MNL world—not an estimate of Prototype 1's Markov transition system.
Read the learning paperWhat this proves—and does not
A working learning loop, inside a disclosed synthetic world.
No Markov-chain recovery, contextual personalization, causal or real-world lift, scalable optimization, theorem reproduction, or safe production exploration is established by this demonstration.
What goes next?