The A/B Test That Made An Algorithm Safe To Try

The Framework

Some products ask the buyer to hand over something they're frightened to lose. Pricing. Payroll. Outbound email to their whole list. The demo goes well, everyone nods, and nothing gets signed, because saying yes means betting the thing that keeps them employed.

The usual answers don't fix it. A free trial still requires them to switch. A pilot with no control group produces an argument about whether it worked. A money-back guarantee refunds the cash and not the damage.

Felix Hoffmann sells software that sets prices for retailers. He got past it by never asking for all of it. 7Learnings prices half the assortment while the retailer's own team keeps pricing the other half.

Same period, same catalogue, two methods, one comparison.

The Four Steps

1. Split the scope, not the timeline. Felix: "We are optimizing half of the prices on our own, I mean together with the customer of course. And then they are continuing their their own method on their end." The buyer never stops doing what they already do, so there's no moment where they're exposed.

2. Agree the readout before you start. Both sides are looking at the same comparison, so "did it work" has an answer instead of an opinion. In Felix's case the number that came back was a 13% profit uplift.

3. Charge for it. They took a monthly fee from the start. A paid pilot is a customer running a test. A free one is a prospect running an experiment on you, and it teaches them your product is worth zero.

4. Pick a buyer who can absorb a bad week. The first live run failed. Prices came back far too expensive on high-priced products, and in e-commerce you know inside a day. It survived because of who the customer was, not because of how good the software was.

Real Numbers

First live pricing run: failed. Felix: "I remember we did the first upload, it was a disaster... we saw that it wasn't working at all. We were far too expensive on high priced products."

Feedback loop: one day. "One day later you know if it's working or not, basically. It's very visible."

Later test: 13% profit uplift. "I think it was thirteen percent profit uplift then later on in the next tests."

Founder involvement: total. Felix explained the failure to the buyer himself. "You as a founder have to be quite close to that buyer... you can't sit back and not communicate yourself."

When It Fails

The A/B test moves the burden onto your delivery team. Felix is blunt that this is a real cost: "it's not like a front end you just simply ship and then it works... you actually promising like profit." He calls it profit as a service, and says it needs its own tooling to run, evaluate and communicate the tests.

He is still not certain it's the right sales motion. Some buyers want the automation and don't need the proof at all, and for them the test is pure overhead.

The signal to switch: your team is spending more hours proving value than delivering it, and the buyers who close were never asking for proof.

Your First Move

Write down the smallest slice of the scary thing your buyer could hand you, where their existing process keeps running alongside it.

Half the catalogue. One region. One customer segment. Then decide the single number that settles it, and say it out loud before the pilot starts.

If you can't name a slice, your ask is still all-or-nothing, and that's why the deals stall.