On Customer Survey Forms
In the olden days, I had a proper job ((Briefly. It didn’t take.)) writing software for customer survey forms. Among other things, our users could ask their customers to rate how likely it was that they would recommend the product to their friends. Typically, this would be on a ten-point scale, from extremely unlikely to extremely likely.
I left the job nearly ten years ago, and hadn’t thought much about it until recently, when I had occasion to fill out just such a survey relating to my stay at the Ritz-Carlton Hotel (!) in Atlanta, Georgia. My stay was pleasant, although very expensive.
Now, back to the survey.
It’s a bad question, for many reasons - for example, a GCSE student would easily spot that it doesn’t specify a time frame (the likelihood of my recommending a product in the next week and the likelihood over a lifetime are very different).
For another example, what does it even mean by ‘extremely likely’? Am I supposed to rate this as a probability from 0 to 1? Is it a log scale? Logit? Because even knowing *that*, it depends critically on the circumstances.
The probability of my being asked for a recommendation for a hotel anywhere in, let’s say, the next year, is small. The probability of them asking me to recommend a hotel in Atlanta are smaller still. The probability of them asking me to recommend a hotel in Atlanta and having the budget for a stay at the Ritz-Carlton is… well, zero, by any sensible measure.
Even if someone with that kind of money did ask me for an opinion about posh Atlanta hotels, my reaction wouldn’t be “I recommend the Ritz-Carlton,” it’d be “I don’t have any frame of reference. It was pleasant, but you presumably know more about what you want from hotels than I do.”
So, even under the most generous interpretation of the question, and even if I’d had the most amazing experience, I couldn’t really do anything other than tick “extremely unlikely.”
This is why it’s generally a mistake to ask mathematicians anything.