• Skip to main content
  • Skip to header right navigation
  • Skip to site footer
Farnam Street Logo

Farnam Street

Mastering the best of what other people have already figured out

  • Articles
  • Newsletter
  • Podcast
  • Books
  • Courses
  • Log In
  • Become a Member
TweetEmailLinkedInPrint
Decision Making|Reading Time: 3 minutes

Promoting People In Organizations

In their 1978 paper Performance Sampling in Social Matches, researchers March and March discussed the implications of performance sampling for understanding careers in organizations. They came to some interesting conclusions with implications for those of us working in organizations.

Considerable evidence exists documenting that individuals confronted with problems requiring the estimation of proportions act as though sample size were substantially irrelevant to the reliability of their estimates. We do this in hiring all the time. Yet we know that sample size matters.

On how this cognitive bias affects hiring, March and March offer some good insights including the false record effect, the hero effect, the disappointment affect.

False Record Effect

A group of managers of identical (moderate) ability will show considerable variation in their performance records in the short run. Some will be found at one end of the distribution and will be viewed as outstanding; others will be at the other end and will be viewed as ineffective. The longer a manager stays in a job, the less the probable difference between the observed record of performance and actual ability. Time on the job increased the expected sample of observations, reduced expected sampling error, and thus reduced the chance that the manager (of moderate ability) will either be promoted or exit.

Hero Effect

Within a group of managers of varying abilities, the faster the rate of promotion, the less likely it is to be justified. Performance records are produced by a combination of underlying ability and sampling variation. Managers who have good records are more likely to have high ability than managers who have poor records, but the reliability of the differentiation is small when records are short.

Disappointment Effect

On the average, new managers will be a disappointment. The performance records by which managers are evaluated are subject to sampling error. Since a manager is promoted to a new job on the basis of a good previous record, the proportion of promoted managers whose past records are better than their abilities will be greater than the proportion whose past records are poorer. As a result, on the average, managers will do less well in their new jobs than they did in their old ones, and observers will come to believe that higher level jobs are more difficult than lower level ones, even if they are not.

…The present results reinforce the idea that indistinguishability among managers is a joint property of the individuals being evaluated and the process by which they are evaluated. Performance sampling models show how careers may be the consequences of erroneous interpretations of variations in performance produced by equivalent managers. But they also indicate that the same pattern of careers could be the consequence of unreliable evaluation of managers who do, in fact, differ, or of managers who do, in fact, learn over the course of their experience.

But hold on a second before you stop promoting new managers (who, by definition, have a limited sample size).

I’m not sure that sample size alone is the right way to think about this.

Consider two people: Manager A and Manager B who are up for promotion. Manager A has 10 years of experience and is an “all-star” (that is great performance with little variation in observations). Manager B, on the other hand, has only 5 years of experience but has shown a lot of variance in performance.

If you had to hire someone you’d likely pick A. But it’s important not to misinterpret the results of March and March and dig a little deeper.

What if we add one more variable to our two managers.

Manager A’s job has been “easy” whereas Manager B took a very “tough” assignment.

With this in mind, it seems reasonable to conclude that Manager B’s variance in performance could be explained by the difficulty of their task. This could also explain the lack of variance in Manager A’s performance.

Some jobs are tougher than others.

If you don’t factor in degree-of-difficulty you’re missing something big and sending a message to your workforce that discourages people from taking difficult assignments.

The importance of measuring performance over a meaningful sample size is the key to distinguishing between luck and skill. When in doubt go with the person that’s excelled with more variance in difficulty.

Read Next

Next Post:The Tragedy Of The CommonsWhat is common to many is taken least care of, for all men have greater regard for what is their own than for what they possess in common …

Discover What You’re Missing

Get the weekly email full of actionable ideas and insights you can use at work and home.


As seen on:

Forbes logo
New York Times logo
Wall Street Journal logo
The Economist logo
Financial Times logo
Farnam Street Logo

© 2023 Farnam Street Media Inc. All Rights Reserved.
Proudly powered by WordPress. Hosted by Pressable. See our Privacy Policy.

  • Speaking
  • Sponsorship
  • About
  • Support
  • Education

We’re Syrus Partners.
We buy amazing businesses.


Farnam Street participates in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising commissions by linking to Amazon.