Between the Numbers
I am a huge baseball fan, and I am especially excited that my Boston Red Sox won the 2013 World Series. Part of my love for the game and math came from following the statistical performance of different players and comparing them. I can remember the endless debates with my friends as a kid about who was better, Wade Boggs or Don Mattingly. My dad and his friends debated their generation’s stars - Ted Williams vs. Joe Dimaggio vs. Mickey Mantle.
Until the past 10-15 years, there was a standard set of stats that people used to measure batter performance. Things like batting average, home runs, runs batted in (RBI), hits, runs, etc. The announcers would always talk about these as the most important measurable aspects for position players (non-pitchers).
I recently re-read a really good book that was re-released in 2005, Baseball Between the Numbers: Why Everything You Know About the Game Is Wrong. A team of data enthusiasts and stats folks (including Nate Silver of NY Times election prediction notoriety) came up with a new set of statistical measures to quantify a single player’s value objectively. It’s a really good read for the serious baseball fan and I picked up a copy for $10 delivered.
The key element is the Wins Above Replacement (WAR) framework, that represents how much better a player is than what a team would available to replace them. The model takes into account hundreds of different quantifiable attributes of a player’s performance (including hitting, defense/fielding, throwing, running) to find a methodology to compare all players and pitchers. The authors went to great pains to make sure the model would normalize differences in baseball team’s home stadiums (it is much easier to hit home runs at Yankee Stadium than Safeco Field in Seattle) and across different eras. This enables an objective frame of reference to determine what player had the greatest single season contribution.
Before reading the book, I’d have guessed Babe Ruth was the greatest player by a huge margin. He not only changed the game as the first great home run hitter, but also was a star pitcher for some World Series winning Red Sox teams. While I knew Barry Bonds put up some amazing numbers from 2001-2004, I never really appreciated just how great he was until reading this book. They make a case that Bonds is the only player even close to Ruth.
One of the most interesting points of the book for me was some of the strategy decisions. In 2004, Bonds was so feared that managers often would intentionally walk him with the bases empty to lead off an inning and rather face the other hitters in the lineup (this is after Jeff Kent had left as a free agent). That year he came up to the plate 617, and walked a record 232 times (with 120 intentional walks). To put that in perspective, Wade Boggs was known for have the ability to generate walks and had a career high of 125 walks in 1988 to lead the American league. Even though the chances for a run to score is quite a bit higher when a runner is on first than with the bases empty, the authors made a strong case that it was better to walk Bonds with the bases empty and face the other hitters.
Applying this to businesses
This got me thinking about how we take a different approach to objective measures to quantify business entities. I often hear executives during earnings calls talking gross sales, cutting expenses, customer acquisition, and customer retention/repeat sales as the most important things for their business. Maybe these are the items Wall Street analysts expect and they are catering to their audience, but those can’t be the only important items to running a successful business.
Can I take a similar approach to can quantify the complete value of an entity compared to the average alternative? Each type of entity would need to context specific variables that would empower a model for calculating the Value Above Replacement (VAR). By context specific, the measures for the variables may be very specific to a company, region, or cohort.
Example: WAR for a Software Developer
A real life example for using this type of model could be to quantify the value of mid-level software developer may have in a startup. I have seen too many managers get excited to focus on metrics that can be easily measured, but don’t necessarily relate to business impact (like number of bugs closed, number of features delivered, percentage of code with test coverage, number of releases, number of hours working on a project).
I have come up with the following list of features that I feel better relate to individual and organizational success:
Metric Name | Description | Baseline | Weight |
Customer Satisfaction | We build products for users. Their feedback and attitude may relate to improved positive outcomes. | ||
Outcomes delivered | We build products to deliver outcomes (more revenue, better efficiency). We should measure how our projects contribute to the goals of the company. | ||
Cycle time / Velocity | The time it takes to deliver an initiative of certain size or complexity. | ||
Features/Products Proposed | I am big fan of encouraging everyone to contribute to product development, and strong engineers can add value. The measure can be the items contributed that make it into a product backlog. | ||
Karma Points | Risk of introducing defects, overhead, technical debt. Facebook has a pretty good framework, but I'd rather start a new person with a low score vs. a perfect score. | ||
Domain knowledge | Come up with a quantifiable measure of business or technical domain knowledge. | ||
Institutional knowledge | Come up with a quantifiable measure of company specific domain knowledge and relationships. | ||
Skill growth | The skills acquired and used during a given period. | ||
Collaboration | There are many types of customer relationships for a developer / team members, sale cycle. | ||
Compensation | Compensation vs. market price |
Assuming that this list is representative of the features needed to assess value for this position and none are mutually exclusive, we could come up with a baseline measure for each metric (based on the average or median level developer we’d consider). If certain features represent higher value to the organization than others, then each other features can be weighted. I left the baseline and weight columns empty as they are organization specific.
Then each employee can be evaluated objectively, comparing their contribution with baseline to calculate the VAR. We now have an objective mechanism to do some important HR related items:
- Justify compensation changes
- Identify performance gaps/excellence
- Compare employees (internally and externally)
- Evaluate new candidates
A high performance company would certainly strive to always hire folks above the baseline. You may find that your superstar is actually worth multiple times more than the baseline candidate.
Other business use cases
Some other places this methodology can be used:
- Product development
- Marketing initiatives
- Sales cycles
- Production/Manufacturing or logistics
- Product sourcing
- Vendor selection
- Investments
- Compensation analysis