Evaluating General Managers

My younger brother, YEAROFCHC2011, came up with the following concepts – I just filled in the logical gaps. He says that this was inspired by the comments cubzfan, fsuapollo, and bdlugz made on the Paul Maholm thread. Let us know what you think!

Winning in baseball is about optimization and efficiency. Sure, you could go out and spend $23 million a year on Prince Fielder - he posted 5.5 WAR, which was valued at $24.6 million in 2011 - but would he bring back any excess value on a 6-year contract? Probably not.

Why is excess value important? Well, if you were looking to build a 90-win team and paid market value for each player, you would end up with the following payroll:

Payroll = Cost of Replacement Level Team + Cost of Marginal Wins

Payroll = (# Replacement Level Players * League Minimum) + (# Wins Above Replacement * Dollar Value of WAR)

Payroll = (25 * $425K) + (42 * $4.47M) = $198.4M

This results in a team payroll that is over the luxury tax of $189 million dollars. While it’s possible to pay market value for players, as long as you’re willing to pay a luxury tax, it’s not realistic. That’s why payroll efficiency is important.

Efficiency

Let’s start with a basic measure of payroll efficiency: dollar per win, or the number of dollars team X had to pay per win.

Below is a chart detailing how much each team paid per win in 2011:

However, we can do better than this. The replacement level team would <link to fangraphs> win 48 games. We want to know how much teams are paying for wins above this replacement level. How much are teams paying for the wins that actually matter – the marginal wins? We arrive at a slightly more advanced measure of payroll efficiency,

Efficiency (rEF) = (rWAR * $/WAR)/Payroll

Where rWAR is WAR calculated by (Team Wins – 48), or the number of wins a team had above replacement level. This is different from summing the WARs of the individual players. I use rWAR instead of WAR because a team’s final win-loss record is more important than a hypothetical win-loss record based on player’s WAR values.

rEF > 1 = Positive Efficiency

An efficiency value that is greater than one indicates that a team’s 25 players are worth more than what they are paid.

rEF < 1 = Negative Efficiency

An efficiency value that is less than one indicates that a team’s 25 players are worth less than what they are paid.

Below is a chart detailing each team’s rEF in descending order.

As we can see, only five teams had an rEF of less than 1: the Mets, Mariners, Cubs, Twins, and Astros – all teams within the top 2/3 of all teams in terms of payroll, and the bottom 2/5 in terms of wins, which is not a winning combination by any measure. However, the takeaway here is that teams are generally efficient; therefore, we should look at relative efficiency.

Relative Efficiency (rREF) = Team’s rEF/League Average rEF

rREF > 1 = Positive Relative Efficiency

A relative efficiency value that is greater than one indicates that a team is more efficient than league average.

rREF < 1 = Negative Relative Efficiency

A relative efficiency value that is less than one indicates that a team is less efficient than league average.

Below is a chart detailing each team’s rREF in descending order.

Relative Efficiency measures how efficient a team is relative to the league. For example, the Rangers had an rREF of 1.237, which means that the Rangers were 1.237 times as efficient – as good at getting more value from players than what they pay players - as the average major league team. Five of the eight playoff teams had an rREF above 1, with two of the remaining three posting REFs above .850.

Another interesting data point: the Tampa Bay Rays. The Rays were 2.563 times as efficient as the average major league team. That is phenomenal. If we look at standard deviations of rREF – one standard deviation is .482 – the Rays were above the 99.7thpercentile in efficiency, i.e. three standard deviations above the average.

General Manager Value

Evaluating the decisions of general managers can be very difficult, but using the efficiency ratings that we have created, we can attempt to place a value on general managers. While this is by no means a comprehensive analysis of a general manager’s value, it can potentially be one among a number of tools used to evaluate a general manager. We can evaluate a general manager’s value, or GMV, by calculating the change in his team’s efficiency rating over his tenure as the team’s general manager. So the GMV formula is,

GMV = (Most Current Year rEF – rEF Prior to First Year)/(Number of Years as GM)

As an example, let’s look at former Cubs GM Jim Hendry. Hendry became the general manager in the middle of the 2002 season, but since he didn’t have much of a hand in architecting the 2002 team, we will use 2002 as Hendry’s baseline efficiency, or if you’re following the formula, the rEF prior to first year. The Cubs 2002 EF was 0.653. The Cubs 2011 EF – the last year of Hendry’s tenure, or Hendry’s most current year – was 0.772. Hendry's GMV is calculated below.

GMV = (.772 - .653)/(9) = .013

In a vacuum, this number isn’t very useful; however, it becomes much more useful once you compare it to the GMVs of other general managers.

Let’s now look at the Rays’ Andrew Friedman. In 2005, the year before he took over, the Rays’ EF was 2.176. In 2011, the Rays’ EF was 4.588. Friedman's GMV is calculated below.

GMV = (4.588 - 2.176)/(6) = .402

We can now compare the two GMVs: Hendry's .013 and Friedman's .402. As we can see, Friedman has been much better at acquiring more value than he pays for. How much better? Friedman’s GMV is 30.4x that of Hendry’s.

While GMV does contain value in and of itself, it is most useful when it’s being used to compare GMs.

General Manager Reasoning

The efficiency rating can also naturally be applied to individual players. For example, Albert Pujols’ efficiency rating in 2011 was 1.425 - he was worth $22.8M and was paid $16M.

Taking this player efficiency rating into consideration, another way to evaluate a general manager is to look at the moves that he makes and evaluate them at that point in time, instead of in retrospect. While some contracts can look very good or bad in retrospect, it is important to evaluate a general manager’s decision based on the information that he had at the time. One way to do this is to project the player the general manager acquires using the ZiPS projection system over the life of his contract. For example, we can use ZiPS projection of Pujols’ value over his 10-year contract and evaluate it against his annual salary. This should help show a general manager’s reasoning when signing a player.

We will call this statistic a general manager’s reasoning, or GMR. GMR evaluates a general manager’s decision given the information that he had at his disposal during the time of his decision. The GMR formula is,

GMR = Present Value of Player’s Projected WAR/Present Value of Contract

The present value of a player’s projected WAR is the value of the player’s production over the course of his x-year contract to the team as it stands today. All things equal, you’d rather have four 5.0 WAR seasons now than in the future. In order to account for this, you have to discount future WAR. I typically use a 5% discount rate. At the same time, you must discount the player’s contract value in order to account for the fact that money today is worth more than money tomorrow. I tend to use a 5% interest rate to discount contracts.

As an example, let’s take a look at Paul Maholm recent contract with the Cubs. Maholm looks like he should post about 2.0 WAR per year during the life of his contract. Let’s assume that the Dollar Value of a WAR stays at $4.5 million over the next two years, and that the Cubs pick up Maholm’s club option. Hoyer’s GMR on the Maholm signing is calculated below.

Present Value of Player’s Projected WAR = [((2.0 WAR * $4.5M)/((1+.05)^0)) + ((2.0 WAR * $4.5M)/((1+.05)^1))] = $17.57M

Present Value of Contract = [$4.5M/((1+.05)^0)] + [$6.25M/((1+.05)^1)] = $10.45M

GMR = $17.57M/$10.45M = 1.68.

While this GMR does tell us that Hoyer paid less than what Maholm will likely be worth, it’s much more useful when compared to the GMRs associated with similar transactions from the same offseason.

Conclusion

GMV and GMR both evaluate a general manager’s value through the use of the efficiency ratings that we’ve outlined. In our eyes, GMV is a retrospective macro-level look at a general manager’s effectiveness, while GMR is a prospective micro-level look at a general manager’s effectiveness. While there are many other factors that could have been considered, and some potential logical problems, (small market teams are more likely to have high efficiency ratings, larger market teams are expected to spend big bucks on big name free agents, depressing their efficiency ratings, etc.), they all add layers of complexity that can’t be completely measured. Even after considering these potential shortcomings, GMV and GMR offer a simplified glimpse of a general manager’s performance.