UZR/150

by Luis on Dec 20, 2009 3:41 PM CST

How reliable is this fielding statistic?

I've seen a lof of people here using it, but I find it hard to interpret. I know what it means. The problem is that there is so much fluctuation for many players that I don't know what to make of it. Look for example at Coco Crisp's UZR/150 (http://www.fangraphs.com/statss.aspx?playerid=1572&position=OF). In 2007 he has 22.4, in 2008 it's -15.4 and in 2009 you have 19.6. So he has gone from a fantastic fielder, to a horrible fielder, to a fantastic fielder in a 3 year period. That's expectable for offensive numbers, but for defense? I see the same pattern for a lot of players and it's a bit disconcerting.

How much should I rely on it? How can interpret these types of results?

This is a FanPost and does not necessarily reflect the views of SB Nation or Al Yellon, managing editor (unless it's a FanPost posted by Al). FanPost opinions are valued expressions of opinion by passionate and knowledgeable baseball fans.

Tweet Comment 33 comments | 1 recs |

Do you like this story?

Comments

it's plenty flawed

generally, an acceptable sample size to get a good reading would be around 2000 innings, or two full season. even then, i’d probably still apply a +/- 5 run margin of error. the main problem with uzr is that the zone that players make plays within/beyond is static. this means that it does not take into effect batted ball type, player positioning or nearby defenders.

for instance if ryan howard is batting and soriano is playing almost in center because of the shift and howard plops a line drive right into soriano’s zone, he is penalized for not making that play, whether it is reasonable to expect him to get to it or not. similarly, if soriano has a full season where the center fielder is, say, colby rasmus, soriano’s numbers will be affected by the fact that rasmus is an elite defender and taking a lot of balls that soriano might otherwise have to get to.

defense can fluctuate from season to season, but you pointing out crisp’s difference is legitimate. it’s probably somewhere in the middle. his career uzr/150 agrees and says he’s right around average if a little better. so, the key to uzr is getting as large a sample as possible. like crisp, over 5000 innings most of that positiong/batted ball kind of noise will cancel out

either way, hopefully in the next few years, with the implementation of hit f/x and field f/x we can get a more reliable defensive metric. for now, though, this is the best we have

Of course, hope means being cut down on some street corner, as you run like mad, by a random bullet.

by prophetjohn on Dec 20, 2009 4:13 PM CST reply actions

The other big problem is that a players' defensive ability does change.

Like you said, the key to uzr is getting as large a sample size as possible (sidenote: this makes looking at uzr for bench and part-time players pretty damn hard). However, when you look at a player over his last 5000 innings, you have to be aware that 5000 innings ago he was a very different player and has likely declined in terms of overall athletic ability since then.

by Poloplaya14 on Dec 20, 2009 4:23 PM CST up reply actions

yeah

i think that defense can slump from year to year, especially when including injuries. it’s also smart to take caution with regard to older players. for instance, it’s probably not reasonable to expect scott rolen to continue to be a +20 defender by the end of his new contract with the reds

i should also point out that noise is going to play a larger role for OF than IF, so the discrepancies will be larger. it’s always good to give the player the eye test. just don’t let rick ankiel fool you into thinking that he’s a good defender just because he dives a lot, etc

Of course, hope means being cut down on some street corner, as you run like mad, by a random bullet.

by prophetjohn on Dec 20, 2009 4:30 PM CST up reply actions

There are even more flaws with the eye test though.

Based on the eye test, Alfonso Soriano would seem like the sorriest excuse for an OF Wrigley Field’s ever seen. But UZR shows that despite the fact that his fundamentals are piss-poor, his raw athleticism (when healthy) makes him an excellent defensive player.

by Poloplaya14 on Dec 20, 2009 4:57 PM CST up reply actions

That was true for 2007 and 2008

Last year he was awful by just about any defensive metric you could find. And a large part of why he was well rated those two years were because of all the outs his arm was getting us, which has dried up quite a bit since then.

by madcow256 on Dec 20, 2009 5:01 PM CST up reply actions

He was hurt last year.

He just wasn’t the same player on either side of the ball, and I bet injury had a lot to do with that.

by Poloplaya14 on Dec 20, 2009 6:27 PM CST up reply actions

His defensive value dropped a lot in 2008 as well, and there was no injury

2007 was a standout year, but it seems like scouting reports are warning against his arm, thus negating some value.

All told, he’s definitely not awful defensively while healthy, but I don’t think he’ll ever top the charts for defensive value again.

by madcow256 on Dec 21, 2009 11:20 AM CST up reply actions

wow

i never knew he was any better than slap your mother embarrassingly bad

Of course, hope means being cut down on some street corner, as you run like mad, by a random bullet.

by prophetjohn on Dec 20, 2009 5:34 PM CST up reply actions

that said

i wouldn’t call him excellent. slightly above average in left

Of course, hope means being cut down on some street corner, as you run like mad, by a random bullet.

by prophetjohn on Dec 20, 2009 5:36 PM CST up reply actions

No LF in 2007 had a better UZR/150 than Soriano

While that may be an aberration, his production was still excellent.

by Poloplaya14 on Dec 20, 2009 6:28 PM CST up reply actions

well

uzr is an attempt to measure defensive value. it’s not necessarily his actual defensive value

Of course, hope means being cut down on some street corner, as you run like mad, by a random bullet.

by prophetjohn on Dec 20, 2009 6:45 PM CST up reply actions 1 recs

Hit F/X won't get you as far as you think.

You don’t get the full trajectory of the batted ball, and because you lack proper spin data you can’t estimate it with a lot of precision. (More precision that the current fly/liner/popup binning? Hard to say for sure.)

One of the problems is that you’re looking at UZR/150 for someone that didn’t play nearly that many games. Random variance increases as the sample size goes down – it is far easier to put up a UZR/150 of +15 in 50 defensive games than it is to do so in 150 DG. So the observed spread of what UZR is seeing for Coco is less than what UZR/150 is making it out to be.

I has a blog. | Ask a sabermagician!

by cwyers on Dec 20, 2009 7:35 PM CST up reply actions

So...

Should I believe that Theriot is an excellent defensive player? His UZR/150 for the last three years are 7.7, 0.7 and 8.3, respectively. I ask because I’ve seen a lot of people here questioning his defense. Is that enought data for me to make a solid prediction?

Luis

by Luis on Dec 20, 2009 8:04 PM CST up reply actions

That should have said "excellent defensive SS"

by Luis on Dec 20, 2009 8:16 PM CST up reply actions

I wish I had a good answer for you.

I’m in the midst of a crisis of faith of sorts as regards current defensive metrics (see here and here).

Given three seasons like that, off the top of my head I think the best estimate of his defensive talent going forward is probably around a +5. What the confidence interval is for that, I can’t say for sure.

I has a blog. | Ask a sabermagician!

by cwyers on Dec 20, 2009 8:31 PM CST up reply actions

It should be good enough

Take a look at other defensive stats (you can find something called Total Zone rating at baseballprojection.com). Here is Theriot’s page.

http://www.baseballprojection.com/war/t/therr001.htm

So, Total Zone has lined up pretty well with UZR. It sees him as a few runs worse over the past 3 years, but there isn’t that much of a difference. He’s probably +5.

by vivaelpujols on Dec 21, 2009 1:31 AM CST up reply actions

that's why

i mentioned field f/x…

Of course, hope means being cut down on some street corner, as you run like mad, by a random bullet.

by prophetjohn on Dec 20, 2009 10:45 PM CST up reply actions

Hit f/x should help UZR for infielders a lot

by vivaelpujols on Dec 20, 2009 11:33 PM CST up reply actions

UZR has been probed a few times already...here are some of the better articles on the subject:

http://www.hardballtimes.com/main/blog_article/how-reliable-is-uzr/

http://www.fangraphs.com/blogs/index.php/uzr-2008-to-2009

by Andronicus on Dec 20, 2009 8:48 PM CST reply actions

There’s almost nothing in either one.

by ol Pete on Dec 20, 2009 10:25 PM CST up reply actions

What does that even mean

by vivaelpujols on Dec 20, 2009 10:26 PM CST up reply actions

They're light on content

by ol Pete on Dec 21, 2009 11:08 AM CST up reply actions

A player's UZR is a combination of 3 things

1) Skill. This is obviously the part we care about the most, and it’s pretty self explanatory. Basically, this is how good a certain defender would be if you gave him 100,000,000 fielding opportunities (and he didn’t get tired!)

2) Sampling error. This also happens with offensive stats. A player could just have a really good defensive year, or a really bad one, compared to his actual skill.

3) Measurement error. This is obviously the problem people have with defensive stats. UZR only makes a guess about how many runs the defender saved by using hit location data. The problem is that A) hit location data isn’t detailed enough (a chopper over the pitchers head, and a harder hit ground ball will be marked the same by the data recorders, even though the difficulty level is obviously harder different) and B) hit location data can be wrong, and evened biased (see the two links that cwyers posted above).

So when you see someone put up a 10 UZR/150, it’s likely that he’s a good defender, who had a better year than he usually does, and he got a bit lucky in terms of how the scorers coded his plays.

That makes UZR, like any stat, unreliable in a small sample size – but if you have 3-4 years, it should be okay.

by vivaelpujols on Dec 20, 2009 9:37 PM CST reply actions

Is that true?

That makes UZR, like any stat, unreliable in a small sample size – but if you have 3-4 years, it should be okay.

In most cases, it should be. The problem is that if there is a systemic, repeatable bias, the errors will never regress out no matter how large the sample size. That’s (possibly) what leads to disagreements between UZR done with STATS data and UZR done with BIS data – Ichiro is a classic example there.

I has a blog. | Ask a sabermagician!

by cwyers on Dec 20, 2009 9:56 PM CST up reply actions

Yeah, of course

by vivaelpujols on Dec 20, 2009 10:18 PM CST up reply actions

I remember a discussion over at baseballfever

Where Tango and the crowd that used to hang there talked about how it was only really good at reviewing a career.

by ol Pete on Dec 20, 2009 10:30 PM CST up reply actions

I've generally been going by the "at least 1,000 innings" sample size...

…when citing UZR numbers. Would you say that’s still OK for straight UZR, but not for UZR/150?

Catch my act on Twitter as @dat_cubfan_dave.

by daver on Dec 22, 2009 4:14 PM CST up reply actions

Well, I would use regressed UZR/150, so it's on the right scale

by vivaelpujols on Dec 23, 2009 5:46 PM CST up reply actions

Well, I'd say it's pretty flawed...

Since some geniuses on here are using it to say that Marlon Byrd is a better center fielder than Reed “Human Highlight Reel” Johnson. That isn’t correct on any planet.

by TheGrinch13 on Dec 21, 2009 2:00 PM CST reply actions