Master post for bashing BP for bashing Chipper

Baseball Prospectus

There was another Prospectus Triple Play slamming Chipper’s defense as “iron-gloved” last week. Same old, same old. Prospectus has gotten into the habit — in the book and in other entries, but especially in the Triple Plays — of claiming that Chipper is one of the worst defensive players of all time.

Now, I am not generally a supporter of Chipper’s defense. (My first piece of negative commentary was an email slamming me for claiming, circa 1998, that Chipper was “average at best” defensively.) But I saw Bob Horner play third base, and Chipper’s miles ahead of him defensively. I’d rate Chipper about a C- defender, ranging from C+ at his best to D or D- at his worst. He’s far from a total butcher, and does some things well at third, but lacks the instincts of a real third baseman and always has.

BP’s own statistics actually bear this out! On his Davenport card, he’s below average every year, but usually above replacement level.

In comments, Bamadan agreed that Chipper is mistreated by BP but thinks my explanation is off-base, that the replacement-level versus average argument is incorrect and that it’s actually a statistical illusion that causes Chipper to rate where he does. He’s right, to a large extent. For most third basemen, the difference between replacement level and average in any season doesn’t matter. But in Chipper’s case, it does, because he’s in a fairly small group: players who have played a lot of games at third base, and who are consistently a little below average. -8 one year won’t hurt, but -8 every year for eight years, with one or two -20s, those add up. Or subtract up, depending upon how you look at it.

But I think there is a statistical illusion at work, caused by an small number of ground balls to the third baseman off of Braves pitching. Chipper’s ratings are always bad; the ratings of Braves outfielders and second basemen are usually good, even for guys like Gary Sheffield and Keith Lockhart who were never confused with Clemente or Mazeroski. When a truly first-class outfielder plays right field for the Braves, like Jordan (in his last tenure) or Drew, he’ll get monster defensive ratings. Nick Green’s a good defensive second baseman, but hardly the tremendous glove some would have you believe. Gosh, Chipper rates as an okay outfielder in 2002 and 2004, and it was all he could manage to not get hit on the head with the ball most of the time.

What you end up with is a situation where:

1. The Braves consistently have good ERAs, better than you’d think from their pitchers.

2. Their third baseman is always rated poorly. Vinny’s rating was about as bad as Chipper’s. Fielding Runs Above Average, by year this decade, Braves seasons in bold: 13 6 0 -12 -4 9

It seems unlikely that the Braves are getting great pitching despite having an all-time worst situation at third base. What is more likely is that the statistics are misleading. If the stats don’t make sense, then there’s probably something wrong with how you’re reading the stats.

29 thoughts on “Master post for bashing BP for bashing Chipper”

  1. It’s just the Braves philosphey at work here. They pitch all righthanders away and play them away, making it very hard for righthanders to hit the ball to thirdbase. The Braves are also very unique in how they position their outfielders. And by unique, I mean good. They have a plan on where to pitch hitters and they know where that hitter will hit the ball if pitched properly. In my opinion this is why the outfielders stats always look so good – great pre-pitch positioning by the coaching staff.

    Obviously, all defensive metrics are going to struggle when they try to analyize the Braves defensive performance, simply because the Braves do things a little differently than everybody else.

    As to why BPro always says Chipper’s D at third sucks even when their own stats tell a different story… Well that’s just BPro being BPro.

  2. Great work Mac. This is just another example why I trust my eyes more than the numbers. If someone sees Chipper’s defense routinely like everyone of us here do, the conclusion will have to be that he is a below average third baseman rather than a butcher.

  3. Mac, I follow you, despite that being the longest post I’ve ever witnessed … from you. I’m fine with Chipper at third and this time last year was advocating the switch back. He’s a good enough athlete, even without the instintcs, to hold the fort, provided his head is in the game (unlike the 2001 NLCS which led to the LF switch and the Vinny reunion).

  4. Does anyone know how BPro actually measures defense? It’s very unclear to me what they are doing, but that is not surprising. What IS surprising is the amount of confidence they place on whatever metric they use. Whatever they have contains about as much information as RBIs, if that much. Dave Pinto, who uses a technique that I think is quite good though not perfect, rates Chipper 3rd among MLB third-basemen.

  5. Mac, I agree with all that you said. I also might add that defensive stastical analysis seems to be way behind offensive stastical analysis. If I understand your premis then Chipper looks worse in BP’s eyes becuase he’s been a below average defender longer than most below average defenders get to play the position and he has had a couple of really stinker years that makes him look worse using their metric. Makes sense to me. The reverse could also be true as well. Its funny that BP will be quick to point out statiscal outliers on the offensive side but apparently not on the defensive side.

    I don’t see why BP continues to make this a big deal. EVERYONE knows that Chipper Jones is in their for his bat and that given the choice between a career .304/.401/.537 hitter or some Brooks Robinson wannabe with a light stick most major league teams would play the bat.

    The only thing that I can think of is that they know that Chipper is blocking Andy Marte’s inevitable rise to stardom.

  6. Well written Mac. When I find time, I’d like to link to some of the Chris Dial / Mike Emeigh stuff discussing the Braves and balls in play distribution.

    I still disagree, though, about the replacement / average dichotomy. I believe that BPro makes a significant mistake comparing non-up-the-middle defenders to a mythical replacement level.

    Having followed minor league and college ball for a number of years, I believe that it is fairly easy to find a player who can field 3B (and the other non-up-the-middle positions) at major league average. That is, the replacement fielder is actually an average fielder. It is largely and almost exclusively his bat that keeps that fungible good from having a professional role, not his glove.

    As such, I believe that Chipper — slightly below average with the glove in my not so humble opinion — is in fact also slightly below replacement defensively. (That isn’t to say as a total player he is anywhere close to average; primarily because of his bat and to a limited extent his baserunning he is and has been excellent overall.)

    Anyway, that is a long way around saying that I think the error in BPro’s metric isn’t the comparison to average vs. replacement; rather, I think its the concept of average vs. replacement altogether.

  7. JC, I’m not sure how familiar you are with the various defensive metrics around, but Michael Humphreys (creator of DRA – defensive regression analysis) thinks that BPro has the best defensive stats; better than UZR, better than PMR, and certainly better than defensive win shares. I hope (but don’t know) that it uses play-by-play data.

    By the way, I looked through a bunch of DT cards, and the difference between “average” and “replacement level” defense seems to average about 15 to 20 runs per position. In essence, what that means is that a team of replacement level hitters who are all average fielders will win between 12 to 16 games more than a team of replacement level players (if we use the rough estimate of 10 runs per marginal win). Hitting wise is about the same: a league average hitter is worth a little more than 15 runs over replacement. So, 8 league average hitters and fielders is already about 25 wins above replacement level. But wait! We still have a replacement level pitching staff. Each league-average starter is worth roughly 60 pitching runs above replacement (if he pitches about 220 innings, which is what a normal starter will do if healthy all year). Add in 4 of those, and we add in another 20-25 wins above replacement.

    There’s something wrong with this. Our team, which by definition now has league average hitting and defense from it’s position starters, and 4 league average starting pitchers, is worth at least 45 runs above replacement. Last I checked, a replacement level team could be counted on to win about 46-48 games (what made the ’03 Tigers remarkable was that they were below that number). Thus, our not-quite-average hypothetical team should win between 91 and 93 games. That’s not an average team! An average team wins 81 games by definition.

    That’s why I don’t place much value on BP’s defensive stats, despite what Michael Humphreys or anyone else thinks.

  8. Thanks Kyle,

    That shed some light on things, because Humphreys won’t tell anyone what he’s doing. He just says, “read my 15-page article on the subject.” Well, I’ve read it, and there’s nothing there that fills me with confidence. What he is doing might be fantastic, but I won’t buy DRA until he open-sources the metric so I can see exactly what he does. The stathead community has a real problem with open-sourcing its metrics. Hopefully, MLB will open-source play-by-play data with batted-ball types soon. They have very little to lose by doing so. It’s not like people are buying lots of it, and every team already owns it. If they would release it maybe statheads could develop a metric in public that everyone agrees is a good thing.

  9. Pingback: Beaneball
  10. Michael Humphreys (creator of DRA – defensive regression analysis) thinks that BPro has the best defensive stats; better than UZR, better than PMR, and certainly better than defensive win shares. I hope (but don’t know) that it uses play-by-play data.

    I’m not sure which “it” this references, but neither DRA nor BPro’s DT are pbp zone based stats but rather adjustments to traditional range factor (number of plays made per game). By making their adjustments (for K’s, Ground ball tendency, etc.) they attempt to closer approximate actual opportunities, but neither have actual opportunities as an input.

    I also seem to recall that Humpries based his entire system on multiple regressions to best approximate UZR. That is for him UZR – and not DT – is the gold standard to which adjusted traditional stats should strive to compare.

    As an aside, I have no problem with closed source statistics. After all, some folks have spent a lot of time working on them and hope to make a buck or two off their investments. On the other hand, I don’t put much credence into systems that I can’t tinker with. I really like the concept behind MGL’s UZR, but without looking at the inputs and the manipulation, I don’t know that he hasn’t done something as simple as reversed a sign somewhere. Because of that, I take these systems with more than a small grain of salt.

  11. There is a difference between open-sourcing data and open-sourcing a metric. I know what Dave Pinto and MGL are doing, I just don’t have the data to replicate it. Humphrey’s data is available, but he won’t share how he calculates it. At least I know what the former are doing, and I know their strengths and weaknesses.

    I also have no problem with individuals wanting to protect their investments. That’s why I wish MLB would make the data available to all to tinker with. I think they have very little to lose and much to gain by sharing. They happen to disagree, which is there right.

  12. Hmmm, I agree with you in concept, but disagree that Pinto or Lichtman offer up the internal workings of the metric. I understand broadly what they are doing, but not only do I lack the raw data, I also lack the precise manipulations to that data.

    Years ago, when MGL was unrolling UZR at fanhome, he had multi-time gold glove winner Torii Hunter as an attrocious fielder. When asked for details, it was revealed that he had Hunter with very good to great road defense, but his park adjusted home performance was abominable. I don’t know what park factor adjustments were made, but generally, once adjusted for parks, there should be little difference between home and road performance. But I have neither the data nor the metric to be able to play around to see if it is a glitch in the system or a real, but unusual, effect of Hunter’s game.

    This isn’t a jab at Lichtman, but rather a reflection that in this information era, information is still not as widely available as I’d like.

    As an aside, one of the things Bill James did was cause the publication of some pretty basic information — batter walks, left / right splits, park data, etc. — to become widely available. His battles with Elias have opened up infinitely more data than was available 25 years ago. And now with the internet, that information is available virtually for free. But having been given a large taste of what is out there, I’m left wanting even more. The default position of MLB and their stats services for decades has been to provide as little information to their customers as is possible. While they have little to lose and much to gain by making PBP data available, that would take a change in corporate philosophy long ingrained.

  13. Sorry for the unclear wording – I know that DRA is not based on PBP data, but wasn’t sure if Bpro fielding stats were or not. If they’re not, than I don’t know why Humphreys said he liked them- IMHO there is no way that having PBP data doesn’t add at least SOME value to defensive analysis. Sure, it’s great to know how many ground balls a pitching staff allowed, but I’d also like to know exactly where they went. How does UZR like Chipper? If the two fielding stats that have at least some ball direction vector believe he’s alright, and the ones (DRA, Bpro, range factor, etc) that don’t have PBP data don’t like him, I’ll trust the statistic that can tell when a ball is actually hit at him. Not to make light of the fine work that Humphreys et al are doing, but it just seems silly to conclude otherwise.

    JC: mlb is not exactly the best at protecting it’s investment. Its biggest asset, despite what anyone in the league may think, is its players. So why is Selig testifying in front of congress, despite the fact that almost every player doesn’t want him to? Just to piss them off? Great idea, bud. No wonder the NFL kicks your ass so badly.

  14. Lol, your going to blame selig? The players and their ridiculous union are the sole reason why we even have this problem to begin with.

  15. Hahaha. I was about to say that, Mac. :) There are plenty of people on Primer to grouse with if you want to place blame for steroids on the players, Grst. I’m curious, though: do you think it’s good policy for management to show the union up? I do not. Embarrassing the players (even if they deserve it) only makes more baseball less popular in the viewing public’s eyes, which is the exact opposite goal that management should have. But as Mac says, we’re not getting into the union thing here, so I won’t say anything else.

  16. Mac, I’m sorry I responded about the Union comment before I saw your post. If you can, please delete my response.

  17. I agree that Pinto and MGL are not quite open-source (Pinto has been pretty open with me though), but at least I understand what they are attempting to do. I’m still not sold on either method.

    I think the main impediment to releasing PBP data is the cost to MLB. Though it’s not all that expensive to keep and update such a database, I’m not sure the returns were high enough to cover it until recently. Hopefully, the success of Moneyball might make it worth their while. MLB is a business, and if they can make money doing something, they will do it.

  18. JC, I’m not sure that MLB is the one who owns the PBP data (at least not the defense-specific ball vector data). I thought that Baseball Info Solutions and STATS Inc. collected this themselves, and Pinto had access to BIS’ data because he worked for them (while MGL paid a few grand out of pocket for STATS’ zone data).

    I agree that there isn’t a lot of impetus to put that data in the open. Other than a few nerds like us, who wants to know how many grounders Chipper faced between 6 and 8 feet to his left hit at such-and-such velocity? That said, if they put it in an Access database and charged 50 bucks a disc, I would consider buying it.

  19. Embarrassing the players (even if they deserve it) only makes more baseball less popular in the viewing public’s eyes, which is the exact opposite goal that management should have. But as Mac says, we’re not getting into the union thing here, so I won’t say anything else.

    Without making a big deal out of this I’d say that, no management should not deliberately show up the players. But I don’t see that it follows where testifying before congress = showing up players. If he goes there and just insults everyone then that’s different. However, if he explains the policies he’s implemented with regard to the minor leagues that have been successful (and the fact remains the union is the reason the same policy is not in the bigs) and where he’s heading with the big leagues then no one has any right whatsoever to say he shouldn’t be there.

    To stop this from being completely off topic, I’ll say it is a bit perplexing why MLB doesn’t release that data. JC is right, however, in that if there is a way for them to make money in it, they will do it. As to open source metrics, it’s somewhat of a catch-22. If you don’t know what someone is doing you don’t know whether to trust the results or not. But if they release their methods than they lose the sole ownership of what was probably the result of countless hours of work.

  20. Total Baseball falls into a similar trap with its TPR. A player who’s slightly below average, but has a long career, is rated as a worse player than someone who stinks up the joint for a few years and is never heard from again. I call it the Ken Reitz Syndrome. Now, throwing Ken Reitz out there year after year may rate as one of the worst management decisions ever, but Reitz was not historically bad, just reliably so.

  21. The best news about Ramirez at this point is that he’s not feeling any pain. If he stays healthy he’ll be just fine.

  22. My back has been acting up and I wound up lying down all afternoon.

    The best news about Ramirez at this point is that he’s not feeling any pain.

    Anyone else notice the strange juxtaposition here? Is it possible that Mac is, in fact, the Bizarro Horacio? When one is healthy, the other is on the shelf? As a fan of both the Braves and Braves Journal, this presents me with quite the moral dilemma….

    Just kidding, Mac. Hope you’re better soon — back pain sucks.

  23. Hello everybody,

    I’m the DRA guy. From time to time I Google to see if the system is mentioned. I just wanted to clear up a few things.

    DRA does *not* use PBP or zone data. All data is publicly available and “traditional”.

    Yes, DRA is still not completely open-source. The November 2003 article (accessible as a PDF file from The Hardball Times) has a *two-page*, ~”twelve-step” summary of the steps used to create DRA, but leaves out a couple of specific techniques–but nothing that couldn’t be figured out, at least in theory, from what is presented. I’m actually surprised no one has figured out the missing pieces. I’m hoping to get a draft of a book done this summer so that DRA will be open-source.

    Yes, BP DFTs are now a mystery. I’m not even sure they don’t use PBP or zone data. Actually, and I won’t go into all the details why, I’m virtually certain they use PBP data at first base, and maybe elsewhere. Even if BP uses only publicly available data, the *way* they use it is completely unknown. Aside from general claims that total team runs allowed, BIP, GB/FB, L/RHP, etc. are factored in, there is no detail at all.

    No, I never said BP DFTs were better than UZR. I said DFTs seem to be pretty good, but not as good as DRA and certainly not as good as UZR, which itself has problems occasionally.

    The biggest problem with UZR is that Lichtman won’t be publishing them anymore. The second biggest problem is that it would cost somebody else about $10,000 and hundreds of hours to buy and program the treatment of the data. The third biggest problem is that the data will never be shareable/public so results reported by a public-spirited sabermetrician will have to be taken on faith. We probably need about three to five people all buying the same data and crunching the numbers their own (fully-disclosed) ways to get a reliable metric.

    Chipper is a bad third baseman. DRA probably rates him too low, and Chris Dial insists that Atlanta has had some weird bip distributions, but even UZR rates him as at best average. Why else would the Braves, who’ve had great team fielding during the Cox years, send him out to left field?

Leave a Reply

Your email address will not be published. Required fields are marked *