A Brief Follow-up on Elite RP and @Ottoneu LWTS

The debate on reliever in fantasy baseball continues to rage in our Ottoneu league – and today, Rotographs’ Brad Johnson joined the fray with an article on the subject inspired by our intrepid commissioner @Fazeorange.  To be fair to Brad, he stated up front that his goal was not to resolve the argument, but rather to present a framework to answering the question.  Given my trademark immodesty, I’d like to offer a high level (and what I consider rather obvious) answer to the question.

Beyond my endless rants on the subject, this question arises from a real world situation:  one of our best and most active owners (and a former champion) has, at least on the message boards, been championing a strategy whereby he stacks his bullpen with “elite” relievers, thereby projecting an enormous bullpen advantage that not only confers extra points, but added flexibility elsewhere (because elite RPs are cheaper than elite position players or starting pitchers).  His bullpen currently includes 3 of the top 5 RPs from last year, and were LWTS scoring retrospective (like a Strat-o-Matic league, perhaps), this makes sense.  However, the question is whether it is likely his bullpen is filled with elite 2014 RPs.  Unfortunately, that is unlikely the case.

To attempt to gather data that might answer the question (if you want the entire spreadsheet, please email me), I went back and gathered the list of Top 10 projected players at each position (25 for the OF) in March 2012 and March 2013 (ie, immediately prior to the beginning of the season).   I used ESPN rankings, though I’m not sure that the lists matter, the variation at the top is minimal each year.  I then pulled each players Ottoneu LWTS point total for that season, and compared that number to the replacement level for his position in that year.  Brad’s replacement level for RPs was 72 (on the assumption that each team rostered about 5 RPs, an perhaps a bench player – in other words, sorting by RP scoring the 72nd ranked player).  For my positions, I defined the “replacement” player as follows:

C 18
1B 24
2B 24
SS 24
3B 18
OF 72
SP 72
RP 72

[** Interestingly, the replacement 3B in each year, finishing 18th each year, was Alberto Callaspo – at the gut level, this gives me some confidence we are defining replacement level appropriately]

Next, I summed the PAR (or, in some cases, the points below replacement) for each position.  Broken down by position and year, the results are as follows:

2012 Total PAR PAR/Player 2013 Total PAR PAR/Player
C 2706 271 C 2018 202
1B 1118 112 1B 1978 198
2B 3301 330 2B 2457 246
SS 3295 330 SS 2476 248
3B 3091 309 3B 3126 313
OF 6805 272 OF 6924 277
SP 3431 343 SP 3537 354
RP 686 69 RP 528 53

What do we see?  Surprisingly, the level of production for the top 10 projected players across the positions versus our replacement level is remarkably constant – except for RP.  If you invest in Robinson Cano, or Adrian Beltre, or Clayton Kershaw, you can be expecting 300-350 Points Above Replacement.  The only outlier here is RP – if you invest in an elite RP, you can expect to receive and extra 50-60 points from that investment.  Why?  Well, because even amongst the top 10, the flame out rate is significant.  In 2012, here is the projected top 10, along with their actual production and PAR:

Player Points PAR
Craig Kimbrel 747 377
Mariano Rivera 79 -291
Jonathan Papelbon 582 212
John Axford 480 110
Brian Wilson 7 -363
Rafael Betancourt 449 79
Joel Hanrahan 439 69
Jose Valverde 498 128
Jason Motte 620 250
J.J. Putz 485 115

Craig Kimbrel was the #1 ranked RP and performed like it.  Mariano Rivera blew out his knee shagging flies, while Brian Wilson wrecked his elbow as pitchers do, and both delivered far below replacement level value.  Rafael Betancourt and Joel Hanrahan managed to deliver replacement level to slightly above performance – but presumably at elite prices.

Was 2012 an outlier?  Here is the same table from 2013:

Player Points PAR
Craig Kimbrel 722 323
Aroldis Chapman 603 204
Jonathan Papelbon 449 50
Rafael Soriano 501 102
Fernando Rodney 553 154
Mariano Rivera 547 148
J.J. Putz 215 -184
Joe Nathan 649 250
Joel Hanrahan -11 -410
John Axford 290 -109

Is there a pattern?  Sort of.  Craig Kimbrel?  Monster – go get him if you can.  3 of the 10 posted wildly below replacement level.  A couple other big names Papelbon and Rodney) managed slightly above replacement, and a couple (Chapman and Nathan) were excellent.

Of course, this is a first level review – as always in Ottoneu, price matters, so a $2 Brian Wilson headed to Tommy John doesn’t affect things very much.  However, the question we’re trying to answer here is whether or not investing in the best RPs pre-season makes sense.  Regardless of price, in my view it doesn’t because there is such little likelihood that can can identify them if they’re not wearing a Braves jersey and closing in Atlanta.  Why does this debate rage, at least in our league?  My suspicion is that, as we look back year over year, it can be difficult to remember which relievers were expected to be elite – those sitting on Koji Uehara now can scarcely remember a time when he wasn’t atop the RP leaderboards.  Nevertheless, if we look at the numbers, RP is the one position that investing in anyone not named Craig Kimbrel makes little sense from the perspective of Points Above Replacement.

Thoughts?  Issues?  Problems with my methodology?  General screeds?  All are welcome, either though the site or via email at bill dot porter at gmail dot com.

Advertisements

About Bill Porter

Bill is an aspiring sportswriter (attorney by day) born in Washington DC, raised in New York, and currently living in San Francisco with his wife Kirsten and two spazzy labs, Fletch and Bear. Follow me on Twitter at @wfporter1972
This entry was posted in Fantasy Baseball, LWTS, Ottoneu and tagged , , , , , . Bookmark the permalink.

8 Responses to A Brief Follow-up on Elite RP and @Ottoneu LWTS

  1. Trey says:

    Bill, this is a great breakdown and I think you’ve made a strong case for RP volatility year over year. Thanks for putting this together.
    I still think a case can still be made for “pre-paying” for elite relievers in the off season, though, so I’ll do my best to lay that out. Brad has already covered part of it well in his “third scenario” here (http://goo.gl/REMpql), but I think there’s more to the story. First, can you email me the spreadsheet so I can use the same data you analyzed? Thanks

  2. Brad Johnson says:

    For injured pitchers, I would consider taking the pts/IP of that pitcher plus a replacement level hurler in order to calculate PAR. That’s part of the reason why your PAR per RP is so low.

    Of course, that’s true of all positions, so it might not change the outcome.

    • Bill Porter says:

      That is interesting. The replacement discussion on Ottoneu definitely gets complicated quickly — ie, what is replacement level in the OF because my five OFs are Allen Craig + Alex Gordon + Carl Crawford vs RHP + Starling Marte v LHP and some RHPs + Coco Crisp v RHP + Matt Joyce v RHP etc . . .

    • Trey says:

      Brad said it before I could run the numbers, but he’s right – if the argument is that RP is the position with the highest replacement value, that should also mean that owners will be quickest to cut higher priced RP in favor of replacement level RP when injuries occur. By making this adjustment, RP actually gain some value because you need to add back that replacement value, which would essentially make the PAR = 0 if a top 10 RP were cut before the season began. In fact, 2012 is a perfect example, as both Mo Rivera and Brian Wilson were lost for the season before it even began, so instead of subtracting their loss of points, you need to add back in the replacement points of 370 each, making PAR zero.
      Should you make this adjustment for other positions too when injury occurs? Yes, probably, but it’s not as simple since replacement value isn’t as high. Using 2012 as the example again, Tulo was the #1 pre-season ranked SS. Well, he was > 100 points LESS valuable than a replacement SS that season. But this is where cost becomes critical to completing your analysis above: the “cost” of losing Mo Rivera (if for some reason I didn’t cut him before the season began) was about $10 (average all-leagues salary in Ottoneu), but the cost for Tulo was $44. It’s much harder to stomach cutting a $44 Tulo than a $10 Mo Rivera, especially when you know for certain that Rivera is done for the season but you’re never really sure if Tulo is out for longer than 15 days. In other words, while it’s certainly reasonable to expect your RP to go down with an injury more often than your SS, the cost of losing them is drastically different. In essence, there’s a higher “carrying cost” associated with non-RP positions because the cost to buy those top 10 players is significantly higher than it would be to buy a top 10 RP.
      All this is admittedly complicated as we’ve all agreed, so here’s an easier way to see it (http://bit.ly/1fHIhIj). I’ve adjusted for the RP replacement costs I’ve described above, and then plugged them in here. The spreadsheet almost becomes like a game: Try to spend $100 the best way possible to achieve the highest PAR you can without going over budget. Going with the elite RP option isn’t always the best strategy, but the main point here is that it CAN be a good strategy sometimes, as it looks like it would have worked well in 2012 and a little less well in 2013. In other words, elite (pre-season Top 10) can often be a good investment because, despite the higher volatility risk, the true risk of loss is quite low. So, see what combination of players you can “buy” that gives you the highest points.

      A few final takeaways I’ve noticed when doing this exercise:
      1) There’s no doubt that RP are volatile, but the lesson here is that if you do decide to buy elite RP, you should essentially have a quicker trigger finger in cutting them than any other position because you can pick up replacements at such a low cost and because the replacement gap can be very thin if you wait too long. In many ways, I think that actually lowers the risk of carrying elite RP into the season, but I’m sure that’s still up for debate.
      2) Paying for elite 1st basemen actually seems to be the riskiest investment you can make, which sort of runs counter to many fantasy “expert” advice. But then again, we’re only looking at two years of data here…

      Again, here’s the spreadsheet (http://bit.ly/1fHIhIj). It’s a great debate; it’s complicated, and there are many factors involved (in particular Brad’s point about surplus roster value), but it’s interesting to look at all the same. Good analysis on all sides.

      • Bill Porter says:

        Neither Rivera nor Wilson were lost preseason, they were both in season losses. I will take a look at your spreadsheet but it sounds like in essence you are ignoring the opportunity cost of spending on these players to begin with, and then arguing in favor of buying elite RPs while simultaneously saying they should be quickly cut and that they are completely replaceable — seems like a strange argument to make. Again, if you can’t identify these guys in advance (and you’re not offering any reason to say we can) then the elite RP investment argument just makes no sense. You’ve got to start with an ability to produce positive EV with the guys you identify as elite. The question you aren’t answering is “what makes you think you’ve rostered an elite bullpen”? If your answer is “well if I haven’t, it’s easy to change course” you’re cooking the books of the argument by essentially assigning zero value to what you would have been able to do with those dollars wasted on supposedly but not in reality elite RP.

  3. Trey says:

    Are relievers volatile from year to year? Yes
    Are relievers more volatile than other players? Probably
    Is it possible to identify and project above-replacement (or even elite) relievers over replacement level relievers with enough confidence to pay a premium? Yes, absolutely.
    How? Using the same rate and skill-based analysis you’d use to identify above-replacement players at other positions, including starting pitchers.

    But don’t take my word for it. Instead, consider why there are currently 26 RP owned in at least 70% of Ottoneu leagues right now for a salary greater than replacement (using $3 or less). If you are saying that it is impossible to properly identify (or separate) above-replacement RP from waiver-fodder RP, it appears that at least 70% of Ottoneu players would disagree. After all, if you cannot predict RP production with any level of confidence, no one would pay more than the replacement price for any reliever. But that isn’t the case, because most owners operate on the understanding that you can indeed evaluate RP like other positions to the extent that you can identify rate-based skills that increase the likelihood an RP will produce at an above-replacement level.
    It shouldn’t come at a surprise that you can use sabermetrics to better identify above-replacement relievers because we all do this already for every player we identify as “own-able”. We use and have access to the same player skills, metrics, and data to make judgments on which players are more likely to perform at an above-replacement level from the very moment we add them to our team. When was the last time you needed to pick up a spot starting SP or a RHP-mashing platoon 6th OF? You probably had three or four players you were considering, and used some advanced data to finally decide which one was most likely to continue their hot streak using advanced data. Well, why aren’t we allowed to acknowledge those same metrics when evaluating potential RP additions, whether in the off season or July? The reality is that there are elite skill sets that set some RP apart from others, and therefore make it more likely that those RP will produce at higher level than what I could get by spending $3 to pick up an RP blindly out of a hat. No one picks blindly, no matter what position, including RP – everyone uses information to make the best possible choice for their team. There is no reason to think this isn’t the case when evaluating RP, either.
    “But the higher injury rate of relievers makes them too risky to own at an above-replacement price. They just aren’t worth it even if you could possibly identify the good ones…”
    Really? First, show me the evidence injuries are more common among RP than others, including SP? However, I’ll concede that point for the sake of this argument and move on, because that risk – the risk of injury and loss of production – is already factored into the RP price, which is a critical element of this discussion.

    Let’s compare Kimbrel vs. Kershaw:
    Both are elite, the very best above-replacement producers at their positions. While Kershaw may add a whopping 800+ PAR to your team vs. the 300+ that Kimbrel does, Kershaw also comes at a cost nearly 3 times higher than Kimbrel. If Kimbrel is more likely to be get injured than Kershaw simply because he is a reliever, that’s fine, but his salary already reflects that risk and serves to bring it into balance. But what is the risk if Kershaw is the one that goes down with injury now that he’s taken more than 10% of your budget? Risk can be evaluated in several different ways. Consider these examples:

    The Top of the Food Chain
    Mike Trout @ $70
    Clayton Kershaw @ $56
    Craig Kimbrel @ $14

    As you’ve stated, RP cannot be predicted with any degree of confidence, but does that automatically make Kimbrel the riskiest contract of the three? Sure it does… if salary is removed from the equation. But it cannot be evaluated that way – cost is always part of the equation, and always calculated in the risk assessment.

    Same Price Special
    David Robertson @ $8
    Carl Crawford @ $9
    Aaron Sanchez @ $7

    All assets here are priced in the same band, but they certainly have different risks. Both Robertson and Crawford have a track record of success, but Robertson is an RP, so he obviously shouldn’t be paid more than replacement level, right? No, that doesn’t make much sense – being an RP alone isn’t enough to remove the cost factor from the equation. In addition, if all RP are too risky to pay above-replacement prices, how in the world can you justify paying a pitching prospect like Aaron Sanchez $7? If there’s anything riskier than a reliever, it’s a pitching prospect…

    The primary point here is that cost MUST always be factored into the risk before you can just designate all RP to be worth $3 and nothing more. Again, no one really operates that way. Premium closers demand a price above replacement, and most people believe and act as if they can identify those above-replacement RP with enough confidence to pay for their services.

    This is also where I think ESPN rankings fall short, though I understand why you used them because it is a common pre-season assessement tool. But the beauty of Ottoneu is that it isn’t setup like most pointless scoring fantasy baseball system. ESPN and other roto-centric rankings are much more reliant upon guesswork – that is, guessing which relievers will be “in a position” to receive Saves, which is nearly as pointless as guessing how many “Wins” a starting pitcher will have a season’s end. Guessing saves is much more of a crapshoot. However, it doesn’t work that way in Ottoneu, thankfully. Our scoring system rewards skills first and opportunity second, though they are closely linked. Generally, those RP with elite skills eventually get the opportunities, which is exactly why guys like Farquhar currently have salaries of $5 – $9 (and yet another example of why most Ottoneu players believe they can rightly identify value above replacement in RP even during mid-season). Elite skills lend themselves to high scoring in this format among RP and, with the exception of injury, I’d argue they are pretty good predictors of future success (and they are the same skills we use to identify the future success of Starting Pitchers, right?). Simply put, there is a mountain of data that suggests that Greg Holland, David Robertson, and Trevor Rosenthal are going to be much, much better than guys like Brett Cecil, Jamey Wright, and Ryan Webb in 2014, so I don’t think it’s unreasonable to pay a premium for those skills sets. Could all three of Holland, Robertson, and Rosenthal get injured? Yes, but there’s no greater likelihood of that than of Cecil, Wright, or Webb getting injured, so the injury risk must be applied fairly across the entire RP spectrum.

    I absolutely think you can identify better RP than others, and obviously to the extent that it’s worthwhile to pay more for those projected above-replacement relievers. The high majority of fantasy players believe this as well, and the evidence is in their actions of paying premium salaries to elite relievers. It’s a slightly different question to ask whether that valuation is a winning strategy; I obviously also think that’s true, but again, a big reason for that is the makeup of my roster and the assumption that I have a lot of surplus value built up. In other words, I have cash to spend, and the numbers (http://bit.ly/1afxbYW ) suggest I can get more bang for my buck if I buy elite relievers. Whether they stay healthy or not is certainly a risk, but again, that risk is already factored into the price.
    Just to test my own hypothesis, I will run some numbers and see if I can predict the top 10 RP for 2014 and post them here on your blog, giving us a chance to review at the season’s end. The real test of course to this strategy is to see which teams end up with the highest P/IP (and thus most points), and for now I think my strategy will work well to accomplish that. I’ve been wrong once or twice before though, so we’ll see what happens. Like they say, that’s why we play the game. Championships aren’t won in January…

    • Bill Porter says:

      Definitely — pick your top 10 and post them here. What would also be interesting is picking top 10 at a few other positions for comparison. The part about injuries obscures the other two main problems with “predicting” RP performance: (a) their scoring largely depends on role, and huge percentages of them lose the valuable role that allows them to score big points (as a contrast — other than injury when is the last time you can remember a top 10 position player merely losing his job), and (b) over 60 innings, the advantage of identifying “elite” through advanced metrics is minimal because of the vagaries of BABIP, HR% etc — over 200 IPs from a starter or 600 plate appearances from a hitter, variance still exists but there is much less.

      With respect to using the fact that lots of Ottoneu owners own RPs, I don’t think that matters much for a couple reasons, (a) we’re approaching the cut deadline, and (b) the herd mentality is strong. We used to see, as recently as a few years ago, big contracts given out to “proven” closers throughout MLB. Outside of Philly, we don’t see it any more, particularly amongst the smartest front offices.

      • Trey says:

        I can see your point about the weight of 200 IP vs. 65, but, using Holland for example since he’s my most expensive RP, I’ve got 200 IP in the last three seasons that suggests he’s good for 8.5+ P/IP. Metrics can certainly change, but it goes the same for SP as well. Both Strasburg and Corbin saw their HR/FB% almost double in the 2nd half last year in about 90 IP, so they aren’t immnune. For guys like Rosenthal, who doesn’t have much of a track record, I’d love to see the data that suggests Kimbrel-like skills can just disappear from season to season (when not related to injury). There will certainly be fluctuations, but even in small sample sizes it’s possible to get the indicators you need to establish value and identify above-replacement production.
        The cut deadline will have an impact, but that’s why I used 70% instead of something like 50%. I’m be surprised if there aren’t at least 20 RP still rostered in 70% of leagues that are over $3 after Jan. 31st. I’m not sure I’m willing to brush something off to a herd mentality when we have a very high percentage of behavior acting in one direction. Owners “act” as if there is value above replacement by paying premium prices in all leagues, even ours, but we can watch for the trends to change after the deadline and start of the season.
        I’ll send you my list of Top 10 players so you can post them in tables rather than through these comments, which should look a little better…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s