One of the key ways to improve any system is to evaluate it against other systems and actual results. I have listed below MORPS versus 2011 actuals, four year average (2010-2007) and four year weighted average (2010-2007). The weighted average is the last 4 years of data with 2010 weighted five times, 2009 four times, etc. The 2011 actuals reflect the 300 batters that had the most plate appearances. In order to create accurate comparisons, all averages also use 300 batters with the most plate appearances during any given year.
|
|
2011 Actuals |
Weighted Average |
4 Year Average |
MORPS |
|
Games |
37502 |
38264 (-1.99%) |
38377 (-2.28%) |
38246 (-1.95%) |
|
ABs |
128897 |
132992 (-3.08%) |
133347 (-3.34%) |
130313 (-1.09%) |
|
Runs |
16931 |
18594 (-8.94%) |
18774 (-9.82%) |
18385 (-7.91%) |
|
BBs |
12110 |
13435 (-9.86%) |
13468 (-10.1%) |
13013 (-6.94%) |
|
Triples |
718 |
756 (-5.03%) |
759 (-5.4%) |
726 (-1.1%) |
|
SFs |
1065 |
1124 (-5.25%) |
1138 (-6.39%) |
1088 (-2.11%) |
|
Hits |
34208 |
36178 (-5.45%) |
36420 (-6.07%) |
35290 (-3.07%) |
|
Doubles |
6833 |
7331 (-6.79%) |
7416 (-7.86%) |
7163 (-4.61%) |
|
HBP |
1254 |
1319 (-4.93%) |
1337 (-6.17%) |
1263 (-0.71%) |
|
Ks |
24852 |
25217 (-1.45%) |
25028 (-0.7%) |
24926 (-0.30%) |
|
HRs |
3850 |
4242 (-9.24%) |
4273 (-9.9%) |
4052 (-4.99%) |
|
SB |
2685 |
2518 (6.63%) |
2504 (7.23%) |
2514 (6.80%) |
|
CS |
1030 |
922 (11.71%) |
903 (14.1%) |
959 (7.40%) |
|
Errors |
2019 |
2082 (-3.03%) |
2087 (-3.26%) |
2030 (-0.54%) |
|
AVG |
0.265 |
0.272 (-2.57%) |
0.273 (-2.93%) |
0.271 (-2.21%) |
|
OBP |
0.332 |
0.342 (-2.92%) |
0.343 (-3.21%) |
0.340 (-2.35%) |
|
SLG |
0.419 |
0.434 (-3.46%) |
0.436 (-3.9%) |
0.430 (-2.56%) |
|
OPS |
0.751 |
0.776 (-3.22%) |
0.779 (-3.59%) |
0.770 (-2.47%) |
|
BABIP |
0.300 |
0.305 (-1.70%) |
0.306 (-2.03%) |
0.305 (-1.70%) |
As you can see, MORPS predicted overall performance for the top 300 batters better than both the four year average and weighted four year average. For those fantasy owners that play in traditional rotisserie leagues or head-to-head leagues that award points based upon individual statistical performance. However, fantasy owners that play in point based leagues or real time simulation leagues like Baseball Manager (BBM) aren’t concerned about each of these statistics individually. These fantasy owners want to plug numbers into their league formulas to figure out individual performance on draft day. If we plug the above numbers into the BBM formula, 2011 saw 18309 runs produced for these players or 0.488 runs per game. This was 10.6% runs less (8.61% runs/game) than the four year average and 9.67% runs less (7.92% runs/game) than four year weighted averages. In comparison, MORPS projected 5.78% runs less (3.94% runs/game). This is encouraging in a year that saw five year lows and highs in a number of statistical categories.
The next question is “how well does MORPS predict individual player performance?” In lieu of focusing on means which are going to be solid based upon the tables above, I decided to compare individual player projections for the top 300 batters against actual performance in 2011. When I saw the initial results, I was concerned that something was wrong with my projection system so I also compared numbers from ZiPS and Marcel. The comparison uses BBM runs per plate appearance. ZiPS did much better a projecting actual BBM runs. Their number of plate appearances projected for each player was much better for the top 300 than MORPS or MARCEL. MORPS will model plate appearances team-by-team in 2012 to narrow the gap between the systems.
|
|
MORPS |
ZiPS |
MARCEL |
|
Within 5% |
52 |
64 |
52 |
|
Within 10% |
100 |
116 |
101 |
|
Within 15% |
146 |
160 |
142 |
|
Within 20% |
183 |
195 |
184 |
|
Within 25% |
213 |
224 |
208 |
|
Within 30% |
233 |
243 |
224 |
|
Within 35% |
245 |
258 |
242 |
|
Within 40% |
260 |
268 |
255 |
The table is cumulative. Thus, the 52 batters within 5% are also part of the next category. According to the numbers, MORPS projected 233 of 300 players (78%) within 30% of actual performance. This was slightly better than MARCEL (75%) but worse than ZiPS (81%). I am hoping that MORPS can narrow the gap with ZiPS in 2012.
Tags: baseball, data, Draft, fantasy baseball, MORPS, Projections, Statistics, Stats
So perhaps using ZiPS with an adjustment for reliability would be a good way to go?
I have to think that ZiPS already uses some type of reliability calculation within their projections. That is how you typically regress to the mean for most projection systems. I’ve been working with historical data over the last several days to see if I could somehow “crack the formula” that ZiPS uses for their batters. I believe the fact that he models the data for each team and doesn’t distinguish between Major League and Minor League ABs accounts for better projections on plate appearances. It doesn’t really help fantasy owners because we only care about MLB ABs, but it does make it easier to project. The one that I don’t get is the ZiPS accuracy on stats per plate appearance. MORPS beat ZiPS with pitchers. MORPS didn’t really come close with batters. One thing I determined late yesterday was that breaking down the stats by first half and second half allows you to give more strength to second half stats going into the coming year. When I applied this to historical data, MORPS made up some significant ground on ZiPS. I believe that their are some correlations that I have not found yet that may make up the rest.