What’s More Valuable: A Sack or A Holding Penalty?

Pass rushers can disrupt an offense’s plan in a number of ways, most obviously by sacking the Quarterback.  More and more, though, the public has realized that there are other ways to disrupt the offense, without actually getting a sack.  Quarterback hurries and hits have appeared in television broadcasts for a few years now, and SI’s Greg Bedard is suggesting a new, yet unnamed stat, that would credit players for assisting in a teammate’s sack, for example.

Interestingly, he also rewards players for drawing holding penalties, something I’ve only previously seen on advanced stats sites.

The average sack sets the offense back by about 6 yards (it was 6.4 yards lost per sack in 2012 according to ESPN) and costs a down, while the holding penalty costs 10 yards with no loss of down.  So how does a holding penalty measure up against a sack?  Which is more valuable to the defense (or put another way, when an offensive lineman gets beat, should he hold his opponent, or allow the sack)?

For this analysis, I am using the Expected Points metric from the excellent Win Probability Calculator on Brian Burke’s Advanced NFL Stats site.  I’m also only going to look at 1st-and-10 situations, since they are the most common, but I recognize that things may be different on 3rd downs.

The chart below shows the loss in a drive’s Expected Points after a sack or holding penalty.

2013-09-14 Sack vs Hold 1st&10 EP

We see that a sack always sets the offense back more than a holding penalty, despite only being a 6-yard loss (again, on average).

What is interesting, is how the relationship between a sack and hold changes.  Most obviously, a holding penalty is a lot less damaging deep in an offense’s own territory, due to the half-the-distance rule.  In these cases, the offense in not actually penalized 10 yards.  A sack is only slightly more damaging than a hold penalty around midfield, likely due to the neutrality in field position, and inside the 10-yard line, likely due to a high field goal make rate.

I also find the spike in a sack’s impact between an opponent’s 10- and 40-yard line to be interesting.  My hypothesis is that somewhere around the 25-yard line (where this spike peaks) is where a few yards can make the biggest difference in field goal success rate.  After a holding penalty (rather than a sack), the offense has an extra play to make up the 4-yard difference, which can usually be done (in 2012, Arizona had the league’s worst yards per play, at 4.1).

This hypothesis is backed up by the another metric on the WP Calculator – 1st Down Probability.  When it’s 1st and 10 at an opponent’s 25-yard line, a holding penalty gives the offense a 39% chance of converting a 1st down, while a sack gives them a marginally smaller 38% chance.  I think the rest of the gap in EP is due to using the extra down to get in better field goal position.

So, when rewarding drawing a holding penalty, how should it compare to a sack?  It seems too complicated to change the value of the drawn penalty based on field position, so to simplify things, we’ll just use this fact: a holding penalty’s impact on expected points is, on average, 78% of a sack’s impact.  It seems that Bedard has it right when he gives a player 0.75 points for drawing a hold.

Upon Further Review, Week 1: Examining Jim Harbaugh’s 4th Down Decision vs. Green Bay

This is the first post in a new weekly series, Upon Further Review, in which I will look at a key coaching decision from the previous week (to challenge a call or not, go for it on 4th down or kick, etc.).  Initially, I wanted to look at the Saints decision to kick a field goal on 4th down, up by three, late in the game against the Falcons, but Advanced NFL Stats beat me to it.  Instead, let’s look at the 49ers’ 4th down decisions late in the GB-SF game.

San Francisco had the ball late in the 4th quarter, leading Green Bay 31-28.  With exactly three minutes to go in the game, and coming out of a timeout, the 49ers faced 4th and 2 on the Green Bay 36.  The options in this scenario are to go for it, attempt a 53-yard field goal, or punt to pin the Packers deep in their own territory.  Coach Harbaugh decided to go for it.

Was it the right decision?  For this analysis, I am using ANS‘s Win Probability metric, which is simply the likelihood that a given team will win (expressed as a %), taking into account the score, time remaining, who has the ball, and down and distance, based on historical NFL data.

2013-09-09 49ers 4th and 2 WP

Source: Advanced NFL Stats’ 4th Down Calculator

According to the 4th Down Calculator, the Niners had a 60% chance of converting the 4th-and-2, and doing so would boost their Win Probability (WP) to 94%, while a field goal would have been good about half the time.  Even then, San Francisco’s WP would only have been 75%.  In either case, failure to convert would leave them with a probability of winning just below 70%.  Between the two options, going for the first down is the better choice.

What is interesting, however, is that the calculator actually recommends punting.  Playing conservatively in this case provides the opportunity to pin Green Bay deep in their own territory, and gives the 49ers an 85% chance of winning the game.

In reality, the 49ers aggression paid off when Colin Kaepernick hit Anquan Boldin for a 15-yard pass, and a first down.  But this wasn’t the end of the 49ers 4th down decision-making.

Two and a half minutes later, the 49ers faced another 4th down, this one 4th and 4 from the Green Bay 15 yardline.  At this point, San Francisco was in good shape, with a WP of 97%.  Here, they opted to kick a field goal, and Phil Dawson converted, putting his team ahead by 6.  Again, though, the numbers say Coach Harbaugh could have made a better decision:

2013-09-09 49ers 4th and 4 WP

Source: Advanced NFL Stats’ 4th Down Calculator

It seems that even by going for it and failing, the 49ers would have further improved their odds of winning.  While this seems counterintuitive, I think this is due to the fact that if the 49ers do not convert, Green Bay takes over deep in their own territory, with no chance of a long kickoff return (and only about 25 seconds remaining).

And wouldn’t you know it, after the field goal increased the 49ers’ lead to six, the Packers did end up with one play from 42 yards out to win the game, but could not convert.

Conclusion: In neither case did Jim Harbaugh make the ‘statistically optimal’ decision, but the margin was pretty thin.  Since the ANS 4th Down Calculator uses league-average data, I think we can say that Harbaugh must’ve felt his team’s odds of converting 4th and 2 were better than average (SF did convert on 8 of 12 fourth downs last year, good for 5th in the league), causing him to decide on going for it.  As for the decision to kick on 4th and 4, perhaps the prevalence of touchbacks that began last year actually made kicking the field goal the right decision.

Were you watching the game?  What do you think Jim Harbaugh should have done in these scenarios?

The Sport of Numbers Returns

After a 5-month break, the blog returns!  Check back soon for a new series, Upon Further Review, in which I will examine coaching decisions from the previous weekend’s NFL games.

In the wise words of Bart Scott: “Cant’ Wait!”

Predicting Final Four Teams Using In-Season Poll Position

I recently saw a tweet mentioning that all of this year’s Final Four participants peaked in the polls in late January.  Louisville hit #1 in the Coach’s Poll on 1/14, Syracuse was #3 on 1/21, and Michigan was #1 and Wichita St. was #14 on 1/28.  This got me thinking – coincidence?  Or trend?

So I set out to answer the question, “When do Final Four teams typically peak during the season?”  Conventional logic would say that these teams’ peaks occur at the end of the season, enabling them to win the four (or five) consecutive games necessary to reach the Final Four, but that’s not what happened this year.

I used StatSheet.com’s NCAA Basketball rankings site to look at the AP Poll trends of Final Four participants this year and over the previous ten seasons.  I included only the AP Polls from January through March in my analysis, in order to assess when teams’ performance actually peaked, without being influenced by early season expectations (pre-season polls).

What I found is that the last two weeks of the season are when the most teams peak, as conventional wisdom would suggest, but overall, there is no clear pattern of peaking at a specific point during the season and making it to the Final Four.

The chart below shows when teams reached their highest spot in the AP Poll during the season they made the Final Four.  Week 1 represents the first poll in January in any particular season.

Final Four AP Poll Peak

And here is a graphical representation of when each Final Four team reached its highest spot in the AP Poll (in dark areas).  As with the first chart, Week 1 represents the first poll in January.

Final Four AP Poll Peak Picture

It looks like March Madness truly can be random.

Official Sport of Numbers Bracket

Here it is, my bracket (courtesy of ESPN’s bracket challenge).

And yes, I have Florida winning it all.  Before you call me a homer (which I am), please read why the Gators’ efficiency margin, which is the highest in 11 years, makes them as good a championship pick as any.

But you should still feel free to mock me endlessly when your co-worker’s dog’s bracket performs better.

Brian Beard Bracket

NCAA Tournament – East Region Preview

This is a preview for the East region.  You can read my preview of the Midwest, West, and South regions here.

The Favorite:

Indiana.  Indiana comes into the tournament with a historically high efficiency margin at 25.8, good for 2nd in the nation this year, and the 4th highest in the last 11 seasons.  Teams with such high efficiency margins during the regular season tend to have a lot of tournament success.  Indiana excels at offensive rebounds and free throws, which should help them avoid an early upsets, and a tough schedule has prepared them for all types of match-ups they may see in this tournament.

The Darkhorse:

Butler.  The Bulldogs are the only team to beat both Indiana and Gonzaga this season, giving them two wins over #1 seeds.  Butler is led by coach Brad Stevens, who always has his team ready come tournament time.  His team dominates on the boards, and tends to wear opponents down during the course of the game.  A tough match-up with Miami looms in the Sweet 16, but if Butler gets past the Hurricanes, they could be just a rematch with Indiana away from 3rd Final Four appearance in just 4 years.

Most Likely First Round Upset:

#14 Davidson over #3 Marquette.  Davidson protects the ball well and relies heavily on threes, making them dangerous should the shots start falling.  Rebounding will be the key in this matchup, as both teams figure to vie for Marquette’s misses (MU is 25th in offensive rebounding, Davidson is 37th defensively) while seemingly ignoring Davidson’s (265th offensively to Marquette’s 243rd defensively).  Also of note – Davidson is the nation’s top free throw shooting team.

Most Likely Double Digit Seed to Make the Sweet 16:

California.  The Cal Bears are a team that played against one of the toughest non-conference schedules in the country this year.  Unfortunately, they lost a lot of their games against top competition.  However, this team has only lost three times since the end of January, against 9 wins and their path to the Sweet 16 is through UNLV, whom defeated California by just one point back in December, and Syracuse, a team that has looked at times like a championship contender and at others like they are capable of losing in the first round.

NCAA Tournament – South Region Preview

This is a preview for the South region.  You can read my previews of the Midwest, West, and East regions here.

The Favorite:

Kansas.  The Jayhawks find themselves with a difficult draw, but this team is up to the challenge.  Kansas rebounds well, makes a high percentage of their free throws, and, led by coach that has had his share of tournament successes (championship in 2008) and  failures (second round upset as a 1-seed in 2010), should be able to overcome the likes of Michigan or VCU and Georgetown or Florida to advance to the Final Four.

The Darkhorse:

VCU.  The Rams may not be a legitimate ‘dark horse’ pick, as many are taking them to reach the Final Four for the 2nd time in 3 years.  VCU will pressure the ball, and leads the nation in forcing turnovers.  The key match-up for them could come in the 2nd round if they face Michigan, who turns the ball over less than any other team in the country.  VCU also protects the ball well, and creates second chances by crashing the boards on offense.

Most Likely First Round Upset:

#11 Minnesota over #6 UCLA.  With the loss of Jordan Adams to injury (which he suffered during the Pac-12 tournament), the Bruins could be in trouble.  As the freshman was 2nd on the team in points and accounted for over a quarter of their steals, the Bruins will miss certainly miss him.  Additionally, Minnesota comes in as the nation’s leading offensive rebounding team, a spot where UCLA has struggled this year.

Most Likely Double Digit Seed to Make the Sweet 16:

Oklahoma.   As a region with quality across the 1-5 seeds, the South doesn’t figure to be the most likely region to produce a double-digit Sweet 16 participant.  That said, Oklahoma presents the best chance for the underdogs.  If the Sooners are able to get past San Diego St., they will face a Georgetown team that sometimes struggles to score.  Additionally, Oklahoma is led by upperclassmen and is the 10th best free-throw shooting team in the country, which could help them succeed in tense, late-game moments.

NCAA Tournament – West Region Preview

This is a preview for the West region.  You can read my previews of the Midwest, South, and East regions here.

The Favorite:

Ohio St.  The Buckeyes’ defense, led by Aaron Craft, excels at getting stops – they are 15th in efficiency and 19th in turnover %.  They also rebound well and bring experience, as none of the team’s 8 regulars are freshmen.  Ohio St. finds themselves with a favorable draw,  and may not face a true test until the Elite 8.

The Darkhorse:

Pittsburgh.  Pitt always seems to lose before their seeding would suggest, so why can’t the opposite happen in a down year for the Panthers?  Pittsburgh boasts the nation’s 3rd highest efficiency margin (and the 22nd highest in the last 11 years) at 23.2, and we already know what a high efficiency margin can mean for a team’s tournament chances.  The Panthers have succeeded this year because they rebound well on both ends of the court, and take care of the ball.  They could surprise Gonzaga in the 2nd round and meet Ohio St. for a spot in the Final Four.

Most Likely First Round Upset:

#11 Belmont over #6 Arizona.  Belmont fits the mold of classic upsetter.  They force turnovers at a high rate (11th in the nation), rely heavily on 3-pt FGs, and shoot well from the free throw line.  On the other side, Arizona is an average ball-handling team, turning the ball over on almost 20% of their possessions, and allows opponents to make 36% of their 3-pointers, good for 277th nationally.  I’ve named this match-up the most likely upset of the first round.

Most Likely Double Digit Seed to Make the Sweet 16:

Belmont.   See above.  In what is probably the weakest region of the bracket, Belmont could be this year’s version of Davidson, shooting their way to the second weekend.

NCAA Tournament – Midwest Region Preview

This is a preview for the Midwest region.  You can read my previews of the West, South, and East regions here.

The Favorite:

Louisville.  The Cardinals are the #1 overall seed for a reason.  They rank 4th nationally in efficiency margin at 23.1, and use a smothering defense that ranks 2nd in turnover rate to keep the pressure on opponents once they get ahead (forcing a turnover on over 27% of their opponents’ possessions).  On offense, Louisville rarely relies on three-pointers, making them less susceptible to an upset, and creates second chances by rebounding over 38% of their misses, good for 16th in the country.  Louisville should be the toughest out in the tournament’s toughest region.

The Darkhorse:

Saint Louis.  The Billikens haven’t gotten much attention this season until recently, when they won the A-10 tournament.  They excel in turnover margin, ranking 37th in offensive TO% and 30th in defensive TO%, something that will come in handy should they meet Louisville in the Sweet 16.  Additionally, Saint Louis is 19-3 since Kwamain Mitchell returned from an injury in late December, with 5 of those wins coming against ranked opponents.

Most Likely First Round Upset:

#11 St. Mary’s over #6 Memphis.  For this to occur, St. Mary’s first has to defeat Middle Tennessee St. in the First Four.  Memphis is upset-prone due to their inability to protect the ball (226th nationally in turnovers) and shoot free throws (258th).  St. Mary’s presents a challenge with their strong rebounding and three-point shooting ability.  You can read more about why I think Memphis is a good upset candidate here.

Most Likely Double Digit Seed to Make the Sweet 16:

St. Mary’s.  See above, though I don’t think this region has any double-digit seeds that are very likely to make it to the tournament’s second weekend.  This is partly due to the fact that the Midwest region has some of the toughest top seeds (St. Mary’s would likely have to go through Michigan St. in the 2nd round, for example).

Could Gonzaga Be the First 1-Seed to Lose to a 16-Seed?

You probably know that a 16-seed has never beaten a 1-seed in NCAA tournament history, and in nearly all cases, the 1 vs. 16 games aren’t even close.  Last week, I examined why 16-seeds never win, beyond the obvious fact that 1-seeds are the tournament’s best teams.  I found that because of the way tournament teams are selected (i.e. conference tournament champions can come out of nowhere to steal a spot), 16-seeds are often much, much worse than their 15-seed counterparts.

In 2013, however, we have an interesting match-up in #1 Gonzaga vs. #16 Southern.

Efficiency Margin indicates that Southern may be the best 16-seed in the last 10 years.  With an EM of 16.1 this season, Southern is far above the average of 2.8 for a 16-seed.  Not only that, but the previous high EM for a 16-seed was 13.3 by UNC-Asheville last year.  In fact, take a look at the first round result of only two 16-seeds from the last 10 years with an EM above 10 (EM in parenthesis):

  • UNC-Asheville (13.3) lost to Syracuse 72-65 in 2012
  • East Tenn. St. (11.0) lost to Pittsburgh 72-62 in 2009

These games were much closer than your typical 1 vs. 16 match-up, where the margin of victory regularly exceeds 20 points.

Throw in the fact that Gonzaga is a bit of an unknown quantity for a 1-seed, having played a relatively weak schedule, and you have the potential for an upset of historic proportions on your hands.  While I certainly wouldn’t say it is likely that Gonzaga loses to Southern, it seems to be a possibility at the very least.

Will you be picking a 16-over-1 upset this year?