Effect of game state and opposition on Expt Pts

Despite creating an Expt Pts model (outputs are here for anyone interested) I am acutely aware of its limitations. There are so many factors that can go into whether a shot is successful or not that only taking three elements (shot type, shot location and whether it is a goal attempt or not) seems arbitrary. It leaves the metric incomplete.

Now don’t get me wrong. I still believe that Expt Pts is a better metric than Conversion Rates (which are better again that just counting wides!) but at the moment it is a tool that allows you to quickly hone in on particular areas of the game rather than being an absolute barometer.

So the question arises. How do we make it better?

The first port of call is to look at what other sports are doing. Both Soccer (through Xg) and the NFL (through DVOA) place emphasis on the game state and the quality of opposition.

We are limited in the GAA by the availability of raw data but at Christmas I was able to post the below graph on Twitter (@dontfoul) which shows that there is definitely a game state effect. (People were generous enough not to point out my spelling of “deficit” – D’oh!)

Was there also a “strength of opposition” effect as well? And if so can we combine the two to create a more refined Expt Pts?

Strength of Opposition
The games used in this review are 2015 – 2017 Championship games. In that timeframe we want to somehow grade/tier all the teams to see if playing up, or down, to your level has an effect on Conversion Rates.

So how do you Grade a team (see Note1)? My approach was a mixture of subjective and objective. From ’15 to ‘17 three teams – Dublin, Kerry and Mayo – were ahead of the rest occupying 9 of the 12 semi-final berths. Go back to 2014 and it is 12 of the 16 semi-final berths with the only non-Big three to make the final in those four years being Donegal. They are our (subjective) Tier1.

It is possible to make a whole army of mini tiers all the way down the 32 counties thereafter but we will then run into sample size issues for any output. Plus it is very rare for those teams in Division4 to compete in TV games, which is essentially what we have to work with, so we do not need a whole host of Tiers. Instead we just need big enough ones that are appropriate.

So I came up with a rules based approach to split the remaining 28 counties (excluding New York & Kilkenny) more or less in two.

All teams in that years’ Division1, outside the Big 3, plus all teams in Division2 are Tier2 as is any team from Division3 and 4 that made the Championship QF. This gives us an objective grading system with the ability to upgrade teams that played well but also stable enough year on year.

It is not perfect. It has an element of subjectivity. But it does the job required.

So we have our teams graded according to their overall relative strength. Does it work? Is there an effect? Intuitively it should. It just makes sense that Dublin will convert more against Antrim than they would against Tyrone. Or that conversely Laois would struggle when playing up a level against Dublin but less so when playing Wexford.

Also we have seen this effect in action previously when we calculated Conversion Rates after overlaying the pressure applied to shots (implication being that “better” teams will apply more pressure to shots and vice versa).

Finally we also have an inbuilt test. If the Grade shows itself up in open play it really shouldn’t from frees. Why would taking a free against Tyrone be any different than trying the exact same free against Offaly? So the expectation is that when we break teams’ Conversion Rates down by Grade attempts from play will vary but those from frees won’t.

And that is exactly what happens.

Irrespective of the opponent Conversion Rates on frees (only frees taken inside the 45 were included to remove as many outliers as possible) are very stable. The volumes are decent as well with just over 1,000 frees included.

The big eye opener however is what happens to attempts from play. As you go up the Grades/Tiers there is steady fall off in Conversion Rates. If, for example, Meath play Dublin then the Conversion Rate is 38.7%. Against Kildare it increases to 45.6% and against Wicklow it jumps again to 51.2%. The same phenomenon can be seen in goal attempts. The “better” the opposition the lower the Conversion Rates.

This all makes sense but the implications for Expt Pts, and anyone looking to use any version thereof, are quite big. Expt Pts currently run off averages … those averages are taken as a whole from all games. They need to be calibrated for opponent.

Game State
So now that we have strength of opposition what about game state? The original chart above showed that there was an affect but that was too simplistic. Registering all scoreboard differences as the same is not right (taking a shot two points down in the first half is not quite the same as having an attempt two points down in the 60th minute). We have to further refine the criteria.

Right now we don’t need to define what the optimum criterion is. We are just trying to show/prove if there is indeed an affect. To that end I have extracted all shots taken under a “clutch” situation – defined as any shot in the 2nd half of a game where there is only three points in it – and compared them to “non-clutch” shots.

Again we could define it differently – say only include games within two points from 60th minute onward – but this will give us volume issues.

So what are we expecting to see? Conversion Rates to be lower across the board in clutch situations. The real test however is that that frees should, unlike when looking at Grading above, be affected by this new scenario. If there is “scoreboard” pressure it should affect frees as well as point attempts.

And once again there is an affect. The Conversion Rates for both frees and point attempts from play disimprove in tighter game scenarios. Again all very sensible.
Finally overlaying the two we can see that the greatest discrepancies happen in, subjectively, the scenarios people are most unaccustomed to.

Lower level teams, with a shot of winning the game in the 2nd half against higher quality opponents, convert only ~31% of their point attempts as against ~40% for the rest of the game. The “deer in the headlights” syndrome.

There is also a minor drop off when teams are playing against opponents of similar, or lower, ability but it is nowhere near as stark.

So there we have it Scoreboard pressure is indeed a thing. The opponent grade matters. And whilst very useful the simplistic Expt Pts model is nowhere near complete.

Appendix
Note1; We could look at grading teams via their defensive performance in that period but some teams rarely, if ever, compete in TV games. How do you grade a defence based on performance metrics if you’ve never seen them?

Note2;

2017 Grade2 teams
Division1 that year; Tyrone, Donegal, Monaghan, Cavan, Roscommon
Division2 that year; Cork, Derry, Kildare, Meath, Down, Galway, Fermanagh, Clare
Division 3 or 4 that made it to the QF; Armagh

Grade 3 – Westmeath, Laois, Louth, Wexford, Limerick, Longford, Sligo, Offaly, Tipperary, Wicklow, Leitrim, Waterford, Antrim, London, Carlow, London