This week the BBI team released two new stats we’re excited about: Offensive Talent and Offensive Optimization. You can find each in our various applications included in our $5/month Data & Tools Package.
TL;DR: Offensive Talent measures what a player’s skills should produce, Offensive Optimization measures how well their environment lets that talent show up.
Offensive Talent seeks to distill a player’s situation-agnostic skill offensively into a single metric by using role, usage, and skills to predict a player’s offensive impact per 100 possessions (O-LEBRON). This metric is designed to separate what a player did given their team’s scheme and roster context from what they are capable of doing, a distinction that impact metrics often blur.
This metric incorporates our skill data for 3PT Shooting, Midrange Talent, Finishing Talent, Playmaking Talent, Offensive Rebounding Talent, Screening Talent, among a few others.
But with 12 different offensive roles, how do we decide weightings? This approach allows us to lean on over a dozen seasons of actual NBA play to tell us what actually matters and how much it matters for each role, as opposed to the BBI team making hundreds of judgement calls subjectively setting our own weightings.
Calculating Offensive Talent
The math is one process using three models trained on the same data: an XGBoost model, a generalized additive model (GAM), and a final ensemble that combines the strengths of both.
First is an XGBoost model. This model is good at finding interactions between variables, as some skills will be important for some roles and not for others. For example, we don’t want to penalize Roll & Cut Bigs for being bad perimeter shooters (using 3PT Shooting Talent) but want to give extra credit to Movement Shooters for excellence in that skill set.
The downside of this XGBoost formulation is it can overfit to the population of really good players. For example, a lot of really good offensive players don’t have very good screening talent. This can lead the model to think things like: “if you are a worse screener you tend to be a better player.”
This may be observed in the data, but we don’t think it’s true and don’t want to penalize good screeners. For this reason we need a second model: a generalized additive model (GAM).
The GAM is monotonically constrained for each of the skills, meaning that being better at each skill will always lead the model to think you are a better player. (We could monotonically constrain the XGBoost but sacrifice predictive power when we do that). The GAM also fits splines on each variable, which allows the increase to be nonlinear. For example, the effect of going from a D to C shooter isn’t as big as the effect of going from a B to A shooter.
These two models separately are able to explain ~70% of the variance in O-LEBRON. We then ensemble the models into a 3rd model that takes the best features of both to get a final estimate for O-LEBRON, which we’re referring to as Offensive Talent. This final estimate explains 78% of the variance in offensive impact (O-LEBRON).
An illustration from the Year Over Year tab of our Leaderboards Tool illustrating some of the top Offensive Talent risers over the years.
Impact vs Talent Data
After years of publishing impact data with the PIPM and LEBRON metrics, the biggest misconception/misuse of that information is interpreting impact data as talent data. Impact data measures results; this is talent data.
What Offensive Talent isn’t considering is on/off-court +/- data (as is used for LEBRON and other impact metrics). If you’re on-court while your team goes on a scoring run but didn’t do anything to contribute, your +/- might see a boost but your Offensive Talent may not see the same rise.
Conversely, if during that scoring run you have strong rim self-creation and difficulty-adjusted shot making, are playmaking for teammates left and right for high quality looks, & are screening to open up better shots, your skill data will see a boost and your Offensive Talent will rise along with them.
In short, this evaluates much more directly what you did and how well you did it, not what the scoreboard implies you might have done.
When we examine year-to-year stability, Offensive Talent is more consistent than O-LEBRON, O-EPM, O-BPM, and Luck-Adjusted O-RAPM. That’s expected: player skills and roles change more slowly than team context, lineup quality, and scheme fit.
Because offensive talent is more stable than single-year impact metric alternatives, it is a better predictor of future impact, especially for when a player changes teams or roles.
When a player changes teams or roles, Correlation drops and accuracy (via RMSE) usually degrades for impact metrics. This tells us that these metrics aren’t just measuring the player, but also measuring the situation the player is in. When the situation changes, the stat becomes less predictable.
Offensive Talent hasn’t fully removed that effect, but performs well compared to single-year impact metric alternatives. A multi-year version of Offensive Talent should similarly perform well compared to multi-year impact metric alternatives.
Impact of Roles
Just as changing a player’s position in NBA 2K will impact their overall rating, altering a player’s offensive role reconfigures the math calculating their offensive talent. How much does having a strong floater game matter? Or strong Playmaking? Or a great pull-up 3-point game? It all depends.
It’s not as important to be a strong 3PT Shooter as a Roll & Cut Big as it is for a Stretch Big, nor as key to be a strong Playmaker for a Movement Shooter as it is for a Primary Ball Handler.
The modeling values skills based on how much they’ve mattered to players over the past decade+ to create impact, and considers how various combinations of skills interact to generate value by amplifying strengths or covering for weaknesses.
And just like in 2K, the same player can look average or elite depending on how well their role matches their skills.
Offensive Optimization
The remaining 22% of variance in O-LEBRON (what Offensive Talent doesn’t explain) is where Offensive Optimization comes in. If we expect you to be a +3 player based on your job and skill sets, but you’re a +1 player, you have a -2 Offensive Optimization rating.
As an Xs and Os guy, I can tell you that while playbooks have a ton of overlap, there’s quite a disparity in offensive scheme in basketball in terms of the strength of a team’s play design, play calling, and tactical adjustments. Each of those impact success, and each trickles down to lineups and players.
Likewise, lineup construction strength/weakness can make the whole be greater than, equal to, or less than the sum of its parts. You can oversaturate a lineup with Shot Creators, or have no bigs and limit ball screen effectiveness, or mash too many Roll & Cut Bigs together and have no spacing. Lineup construction matters, and you can say the same for roster construction generally. These are the three key areas that will drive this optimization data, and help us answer questions like “how well optimized is player X for a Shot Creator?”
All together, it’s possible for players to rate highly in Offensive Talent but show modest impact, and vice versa.
Another element of optimization not measured by comparing Offensive Talent and O-LEBRON is role fit, which we can now calculate by comparing Offensive Talent in a player’s current role with the maximum Offensive Talent value calculated for the same player among our 12 Offensive Roles. This helps us answer “is player X in the best role for their skills?”
For Overall Offensive Optimization, we compare a player’s actual offensive impact (O-LEBRON) to the maximum Offensive Talent value they would have in any of our 12 offensive roles, encapsulating both forms of optimization described above.
Offensive Talent captures what tends to persist. Offensive Optimization explains why impact doesn’t always follow. Optimization is structure, not noise. Higher stability suggests Offensive Talent is a strong baseline for future offensive expectations, though optimization and context still matter.
A look at the Offensive Talent x Offensive Optimization landscape among 2019-20 starters, from our Headshots & Scatterplots Tool
Wrapping Up
We’ve had opportunities to work on a lot of cool data. This may be the coolest. It’s easy to lose track of time while deep diving players on old teams and leaderboards for prior seasons.
Is it perfect? We won’t claim so, but we put a lot of work, heart, and time into creating a single metric we feel we can stand behind for the >6,800 players over 12 seasons (so far) we’ve calculated this for.
After countless hours exploring the data, I haven’t found meaningful biases or systematic gaps that lead to distorted evaluations. This matches my eye test, and I hope it aligns with what you see on the court as well.
Next Steps
Among what you can expect to see in the future related to this:
- Multi-Year Talent metrics for each skill set that consider more than just the current season and are rolled up into a multi-year version of Offensive Talent.
- A dashboard where subscribers can analyze team seasons and compare coach optimization data.
- A Defensive Talent metric that matches this approach for the defensive end of the court.
- A metric similar to LEBRON that utilizes Offensive and (when available) Defensive Talent in place of boxO-LEBRON and boxD-LEBRON for the prior.
- A version that includes our Teammate Context stats in the math for purposes of inclusion in trade tools showing how players, lineups, & teams would be expected to shift as you move pieces around.