Artificial Intelligence: Meet The Potential Scouting Director Of The Future

One of the biggest ways AI can help baseball teams over the next decade is by improving team’s draft and player valuation models.

And there may be few more difficult tasks in baseball than improving a draft model, because the very nature of the baseball draft makes it difficult to know for years whether a pick was good or bad.

In other sports, it doesn’t take long to know how a draft should have lined up. And even in baseball, the best picks become obvious quickly. The Angels didn’t need to wait long after the 2009 draft to know that picking Mike Trout 25th overall was a great choice.

But all too often, the success or failure of a pick isn’t known for years. Consider the top of the 2017 draft. The Twins took high school shortstop Royce Lewis No. 1 overall. Even six seasons later, it’s hard to say how astute that pick was, largely because Lewis has battled injuries. 

Now, it looks like the Reds choose wisely at No. 2 by picking high school righthander Hunter Greene one spot ahead of the Padres’ pick of prep lefthander MacKenzie Gore. Heading into 2020, however, Gore was arguably the best pitching prospect in baseball, while Greene was struggling with injuries and getting squared up.

The Braves’ selection of Vanderbilt righthander Kyle Wright at No. 5 overall has similarly waxed and waned. He struggled for several years but broke out in 2022, his sixth pro season.

Six years later, we’re just getting some clarity on the 2017 draft. Properly evaluating the 2021 and 2022 drafts will need to wait for several more years.

If teams are trying to see how well their draft model lines up a draft board, they have to look at drafts from a number of years ago. So they are looking at drafts from the 2010s to best project who they want to draft to be in their lineup in the 2030s.

The challenge of that task is obvious, especially in a sport where the conditions of the game change. The liveliness of the baseball appears to have changed multiple times over the past decade, which affects which players succeed. Stolen bases were exceedingly rare in the 2010s, but they’ve come back because of rules changes in 2023.

So whenever a team decides to let AI take a run at fashioning its own draft model—based on potentially millions of iterations that are tested over and over to find the proper weighting for potentially hundreds of thousands of different data points—a team willing to use an AI-based draft model will be taking a leap of faith, knowing it won’t know how well the model did until five or more years in the future.

WHAT IS A DRAFT MODEL?

Long before there were draft models and player-evaluation models, there were scouting directors. And in their heads, scouting directors had to have a form of a model, long before the first team ever bought a computer.

To line up a draft board, a scouting director had to weigh how strongly different scouts lined up their pref lists. The opinion of a veteran scout who always seemed to have pitchers pegged needed to be weighed more strongly than a first-year scout who was still getting established.

Then, the director must decide how to line up a less athletic, but more productive college player versus a more promising, toolsier college player who hasn’t had nearly as much success in his career.

Oh, and the top player is looking for $2 million more in signing bonus than the player the scouting director believes is nearly equivalent in talent. That extra $2 million in bonus pool money could lead to signing a couple of late-round picks.

When a scouting director was lining up a team’s draft board, he was building a draft model in his head. Figuring out how to weigh a multitude of variables has long been part of the scouting director’s job.

But we humans struggle to hold too many different pieces of information in our brains at the same time when making decisions. That’s where computer models become useful. They can comfortably deal with a large number of variables and data points at the same time, helping to synthesize a multitude of weights for different factors before bringing them all together into a numerical value or ranking order.

That’s why most MLB teams now use algorithms and computer modeling to help line up their draft boards. The difference between current draft boards and AI-driven boards of the potential future is that the current models are programmed by humans who determine the way different pieces of information are weighted. 

In the future, it’s possible that draft boards will be fully built by computers.