What Did We Even Write About Before AI?
Directionally Correct Newsletter, The #1 People Analytics Substack
By: Zach Williams & Cole Napper
Introduction
HR’s core problems haven’t meaningfully changed in a hundred years. Before AI, HR and people analytics had problems worth solving. We still do. What’s changed isn’t the questions HR is asking, it’s the toolbox we’re using to answer them. And the reality is it’s the constraints that have shifted.
Constraints
Pre-GenAI – Scarcity: Characterized by limited data, limited resources, slow cycle times, and manual work.
Post-GenAI – Abundance: Characterized by easy-to-generate insights, but difficulty in trusting and organizing them.
Not long ago, a people analytics project meant six months of data wrangling from disparate sources, stakeholder alignment, and slide-building before a single insight reached a decision-maker. Advances in data warehousing and visualization collapsed that timeframe to six weeks, then modern dashboarding brought it down to six days, and as teams matured and machine learning entered the mix, six hours became the new benchmark. Now, with tools like Claude handling data analysis end-to-end, we can even operate in six minutes on occasion. Which raises the obvious logarithmic improvement question: how long before it’s six seconds?
The New Bottlenecks
AI has changed what’s measurable, shifted where the bottlenecks live, and amplified the cost of mistakes. The “intelligence” layer on which people analytics teams have built their value proposition is collapsing before our eyes. However, the competitive advantage was never the insight itself. In Cole’s book People Analytics he terms the key contributions the Tree of Value: The integration of people analytics, workforce planning, talent intelligence, and behavioral science into something that actually changes decisions. AI can commoditize the analysis. It can’t commoditize the judgment, the organizational trust, or the human expertise required to combine those disciplines in ways that hold up under scrutiny.
Back to Basics
The core components of people analytics have evolved. Let’s look through four examples of use cases from people analytics about how the future looks compared to the past.
1. Attrition & Retention
At some point in the last decade, almost every people analytics team built a flight risk model. Most of them also have a quiet story about what happened when they tried to actually use it.
The reason retention always mattered is obvious. Attrition is expensive (except for, apparently, right now…) in ways that are easy to calculate and painful to explain to a CFO. Replacement costs, productivity drag, institutional knowledge walking out the door, and the quiet toll it takes on the people who stay and watch it happen. It was never a soft problem. It always had a quantifiable price tag.
The Old Way
The classic approach for understanding attrition was survival analysis and flight risk modeling. You’d pull together tenure, performance ratings, compensation data, maybe some engagement scores, if you had them, and build a model that scored employees likelihood to leave. The model would run, the list would get generated, and then things would get murky.
The prediction was never really the hardest part. We could generally get in the ballpark for actionable decisions. The hard part was what came next. Who do you share the list with? What can they do with it? What happens if a manager reads between the lines of the sudden interest from their HRBP and begins to box out an employee preemptively instead of retaining them? What’s your legal exposure if you knew someone was a flight risk, did nothing, and they later filed a discrimination claim? What’s your ethical exposure if the model is systematically flagging certain demographics as higher risk because of historical patterns baked into the training data?
The bottleneck was never really prediction. It was intervention design, organizational will and risk appetite, and the fact that acting on uncomfortable signals at scale is genuinely hard in ways that a model output doesn’t solve for you. By the time most organizations had worked through even half of those questions, the person on the list had already accepted another offer.
Why It Still Matters
AI doesn’t change the question. It dramatically raises the stakes of the unanswered ones.
You can now predict attrition with a level of granularity and speed that would have seemed like science fiction ten years ago. Passive signals, communication patterns, calendar behavior, performance trajectories, external labor market conditions. The inputs available to a modern attrition model are orders of magnitude richer than anything we had before. The insight is cheaper and faster than it has ever been.
Yet the dilemmas that were always lurking underneath this problem haven’t kept pace. We’ve gotten much better at generating the list. We haven’t gotten meaningfully better at knowing what to do with it, who’s accountable for acting, or what guardrails belong around a signal this consequential.
The abundance problem in attrition is different from other areas of people analytics. In engagement, the risk of over-production is mostly about attention and action capacity. In attrition the risk, you can now generate predictions granular enough to influence individual employment decisions, at scale, faster than any governance structure most organizations have in place can reasonably review. That’s not a reason to avoid building better models. It’s a reason to be honest that the hard work was never the modeling. It was always the organizational processes around what happens after the model runs.
The Upgraded Mindset
What does a realistic, ethical, legally defensible retention action actually look like? Who owns it? What’s the minimum signal required to trigger it? What actions are off the table regardless of what the model says? What’s the review process when the model flags someone from a protected class at a higher rate? Those questions aren’t new. When predictions were slow, expensive, or didn’t really work, organizations could afford to treat governance as a future problem. When predictions are fast and cheap, governance is a right-now problem.
It’s also worth naming the connection between attrition and the hiring decisions that came before it. The best retention strategy isn’t a flight risk model. It’s a hiring process that selects for people who are genuinely set up to succeed in the role, the relationship with the manager, and the culture they’re walking into. Attrition and quality of hire were in a lot of ways the same problem wearing different clothes. AI makes that connection easier to see and easier to measure, if you’re looking for it.
Keeping good people is still about the quality of the environment you build for them. That’s the real problem to solve.
2. Quality of Hire
The goal of quality of hire has always been more or less the same: Did we hire someone who actually made the business better? Not someone who interviewed well. Not someone who “felt like a culture fit.” Someone who ramped quickly, performed strongly, and stayed long enough to compound their impact.
Quality of hire matters because hiring is one of the most expensive and costly to reverse decisions an organization makes. A bad hire costs money, kills culture, and wastes time. A great hire compounds value for years (i.e., see employee lifetime value).
The Old Way
People analytics tried to measure quality of hire with lagging indicators alone:
First-year performance ratings
12-month retention
Hiring manager satisfaction and onboarding surveys
Time to productivity proxies
Sometimes we would build models to see which interviewers were good at consistently selecting high performers. Depending on the role and company, various assessments and psychometric assessments might have been applied. But quality of hire has always been messy and squishy. Performance ratings were subjective. Manager surveys were biased and inconsistent. By the time we knew if someone was a high performer, it was often 12–24 months later. We weren’t short on questions, we were short on signal and speed which created an inability to confidently act at scale.
Why It Still Matters
AI doesn’t change the question quality of hire intends to address. But it does give us an entirely new toolset to tackle the problem. Yet, speed isn’t the value-add here like most places AI is deployed. Speed without context just scales regret. You can’t know if someone is a star performer after one day on the job. Hire the wrong people faster and you’ve industrialized a bad process.
The smart way to deploy AI for predictive quality of hire is grounded in the decades of hard lessons that came before it. When you strip away the noise, the champion of hiring for performance has been: structured interviewing.
Schmidt & Hunter’s 1998 meta-analysis covered decades of research across 19 selection methods and put the validity coefficient for structured interviews at .51, compared to .38 for unstructured. That’s not a marginal difference. Huffcutt & Arthur (1994) found that highly structured interviews approached the predictive validity of cognitive ability tests. Structure doesn’t just incrementally improve the interview, it transforms it into a categorically different instrument. McDaniel et al. (1994) analyzed over 245 validity coefficients and confirmed the gap held across thousands of studies. Sackett et al. (2022) brought the most comprehensive reanalysis of personnel selection research in decades and the structured interview held its position as one of the strongest predictors of job performance. The evidence has had 25 years to fall apart. It hasn’t.
The problem was never that we didn’t know what worked. The problem was that structured interviewing is hard to scale and was almost impossible to score adherence.
That’s exactly where AI changes the calculus. Not by replacing the structured interview, but by making it far easier to monitor, nudge, and score adherence at scale. Zach calls this the Structured Interview Integrity Score (SIIS). In this framework you’re actually scoring the interviewer, not the interviewee, on how well they adhered to structured interviewing best practices. Using transcripts or written feedback, you can score the integrity of the interview by looking at things like:
Used standardized questions – Did the interviewer stick to the same core questions for each candidate?
Captured behavioral evidence – Is feedback based on specific past actions (what they did) instead of impressions?
Stayed job-relevant – Are comments tied to defined and assigned competencies rather than “culture fit” or personality?
Aligned ratings to evidence – Does the score match the written justification?
Avoided bias signals – No comparative language, personality-based adjectives, or irrelevant traits.
Showed scoring discipline – Clear consistency across interviewers
The SIIS framework is a new opportunity to finally leverage structured interviewing as part of the predictive measures of hiring outcomes. (If you want to learn more about this topic, Sun Park and Cecil Wolfe from Salesforce will be presenting it at SIOP 2026 on the work they’ve pioneered at Salesforce).
3. Listening & Engagement
Leaders have sometimes wanted to know what is going on with their employees. Not just to know if employees were unhappy. To know why, where, and what to do about it before the unhappiness becomes attrition, “retiring on the job”, or lower productivity.
Engagement matters because the research connecting it to business outcomes is about as solid as anything in organizational psychology. Productivity, safety, absenteeism, customer satisfaction. “Engaged” workforces outperform disengaged ones on nearly every metric that matters to a CFO, or at least that’s what we’d like to think.
The Old Way
The engine of organizational listening was the annual engagement survey, peppered with additional employee lifecycle surveys (New Hire, Exit, Pulse, etc.). You would field a psychometrically validated survey instrument – from reputable vendors or something homegrown – wait six to eight weeks for results, spend another month building all the decks, and then roll findings out to managers with self-service tools who had, at best, a mixed track record of actually doing anything with them.
The model had problems that were structural. Annual surveys created a fundamental mismatch between survey results and action. By the time results landed, the moment had passed. This lack of action made disengaged employees less likely to answer honestly next time.
Open-ended survey comment data only made it worse. Processing thousands of comments at scale was expensive and slow, so many executives and HR leaders would spend weeks going through the character-building exercise of reading all the comments. And many organizations either ignored them, or summarized them into broad themes and then didn’t do much else with them.
Action planning was a multi-month, sometimes even multi-year process. The survey said morale was low in a region. Leadership acknowledged it. Managers were given a toolkit. And then twelve months later the follow-up survey asked whether anything had changed, and the scores were flat, and everyone agreed the survey process was to blame and needed to be redesigned. It definitely wasn’t the system that was broken…
Why It Still Matters
AI doesn’t change the problem surveys are trying to address. It obliterates the excuse for why we can’t do anything to change the problems unearthed in a timely manner.
Continuous listening, always-on pulse tools, NLP aren’t new anymore and existed with pretty wide adoption even before many AI tools that are entering the market. You could process thousands of open-ended comments in seconds, sentiment analysis that can surface themes by team, manager, tenure band, or demographic cohorts. You can know what your employees are experiencing closer to when they’re experiencing it than at any point in the history of organizational measurement. AI tools are even allowing you to do this without surveys at all.
Which means the bottleneck has moved. It was never really about survey frequency or comment volume or theme categorization. It was about whether the organization had the managerial capability and the leadership will and accountability structures to actually act on what employees were telling them.
AI-powered listening at scale, whether through active or passive methods, creates a new version of an old problem. You can now generate team-level sentiment breakdowns, manager-specific feedback signals, and real-time engagement trajectories faster than any HR team or manager population can absorb and respond to them. Listening was never the constraint. The constraint was always what your managers did after the results came out. It’s why some organizations stopped, temporarily, listening to employees altogether. “Survey theater” can’t be tolerated when results are almost immediate and continual.
Organizations that get this right won’t be the ones with the best AI tools. They will be the ones who figure out how to redesign leader and manager accountability to take action in real time.
4. Workforce Planning & Skills
Leaders have always wanted the same thing with workforce planning. Do we have the right people, in the right roles, in the right places, with the right capabilities, to execute the business strategy now and over the next few years?
Workforce planning matters because the consequences of getting it wrong are expensive, and by the time they’re obvious, already difficult to reverse. Over-hire into a business unit that contracts and you’re restructuring. Under-invest in a capability that becomes strategically critical and you’re paying a premium to acquire it from the outside or watching a competitor pull ahead because they saw it coming. The stakes were always high. They were just spread across a long enough time horizon that organizations could tell themselves the problem was still manageable.
The Old Way
The standard approaches were always headcount modeling, operational short term plans, and sometimes strategic plans made a few years into the future. Finance (FP&A) owns a financial and FTE number. HR owns a human-based supply model and capability gaps. And somewhere in the delta between the two successfully, or unsuccessfully, was implemented before the data went out of date.
You start with attrition assumptions. You would layer on growth projections from the business that even they were privately skeptical of. If you were lucky, you would model a few scenarios – base case, upside, downside – and produce a document that HR and Finance would argue about in a room while the business moved in a direction that the board, the external environment, or the stock market dictated.
Then skills-based organizations entered the workforce planning equation. The job architecture most organizations were working from was a product of how compensation classified roles, not a reflection of the actual capability clusters that drove performance. A “Senior Analyst” in one business unit shared almost nothing with a “Senior Analyst” in another except the title. Trying to do supply/demand analysis on top of a taxonomy that was built for compensation banding rather than capability mapping produced numbers that were at best, directionally correct. External labor market data exists and is helpful, but accessing it takes time and costs money, so only the most well-resourced teams use it.
The honest summary is that most workforce planning was really headcount planning dressed up in strategic language. It told you how many people you made educated guesses about how many you would need. And frankly there was nothing wrong with that. But, it didn’t tell you whether those people had the capabilities required to do what the strategy actually demanded.
Why It Still Matters
AI changes what’s possible on both sides of the supply and demand equation simultaneously, while also bringing in new data capabilities while reducing time and labor constraints.
The supply picture is clearer than it has ever been. Internal skill data can now produce capability maps of your workforce that are orders of magnitude richer than any self-reported skills inventory. You can see what your people can actually do, not just what their job description says they’re supposed to do. And if you invest in it, even validate those capabilities at scale.
The demand picture is harder and more urgent at exactly the same time. AI is compressing the half-life of skills in ways that make traditional three-to-five year planning horizons feel almost comically long. The capability that’s scarce and strategically critical today might be partially or fully automated in 18 months, or it might not. The workforce planning question has always been about the future, but the future is now arriving faster than the planning cycle was designed to process.
You can now generate granular, real-time skills gap analyses using AI faster than your talent development infrastructure, your recruiting pipeline, or your internal mobility processes can respond to them. This creates real abundance issues. Knowing that you have a critical shortage of a capability means nothing if the organizational systems for addressing that shortage. The insight has gotten faster. The response mechanisms mostly haven’t.
AI-inferred skills assessments carry real consequences for individuals. Who gets considered for an opportunity, who gets flagged for upskilling, and whose role gets redesigned versus eliminated. And what if we’re wrong? The teams getting this right have stopped treating workforce planning as a once-a-year exercise that HR does for Finance. They are building continuous capability monitoring and change management mechanisms, connected to strategy and responsive to labor market signals. What we have to be honest about is the gap between the insight the model can generate and the organizational capacity to act on it.
The Value Proposition Shift
HR problems haven’t changed, but the value proposition and friction points of people analytics have. AI has made “insight” cheaper and insights are now commoditized, while making trust more expensive. None of the opportunity is reachable if we lose the trust of the people the data is about. The “worker-first” ethos of the 2010s didn’t survive the end of cheap money, and in a high performance culture moment we are in, someone in HR needs to own the line between performance intelligence and surveillance culture before it legislatively gets drawn for us, which is exactly the future warned about in The Camera™.
The Abundance Trap
We can now produce insights faster than humans can absorb, prioritize, or act on them. The cost of generating insight is approaching zero. The cost of acting on one hasn’t moved, yet. Humans can only hold two or three insights in active attention at a time. That’s not a technology problem, that’s our cognitive architecture, and it doesn’t scale like a compute budget. When insight supply grows by an order of magnitude and demand stays flat, you get paralysis dressed up as productivity improvements.
Ruthless prioritization of the right problems beats comprehensive coverage of all of them. Our ability to detect opportunities vastly exceeds my ability to influence decisions and drive action, and that gap is only widening. The constraint is no longer analytical horsepower. It’s attention, trust, relationships, and time. Every insight should connect to a decision, every decision to an owner, every owner to a realistic path to action. That is part of the pathway to winning.
The Winning Strategy:
The winning teams won’t be the ones who generate the most analysis. They’ll be the ones who can prove, end-to-end, that work decisions got better consistently.
Faster cycle times
Fewer mistakes
Less rework
Measurable lift
This is the winning strategy.
People analytics becomes the team that designs the “decision machine” and proves outcomes added value without breaking trust. Decision outcomes and accountability are now the measure of value.
Create a decision contract within your organization that measures the rework tax, trust, and impact. Kill projects below your threshold to keep enough bandwidth for attention and impact.
Trust and rework scorecard – Start measuring impact to trust and the hidden rework it takes to continuously review and keep AI generated insights on track.
Focus on the problems that AI can’t solve for us – Fundamentally keep work human.
I hope you like this article. If so, I have a few more articles coming out soon. Stay tuned. If you are interested in learning more directly from me, please connect with me on LinkedIn.
Cole’s recent articles
For access to all of Cole’s previous articles, go here.
 | A guest post by
|