Challenges of measuring learning impact
What stops learning teams from measuring more than completions and scores? Here are four key challenges we uncovered in our recent roundtable that learning managers say get in their way:
- Measurement is inconsistent – When working at scale, there are so many types of learning for many different teams. It can be hard to find a success measure that works for all. (Perhaps this ‘magic metric’ doesn’t exist?)
- Stakeholders rarely ask for measurement data – Are stakeholders proactively asking for your learning data, or only when something (like a compliance incident) goes wrong? Learning managers shared they are rarely asked for data. (But maybe that’s because it’s not data business leaders are interested in?)
- Data is often trapped in different systems – Making it hard to get a clear, joined-up picture of learning effectiveness.This is where AI and AI-driven data tools can now really help.
- Evaluation is often reactive, not proactive – Measuring the effectiveness of learning can often be driven by when there’s a key interest in the learning from the business. For a business critical learning programme for example. Where does that leave other learning programmes? (Hint – perhaps they shouldn’t be developed if they aren’t driving a business outcome?)
How to measure learning impact effectively
Here’s how to take a practical approach to measurement and evaluation, without getting lost in data.
1. Be proactive, not reactive
Many L&D teams only measure when they’re asked to or when a problem arises. Others are focused primarily on completions and scores. Instead, start embedding performance measurement into the learning design process from the beginning.
Ask before design work starts:
- What business problem are we solving?
- How do you know it’s a current problem or gap that needs to be addressed?
- What existing business data could indicate success?
- What will stakeholders care about most?
- How can we make it easy to track?
When you start with business goals and create a collaborative conversation, measuring impact becomes easier and more meaningful.
2. Focus on one or two metrics
Some L&D professionals hold back from evaluation because they assume it’s all or nothing. Full blown ROI evaluation or nothing. This procrastination can keep us in the completion tracking box! Instead, consider a leaner, more sustainable model.
Measure one or two outcomes, rather than trying to do everything. And try to hone in on a metric that already exists!
For example:
- What’s the business goal? (E.g., “Reduce new hire onboarding time from 60 days to 30 days.”)
- Who is affected? (Target audience: “Customer service reps in Europe.”)
- What should change? (Behavioral goal: “Reps should resolve complaints 30% faster.”)
- How will we measure success? (KPI: “Complaint resolution time drops from 5 to 3 days.”)
3. Use what exists already
You don’t need to invent new metrics or data! Ask your stakeholders to show you the metrics they already track in that area – For example:
- HR data – Retention rates, churn, time to productivity – e.g. Workday, SAP
- Sales data – Revenue uplift, deal size, conversion rates – e.g. Salesforce, Hubspot
- Customer metrics – CSAT, product adoption, call resolution times – e.g. Qualtrics, Jimminy
4. Utilise AI to help you integrate and evaluate data sources
Many L&D teams struggle to connect training data with business KPIs because data is siloed and in different systems. AI tools like Tableau, Looker, Power BI, or xAPI-enabled learning platforms can integrate multiple data sources.
For example: You can merge LMS data (course completions, quiz scores) with HRIS data (performance ratings, retention rates) to see if training is improving employee performance and retention. (Or just look at them side by side – it’ll be clear if there’s a correlation!)
5. Test small, then scale
Instead of tracking every learner, find top-performing teams & compare them to low performers.
Look for patterns: What did the successful teams do differently? Did they engage more in learning? Did they have coaching conversations with their managers?
Use these insights to scale up best practices rather than trying to look at all the data.
6. Tell the story, not just numbers
Leadership doesn’t care about completion rates – they care about business results. Use data storytelling to show impact e.g., “We cut onboarding time by 30%, saving $X per new hire.”.
Present data visually and bring it to life with quotes and short case studies.
If you’re looking for even more in-depth advice around learning impact, check out these 20 learning measurement ideas to show impact and improve performance.