How IBM HR leverages IBM Watson® Knowledge Catalog to improve data quality and deliver superior talent insights
People analytics is at the center of human resources (HR) strategy and planning. Companies rely heavily on data and analytics to find and retain talent, drive engagement, improve productivity and more across enterprise talent management. However, analytics are only as good as the quality of the data, which must be error-free, trustworthy and transparent.
According to a Gartner report, poor data quality costs organizations an average of USD 12.9 million each year. Poor quality data compounds the complexity of data ecosystems which can lead to inaccurate results that lead to poor business decisions.
A long-standing partnership between IBM Human Resources and IBM Global Chief Data Office (GCDO) aided in the recent creation of Workforce 360 (Wf360), a workforce planning solution using IBM’s Cognitive Enterprise Data Platform (CEDP).
Wf360 delivers one integrated HR profile spanning career, skills, performance, learning and compensation, incorporating daily snapshots and historical data. Built on IBM’s Cognitive Enterprise Data Platform (CEDP), Wf360 ingests data from more than 30 data sources and now delivers insights to HR leaders 23 days earlier than before. Flexible APIs drive seven times faster time-to-delivery so technical teams and data scientists can deploy AI solutions at scale and cost.
Wf360 offers a wealth of data and AI-powered HR experiences on one platform, eliminating the need for dedicated infrastructures. For instance, the Job Recommendation function assists 180,000 employees in finding internal career opportunities; the Compensation Advisor, recommends Employee Salary Program increases to managers across the company; and the Performance to Skill indicator measures the scarcity of skills in the market.
Despite this solution’s ability to effectively ingest data and deliver insights to HR and other IBM business units, addressing data quality and reducing manual checks of the data, which can be labor-intensive and error-prone, remained a challenge.
To address this problem, IBM HR and the IBM Data Governance team within GCDO are building a solution that automates data quality rules, while enhancing trust on the platform, using IBM Watson® Knowledge Catalog (WKC) which operates in Cloud Pak® for Data.
What is data quality?
Data quality measures how well a dataset meets criteria for accuracy, completeness, validity, consistency, uniqueness, timeliness and fitness for purpose. Data quality is also critical for data governance. Data quality standards make sure that organizations are making data-driven decisions to meet their business goals.
Data quality is not only essential for smooth, daily business operations, but is also crucial for adopting and integrating artificial intelligence (AI) and automation technologies.
High data quality gives organizations confidence that they can accurately interpret the data and derive meaningful insights that improve overall business performance. This helps drive efficiency and create intelligent workflows that free staff to devote their time to high-value tasks.
Data quality and people analytics
Historically, HR departments have been heavily reliant on manual record-keeping and paper-driven processes. HR teams used to spend countless hours manually combining data from different sources and comparing insights from various teams. While automated workflows are more commonplace, people analytics remains complex, with data quality representing a key challenge.
IBM HR turned to the data governance team because they needed a more proactive way to monitor data quality, especially in relation to business integrity surrounding trust and transparency. For instance, weekly talent reports generated for IBM’s CHRO and CEO needed to be 100% clear of inaccuracies in the data. What’s more, while the HR team members had scripts to check for data ingestion errors and data integrity, they lacked a solution that could proactively identified business errors within the data.
Data quality is a key component for trusted talent insights. Take this example: there is a 1% spike in IBM total headcount. While this change might be justifiable from a business standpoint—such as IBM just acquired a new company which led to an increased number of headcount—the HR team member analyzing the data still needs an approach to monitor the cause for this spike.
First they need to know if the data is accurate. If so, what caused this spike?
IBM Watson Knowledge Catalog can be used to automate a data quality rule that flags this spike. This enables the HR team member to derive trusted talent insights that accurately tell the story behind what has occurred in the company.
With the help of WKC, IBM HR has established hundreds of data quality rules using a simple interface that allows them to consistently monitor for errors. Data quality rules are run daily against all of IBM’s 250,000-plus employee population span across 172 countries in critical areas such as compensation, diversity and overall attrition.
How the Watson Knowledge Catalog monitors data quality
Data quality rules in the Watson Knowledge Catalog are built using an easy-to-use graphical interface, which facilitates the discussion between the technical team and the business users, and makes it easier to spot and correct mistakes in logic. In addition, the solution not only uncovers potential errors, but also helps to solve them. It does this by automatically providing sample records that did not meet the criteria (when allowed by privacy regulations). Furthermore, rules for a data element such as an employee’s base country can be defined once and applied dozens or hundreds of times across all tables and databases being monitored, which not only saves time, but also promotes constancy. And if the business requirements change, the rule only needs to change once and the change is reflected everywhere it was used.
WKC is also able to tackle various levels of rule complexity. For example, using a simple rule, it can pick up if an employee receives a salary increase outside of an employee salary plan cycle. Or, if a salary increase falls outside of a defined range, which could indicate that an HR professional mistakenly entered the amount in local currency instead of US dollars. A more complex example would be detecting a variation in headcount over 1% in any business unit, as described earlier.
Automated data insights increase productivity and drive efficiency. In the past, manually capturing the data related to the examples laid out above would have been carried out by at least half a dozen employees all looking at various scenarios. What used to take weeks for an HR team to complete, can now be accomplished in mere minutes. HR is now in the driving seat to create and manage data quality rules, working in tandem with the IBM data governance team who drive conformance to data regulations and enterprise data standards.
Next steps
Through its work with the GCDO, IBM HR is at the forefront of people analytics work, using AI to improve decision making and successfully implement new workforce planning strategies. The next phase of the collaboration will continue modernizing business-ready data that is trustworthy and transparent.
This includes:
- Expanding data quality rules
- Adding more predictive rules
- Sending notifications directly to HR and BU leaders when errors occur
Looking ahead, the GCDO’s goal is to replicate this data quality solution for other parts of the IBM data ecosystem, including finance, sales, procurement, real estate, products and services, for trusted AI-driven data insights throughout the enterprise.
Learn more about IBM data governance using Cloud Pak for Data
Director, Distributed Governance
Chief architect, people data platform and analytics STSM
