Human in the loop: Understanding employee perception in AI-based performance management system

Abstract

As organizations increasingly deploy algorithmic management systems to support performance evaluation, employee perceptions of AI-based performance management systems remain underexplored, particularly in knowledge-intensive environments where performance cannot be reduced to standardized metrics alone. This study examines how employees perceived and experienced an internally developed AI-based performance management system structured around the planning, monitoring, and review stages of the performance management cycle. Using a qualitative case study design, 19 semi-structured interviews were analyzed through Braun and Clarke’s reflexive thematic analysis, supported by MAXQDA. Four themes were identified: emotionally-disconnected efficiency, task-bound trust, human-in-the-loop safeguard, and algorithmic fairness tension. Employees valued AI for speed, objectivity, documentation, and reduced managerial favoritism, but raised concerns about impersonal feedback, surveillance-like monitoring, contextual blindness, hidden bias, and opacity. Extending prior task-dependent algorithm-trust research, the findings show that AI-PMS legitimacy is not shaped by task type alone, but by the combined configuration of task type, performance-management stage, system transparency, and human oversight. AI was perceived as more legitimate for quantifiable KPIbased evaluation, but less legitimate for qualitative, relational, developmental, and careerconsequential judgments. The study contributes to algorithmic management and AI-HRM research by moving beyond binary acceptance or resistance models toward a conditional legitimacy framework. For managers responsible for developing and implementing AI-based performance management systems, the findings highlight the importance of task-calibrated AI deployment, human-in-the-loop governance, transparent evaluation logic, and empathetic feedback practices to support the effective use of AI in critical HR functions such as performance management.