Pitfalls of simplified measurements

Pitfalls of simplified measurements

In today's post, we venture to analyse a measurement methodology applied by Amazon and discover the downsides of such one dimensional measurements. Amazon implemented a system to constantly observe warehouse employees and assess their performance. Yet, the system was given far greater power than it should have: it could automatically make decisions regarding employee performance, and also to apply detention - or even expel a given employee. The main indicator observed by this system was the total amount of breaks taken by any given employee, and once the system decided it was too much, it started to penalize the employee in question. Because of this penalization, employees soon did not dare to go out to the washroom or take their normal breaks. The situation led to a scandal about Amazon, and the cruelty of its CEO Jeff Bezos, all because the system had too much power combined with a less sophisticated measurement algorithm. Of course it all happened back in spring 2019, and in the time being, these issues were resolved from what we know. However, it is a great example of applying measurements, yet not in the way beneficial for the organization. Let's see the factors and the outcomes!

Working time as a performance indicator

At a glance, measuring time spent directly with labour seems to be an accurate performance indicator. However, being only one dimension of the whole picture, and therefore is unable to provide an accurate overview of employee performance, relying on this indicator alone might lead to deficiencies in measurement. First, in a manual labour environment, at least a couple more dimensions are decisive when measuring employee performance and efficiency. 

Indicator examples

  • Speed - how many units are produced in a shift?
  • Errors - how many units has deficiencies in a shift?
  • Accuracy - has attention been paid to details and all were performed correctly?

Needless to say that there are many other KPIs for measurement. In order to measure these indicators, data is needed from operative processes. Therefore it is necessary to model such processes and make them measureable. During the analysis, the exact "locations" of errors are becoming visible: detailed metrics show exactly which tasks are prone to error. Hence, a good example for modelling could be the number of errors, enabling process understanding.

Number of errors - decrypting the process

In a manufacturing or logistics environment, decrypting operational processes starts with tracing back the final product's whole lifespan. Lifespan in logistics starts with ordering a certain product and ends with the product purchased by a customer. Between these two ends, a lot is happening. Today's barcode technology however makes it possible to store every detail about a product throughout time. The barcode is scanned when the product arrives, when it is moved, and many times more before the purchase. Therefore, every detail regarding the product is stored and can be analysed. Products and tasks performed pertaining to them can be linked to employees too, making complex, more involving tasks transparent and measurable.
This way all errors in the product's lifecycle can be discovered, and the deficiencies abolished as much as possible.

Speed, accuracy and change - the employee perspective

Implementing solutions for the discovered deficiencies will result in changing working habits and environment to employees. Companies tend to constantly raise previously set benchmarks, which might easily lead to frustration within the organization. Fluctuation therefore may rise, resulting in hard costs besides endangering normal daily operation. Process optimization shall support the organization from an employee perspective too, by finding the sweet spot between measurable business performance and human needs. Automation can serve as an enabler for decreasing employee workload, while also help decreasing the number of errors in certain areas.

Results

As different metrics are also taken into consideration, it might happen that decreasing amount of goods produced leads to higher profits, as error correction might result in a slower pace of work. On the other hand, accuracy increases and less problems occur either on the business or the consumers' side, hence eliminating unnecessary costs - and resulting in higher profits, compensating for the lower amount of goods sold.
Of course, each case has its individual characteristics, and measurements must be designed individually - just as the underlying business processes. 

Conclusion

Considering the above, measuring performance based on one indicator seems to deliver poor results in many aspects. More complex analysis of employee performance shall be applied in order to learn about, and understand not only the company's operations, but also the human side of employee performance. Process modelling and later automation could prevent fluctuation and make change management easier too, as employee acceptance raises. 
Most importantly, applying multi-dimensional and individually designed performance measurement models could be a key to gain in-depth knowledge about employee performance, instead of considering one dimension only.