App builders take note: You are responsible for the decisions your users make

At a minimum, creators of business intelligence and analytics applications bear responsibility for ensuring that people can reach the right conclusions from the facts and the way they are presented

decision point
Credit: flickr/Daniel Oines

Most enterprise apps are built to support the decision making of their users. Thus the question is: what responsibilities do the creators of apps have? Not just the developers, but anyone who is involved, from defining the concept, to requirements gathering, to final implementation. Most recommendations are focused on security and privacy aspects, omitting the critical ethical implications of building and providing an app used for decision-making.

The builders of the app are in fact implicitly assuming some of the responsibility for other people’s decision making. So, to evaluate the ethical responsibilities of the app creators we must begin by looking at how decisions can go wrong and making sure that we have a check list to minimize all objective factors that can contribute to a wrong decision. The creators of the app should ensure that people can reach the right conclusions from the facts and the way they are presented in the app.

So let us look at the issues.

The first issue is about data. The two basic questions to ask are, “is the data accurate?” and “is the data complete?” The user presumes that the answer to both these questions is “yes.” Data issues have been discussed a lot in the BI community, but the point here is that it would be unethical to leave the judgement about the data to the end user. An analogy with retail might be helpful here. If a retailer publishes an incorrect price (perhaps due to bad data), it has to honor it. Hence, the retailer incurs the cost of the wrong data. Since it is difficult to track the cost of wrong decisions, we should impose stricter standards on data assurance and make it clear to the data creators that it is their responsibility to provide such assurance.

The second issue is about analytics. Are the analytic methods valid and do the measures have an impact on the desired outcomes? Selecting the right method of analysis is very important, especially in applications that embed predictive and prescriptive analytics. The advanced analytics space is full of statistical paradoxes that are hard to discern, hence the selection of the method has to be done with complete scrutiny. It is not by chance that clinical trial research has so many checks and balances. This type of error is analogous to a doctor prescribing the wrong test and basing the diagnosis and the treatment on the test results. Hence, we recommend formal review and sign off process for the analytical methods deployed in apps. Another frequent problem occurs when the measures used to guide behaviors and achieve outcomes do not correlate with the desired outcomes. In other words, the measures selected by the app creators do not have a causal relationship to the desired outcome. This type of error is analogous to a doctor treating a symptom with the wrong medication.

Finally, how information is presented matters as it directly impacts the perception of the data and consequently the conclusions made from the visuals. It must be noted that people use apps to make decisions quickly. In many cases they make decisions instantaneously based on what they see. So any embellishment or design decoration that can lead to incorrect perceptions are unethical to use as they are diverting the attention from the main purpose of the application. The goal is absolute clarity and lack of ambiguity so that decisions can be made quickly. Visual effects can obscure the facts and misrepresent proportions and ratios, thus leading to incorrect conclusions.

In conclusion, we need to create awareness among the creators of apps about the ethical aspects of its usage and their implicit responsibilities. While the answers to many of the issues raised above are soft and many issues may fall in gray areas, overall awareness would help the creators make better choices about data, analytics and presentation of information.

This article is published as part of the IDG Contributor Network. Want to Join?

ITWorld DealPost: The best in tech deals and discounts.
Shop Tech Products at Amazon