A formal structure for the evaluation of decision support system generators (DSSG): the systems approach

Decision Support Systems (DSS) are future-oriented decision-support tools used mostly for strategic planning tasks which are ill structured. It follows that most DSS benefits are qualitative and therefore their impacts on organizations are difficult to measure. Various approaches such as cost/benefit analysis, value analysis, and other utility methods have been used to evaluate a DSS, but unfortunately, qualitative benefits are handled in a very arbitrary and subjective manner.

This dissertation attempts to construct an ex ante evaluation model for a DSS generator (DSSG) with explicit consideration of all qualitative benefits associated with a DSS. The qualitative benefits are recategorized according to user effectiveness criteria given by Alter (1980) and are incorporated along with system capabilities into a hierarchical structure using the systems approach. The Analytical Hierarchy Process (AHP) developed by Saaty (1980) and Worth Assessment (WA) described by Sage (1977) are used to determine the weights of the evaluation criteria. Then worth scores (Sage, 1977) are assigned to the criteria of system performance in order to determine the DSS Quality Index (DQI), which represents the quality of a DSSG to a given organization.

This dissertation also constructs a prototypical system which can support users in their DSS evaluation process. The model and methodology addressed above become a conceptual framework for constructing the prototypical system. The system is designed to provide guidelines for users to construct their own evaluation model and to determine the priorities of the objectives and evaluation criteria in order to assess the worth of a DSSG. This evaluation process and the outputs of the system (the worth scores, the weights of the evaluation criteria, and DQI) will become a tool to monitor the development of an evolving DSS.

The DQI allows an organization to assess the value of a DSSG before its implementation. The model is built as a planning mechanism that provides a direction rather than a post-audit examination. Since this model intends to support the judgments of users in the evaluation process, yet not to replace them, it will provide a convincing way of obtaining the worth of a DSSG.