1803ICT Information Systems Foundations
1803ICT Information Systems Foundations
Module 7
Information system assessment, impact and success
By the end of this Module you will:
Discuss the success of IS in organizations using appropriate parameters
Describe and apply the IS success model
Explain context of use in terms of IS success
Discuss the productivity paradox, what it is and the possible reasons for it
Explain development and use failures
Apply a model to predict system failures
Apply Mintzbergs organisational forms
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
1
Although the opportunity for an information system has been identified and potential benefits are obvious, things can go wrong and sometimes it doesn’t seem that the information system implementation was worth it.
Once an information system is introduced into an organization it will have effects.
People may not want to use the system or love to use the system, people might find the system easy or hard to use etc. These effects can be categorised as first order effects – they are issues related use.
The other effects of the information system are those that are felt at the level of the organization – is the information system making things more efficient? Is it increasing effectiveness? Is it helping to create value and supporting the organizational goals? – these second order effects; these have to do with the impact on the organization itself.
It is necessary to be familiar with frameworks that enable us to evaluate information systems, recognize success, understand impact, and guard against system failure – this is the focus of the present module.
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Although the opportunity for an information system has been identified and potential benefits are obvious, things can go wrong and sometimes it doesn’t seem that the information system implementation was worth it.
Once an information system is introduced into an organization it will have effects.
People may not want to use the system or love to use the system, people might find the system easy or hard to use etc. These effects can be categorised as first order effects – they are issues related use.
The other effects of the information system are those that are felt at the level of the organization – is the information system making things more efficient? Is it increasing effectiveness? Is it helping to create value and supporting the organizational goals? – these second order effects; these have to do with the impact on the organization itself.
It is necessary to be familiar with frameworks that enable us to evaluate information systems, recognize success, understand impact, and guard against system failure – this is the focus of the present module.
2
How can the value or worth of an IS for an organization be determined?
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
As organisations become ever more reliant on information, the question of quality of an information system becomes increasingly important.
Organisations are spending lots of money and time to develop and implement information systems. In view of such investments it is necessary to ask was it really worth it -or – how can the value of an IS be determined?
3
IS Success Measures
DeLone and McLean performed a systematic review of available published material on the question of the success of information systems in organisations. They subsequently developed a model…
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Delone and Mclean undertook an extensive review of existing literature on the topic. They collected what factors various researchers had identified as influencing the perceived quality/value of an IS and on the basis of those factors proposed a model of how they factors interacted. Using statistics they were then able to show which factors were most significant in influencing perceptions of quality and value.
The end result was the following model:
,
4
DeLone and McLean’s IS Success Model (1992)
Read two of DeLone and McLean’s papers
Delone original paper (1992).pdf
IS Success Measures
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
When DeLone and McLean performed a statistical analysis on factors they found in literature which have an influence on perceived value or worth of an IS, they found that there were six categories of factors which influenced system quality….
Information quality, a concept that is related to the quality of information system outputs, can be described in terms of outputs that are useful for business users, relevant for decision making, and easy-to-understand (representing IS quality as value) as well as outputs that meet users’ information specifications.
System quality represents the quality of the information system processing itself as carried out by technology, which includes software and data components, and it is a measure of the extent to which the system is technically sound. System qualityis concerned with whether there are bugs in the system, the consistency of user interface, ease of use, quality of documen-tation, and sometimes, quality and maintainability of program code.
Organizational impact represents the firm-level benefits received by an organization because of IS applications. The orga-nizational impact of IT is realized through business performance
Use – extent of use, frequency of use, what it is used for etc. ease of use
Satisfaction – the experience was the user able to do what they needed to do, easily and well
Both the quality of the system and of the information it produces is likely to influence the use of the system and perceptions of user satisfaction with this information system.
In turn, the use of the system will have an impact on the individual within the organisation, which in turn will have an impact on the organisation as a whole.
The six factors can be categorised into three constructs:
The domains of functionality, usability and utility
Two issues of the use of information systems and the impact of its use are associated with assessments as to the worth or value of an IS
Elements from the DeLone and McLean model can be grouped into the domains of functionality, usability and utility:
VALUE
IS Success Measures
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
functionality of an information system is normally determined by a close examination of organisational requirements. It is what the system does, or should do. Specifying the core functionality is a critical aspect of information systems development (see Chapter 12). For instance, to describe a system as an order processing system indicates that it, in some way, captures, stores and manipulates data associated with the processing of orders.
Usability is evident in the way in which an information system embeds itself in activity. It is a measure of how easy it is to use for the purpose for which it has been constructed. Usability is evident at the human–computer interface (see below), the point at which the user interacts with the ICT system. An order processing system’s usability will be determined by how easy it is for users, such as order clerks or customer service representatives, to input and extract data about orders.
functionality defines what an information system does
usability defines how it is used, and is evident in the user interface
utility defines how acceptable the information system is in terms of doing what is needed. It judges the system by the contribution it makes to its activity system and the organisation as a whole. An order processing system’s utility might be de ned as the contri- bution it makes to the e cient handling of customer orders. It might contribute to signi cant cost savings in order processing, and/or to improvements in organisational efectiveness. For instance, if it leads to customers getting an improved service, that should be apparent in their level of satisfaction, which feeds through to the level of customer retention.
The IS success model (2003)
IS functionality
IS usability
IS utility, or benefit
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
10 years later, Delone and Mclean revisited their model and updated it.
The updated model also consists of six interrelated dimensions of IS success: information, system and service quality, (intention to) use, user satisfaction, and net benefits.
There’s two main differences with the original model
The addition of the service quality dimension. Service quality relates to the experience of IT support services in an organisation and are measured by reliability, responsiveness, assurance,and empathy.
This was likely an addition due to the growing recognition of IT services as an asset to the organisation and a recognition that IT support services make a contribution to the creation of value – a topic we cover in more depth in a course called IT services management.
The second difference between this model and the earlier is the the bringing together of individual impact and organisation impact into a construct called – net benefits.
The arrows demonstrate proposed associations between the success dimensions and show the inter-relatedness of the various factors..
The model can be interpreted as follows:
A system can be evaluated in terms of information, system, and service quality; these characteristics affect the subsequent use or intention to use and user satisfaction. The intention to use and actual use are inextricably linked,. User satisfaction and use influence each other.
As a result of levels of use of the system, certain benefits will be achieved. If users use the system or are satisfied with it then it is likely that there will positive benefits. Conversely, if system is not used then the benefits may not be realised.
The success dimension service quality represents the quality of the support that the users receive from the IS department and IT support personnel, such as, for example, training, hotline, or helpdesk.
This construct is an enhancement of the updated D&M IS Success Model that was not part of the original model.
As one of the few truly IS-specific pieces of theoretical knowledge created by IS scholars in the last decades, work using the D&M IS Success Model will remain popular in the years to come.
.
7
The IS success model (2003)
IS functionality
IS usability
IS utility, or benefit
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
We will now explore further Use and usability followed by utility or benefit..
.
8
IS Use and Usability
The success or otherwise of an information system is embedded in the context of use.
The context of use of an IS is defined by :
The user interface
Usability
Stakeholder involvement
Stakeholder satisfaction & resistance
We will look at each in turn:
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Following on from Delone’s and McLean’s model, we briefly focus in on usability,, and utility
The success or otherwise of an information system is affected by many different factors, not the least of which is user perceptions. With this in mind we define the context of use in terms of user interface, usability, use settings and stakeholder involvement and satisfaction –
We’ll look briefly at each of these before turning to utility and impact of IS leading into a discussion of productivity paradox and system failures.
9
User Interface
An interface defines how a given technology can be used
Every technology has an interface, including ICT
An ICT system’s interface is called the user interface or the human-computer interface
Use and usability
The user interface
Usability
The Use setting (environment)
Stakeholder involvement
Stakeholder satisfaction
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
The user interface is where use happens:
Members of an organisation use the interface to input data into the system and to receive data output from the system
The use activities of the ICT system form part of the larger IS
Decisions are made on the basis of information interpreted from the IS and action is taken within the human activity system
More in 1805ICT Human Computer Interaction, an introductory course on the human computer interface.
User Interface
Use and Usability
The user interface
Usability
The Use setting (environment)
Stakeholder involvement
Stakeholder satisfaction
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Usability
What is do understand by the term “usability”?
Use and Usability
The user interface
Usability
The Use setting (environment)
Stakeholder involvement
Stakeholder satisfaction
Think back to your experience of using the class enrolment system. Was it a good experience or not? How do you feel about it and why?
Now think about usability – if you needed to explain the term to a friend or relative who has no knowledge of computers, design etc what would you say to explain the idea of usability?
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
The usability of an information technology system is how easy an system is to use for the purpose for which it has been constructed. Usability applies not only to the user interface in Ict but to the whole of the information system. How much effort is required for people to get the right information at the right time? How well can the information be understood? Do they have all the information required? Is it accessed efficiently?
How much effort does it take for people in the organisation to learn to use and interact with an information system.
Effectivenesss efficiency satisfaction learnability
Learnability: how easy is it to learn to use
Memorability: ease for a user to remember
Efficiency of Use: no time delays for input or output
Reliability in Use: Help users to make fewer errors
User Satisfaction: pleased to use it
Usability as the amount of mental effort users must expend in order to meet their objectives.
There are five dimensions to usability:
Usability
Use and Usability
The user interface
Usability
The Use setting (environment)
Stakeholder involvement
Stakeholder satisfaction
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
The usability of an information technology system is how easy an IT system is to use for the purpose for which it has been constructed. Nielsen considers five facets of usability:
Learnability An IT system should be easy to learn how to use
Rememberability. Having learned how to use a system the user should be able to remember how to use it easily
Efficiency of Use. The system should be efficient to use. For instance, input operations should not be subject to delay
Reliability in Use. The IT system should promote reliable human performance in the sense that it should lead the user to make fewer errors
User Satisfaction. The interface should satisfy the users in the sense that it leaves them subjectively pleased with using the interface
Stakeholder Involvement
Stakeholder
Satisfaction
Stakeholder
Involvement
A number of critical principles affecting use arise from the concept of stakeholder groups:
It is important to identify stakeholder groups that are likely to influence the development process as part of the planning for an information system
It is important to involve representatives of various stakeholder groups in the development process
It is important to identify differences in meanings assigned to technology between stakeholders
Stakeholder involvement in the development of information systems is seen to improve system acceptance and the satisfaction with systems (Newman, 1996).
Leads to
Use and Usability
The user interface
Usability
The Use setting (environment)
Stakeholder involvement
Stakeholder satisfaction
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Recalling that in Delone and mcleans model, one factor for IS success is satisfaction.
Satisfaction is a perception of the people using it i.e. statkeholders.
Hence in designing and implementing information systems we need to consider stakeholder satisfaction. And this can be achieved largely through the involvement of stakeholders throughout design development and implementation.
A number of critical principles affecting use arise from the concept of stakeholder groups:
It is important to identify stakeholder groups that are likely to influence the development process as part of the planning for an information system
It is important to involve representatives of various stakeholder groups in the development process
It is important to identify differences in meanings assigned to technology between stakeholders
Stakeholder involvement in the development of information systems is seen to improve system acceptance and the satisfaction with systems (Newman, 1996).
Clearly satisfaction can be assessed at a number of levels:
Satisfaction with the interface. This may serve as one measure of usability
Satisfaction with the ICT system. This may serve as one measure of functionality
Satisfaction with the information system. This may serve as one measure of utility
Stakeholder Satisfaction
Use and Usability
The user interface
Usability
The Use setting (environment)
Stakeholder involvement
Stakeholder satisfaction
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Stakeholder satisfaction refers to a subjective assessment of the success of some IS. Determining the levels of such satisfaction is an important part of the assessment of the worth of some information system. Clearly satisfaction can be assessed at a number of levels:
Satisfaction with the interface. This may serve as one measure of usability
Satisfaction with the IT system. This may serve as one measure of functionality
Satisfaction with the information system. This may serve as one measure of utility
Some stakeholder satisfaction criteria with systems include:
Output assessments such as accuracy, quality, completeness and relevance of output
Process assessments such as availability of service, mean time between failure, down-time and number of security breaches
Input assessments such as ease of use and response time
The assessment of the satisfaction with interfaces is frequently assessed via questionnaires consisting of a series of attitude questions.
Stakeholder Satisfaction
Use and Usability
The user interface
Usability
The Use setting (environment)
Stakeholder involvement
Stakeholder satisfaction
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Since the interface is the major focus of the Module of use we concentrate on user interface satisfaction here, although it is somewhat difficult to separate out issues of functionality from this assessment. Some user satisfaction criteria with systems include:
Output assessments such as accuracy, quality, completeness and relevance of output
Process assessments such as availability of service, mean time between failure, down-time and number of security breaches
Input assessments such as ease of use and response time
The assessment of the satisfaction with interfaces is frequently assessed via questionairres consisting of a series of attitude questions. One popular approach is QUIS (Chin, Diehl et al. 1988) – Questionairre for User Interface Satisfaction – consisting of 27 items using a nine-point Likert scale.
Stakeholder Resistance
Resistance Strategies
Lay Low
Rely on Inertia
Keep the project complex
Minimise Implementer’s legitimacy
Exploit lack of insider knowledge
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
One of the key ways in which organisational politics may affect the post-implementation trajectory of an IS is through user or stakeholder resistance. Keen details a number of counter-implementation strategies that users may take to impede the development of the system or its implementation:
· Lay Low. If you do not want a system to succeed, then the more you keep out of the way and do not give help and offer encouragement, the more likelihood there is of failure
· Rely on Inertia. If you can be too busy when asked then the implementation process may come to a halt
· Keep the project complex, hard to co-ordinate and vaguely defined. If the goals are ambiguous or too ambitious there is every chance of failure as energy is dissipated in many different directions
· Minimise the implementers legitimacy and influence. If the designers are kept as outsiders, other users will probably not allow them to work effectively
· Exploit their lack of inside knowledge. The design team probably know very little about the detailed nature of the work and if they are denied this knowledge, the system will probably prove to be inadequate when it is implemented
One of the key ways in which organisational politics may a ect the post-implementation trajectory is through user or stakeholder resistance. Hirschheim and Newman (1988), for instance, provide a case study that illustrates how user resistance depended on the amount of stakeholder involvement in implementation. Keen (1981) details a number of counter- implementation strategies that users may take to impede the development of an information system. One of these strategies is to lay low or to be too busy to be involved. In other words, if a stakeholder does not wish a system to succeed, then keeping out of the way and not giving help and encouragement can help make it a failure. Another strategy is to exploit the devel- opment team’s lack of inside knowledge. e technical sta in particular will probably know very little about the detailed nature of the work involved in a particular business area and, if users do not help by telling them what goes on, the system they design will probably prove to be inadequate.
recap: After information systems are introduced in an organisation, the context of use begins to affect the activity system. Information systems may be closely aligned with their activity system, through design or by accident, generating both intended and unintended positive effects. If they are misaligned they can have a negative effect on organisational activity. They are more likely to be well aligned if the developers identify the stakeholders, understand the differences in their technological frames, and involve representatives of stakeholder groups in the development process.
IS functionality
IS usability
IS utility, or benefit
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Having discussed usability, we now turn to utility or benefit. Discussion begins with a consideration of impact of the introduction of information systems.
A system can have a Positive or Negative impact
Usually not neutral as the system is introducing change
IS has an impact on groups and individuals inside organisation:
Individuals
Groups (HAS)
Organisation as a whole
IS also has an impact on groups and individuals outside organisation
Impact of Introduction of IS
Utility
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Examples of potential positive impact:
increased level of work monitoring and control by managerial group
enriching certain activities and empowering certain groups of workers
empowered decision making by managers.
Examples of potential negative impact:
An online system developed for customer support which has been slow and down many hours every week
an online system generated to access patients medical history lacks data accuracy and completeness.
Employees losing their job due to automation, or de-skilling of employees.
Impact of Introduction of IS
Utility
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
In your assessment you will: Describe measures of efficiency, efficacy, and effectiveness for your proposed new IS.
The level of performance an organisation attains is a function of the efficacy, efficiency and effectiveness of the actions it has undertaken.
If the IS supports
These then there will be a positive impact.
Impact of Introduction of IS
Utility
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Performance of systems in organisations, have three main types of measures:
Efficacy (does it work? Output oriented). Efficacy: is a measure of the extent to which a system achieves its intended transformation. Efficacy is primarily focused on the outputs and producing appropriate value.
Efficiency (does it work well? Outputs / Inputs) with minimal resources, time, effort, money etc.
Effectiveness (does the system contribute to a higher-level system?). Does the IS help achieve organizational goals?
Efficacy is primarily focused on the outputs and producing appropriate value.
Examples:
how durable are specific batteries produced by Samsung?
How many years can an average Toyota Corolla travel without needing a major service?
How tasty is a specific food product (as measured by its customers’ perception)?
Efficiency (Outputs/Inputs):
The extent to which the system achieves its intended transformation with the minimum use of resources?
Examples:
how many person hours does it take to produce one unit of product?
Average time it takes to provide a specific service to a customer?
How much does it cost to produce one unit of a product?
The productivity paradox
Does the introduction of an information system always have a positive impact?
Evidence suggests that there are some inherent paradoxes or contradictions involved with the introduction of information systems into organisations. One of the most significant is known as the productivity paradox.
A paradox: “something is inconceivable, but happens to be true”. A puzzle.
What do you understand by the term ‘productivity’? Write in your own words an explanation of productivity as you understand it.
Impact of Introduction of IS
Utility
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Productivity within an organisation can be understood as a gain in effectiveness.
Recall from the previous section on the Performance of information systems in an organisation, that effectiveness is about meeting organizational goals.
Achieving effectiveness also requires Efficiency (minimal use of resources to achieve a goal) and efficacy (things work as they need to).
Effectiveness gains normally involve delivering greater value to the customer of some organisation. Hence, effectiveness is primarily focused on the outputs from an organisation. Effectiveness gains can be measured in terms of improvements in the quality of the service or product or measures such as improvements in customer satisfaction
22
One key reason organisations use information systems, particularly IS in which ICT is used as a tool, is the expectation that employing such systems will raise the productivity of the workforce.
However, over a number of years, this link between information system usage and productivity has been questioned. This has become known as the productivity paradox.
The productivity paradox
Impact of Introduction of IS
Utility
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
23
Explanations – Productivity Paradox
Mis-measurement
Lags
Re-distribution of Profits
Mis-management of ICT
Mis-management: This is the basic IS argument. It proposes that organisations have systematically mismanaged IT investments, and have not planned systematically for the introduction and use of sophisticated ICT systems.
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
24
Brynjolfson considers four main explanations for the productivity paradox:
· Mis-measurement of inputs and outputs. A proper indicator of IT impact has yet to be formulated and analysed. Traditional measures such as the number of service transactions multiplied by their unit value, tend to ignore non-traditional sources of value such as increased quality and speed of customer service
· Lags due to learning and adjustment. The long-term lag between cost and benefit may be due to the extensive learning required on the part of the individual and the organisation to fully exploit IT
· Redistribution and dissipation of profits. This explanation proposes that those investing in technology benefit at the expense of others in a particular industry, Hence, there is no aggregate benefit to an industrial sector such as financial services
· Mismanagement of IT. This is the basic informatics argument. It proposes that companies have systematically mismanaged and have not planned systematically for the introduction of IT. This explains the lack of piecemeal benefit from the introduction of IS/IT
Explanations –
Productivity Paradox
Brynjolfson considers four main explanations for the productivity paradox:
Mis-measurement of inputs and outputs. A proper indicator or measure of IT impact has yet to be formulated and analysed.
Lags due to learning and adjustment. The long-term lag between cost and benefit may be due to the extensive learning required on the part of the individual and the organisation to fully exploit IT.
Redistribution and dissipation of profits. This explanation proposes that those investing in technology benefit at the expense of other types of capital expenditure in a particular industry. Hence, there is no aggregate benefit to an industrial sector.
Mismanagement of IT. This is the basic IS argument. It proposes that organisations have systematically mismanaged IT investments, and have not planned systematically for the introduction and use of sophisticated ICT systems.
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
25
Information systems failure and strategies for avoiding failure can be considered in terms of horizontal and vertical dimensions
The horizontal dimension involves development failure and use failure
The vertical dimension:
Failure at the technical level of ICT systems, information system projects, organisations or the external environment
Two dimensions of systems failure
Systems failure
Utility
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Thinking about the productivity paradox definitely brings into focus the issue of system failure as a contributer to why the potential of the IS isn’t realised.
Development failure occurs when all or part of a system is abandoned prior to implementation
Use failure occurs during the post-implementation trajectory
E.g. when a system is abandoned after a period of use or if a system is subject to large amounts of adaptive maintenance
Development failure occurs when all or part of a system is abandoned prior to implementation
Use failure occurs during the post-implementation trajectory
E.g. when a system is abandoned after a period of use or if a system is subject to large amounts of adaptive maintenance
Systems failure
Utility
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Knowing that systems can fail is important but even more important is knowing why they fail.
Examples may be quoted of systems failing:
to meet a pre-determined specification;
failing to be completed at all, or
failed to be completed on time; or
completed but remaining unused; and so on.
Systems failure
Utility
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Respondents see failure in a number of different ways according to their preceptions
Respond according to their own perceptions
Lyytinen & Hirschheim (1987) see systems failure grounded in social reasons –
In other-words, “systems do not fail primarily because of technical shortcomings, but because those involved in & affected by the system are not adequately considered”.
Systems fail for people reasons primarily, not technical reasons.
Systems failure
Utility
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
The reason for failure is not so straight forward.
This is because the reason is often a perception. The concept of failure (like the concept of quality) in information systems can be difficult to pin down, mostly because it is subjective and it is viewed from the varying perspectives of different stakeholders.
For example, an IT technical person may consider the system as a failure because it runs somewhat slower than anticipated
A user of the system may have different perspective – they want it to be easy to use but perhaps the interface is not intuitive so they will deem it to be a failure
The CEO of a company may view the same system very differently, and see it as a failure because profits are not rising since it was introduced.
These observations make very clear the need for stakeholder involvement in design and development, as was noted in the earlier part of the module.
Lyytinen & Hirschheim (1987)
Four categories of information systems failure:
Correspondence failure: failure to meet requirements set in advance
Process failure: system not produced within given budget or time
Interaction failure: system not used or under-used by target
Expectation failure: failure to meet user expectations (a super set of the above 3)
Systems failure
Utility
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
In trying to identify the possible reasons for system failure, Lytman and hirscheim reviewed literature and propose four categories of system failure in IS
Lyytinen and Hirschheim – four major theoretical categories of such phenomena:
· Correspondence Failure. This is the most common form of IS failure discussed in the literature and typically reflects a management perspective on failure – design objectives are first specified in detail – evaluation is conducted – lack of correspondence between objectives and evaluation the IS is regarded as a failure
· Process Failure. This type of failure is characterised by unsatisfactory development performance. It usually refers to one of two types of failure – cannot produce a workable system – development process produces an IS – over budget in terms of cost, time etc
Interaction Failure. a mismatch of requirements and system or poor development performance to a consideration of usage of a system.
· Expectation Failure. Lyytinen and Hirschheim describe this as a superset of the three other types of failure. the inability of an IS to meet a specific stakeholder group’s expectations.
Correspondence Failure
Process Failure
Interaction Failure
Expectation Failure
(Lyytinen & Hirschheim,
1987)
Termination Failure (Sauer, 1993)
Sauer (1993) criticises L&H (1987) for the concept of expectation failure. All systems would be a failure.
Sauer (1993) proposed a more conservative definition: a system has failed only when it has terminated.
Systems failure
Utility
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
While Lytinen and hircheim’s idea seems neat, it is not without criticism, especially in relation to expectation failure – some authors saying that if we talk about expectation failure then the system will always be a failure because there’ll always be someone for whom the system dosen’t meet expectation. Instead, a more conservative definition might be that a system has failed only when it is terminated and no longer used.
Digging further into system failure, some researchers have asked the question of whether different types of failure are more likely in different types of organisations.
Five organisational forms (Mintzberg, 1991)
Entrepreneurial: start-up firms, visionary leadership, single executive, i.e. SMEs
Machine: mass production, inward looking, standardisation i.e. factories
Professional: accountancy practices, law firms, university schools – concentrate on levels of skill & proficiency
Diversified: range of products or services, autonomous business units
Adhocracy: skilled experts, fluid structure, informal communication i.e. R&D arm of company
To do: Classify the Commonwealth games
Organisation committee?
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Mintzberg identified a number of different organizational forms based on the way the organization is structured and what it does.
5 forms are identified.
Organisational Forms and Systems Failure
The grid on the following slide represents a summary of empirical evidence from consultancy interventions in systems development, where it has been observed that “certain forms of organisation appear to make the organisation more susceptible to a particular type of system failure”.
The grid indicates the relationship of systems failure concepts to the different forms of organisation in Mintzberg’s (1991) classification: a double asterix indicates a stronger relevance; a single asterix indicates some relevance; blank indicates no association.
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Organisational Form and Systems Failure
Correspondence
Failure
Process
Failure
Interaction
Failure
Expectation
Failure
Organisational
Form:
Machine
**
*
Entrepreneurial
*
*
*
Professional
**
**
Adhocratic
**
**
Diversified
*
*
*
Double * = stronger relevance
To do: Predict the kind of information system failure most likely to occur in the commonwealth games? Provide advice so that failure does not occur.
1
2
3
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
The grid on the following slide represents a summary of empirical evidence from consultancy interventions in systems development, where it has been observed that “certain forms of organisation appear to make the organisation more susceptible to a particular type of system failure”.
The grid indicates the relationship of systems failure concepts to the different forms of organisation in Mintzberg’s (1991) classification: a double asterix indicates a stronger relevance; a single asterix indicates some relevance; blank indicates no association.
Classify your organisation according to one of Mintzberg’s 5 organisational forms.
For that organisational type row, read along the table until you come to a cell with ** or * in it. If there is a cell with ** this means ‘stronger relevance’.
Read up the column with the ** in it to point to the failure type predicted.
Whatever the failure type is pointed to, your next task is to make recommendations to guard against this type of failure happening.
How to Use the Grid
Classify your organisation according to one of Mintzberg’s 5 organisational forms.
For that organisational type row, read along the table until you come to a cell with ** or * in it. If there is a cell with ** this means ‘stronger relevance’.
Read up the column with the ** in it to point to the failure type predicted.
Whatever the failure type is pointed to, your next task is to make recommendations to guard against this type of failure happening.
Organisational Forms and Systems Failure
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
Classify your organisation according to one of Mintzberg’s 5 organisational forms.
For that organisational type row, read along the table until you come to a cell with ** or * in it. If there is a cell with ** this means ‘stronger relevance’.
Read up the column with the ** in it to point to the failure type predicted.
Whatever the failure type is pointed to, your next task is to make recommendations to guard against this type of failure happening.
35
These four concepts of systems failure can be related to organisational forms (Mintzberg, 1991)
Certain types of organisations appear more susceptible to a certain type of systems failure.
Predicting failure type gives a warning or a flag to manage this situation so that the IS is a success.
Systems failure
Utility
Module 6
IS Assessment,
Impact and Success.
6.1 IS Success Measures
McLean & Delone’s
Success Model
6.2 Use and Usability
User interface
Usability
Stakeholder involvement & resistance
6.3 Utility or Benefit
6.4 Utility or Benefit
Impact of introduction of IS in an organization
6.5 Utility or Benefit Productivity and the productivity paradox
6.6 Utility or Benefit
System Failure and Mintzberg’s organisational forms
In summary
/docProps/thumbnail.jpeg