A Relationship-Based Approach To IT Investment Appraisal

Neil McBride

Centre for IT Service Management Research

De Montfort University

Leicester

1st September 2001

nkm@dmu.ac.uk

 

Using concepts from the practice of relationship marketing, this paper outlines an approach to IT investment appraisal which centres on the identification and evaluation of relationships of importance to an organisation. This approach aims to evaluate each relationship which is mediated by IS; assign a value which represents the added benefits brought to the relationship by the IS; support the aggregation of values for individual relationships to give an overall organisational value for the system.

1. Introduction

In 1954, A.R. Zipf prepared a detailed cost and benefit analysis for an Electronic Recording Machine project for the Bank of America. Zipf believed that the computer’s greatest potential was for improved management reporting, but knew that the cost of large-scale computing could not be justified on that basis alone. He therefore searched for areas of the bank’s credit operations that were characterised by high volume, repetitive clerical activities and showed that return on investment would be achieved after two years based on staff reductions (McKenney et al, 1997). The computer system procured on the basis of this investment appraisal had a memory, which could contain 4,000 characters, a cycle time of 32 microseconds, and consisted of 8,200 vacuum tubes and 34,000 diodes. Forty-seven years on, computer systems have evolved beyond recognition but approaches to investment appraisal remain unchanged. Ballantine and Stray (1997) found that financial cost-benefit analysis and payback remain the primary approach to IT investment appraisal in large firms. The philosophy of IT investment appraisal in practice remains one of cost savings, analysed at an organisational level and often focused on staff savings.

Extensive research over the past ten years has shown that different evaluation approaches are needed for different information systems (Farbey et al, 1993). Fewer new applications concern the automation of clerical tasks. Applications are likely to support information provision, decision making and communication. Information systems evaluation research has now highlighted the importance of understanding in the evaluation process as well as efficiency and effectiveness (Smithson & Hirschhiem, 1998). Context, culture and politics are important in information system evaluation (McBride & Fidler, 1994). Researchers have recognised that IT investment decisions are not wholly rational and are influenced by the organisational environment. Interpretive approaches to evaluation recognise the importance of social and political issues (Walsham, 1993), but remain firmly in the academic arena. While a key benefit of an IS may be culture change, few companies will pay for this. While practitioners may be aware of the irrationality of IS investment decisions, there remains a reliance on accounting approaches which may be ritualistic and inaccurate, but are still acceptable to managers and decision makers. This may reflect a culture of numbers within most organisations. Value is often interpreted as a quantity, expressed in monetary terms, rather than something qualitative involving personal and environmental improvement and requiring morality, commitment and trust.

Currie (1995), describes a case where designers were justifying the procurement of a Computer-Aided Design (CAD) system. While the real value of the system lay in the ability of designers to communicate using CAD and to improve the quality of designs, the justification of the procurement was presented in terms of an increase in the number of drawings produced by each designer. Similarly, in evaluating executive information systems, benefits were presented in terms of time savings in information searches and savings in paper costs, thus overlooking the actual purpose of the system (Beltcher and Watson, 1993). These examples illustrate the continuing gap between the way in which evaluation actually occurs in organisations and the way evaluation decisions are represented.

A renewed view of information systems and their organisational role may be required. Information systems could be viewed as organisational infrastructure, incurring continuous costs. IS development should be seen as continuous and evolving, rather than a discrete product development.

There is a need to construct new approaches to IS investment appraisal which:

Such approaches to IT investment evaluation should focus on the people and their use of the information systems. As employees are seen more as assets to the organisation, rather than costs, the valuing of an information system may be viewed differently. The effect of the information system on knowledge accumulation and on relationships with other employees and with customers may give a helpful indication of the value of a particular system and the rationality for investment in the IS.

The paper outlines one such approach, relationship-based evaluation, in which the contribution of the IS to each organisational relationship is assessed. Firstly, some concepts from relationship marketing are described. The shift occurring in marketing from product-based to customer-based provides an example which might well be followed in information systems evaluation. The idea of information systems as relationship-mediators is then examined, using some concepts from organisational structuration as a starting point. The application of these concepts is expressed in the form of a methodological approach to information systems evaluation, described as a number of interlinking steps. Some of the advantages and the problems raised by this approach are discussed. Research will be carried out to test the validity of this approach in evaluated Internet-based public domain information systems (McBride, 1999).

2. The Application of Relationship Marketing to Information Systems Evaluation

In the last few years, there has been a shift in marketing approaches towards relationship marketing. A brief examination of this shift provides a parallel to the shift in thinking that is needed in information systems evaluation.

Traditionally, marketing has taken a transaction-oriented point of view, focusing on selling a product or a service to a customer and measured by the number of products sold. Marketing has been organised around product, price, promotion and place. Marketing has been short-termist, concerned with today’s sales and this year's top and bottom lines. As such it has concentrated on the product and not the customer.

Relationship marketing has provided an alternative focus in marketing. It is concerned with attracting, developing and retaining customers (Buttle, 1996). Instead of focusing on the product, relationship marketing focuses on the customer and the development of a lifetime relationship. A variety of products may be sold over the lifetime of the relationship. Measures of effectiveness concern the on-going revenue from a particular relationship. This view is distinctly long-term. Yields from investment in a customer may be negative in the short term, but build up over a number of years as the fruit of continued customer loyalty. The focus is on cultivating each of those relationships and the measure of value is on a basis of revenues from the relationship rather than revenues from a product. Such a long-term view enables managers to take an initial loss on a customer in the light of anticipated revenues from the long-term customer relationship.

Relationships do not only encompass customer and supplier. A company will have a wide variety of relationships - including supplier partnerships, customer partnerships, internal partnerships including employees, business units and departments and lateral partnerships including competitors, non-profit organisations and government. All these relationships need cultivating and addressing through marketing. The relationship develops through stages - awareness, exploration, expansion, commitment and dissolution. During each of these stages, different approaches to relationship marketing are required, different types and volumes of transaction may take place and the value of the relationship to the company changes.

Current popular approaches to IT investment analysis may be considered analogous to traditional marketing. The focus of approaches such as capital appraisal investment techniques is the information system and the organisation not the people using it. Information systems are judged on the basis of cost savings and efficiencies, involving staff reductions and increased throughput. Such a focus on cost savings means that once the information system has been installed and the savings made through redundancies and redeployment, the search moves elsewhere for new systems to save cost. Such an approach is analogous to the short-termism of traditional marketing and does not consider the accumulating benefits of the information system as its position in the organisation is consolidated and its use evolves.

Furthermore, the focus on the information system as a capital product ignores the role of people in harvesting benefits from the system. The information system yields benefits from the way the users apply the information to gain new knowledge, support business relationships, and influence decisions. Furthermore these benefits emerge over a long period of time and may change significantly during the lifetime of the information system. This paper advocates a move away from a focus on the information system’s organisational benefits to a focus on the information system’s relational benefits. We ask: ‘How will the information system benefit each individual relationship?’ and ‘How can such benefits be analysed and accumulated to give an overall value of the information system for the organisation?’ The focus of the information system must be moved from short-term cost savings to long-term returns yielded from its role in supporting organisational relationships.

  1. Organisational Structuration: The Organisation as a network of Relationships.

Structuration theory (Giddens, 1984) posited an integration between organisational structures and individual actions. Organisation structures are created and changed as a result of individual interactions and do not exist independently of those actions. Giddens (1994) further identified mediating forces that linked organisational structures and individuals. Power, rules and norms, and social interpretation enable organisational structures to influence individual behaviours and individuals to affirm or deny organisational structures. Structuration theory suggests that the study of individual interactions will provide insight into the derivation of organisational structures.

Harris and Taylor (1998) developed these ideas further, concentrating on relationships and interactions and the role of conversation. They suggested that an organisation consists of a network of relationships between actors. The organisation may be a company, a self-help group, an institution or a market. In any case, the basic building block of the organisation that Harris and Taylor focus on is the dyadic relationship. Organisations are maintained by the interaction of individuals in dyadic relationships. These relationships involve interpretations of phenomena, the distribution of power and the use of rules and norms.

Within a dyadic relationship, the individuals take on roles. These roles are assumed within the relationship and have no value outside the relationship. Mutual expectations are attached to the roles, defining what each participant in the relationship expects from the other and from themselves (Vickers, 1967).

There is a strong coupling between the relationship and the interaction that takes place within the relationship. In other words, we identify the presence of a relationship because of observed behaviour. The interaction occurring demonstrates the presence of a relationship and that interaction can be discovered, observed and analysed. Harris and Taylor (1998) suggested that the interaction consists of four parts: preparation, negotiation, performance and assessment, all requiring conversation. Often these interactions take the form of processes. Processes can be seen as routine or ritualised interactions within relationships. Often involving identical interactions occurring within many dyadic relationships (for example, a retail transaction, an order enquiry, a bank payment).

4. The Information System as a Relationship-Mediator

If organisations consist of networks of dyadic relationships, involving interactions between two actors taking on roles, then we can suggest that information systems operate within the context of relationships. The information systems are grounded within the relationship; recording roles and interactions, enabling the control of resources associated with these roles and relationships and adding value to relationships. The information system records the interaction that reveals the relationship. Indeed, without the dyadic relationship there may be no need for an information system.

This argument leads us to suggest that the value and benefit of the information system may lie in the way it supports and enriches each relationship that makes up the organisational social network. Therefore, the real value of the information system should be revealed at the relationship level.

Information systems perform a variety of roles in organisations, for which a variety of evaluation techniques are required (Farbey et al, 1999). While some of these roles may be focussed purely on resource control, many can be viewed directly or indirectly as concerning the development and maintenance of relationships. Information systems store data about relationships and are used by stakeholders within the context of a relationship.

Even a simple payroll system, for example, stores data about the relationship between companies and its employees. Each employee will be present in the system as a series of records concerning salary and expense payments made by the organisation. The payroll system mediates the relationship between the employee and the organisation. Therefore, those who have data in the system have an interest in the system and have associated costs and benefits. Besides the actual information involving relationships, the use of the system will involve relationships. The payroll clerk uses the payroll system in pursuance of a relationship with the organisation. This relationship involves tasks, goals, deliverables and deadlines. The clerk must enter the data and generate payroll output by set deadlines. Many further secondary relationships exist, in the case of the payroll between individuals or other organisations and the host organisation. There will be secondary relationships with the employee’s tax office, and with social security. There will also be secondary relationships with the organisation’s accountants and with managers within the organisation. All these relationships, mediated by the information systems, whether due to information presence or information usage, involve costs and, more importantly, benefits.

The traditional approach to justifying investment in a payroll system involves analysing the cost of the system - hardware, software, implementation and ongoing costs and comparing this to benefits. Benefits are calculated in terms of cost savings - staff savings in terms of payroll clerks, reduced usage of paper, less time needed per employee. These savings are in a sense negative. They look at what can be lost - reduced salaries, reduced paper requirements and less telephone calls. They are one-off savings. Once the system has been installed and the cuts made there are no further accountable benefits from the system. Relationship-based evaluation looks at the accumulating benefits to those using the system.

It could be suggested that most web-based systems focus on relationships, whether B2B or B2C. The systems enable interaction between participants in the relationship to achieve agreed goals, to fulfil mutual needs or to conduct an exchange of goods or information that has some value to the participants. A Customer-Relationship Marketing (CRM) system, for example, is exclusively concerned with the development and maintenance of relationships between the organisation and its customers. The entire value of such a system lies in the extent in which it adds value to the relationship as perceived by both the organisation and the customer. This value added by the CRM will be different for the organisation and the customer.

The following approach to IT investment appraisal centres on the identification and evaluation of relationships of importance to an organisation. Each IS-mediated relationship is evaluated and assigned a value which represents the added benefits brought to the relationship by the IS. The aggregation of values for individual relationships gives an overall organisational value for the system and enables the identification of new benefits and added value that may be provided to the relationships through the IS. The following sections outline the proposed steps in a relationship-based approach to IT investment appraisal. These are scoping and preparation, stakeholder identification, relationship identification, analysis of IS involvement, validation, aggregation and testing (see Figure 1).

 

5. Aims and Objectives of Relationship-based IT

The principle aim of evaluation is to identify the benefits and therefore justification of an IS based on the value it adds to organisational relationships. It is also important to provide investment figures that will be acceptable to finance directors and executives within the organisation. These figures may be obtained from an aggregation of the values determined from individual relationships. The figures will provide the justification for initial or further investment in the IS. Furthermore IT investment will require an examination of the costs. Individual costs within relationships - training, maintenance time, and workstation costs may be determined within the context of relationships. Overall costs of buying the system may be incorporated into the aggregate view or broken down and allocated to individual relationships. Examining IS value from a relationship point of view will, ex-ante, identify improved requirements and ex-post identify improvements in usage and new uses for the IS.

The objectives of a relationship-based approach to IT evaluation include:

6. Overview of the expected stages in relationship-based evaluation.

 

6.1 Scoping and Preparation

Scoping will involve stage, system and stakeholder. The stage of evaluation refers to whether it is ex-ante or ex-post. Relationship-based evaluation can be used to provide justification for an IT procurement, or to examine an implemented system for benefits. Ex-ante will involve an understanding of the problem that the IS will address and understanding of the tasks involved. Ex-ante may be combined, or part of requirements analysis. Ex-post will examine individual or group use of the IS and identifying the benefits accumulated. Note that most of the system lifecycle is in the maintenance phase. Evaluation at this stage will be an important contribution to the development of the system. It is important that the goals of the evaluation are understood. Ex-ante the prime aim will usually be to produce a justification for investment in an information system. This analysis should feed into the IS selection process. Ex-ante, we may be reviewing to see whether the benefits have been delivered, or analysing whether in information system should be retained in the organisation.

System scoping considers what information systems should be included or excluded. We may focus on the procurement of a particular system, we may examine a group of systems within a department, or we may carry out evaluation at a strategic level, examining the effect of IS on relationships between groups. For example, we could focus on a particular system such as an online shopping system or we could look at all IS usage within a department.

Stakeholder scoping involves choosing which stakeholder groups will be considered in the evaluation. We could examine the use of IS by every individual inside and outside an organisation and the effect on every relationship. But this would be an immense task. We need to consider at what level of detail we carry out the evaluation. We may select sample from user populations or consider ‘generic’ users for various populations. We may select particular relationships we are interested in as well. The greater of level of detail may increase the accuracy of our aggregated evaluation, but may not be cost-effective.

Large populations can be sampled using surveys, but detailed information and interpretation will inevitably require interviews and possibly observation. The involvement of stakeholders in the evaluation process, in which they examine their relationships with other stakeholders may also be important.

The output of this stage will include a plan which identifies the systems to be examined and the number and level of stakeholders. These plans will be modified as new stakeholders and relationships are discovered.

6.2. Stakeholder Identification.

The first major step is the identification of stakeholders who have or will have direct or indirect stakes in the IS. Direct stakeholders will be users and managers who enter information into the IS or extract and use information. Indirect stakeholders include those who have an interest in the IS because information about them is in the IS.

Firstly, stakeholders will be analysed in terms of roles, responsibilities, routines and relationships. Each stakeholder may have several different roles, each influencing the use of information systems. Responsibilities will be towards other stakeholders. Routines concern tasks needing to be done for the stakeholder to achieve her objectives. An analysis of formal roles and tasks will help build up a relationship map. Relationships will emerge from interviews with stakeholders. However, attention will need to be paid to informal roles and tasks, which may equally be mediated by the IS.

Secondly, stakeholders attitudes to IS and ability to use IS must be evaluated. An IS may be mediating a relationship ineffectively because of resistance from the stakeholders to the IS or inadequate training. This would reduce the value of the IS and may indicate that training would increase the value of the IS and should be included in the costs of the IS.

Thirdly, the objectives of the stakeholders should be examined. Objectives may have a political influence and may enhance the value of IS if, for example, it is politically expedient to use the IS, or reduce the value of the IS if individual objectives detract from its use. Rational, organisational and individual objectives should be considered (Vidgen and McMaster, 1995).

6.3. Relationship Identification

Having identified stakeholders, relationships between stakeholders should be explored. While described here as a separate step, it is likely that stakeholders and their relationships will be identified in tandem. A variety of relationships are possible between stakeholders. Table 1 identifies some of the types of relationship possible. Once a relationship is identified, questions need to be asked about it: How strong is that relationship? What is the level of commitment? What level of trust is involved? What are the interactions that indicate the presence of the relationship? What is the nature and frequency of these interactions? What value do the participant stakeholders attach to the relationship? What needs does each stakeholder consider to be met from the relationship? Are there unmet needs that could be derived from the relationship? How does information support that relationship? What value do the stakeholders put on the role of information in the relationship? What is the stage of maturity of the relationship?

At this point we have established, in the context of the use or proposed use of one or more information systems, a social network of stakeholders and their organisational relationships. The next task required is to identify and characterise the information system involvement in the social network, so that we can assign some value to each relational element of the social network and hence establish a value for the information system within the organisation.

Relationship

Description

Example Information Systems

Customer-Business

Customer purchases goods or services from business.

Shopping web site, Customer relationship management systems. Billing system.

Supplier-Business

Supplier provides goods to business.

EDI link, Extranet. Warehousing and distribution system

Employee - Employee

Employee interacts with another employee to achieve a business goal.

Intranet, Knowledge management system, Financial spreadsheet. Computer Aided Design

Agent-Business

A principal (owner) empowers an agent (manager it act on her behalf.

Decision support systems, management accounting system

Business Partner - Business

Two equal partners co-operate to achieve a business goal which individually they would not be able to do.

Extranet, E-mail. Global Information System

Table 1. Organisational Relationships

 

 

6.4. Identifying Information Systems Involvement

Identifying the organisational relationships of relevance to the IS under evaluation provides a framework for valuing the IS. While identifying IS involvement may be straightforward, allocating a value to the involvement is much more difficult. The stakeholder should be asked: what will happen to the relationship if you don't have this IS?

It should be noted that any value given to the IS is to a large extent based on a subjective judgement. Even a company's share value, for example, does not represent an objective valuation, but is based on the market's perception of the company's trading position and prospects. While analysis of the company's accounts, assets and market position can suggest its value, ultimately its valuation is a matter of judgement and may be a complete misjudgement, as has recently been seen in the case of dot com companies. Therefore, although we want our stakeholders to be as subjective as possible, the valuation of a relationship and the added value of an IS is a matter for their judgement, and this may change with time.

Our aim is, through discussion, to identify the value-adding contribution of the IS and to move towards some qualification of this value. We must ask: what does the IS offer the relationship? Table 2 offers some possible headings and examples of how IS might add value to a relationship. The step will involve identifying the ways the IS potentially offers value, deciding whether each value is negative or positive and assigning figures to the value.

Cost Reduction Value

Reduction in cost of interaction

 

Increased efficiency of transaction

 

Less duplication of effort/work

Less non-productive communication

 

Cheaper goods and services

Revenue Increasing Value

More sales

 

Increased number of interactions

 

Greater customer loyalty

 

Increased relationship lifetime

Quality enhancing value

Better products through better design co-operation

 

Assurance of professionalism of actors in relationship

 

Increased consistency of interaction

Goal-satisfying value

Enables satisfaction of stakeholder rational, organisational or individual goals

Table 2 : Classification of IS involvement in relationships.

 

6.5. Validation and Justification.

 

Validation and justification may be the most difficult stage of evaluation. The aim is, at the level of the individual relationship, to provide evidence of beneficial IS involvement in a relationship. IS involvement was identified in the previous stage, but needs to be backed up by evidence. Evidence may be drawn from stakeholder interviews, analysis of the interaction within the relationship and financial information as well as other sources. Development of a typology of benefits may enable the generation of ideas and hence the search for evidence. A coherent and rational case should be constructed for how the IS will benefit the relationship.

The evidence may include bottom-line financial data, but may also illustrate value judgements made by relationship participants. Statistical, logical, expert and anecdotal evidence should be gathered. Primary evidence, which is direct evidence from stakeholders, documents, or statistics and secondary evidence, which may include case studies and examples from the information systems working in a different organisation should be used. Arguments should be constructed that make a convincing case that relationship benefits will or will not result from the IS. An evidence file should be developed which documents evidence for IS benefits for each different type of relationship.

Validating the assertion that an IS contributed in a stated way to a relationship will depend on the strength of the evidence. We need to ask:

 

6.6 Aggregation.

The results of relationship based evaluation will need to be aggregated to reflect the total value to the organisation. For example, the evaluation may have involved two or three representative relationships from four or five types of relationship which may be supported by the IS. These will have to be multiplied to reflect the size of the user population and then summed to give a full picture of the organisational benefit.

6.7 Test

The resulting evaluation needs to be acceptable to the stakeholders and senior decision- makers. A workshop should be run to consider the relationships, the interactions and the evidence of benefit. This workshop should have the final say as to whether the results of the appraisal are acceptable and whether a sufficient case has been made to justify investment in the IS. In acting like a jury, different viewpoints are brought to bear on the evaluation and soft data is aired and given consideration in addition to the hard data.

 

 

7. Advantages of the approach.

A relationship-based approach to IT investment appraisal may have some significant advantages. Firstly, the scope of the exercise can easily be varied. A quick evaluation of one relationship and the effect of an IS can be carried out, or a more extensive exercise entered into. The effect of one IS on various relationships may be examined, or the effect of all IS on one relationship may be studied.

Secondly, the approach cannot be undertaken remotely. It requires interaction with stakeholders and an investigation of the effect of the IS at the coalface. It provides an opportunity for identifying strategic IS and does not differentiate strategic IS from factory IS.

Thirdly, it provides an opportunity for user education and increasing the understanding of users concerning the potential of IS to add value to business relationships. It may also provide an easy bridge to requirements analysis that does not necessarily have to be a separate exercise.

Finally, while focusing on people rather than capital, it may provide a bottom-line financial valuation that may satisfy the finance directors and accountants within the organisation.

 

8. Problems with Approach

While relationships may provide a useful focus for IS evaluation, the problem of assigning values to intangible benefits remains. It will be easy to identify the effect of the IS on transaction costs, for example, but the effect on intangibles such as loyalty, trust and quality of communication remains difficult to quantify. While the evidence built up may make the case for soft benefits, a case which may be good enough to convince the workshop attendees, any requirements for actual hard figures will involve quantifying the intangible benefits of the IS in the relationship. Some kind of value analysis (Keen, 1981) may be required where stakeholders in the relationship are asked 'how much are these benefits worth to you in this relationship?'

Furthermore a decision has to be made as to the granularity of the approach. Interviewing all stakeholders and analysing all relationships and interaction may be impractical. Representative stakeholders should be interviewed and an average of several interviews used in aggregation.

Also, while this paper has focussed on benefits appraisal, some cost allocation will be required, in which costs are considered at the relationship level.

 

9. Conclusion

The approach outlined above may be applied to the ex-ante evaluation of organisations’ web-sites. While cost-savings should be considered as a deliverable benefit from a web-site in terms of reduced transaction costs, the real value of a web-site will lie in its ability to act as a channel for improving and developing relationships with customers and other stakeholders. The value of the relationship should be increased through the use of the site and the application of a relationship-based evaluation approach, if successful, should enable some demonstration of that increase in value.

A new approach to IS evaluation which can measure the effectiveness of web sites is certainly needed. But this is not an easy problem. Charles Dunstone (1999), managing director of Carphone Warehouse has commented, ‘It is almost impossible to measure the effectiveness of the web sites, because there is no way to know how many customers follow up an electronic enquiry with a purchase by mail order or at a retail store. All we know is that a lot of people are hitting the site and making in-depth enquiries, and our overall profit figures are up, and we have to assume the two are connected.’ This indicates a lack of understanding of the relationship between IS and organisational benefits. Research is needed on the classification benefit types and the traceability of benefits from the IT to the organisation. A focus on relationships may help in developing such research.

The focus of information systems evaluation should be moved from cost-cutting to value- adding, from the organisational to the individual, from efficiency to productivity, from transaction costs to information value, from short-term to long-term and from capital to people. Taking a relationship-based approach to evaluation which treats information systems as relationship-mediators may provide the key to such a shift in focus.

10. References

Ballantine, J and Stray,S. (1998) Financial appraisal and the IS/IT investment decision making process. Journal of Information Technology, 15, 3-14.

Belcher L W and Watson H J (1993) Assessing the value of Conoco's EIS. MIS Quarterly 17, 239-253

Buttle, F. (1996) Relationship Marketing in Relationship Marketing, Theory and Practice ed Buttle, F, p 1-16.

Dunstone C. (1999) Quoted in Microsoft OnLine Spotlight, VNU Business Publications, London.

Giddens, A. (1984) The Constitution of Society. Polity Press, Cambridge.

Farbey B, Land F and Targett D (1993) IT investment. A study of methods and practice. Butterworth-Heinemann, Oxford.

Harris, G.B. and Taylor ,S.S. (1998) Organisational Structuration: Interaction and Interrelation.

Keen, P.G.W. (1981) Value analysis: justifying decision support systems. MIS Quarterly 5, 1-15.

McBride, N.K. (1999) Public Domain Information Systems. Presented at the 9th Annual Business Information Technology Conference, Manchester Metropolitan University, 4-5 November 1999.

McBride N K and Fidler C (1994) An interpretive approach to the justification of investment in Executive Information Systems in Proceedings of the First European Conference on Information Technology Investment Evaluation. (Brown,A and Remenyi,D. Eds.) Henley on Thames, UK, 13-14 September.

McKenney, J.L., Mason, R.O. and Copeland, D.G. (1997) Bank of America: The Crest and Trough of Technological Leadership. MIS Quarterly, September 1997, 221- 352.

Smithson,S and Hirschheim,R. (1998) Analysing information systems evaluation: another look at an old problem. European Journal of Information Systems 7, 158 - 174.

Vickers,G. (1967) Stability, Control and Choice. In Towards a Sociology of Management, Chapman and Hall, London.

Vidgen,R and McMaster,T. (1995) Black-boxes, Non-human stakeholders and the translation of IT through mediation. In Information Technology and Changes in Organisational Work, edited by Orlikowski, W, Walsham, G, Jones, M.R. and De Gross, J.I. Pp 250 - 270. Chapman and Hall, London.

Acknowledgement

I thank Mohamed Sheriff for comments on an earlier draft of this paper.