FORUM

 

Ambivalent implications of health care information systems: a study in the Brazilian public health care system

 

Implicações ambivalentes de sistemas de informação de saúde: um estudo no sistema Brasileiro de saúde pública

 

Implicaciones ambivalentes de sistemas de información de salud: un estudio en el sistema Brasileño de salud pública

 

 

João Porto de AlbuquerqueI; Edmir P. V. PradoII; Gabriel Raja MachadoIII

IProfessor at Departamento de Sistemas de Computação do Instituto de Ciências Matemáticas e de Computação, Universidade de São Paulo - São Carlos - SP, Brazil. jporto@icmc.usp.br
IIProfessor at Escola de Artes, Ciências e Humanidades, Universidade de São Paulo - São Paulo - SP, Brazil. eprado@usp.br
IIIB.S in Sistemas de Informação at Escola de Artes, Ciências e Humanidades, Universidade de São Paulo - São Paulo - SP, Brazil. rajamachado@yahoo.com.br

 

 


ABSTRACT

This article evaluates social implications of the "SIGA" Health Care Information System (HIS) in a public health care organization in the city of São Paulo. The evaluation was performed by means of an in-depth case study with patients and staff of a public health care organization, using qualitative and quantitative data. On the one hand, the system had consequences perceived as positive such as improved convenience and democratization of specialized treatment for patients and improvements in work organization. On the other hand, negative outcomes were reported, like difficulties faced by employees due to little familiarity with IT and an increase in the time needed to schedule appointments. Results show the ambiguity of the implications of HIS in developing countries, emphasizing the need for a more nuanced view of the evaluation of failures and successes and the importance of social contextual factors.

Keywords: Health care information systems; social implications; Brazil; public health care; information technology.


RESUMO

Este artigo analisa implicações sociais do Sistema de Informação SIGA em uma instituição pública de saúde na cidade de São Paulo. A avaliação foi realizada por meio de estudo de caso em profundidade com pacientes e funcionários da instituição pública, utilizando metodologia qualitativa e quantitativa. Por um lado, o sistema teve conseqüências percebidas como positivas como a melhoria na comodidade e democratização do atendimento especializado aos pacientes e melhorias na organização do trabalho. Por outro lado, resultados negativos foram relatados, como dificuldades enfrentadas pelos trabalhadores devido a pouca familiaridade com TI e um aumento no tempo necessário para agendar consultas. Os resultados mostram a ambigüidade das implicações dos sistemas de informação de saúde em países em desenvolvimento, enfatizando a necessidade de uma visão mais apurada em relação à avaliação de fracassos e sucessos e a importância de fatores sociais contextuais.

Palavras-chave: Sistemas de informação de saúde; implicações sociais; Brasil; sistema de saúde pública; tecnologia da informação.


RESUMEN

Este artículo analiza las implicaciones sociales del Sistema de Información SIGA en una institución pública de salud, en la ciudad de São Paulo. La evaluación fue realizada por medio de un estudio de caso exploratorio en profundidad con pacientes y funcionarios de la institución pública, utilizando una metodología cualitativa y cuantitativa. Los resultados obtenidos fueron ambivalentes: por un lado, el sistema tuvo consecuencias percibidas como positivas, tales como la mejoría en la comodidad y la democratización de la atención especializada para los pacientes, así como mejorías en la organización del trabajo. Por otro lado, no obstante, fueron relatados también resultados negativos, tales como las dificultades enfrentadas por los trabajadores, debido a su poca familiaridad con la tecnología de la información y, más sorprendentemente, un aumento en el tiempo necesario para marcar consultas médicas. Los resultados muestran la ambigüedad de las implicaciones de los sistemas de información de salud en países en desarrollo, enfatizando la necesidad de una visión más aguzada en relación a la evaluación de fracasos o éxitos, así como la importancia de factores sociales contextuales.

Palabras clave: Sistemas de información de salud, implicaciones sociales, Brasil, sistema de salud pública, tecnología de la información.


 

 

INTRODUCTION

Recent years have seen a growing dissemination of Information and Communication Technology (ICT) in a wide variety of areas and activities of human endeavor. The situation is no different in the health care sector. The introduction of Health Care Information Systems (HIS) is commonly explained by the need of improving treatment and integration of medical information through digitalization of work processes in health care so as to generate productivity gains and improve health services (MINISTÉRIO DA SAÚDE, 2004).

In the medical context, information is indeed particularly important, not only due to its critical role in medical treatment but also due to the complexity and importance of health care activities. Indeed, economist Amartya Sen in his theory of development as freedom (SEN, 1999) includes the opportunity to receive appropriate health care assistance as one of the constitutive components of development. The use of Information Systems to improve health care is therefore an important research topic, and so the potential of positively contributing to development can be effectively realized.

In the context of health care organizations, it is expected that the immediate access to information provided by Information Systems will allow for better quality in patient assistance, together with cost reduction. As a consequence, the use of Health Care Information Systems (HIS) worldwide has steadily increased in recent years. According to Carvalho and others (2008), Brazilian hospitals are also in the process of gradual computerization by implementation of health care-specific Information Systems. However, many have questioned the efficacy of automated systems in health care. Borba and others (2003) point out that Brazilian hospitals are increasingly computerized, but most HIS are still directed towards administrative functions rather than towards improving assistance. Even worse, although computerized systems are often considered drivers of change and innovation, HIS are in practice frequently not able to fulfill these expectations. Heeks, Mundy and Salazar (2000) believe that not only many, but most HIS are failures. Regarding the specific context of developing countries, despite the inexistence of strong evidence, several authors point out to the high rates of failure of HIS in developing countries (AVGEROU, WALSHAM, 2000; HEEKS, 2002).

Therefore, it is important to perform studies to evaluate the successes and failures of HIS in the particular context of developing countries, and particularly in Latin America, in order to improve our understanding of the practical results and effects of HIS. Furthermore, we must go beyond an evaluation based solely on technical parameters towards a broader consideration of the social and technical imbrications in the development and use of HIS in order to obtain a deeper comprehension of the implications of the growing computerization of organizational processes in the health care sector (ALBUQUERQUE, 2009).

In this context, this article aims at evaluating the social implications of the deployment of a HIS known as "SIGA Saúde" (henceforth referred to as SIGA) for hospitals of the Brazilian public National Health Care System (SUS) in the city of São Paulo. The city of São Paulo has a population of approximately 10 million today and consequently faces several problems due to its large dimension. Providing health care for this population is a major challenge for the public health sector, which has more than 500 health care facilities of different types. Therefore, the management of a public health system of this size is far from being a trivial task. In this context, the SIGA system was implemented by the Municipal Government of São Paulo in 2004 with the purpose of integrating the Information Systems of municipal public health care facilities in order to improve the health care services for its users.

Accordingly, the main objective of this article is to evaluate the implications of the SIGA system in hospitals of the São Paulo public health care system by means of an in-depth case study in a health care organization. This evaluation was made from the point of view of patients and staff of the public health care organization selected to better understand the perceptions of the effects brought about by the system for people directly involved with it.

The remainder of this article is organized as follows: Section 2 presents the background of this research, while Section 3 discusses the methodology employed. Section 4 presents the results, which are then discussed in Section 5. Finally, Section 6 concludes this paper with final remarks.

 

BACKGROUND

The health care scenario in Brazil

In recent decades, we saw, both in Brazil and worldwide, a decline in infant mortality, an increase in the aging population and a reduction of fertility. This led to an increase in the consumption of health care services. For instance, the population over 65 years old demands up to four times more hospitalizations than the average population (WORLD HEALTH ORGANIZATION, 2000). In addition, prices of health care services are growing faster than the average value of other services in the economy. This is, in part, a result of two factors: the increasing demand for health care services, which cannot be met by the supply; and the pressure for the use of new technologies, which generates investment depreciation costs and problems regarding their viability in the market (INSTITUTO DE ESTUDOS DE SAÚDE SUPLEMENTAR, 2006).

Most of the Brazilian population and many health care institutions depend on public health care services. In Brazil, the federal government established in 1988 the National Health Care System (Sistema Único de Saúde - SUS). SUS is one of the largest public health care systems in the world, covering a wide spectrum of services, ranging from simple outpatient health care to organ transplants, and ensuring free access to health services for the Brazilian population (PORTAL DA SAÚDE, 2010). According to the World Health Organization, despite its many problems, SUS has brought quality health care to millions of poorer inhabitants who were previously denied even basic health care (WORLD HEALTH ORGANIZATION, 2008).

According Maudonnet (1988), Brazilian Health Organizations can be classified into four categories: a) traditional, supported by religious and philanthropic institutions; b) supported by religious institutions but not for profit; c) private for-profit institutions audited by the State; and d) government-supported public institutions. The analysis of this article is based on a public organization of the Brazilian National Health Care System (SUS).

Information systems in health care organizations

Information has a significant importance for health care institutions. For this reason, the management needs IS to organize and manage information regarding the operations of a hospital. As Amaral (1998) points out, access to information on users of health care services improves the quality of these services. It also enables cost reduction and proper management of services, because it avoids the repetition of unnecessary and costly medical diagnostic procedures. In a wider context, IS can also be used to control medical practices as a whole, e.g. by recording the occurrence of specific diseases so as to allow control of epidemics and provide data to substantiate public health policies. In this paper, however, our focus lies on the implications of IS for health care organizations, i.e. hospitals.

According to Carvalho and others (2008), a hospital is a structure of remarkable complexity with a goal of integrating its business processes through information management. Gurgel Jr. and others (2002) corroborate this view. For these authors, a health care institution is one of the most complex organizations, because it has a multidisciplinary team with a high degree of autonomy in the provision of health care. Moreover, these people work in an environment that employs several different technologies and has a practice of teaching and learning, as well as scientific production.

The development of Health Care Information Systems (HIS) has increased as a result of competition and of the demand for competitive advantages. According to Carvalho and others (2008), a HIS must support different hospital services, ranging from technical service to management tasks. As a result, the health care sector uses different types of HIS with different goals. Borba and others (2003) highlight the Electronic Patient Record, Electronic Prescribing and Integrated Management Systems among the main trends in the industry.

In Brazil, particularly in the public sector, there is a major shortage of IT resources. According to Malik and Neto (2007), the main reason for this lies in the fact that there are managers who claim that investment in IT is a luxury that diverts resources from health care. The same authors recognize that a hospital are one of the most complex organizations ever created, and therefore it needs a tool to make data regarding its operations readily available so as to allow an efficient management process.

Benefits of IS in health care organizations

In general, organizations benefit from the use of IS. Earl (1990) highlighted a number of advantages: improvement of the quality of information for decision-making; automation of routine tasks; improved control of operating activities and customer service; increased ability to detect problems early; and increased productivity and competitiveness.

These benefits are also obtained by health care organizations. A study by Galvão and Sawada (1996) showed benefits in the use of IS by these organizations. Significant benefits are related to the agility in obtaining information, streamlining of services and cost reduction. However, some factors influence the achievement of these benefits. Davenport (1998) mentioned the high investments required by IS, while Ball (2002) associates the benefits of IS to how IS are used to generate improvements in the health care area. Other authors such as Mabert, Soni and Venkatraman (2003) highlight the influence of the size of the organization. According to these authors, the size of the organization can affect the results due to the complexity of the organizational change required. Carneiro and Dias (2006) corroborate this view and add two factors that improve the results of the implementation of an IS: the availability of resources and the early adoption of IS. Goguen and Linde (1994) point out that the training of users is a critical factor for successful IS deployments and stress that the involvement and participation of users increase the chance of success.

 

RESEARCH METHODOLOGY

The research is characterized as an exploratory study. According to Selltiz, Wrightman and Cook (1987), exploratory studies aim at familiarizing the researcher with the phenomenon or obtaining a new understanding. As such, this research consists of an exploratory study with the main objective of evaluating the results and effects of implementing the SIGA HIS in hospitals of the São Paulo municipal public health care system.

In order to evaluate the results and effects of the deployment of SIGA, we sought to identify the implications of the implementation of the system for people in the health care organization. According to Yin (2009), three factors determine the type of research strategy to be used: the research question, the degree of control that the researcher has over behavioral events and the degree of focus on contemporary or historical events. Thus, we adopted in-depth case study as the main research strategy, since it is considered well-suited for contemporary exploratory studies in which there are no clear-cut boundaries between the phenomena being studied and the context and there is limited control over behavioral variables (YIN, 2009).

The main unit of analysis of this study is a Specialty Clinic of the São Paulo municipal public health care system. Among the several types of public health care facilities in São Paulo (see below in Document Analysis, the Specialty Clinic was selected as a research object for this study because this type of health care facility provides many medical specialties, and thus receives different users sent from various other health care units that do not provide specialized medical care. Therefore, Specialty Clinics present a higher diversity of patients and perceptions, which this study intends to analyze, and may be viewed as a typical case in this context. Furthermore, this research has other two embedded subunits of analysis: one comprises the employees of the health care organization studied, while the other consists of the patients who are users of the health care services.

As for the data gathering, two different strategies were employed, one for each stage of the research. The first stage was of a qualitative nature and had the primary goal of identifying the key features of the system and the perceived improvements resulting from the implementation of SIGA to employees and patients. In this task, we performed an analysis of the existing documentation and observation of participants at a Specialty Clinic to find both positive and negative implications of the system.

The second stage of the research was quantitative and aimed at examining whether the issues identified in the qualitative stage were confirmed by the perceptions of users of the health care service and employees who operate the system. At this stage, we performed a survey of patients and employees of the Specialty Clinic selected.

The results of the two stages were analyzed and compared in two types of triangulation (YIN, 2009): data triangulation (i.e. comparison of the empirical evidence collected at the different stages) and methodological triangulation (i.e. contrasting the results obtained by the qualitative methods with the results of the quantitative methods). Figure 1 schematically shows the design of this research and its different stages.

 

 

Qualitative stage

The qualitative stage involved two steps:

a) Documentation analysis. In this stage, we analyzed several secondary sources, i.e. existing documents about the SIGA system, including training manuals (ATTI-SMS-SP, 2008), press notices (ESTADO DE SÃO PAULO, 2006; CAMPOS, 2008) and the websites of the São Paulo Municipal Secretary of Health Care (SMS, 2009) and of the company who developed the system (ATECH, 2009). These documents were carefully considered in order to obtain an idea of the features and advantages of the system, with the caveat that this information should be regarded as evidence to be confirmed by other sources, and not as conclusive findings (YIN, 2009, p. 114).

b) Observation of participants. In this stage, one of the authors of this paper acted as a system facilitator in the health care unit under analysis for a period of 12 months, and also helped in scheduling medical appointments with the system. As such, this experience of immersion in the system environment offered important insights on the implications of the system. According to Yin (2009, p. 122), observation of participants provides the benefits of facilitating access to data, in addition to the interesting opportunity to perceive the reality from the perspective of someone "inside" the Case Study rather than from an external perspective. This opportunity was used to observe the positive and negative implications of the implementation of SIGA from a direct experience of system use. The observations of the researcher were conducted in an informal way, and the impressions gathered were controlled in several informal conversations with staff and patients. However, one of the major disadvantages of the observation of participants is the possibility that the researcher introducing bias due to his closeness to the phenomena. To address this problem, the observations of the researcher were also discussed with the other researchers and triangulated with other types of data gathered (i.e. direct analysis of the system and of its documentation). Moreover, the results of this stage were used as a basis for the quantitative stage as explained below.

Quantitative stage

The quantitative stage of this research had the objective of compiling and analyzing data in order to confirm the evidence gathered in the qualitative stage. For this reason, we applied a survey with the following characteristics:

a) Population sample. The population studied includes users of SIGA, including both employees of the health care unit analyzed and users of the São Paulo public health care system. The selection of users has been made on different samples. One of the selection criteria used was to include only users who also had contact with the old manual medical appointment system. The statistical technique adopted in the data analysis was Fisher's Exact Test (SPSS, 2001). Due to time and cost constraints, we opted for a non-probabilistic sampling.

b) Data Gathering Tool. The data gathered in the quantitative stage are primary, i.e. they had never been gathered before and are in possession of the researchers. A structured questionnaire was used as a data gathering tool. The advantages of this tool are its low cost of handling and the uniformity of measurement. Selltiz, Wrightman and Cook (1987) also point out that the questionnaire is the best way to gather information from a large number of respondents at a low cost. Data were gathered in the first half of 2009. Before the data were gathered, the tool was pretested with five health care system users and five employees. The pretest was aimed at checking the clarity of the questionnaire and its proper understanding by respondents.

c) Data Analysis Procedures. Data analysis was performed in two steps. In the first step were used descriptive statistics, which includes frequency tables. The objective of this step was to describe the characteristics of the sample. In the second step, we used Fisher's Exact Test to evaluate the statistical significance of the relationships found.

 

PRESENTATION OF RESULTS

Qualitative analysis

Qualitative analysis was performed using documentation analysis and observation of participant. The documents analyzed were obtained at health care units and supplemented with secondary information sources related to the São Paulo municipal administration. Observation of participants was conducted by one of the authors of this paper, who experienced the process of deployment of SIGA. These two activities are detailed in the following sections.

Document analysis

According to data obtained in 2008 directly from the São Paulo Municipal Health Care Secretary (SP-SMS), the city has 396 Basic Health Units (Unidades Básicas de Saúde - UBS), 28 Medical Clinics (Assistências Médicas Ambulatoriais - AMA), 12 Emergency Clinics (Prontos-Socorros - PS), 5 Emergency Ambulatories (Prontos-Atendimentos - PA), 23 Specialty Clinics (Ambulatórios de Especialidades - AE), 65 Mental Health units, 44 support centers, 16 dental care units, 5 occupational health units and dozens of others in areas such as specialized health care in sexually-transmitted diseases and substance abuse. In 2004, the São Paulo Municipal Health Care Secretary (SMS) decided to implement "SIGA Saúde" (referred to as SIGA), a Healthcare Information System (HIS) with the task of linking and organizing the city's health care facilities. The system was developed by an IT provider called ATECH, with the help of other organizations to provide training, support and technological infrastructure. The implementation process took about three years, and SIGA is currently available in all health care facilities in São Paulo, having become the main information system of the SMS, under the responsibility of its Information Technology Advisory Department (ATTI-SMS-SP, 2008).

According to ATECH, SIGA consists of "a set of technologies that integrates and streamlines the work of the health care public sector, allowing data gathering and providing information about patients, as well as linking appointments to physicians and requests for medical examinations" (ATECH, 2009). The system provides support to internal functions such as registration of patients with the Identification Card (IDC) used in the National Health Care System (SUS). Among the benefits offered to the patients, the identification card allows them to have medical care in any public health care facility.

• SIGA was initially developed with the following modules (MEDEIROS, CORAL, 2006):

• Users Management ("Cadastro de Usuários") controls and manages the national ID Card of patients;

• CMES ("Cadastro Municipal dos Estabelecimentos de Saúde") controls and manages municipal health care facilities;

• Local Scheduling ("Agenda Local") manages the medical scheduling process of a given health care facility;

• Integrated Scheduling ("Agenda Regulada") manages the medical scheduling process across different health care facilities;

• APAC ("Autorização de Procedimentos de Alta Complexidade") manages authorizations for high-complexity medical procedures;

• Assistance ("Atendimento") allows registration and tracking of health care services provided to patients (related to electronic patient records).

Before SIGA, patients who needed specialized health care which was not available in the health care facility nearest to their home would receive an authorization for medical treatment in another facility that could provide it. After receiving such authorization, the patient had to seek an adequate facility, hoping that there would be available slots for an appointment. Finally, the patient went back to the selected facility on the scheduled date for the medical appointment. A major goal of SIGA was to improve this inefficient process.

The SIGA system should considerably improve this medical scheduling process. Whenever the need for specialized treatment was identified by an employee of any health care facility, the employee could use the Integrated Scheduling module to find out a facility that provides the special treatment required and scheule an appointment for the patient on a suitable date. This prevents the patient from going through many health care facilities until he found one that could offer the special treatment needed and had available slots for an appointment, usually having to wait in long lines (O ESTADO DE S. PAULO, 2006).

Statistics show that the use of the SIGA system has been increasing steadily in health care units. Just two years after its deployment, 372 (out of 404) health care units were already using the system. About 14 million public health care users were registered (more than the official population of São Paulo, indicating that the system also serves neighboring cities), and in 2006 more than 1.7 million medical treatments per month were being recorded in the system (O ESTADO DE S. PAULO, 2006).

A difficulty reported in implementing the system was the poor knowledge of Information Technology (IT) by the staff of health care facilities. According to information obtained directly from the SMS, the majority of the staff had never used a mouse or accessed the Internet before. Medeiros and Silva (2006) also emphasize the lack of culture in using IT tools in the Brazilian public health sector as a whole, since a good part of the processes in this sector, in particular all the administrative procedures, would still be performed manually. Thus, there was a considerable need of training.

To address this problem, the SMS started a process of training users ,and also created a task force with the recruitment of 300 trainees (students in IT-related courses) to act as facilitators in the SIGA deployment process. The choice of using students was justified by the visibility given to the project, in addition to cost savings (MEDEIROS, SILVA, 2006).

The SMS is also interested in spreading the use of SIGA in order to create user communities in other cities and states so as to share the costs of new developments. In this regard, a technical assistance agreement was signed with the Brazilian Ministry of Health, so that the system can be expanded to the rest of the country. In addition to the city of São Paulo, the cities of Campinas and Diadema, both in the state of São Paulo, and the city of Camaçari, in the state of Bahia, are already using SIGA (CAMPOS, 2008).

Observation of participants

One of the authors of this paper acted as a system facilitator for more than 12 months in the health care organization under analysis. This long-term observation revealed that the SIGA system has brought benefits to the users, but also posed some problems. Among the main advantages observed, we highlight the improved convenience for patients to schedule medical appointments. As mentioned above, before the deployment of SIGA, patients would go to the clinic with only an authorization to schedule an appointment, but they should come back on another date for the consultation with a specialist physician. The Integrated Scheduling module provides greater convenience in allowing patients to go to the health care unit nearest to their home and, after confirmation of the need of specialized health care, a medical appointment is automatically scheduled by the system in any public health care facility that offers the required treatment.

Other benefits reported by employees and by some patients were an improved work organization and workflow and improved procedures for scheduling medical consultations. Some patients reported that, prior to the implementation of SIGA, employees used a lot of forms and printed documents, which impaired the quality of the service. By automating the scheduling of medical consultations, the system eliminated the need for these documents and forms.

Another positive aspect is that SIGA has increased the access of patients to specialized medical care, thus making the public health system more democratic. Before the implementation of the system, the consultation schedule of a physician at a health care unit that provided special treatment usually contained few patients from distant neighborhoods. With the use of SIGA, the schedule of specialist physicians began to include more patients from various neighborhoods of the city, thereby improving access to specialized medical care for people living in (usually poorer) areas that do not have health care facilities offering more complex treatments.

Regarding the use of the SIGA by the staff, we also found some problems caused by the system deployment. Knowledge of IT by employees was very poor before the implementation of the system and so remained even after the implementation of SIGA. This was one of the causes pointed out by some employees for their difficulties in the operation of the system. Some of these employees did not realize any significant gain in their tasks using the system, preferring to use a manual physician scheduling process. It was also noted that the average age of the employees influenced the acceptance of the system, i.e. older employees had more difficulty in using the system and developed resistance to its use.

Another problem observed is that, despite the system's goal of scheduling appointments more agile, we actually noticed an increase in the time to schedule medical appointments with the use of the system's scheduling module. This increase was seen not only when the staff of the hospital was scheduling appointments, but also when the researcher himself was performing this task. Based on the concept of usability - i.e. the way that a product can be used to accomplish specific goals with effectiveness, efficiency and user satisfaction (ISO, 1998) - we can say that the SIGA System was often effective (since it achieved the goal of scheduling an appointment), but not efficient. In this respect, users reported difficulties with the system's appearance (e.g. some icons do not resemble their meaning), and they also found the system difficult to operate. These usability problems reported by the users were frequently observed by one of the researchers, who, based on his own system use, also found the software interface rather clumsy.

Another observed cause for the increase in the time to schedule appointments was that the system was down too often, which caused various problems for patients and staff at the health care units. In these situations, employees were required to schedule medical appointments manually and then retype them into the SIGA system when it became operational again. This implied double work and a consequent waste of time for employees. Another related situation faced by users is network latency in scheduling operations, which makes the service offered to patients more time-consuming. This problem is probably a result of an insufficient network infrastructure overloaded with the heavy traffic generated by the system.

The implications of the SIGA system can be sorted, therefore, into positive aspects (perceived benefits) and negative aspects (perceived difficulties), which are summarized in Exhibit 1. The next stage of this work aims at studying the perceptions of staff and patients about each of these features, using a quantitative analysis of questionnaires answered by both staff and patients. Only the feature named Democratization of Specialized Health Care will not be evaluated in the next stage, since it depends on a more detailed observation in the schedule of specialist physicians and their respective group of patients and cannot be analyzed only through questionnaires submitted to patients or employees.

Quantitative analysis

The quantitative analysis was performed in two steps. First, we used descriptive statistics to present the characteristics of the sample. Secondly, we analyzed some of the evidences using Fisher's Exact Test. The results of the two steps are explained below.

Characteristics of the sample

The sample consists of 79 users, including patients and staff who operate the system. All questionnaires were answered and the data that summarize the characteristics of the sample are presented in Table 1. Most of the sample comprises patients from a health care unit that offers more complex treatments. Most of these patients are more than 20 years old and their annual salary is less than US$ 38,000.

 

 

Analysis of the evidence

During the qualitative stage of this research, evidences of four changes in the process of physician scheduling were identified. These evidences were analyzed quantitatively through the submission of questionnaires to patients and staff and are described as follows.

a) Convenience for patients. There was an improvement in convenience for patients in having medical consultations due to the implementation of SIGA system, as shown in Table 2. Currently over 60% of patients (61.6%) and employees (63.2%) have a good perception of the new physician scheduling process. However, there was no statistically significant difference in patients and staff perception regarding age.

b) Knowledge of IT. There was an improvement in the knowledge of IT by employees when compared with the period before the implementation of SIGA. This was a contribution of SIGA. Prior to the implementation of SIGA, a difference in perception about IT knowledge was observed among employees under 40 years old and those over 40 years old. This difference is statistically significant at 10%, using Fisher's Exact Test. Table 3 shows this result, i.e. younger employees, before implementation of SIGA, had a greater knowledge of IT. Currently there is no longer a significant difference.

 

 

c) Increased time for the scheduling process. There was a worsening in the processing time for physician appointment scheduling. Table 4 shows that this worsening was more noticeable for the employees who operate the system. Before the implementation of SIGA, 89.5% of them felt that the processing time was good, and currently only 31.6% maintain the same opinion. The results are not statistically significant when analyzed by age group.

d) Work organization. There was improvement in work processes. This was a contribution perceived by both patients and staff, and is illustrated in Table 5. It is worth noting that only patients and staff over 40 years old have evaluated the work organization before the implementation of SIGA as "good." This perception is statistically significant at the 10% level using Fisher's Exact Test.

 

DISCUSSION

This work evaluates the implications of the introduction of "SIGA Saúde", a Healthcare Information System, in a hospital of the São Paulo municipal public health care system. In the first stage of the research, the following items were observed as positive aspects of the system: improved convenience for system users; increased democratization of specialized treatment; better work organization for the s health care unit staff. As negative aspects we observed: difficulties faced by employees due to their poor familiarity with Information Technology; increased time to schedule medical appointments in comparison with the previous manual procedure.

These aspects were then checked against the perceptions of staff and patients by triangulating the results of the qualitative stage with a quantitative study based on a survey. The results of this triangulation confirmed that SIGA has afforded a significant improvement in convenience for patients, mainly due to the easiness in scheduling appointments at other units of the health care public system. This was indeed a major motivation for the implementation of SIGA, which was successful in this regard.

As for the work organization of employees, the perception of patients and employees confirmed an improvement in comparison with the situation before the use of SIGA. An interesting observation in this matter was that only patients and employees who were 40 years old or older had a positive perception of the work organization before the implementation of SIGA. This may reveal a more conservative feature of this age group, which was also noted in the observation of participants.

One of the negative aspects identified in the observation of participant was the difficulty of users in using the system due to their poor familiarity with Information Technology (IT). However, the quantitative stage has shown that the implementation of SIGA has offered a contribution in this regard, because the percentage of employees that view their knowledge of IT as "good" has raised significantly after implementation. This effect was even greater for older employees. Before SIGA was used, younger employees had better familiarity with IT compared to older ones; after the implementation of the system, the difference between the two groups ceased to be significant. We could say that the system contributed for the digital inclusion of employees, a fact that probably was also due at least in part to the work of student trainees as system facilitators.

Perhaps the most surprising finding of this study was the time to schedule medical appointments with the SIGA system. Contradicting one of the main objectives of the system, i.e. making this procedure more agile, we observed already in the first stage of the research an increase in the average time required to schedule an appointment. This increase was also noticed by patients, as confirmed by the results of the survey, because there was an increase in patient perception of the time to schedule appointments as "bad" in comparison with the previous manual procedure, which even more prominent amongst employees.

We detected two main causes for this problem. First, the staff's difficulty with the system operation seems to be a major issue here. In spite of the observed enhancement of their familiarity with IT after SIGA was deployed, employees still face major problems using the system. Indeed, we noted that after the student trainees that worked as system facilitators had left, many employees had major problems to continue using the system. Secondly, another factor that seems to have contributed to the increase in the time to schedule appointments is the latency of the computer network. The bandwidth of the network link (256 kbps in most health care facilities) was clearly underestimated for the intense data traffic required by the system, thus imposing long waiting times during regular use of the system.

Comparison with previous work

Regarding the different hospital services that a HIS must support listed by Borba and others (2003) and Carvalho and others (2008), the SIGA system under analysis proved to be relatively comprehensive, since it includes modules for user management, healthcare facility management, scheduling medical appointments (both locally and in other organizations of the public healthcare system), handling authorizations for medical procedures and electronic patient records. Within this context, our analysis was focused on the functionalities related to the scheduling of medical appointments, which was identified by users as one of the most important functions of the system.

By comparing the results of our study with the aforementioned work by Galvão and Sawada (1996), we find that, of the three main benefits pointed out by those authors - namely, increased agility in obtaining information, streamlining of services and cost reduction - only the second (i.e. streamlining of services) seems to have been achieved as a result of the detected increased convenience for patients and improved general organization of the work in health care units. The other two objectives (increased agility in obtaining information and cost reduction) seem to have not been completely achieved, since this study reveals an increase in the time to schedule appointments after the deployment of SIGA, according to the perception of patients and employees. The main causes that we observed for this were related to contextual factors, such as the difficulties of the organization's staff in using the system and insufficient network infrastructure available. Thus, this enhanced understanding sheds light on dimensions which are usually neglected in several HIS projects, which are still mostly based on a narrow technically-centered view of Information Technology and Information Systems in the health sector (MORAES, GÓMEZ, 2007).

By analyzing the perception of employees in different age groups, one can also notice a curious fact. While electronic scheduling has raised the perception of the time required to schedule appointments as "good" among younger employees, the rate of good perception was lower than before among older ones. This discrepancy in perceptions between the two different age groups may indicate the need for special training and capacitating measures - a critical success factor according to the aforementioned study by Goguen and Linde (1993) - in order to address the difficulties of these groups in a separate manner.

Limitations and future work

The results of this study should be appreciated with regard to their limitations. First, the intent of this study was not to evaluate the implications of the SIGA system for the public health care system as a whole, but to verify the social implications of its deployment for the public hospitals of São Paulo from the point of view of staff and patients. The SIGA system encompasses many more functionalities and has much more complex goals than the focus of this article was aimed at analyzing. As such, a more comprehensive research is required to fully evaluate the effects of SIGA, such as, for instance, regarding the benefits brought about by the public health administration in the several administrative levels (e.g. local, regional and national).

Furthermore, we do not claim that the results found can be statistically generalized to all hospitals in the São Paulo public health care system, given that we analyzed a single health care unit among more than 500 other public health facilities in São Paulo. Future work could be done to replicate the same model of this research at other public health organizations in order to confirm results. Additionally, random samples could be used to extend the study to other municipalities that use the SIGA system.

Notwithstanding these limitations, many of the problems observed and their underlying causes can be expected to be common to many, if not all, health care organizations of the São Paulo municipal public health care system, since we studied a typical case in this context. As pointed out by Flyvbjerg (2006), formal statistical generalization should not be seen as the only way to achieve relevant scientific results, and in-depth single case studies can also be used to obtain general results by carefully choosing typical situations and reasoning about the general factors present in that particular context.

Therefore, we believe that the case study conducted here can indeed offer insights into some important implications of SIGA for the staff and patients of the São Paulo municipal public health care organizations. Moreover, the ambivalence of the effects of system deployment and the social contextual problems elucidated in our analysis of this case (e.g. poor digital literacy and insufficient infrastructure) are expected to be common challenges to many public health care organizations in developing countries, particularly in Latin America.

 

CONCLUSION

This paper evaluates the implications of the deployment of the SIGA Healthcare Information System (HIS) for hospitals in the São Paulo municipal public health care system. This evaluation was carried out through an in-depth exploratory case study at a health care unit by using a mixed-methods approach. First, a qualitative stage was conducted, based on documentation analysis and observation of participants and identifying positive and negative consequences from the deployment of SIGA. Later, a quantitative stage was conducted through a survey of employees and patients, in order to check if the perception of these authors coincided with the results obtained in the qualitative stage.

After evaluating the results obtained in both the qualitative and quantitative stages of this study, we can return to our main research question regarding the implications of the SIGA system for patients and employees of the public health care unit observed. The evaluation clearly points out to an ambivalent result of the deployment of SIGA in the analyzed organization: on the one hand, there was a significant improvement in the convenience of patients, who are now able to schedule appointments at any facility in the public health care system much more easily. Furthermore, this improved convenience has also resulted into increased democratization of specialized treatment, because, after verification of the need of specialized care, a medical appointment is automatically scheduled by the system at another health care unit that offers more complex treatment. On the other hand, however, the time needed to schedule appointments has increased with the deployment of SIGA, which certainly results in discomfort for both patients and staff, in direct contrast to one of the major goals of the development of the system.

These results show the ambivalence of the implications of HIS in the face of the challenges imposed by a reality that is common to many developing countries, particularly those in Latin America. While we could certainly argue that the implementation of SIGA in the analyzed organization has succeeded in several aspects, according to the perception of both staff and patients and our own observation, the system has also failed to deliver the benefits of agility in scheduling appointments in the health care organization analyzed. This implies the need of a more nuanced view regarding the evaluation of the failures or successes of HIS in order to capture the multifaceted implications of Information Systems in the reality of developing countries.

Furthermore, the two main causes for the increased time to schedule appointments - i.e. poor familiarity with IT and problems with the network infrastructure - are typical contextual factors of Latin American countries, in which a large part of the population has difficult access to quality education and network infrastructure costs are still usually high. As such, our study indicates the need for carefully considering such contextual factors and particularities in HIS projects in developing countries, particularly in Latin America, in order to better tailor them to the specific needs of people in these contexts.

 

REFERENCES

ALBUQUERQUE, J. P. de. Rethinking formalization processes in computerized systems: analyzing the co-evolution between software and organizational practices. RECIIS - Electronic Journal of Communication Information and Innovation Health, v. 3, n. 2, p. 7-15, 2009.         

AMARAL, M. B. Na direção do prontuário eletrônico de pacientes do HCFM da USP. In: FÓRUM NACIONAL DE CIÊNCIA E TECNOLOGIA EM SAÚDE, 1998, Curitiba. Anais. Curitiba: Fórum Nacional de Ciência e Tecnologia em Saúde, 1998.         

ATECH. Casos de Sucesso: SIGA Saúde - Secretaria Municipal de Saúde de São Paulo. Available at: http://www2.atech.br/Saude.atech. Access on: 04.04.2009.         

ATTI-SMS-SP. Projeto de Informatização da Secretaria Municipal de Saúde do Município de São Paulo: SIGA-SAÚDE Versão 3.0 a. Manual das Alterações Implantadas em 06/02/2008. Assessoria Técnica de Tecnologia da Informação, 2008.         

AVGEROU, C; WALSHAM, G. Information technology in context: studies from the perspective of developing countries. London: Ashgate, 2000.         

BALL, M. J. Hospital information systems: perspectives on problems and prospects, 1979 and 2002. International Journal of Medical Informatics, v. 69, n. 3, p. 83-89, 2003.         

BORBA, G. S. and others. Saúde integrada? Um estudo exploratório obre o uso de sistemas integrados de gestão (ERP) nos hospitais gaúchos. In: ENCONTRO NACIONAL DA ASSOCIAÇÃO DE PÓS-GRADUAÇÃO EM ADMINISTRAÇÃO, 27, Atibaia, 2003. Anais. Atibaia: ENANPAD, 2003.         

CAMPOS, M. V. Saúde chega a Diadema. Revista Vigor, s/n, 2008. Available at: http://www.revistavigor.com.br/2008/05/27/siga-saude-chega-a-diadema. Access on: 25.03.2010.         

CARNEIRO, T. C. J; DIAS, D. S. Sistemas integrados de gestão e a indústria farmacêutica no Brasil: segmentação e performance do negócio In: ENCONTRO NACIONAL DA ASSOCIAÇÃO DE PÓS-GRADUAÇÃO EM ADMINISTRAÇÃO, 30, Salvador, 2006. Anais. Salvador: ENANPAD, 2006.         

CARVALHO, R. B. and others. Avaliação da Implantação de ERP: Estudo de Caso de um Hospital de Grande Porte. In: ENCONTRO NACIONAL DA ASSOCIAÇÃO DE PÓS-GRADUAÇÃO EM ADMINISTRAÇÃO, 32, Rio de Janeiro, 2008. Anais. Rio de Janeiro: ENANPAD, 2008.         

DAVENPORT, T. H. Putting the enterprise into the enterprise system. Harvard Business Review, v. 33, n. 6, p. 121-131, 1998.         

EARL, M. Information management: the strategic dimension. Oxford: Clarendon Press, 1990.         

FLYVBJERG, B. Five misunderstandings about case-study research. Qualitative Inquiry, v. 12, n. 2, p. 219-245, 2006.         

GALVÃO, C. M; SAWADA, N. O. O uso da informática na rede básica e hospitalar da cidade de Ribeirão Preto (SP). Revista Latino-Americana de Enfermagem, v. 4, n. 1, p. 51-59, 1996.         

GOGUEN, J. A; LINDE C. Techniques for requirements elicitation. In: Proceedings of the first IEEE International Symposium on Requirements Engineering, San Diego, Ca, IEEE Computer Society Press, 1994, p. 152-164.         

GURGEL JR, G. D; VIEIRA, M. M. F. Qualidade total e administração hospitalar: explorando disjunções conceituais. Ciência & Saúde Coletiva, v. 7, n. 2, p. 325-333, 2002.         

HEEKS, R. Information systems and developing countries: failure, success, and local improvisations. The Information Society, v. 18, n. 2, p. 101-112, 2002.         

HEEKS, R; MUNDY, D; SALAZAR, A. Understanding success and failure of healthcare information systems. In: ARMONI, A. (Ed). Healthcare information systems. Hershey, PA: Idea Group, 2000.         

INTERNACIONAL STANDARDS ORGANIZATION. ISO 9241-11:1998. Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11: Guidance on usability. 1998.         

INSTITUTO DE ESTUDOS DE SAÚDE SUPLEMENTAR. Saúde suplementar frente às demandas de um mundo em transformação. Série IESS 001/2006. São Paulo: IESS, 2006.         

MABERT, V. A; SONI, A; VENKATRAMAN, M. A. The impact of organization size on enterprise resource planning (ERP) implementations in the US manufacturing. Omega, v. 31, n. 3, p. 235-246, 2003.         

MALIK, A. M; NETO, G. V. Tendências na assistência hospitalar. Ciência & Saúde Coletiva, v. 12, n. 4, p. 825-839,          2007.

MAUDONNET, R. Administração hospitalar. Rio de Janeiro: Cultura Médica, 1988.         

MEDEIROS, R. A; CORRAL, H. H. A. A saúde de São Paulo entra na era do Bit! In: CONGRESSO BRASILEIRO DE INFORMÁTICA EM SAÚDE, 10, Florianópolis, 2006. Anais. Florianópolis: CBIS, 2006.         

MEDEIROS, R. A; SILVA, L. C. A importância dos estagiários na implantação do sistema de informação da Secretaria Municipal de Saúde de São Paulo. In: CONGRESSO BRASILEIRO DE INFORMÁTICA EM SAÚDE, 10, Florianópolis, 2006. Anais. Florianópolis: CBIS, 2006.         

MINISTÉRIO DA SAÚDE. Política nacional de informação e informática em saúde: proposta versão 2.0. Brasília: Ministério da Saúde, 2004.         

MORAES, I. H. S; GÓMEZ, M. N. G. Informação e informática em saúde: caleidoscópio contemporâneo da saúde. Ciência & Saúde Coletiva, v. 12, n. 3, p. 553-565,          2007.

O ESTADO DE S. PAULO. Editorial: A saúde na era da TI. O Estado de S. Paulo, São Paulo, p. 2, 23.08.2006.         

PORTAL DA SAÚDE. O que é o SUS. Available at: http://portal.saude.gov.br/portal/saude/cidadao/area.cfm?id_area=1395. Access on: 25.03.2010.         

SELLTIZ, C; WRIGHTMAN, L. S; COOK, S. W. Métodos de pesquisa nas relações sociais. São Paulo: EPU, 1987.         

SEN, A. Development as freedom. Oxford: Oxford University Press, 1999.         

SPSS Inc. Exacts Tests 7.0. Chicago, 2001.         

WORLD HEALTH ORGANIZATION. The World Report 2000. Health systems: improving performance. Geneva, 2000.         

WORLD HEALTH ORGANIZATION. Flawed but fair: Brazil's health system reaches out to the poor. Bulletin of the World Health Organization, v. 86, n. 4, p. 241-320, 2008.         

YIN, R. K. Case study research: design and methods. 4th Ed. London: Sage, 2009.         

 

 

Submitted 31.03.2010.
Approved 14.10.2010

 

 

Evaluated in a double blind review
Scientific Editor: Eduardo Diniz, Marlei Pozzebon e Nicolau Reinhard