Perspectives in Clinical Research

: 2019  |  Volume : 10  |  Issue : 2  |  Page : 51--56

Conducting real-world evidence studies in India

Arun Bhatt 
 Consultant – Clinical Research and Development, Mumbai, Maharashtra, India

Correspondence Address:
Dr. Arun Bhatt
303-4, Dheeraj Valley 3/C, Mohan Gokhale Road, Goregaon (East), Mumbai - 400 063, Maharashtra


Real-world data (RWD) are data relating to patients' health status and/or the delivery of health care routinely collected from a variety of sources. Real-world evidence (RWE) is the clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of RWD. RWE can provide information on diverse areas, for example, natural history and course of disease, effectiveness studies, outcome research, and safety surveillance. India has unique opportunity of conducting RWE studies in several interesting areas, for example, natural history of communicable and noncommunicable diseases and rare disease and Health Economics and Outcomes Research in public versus private hospitals. However, utility of RWE is challenged by diversity of information, large data sets of uncertain quality, and methodological rigor. For India, there are additional challenges in conducting RWE studies – Indian clinical practice pattern and physicians' interest in RWE studies. It is recommended that RWE studies in India should focus on health management issues of relevance to the country India and should be well planned to generate high-quality data.

How to cite this article:
Bhatt A. Conducting real-world evidence studies in India.Perspect Clin Res 2019;10:51-56

How to cite this URL:
Bhatt A. Conducting real-world evidence studies in India. Perspect Clin Res [serial online] 2019 [cited 2020 Feb 27 ];10:51-56
Available from:

Full Text


Real-world data (RWD) are data relating to patients' health status and/or the delivery of health care routinely collected from a variety of sources.[1] Real-world evidence (RWE) is the clinical evidence regarding the usage and potential benefits or risks of a medical product derived from the analysis of RWD.[1] RWD includes variety of data – electronic health records (EHRs), insurance data, product and disease registries, etc. RWD has potential to complement the knowledge available from conventional randomized clinical trials (RCTs), whose design limitations make it difficult to generalize findings to population at large.[2] RWE can provide information on diverse areas, for example, natural history and course of disease, effectiveness studies, outcome research, and safety surveillance. However, utility of RWE is challenged by diversity of information, large data sets of uncertain quality, and methodological rigor. These issues are relevant to conduct of RWE studies in India.

 Real-World Evidence Study Versus Randomized Clinical Trials

RCTs are designed to provide robust evidence of premarket evaluation of efficacy and safety of an investigational product. RCT design focuses on attaining internal validity, which may sometimes compromise generalizability.[2] In contrast, RWE studies are planned to provide data with relevance to clinical practice. Differences between RWE study and RCT [3] are summarized in [Table 1].{Table 1}

 Real-World Evidence Studies: Potential Benefits

RWE studies are valuable across product lifecycle and can provide multiple benefits.[2],[3],[4]

Prior to product development

Natural history of disease – prevalence, incidence, and unmet medical needCurrent treatment patterns and standard of careUnderstanding current service structures, especially if they may need to change to deliver the new medicineGenerate hypotheses for prospective trials.

At or following market authorization

Insights into early and/or long-term clinical experienceProspective studies for safety surveillanceEvidence to support health outcomes, reduction in resource use, and patients' acceptabilityDetection of untreated/undiagnosed patients (“unmet need”).

After product launch

Comparative effectiveness and health outcomesCompliance with product label or treatment guidelinesIdentification of suboptimal dosing or treatmentSupport for expansion of indicationsCharacterizing appropriate treatment subgroupsMonitoring safety, especially uncommon/rare adverse reactions.

Despite many potential benefits, RWE studies are primarily conducted to meet demands or expectations of reimbursement or insurance agencies and patients and to satisfy drug regulatory agencies.[3] The payers and health-care decision-makers have been increasingly demanding postapproval evidence to support reviews of pricing, reimbursement, licensing for new therapies, and formulation and indication changes.[3] However, according to a recent survey of 30 companies, the objectives of most RWE studies were postapproval safety and commercialization strategies.[5]

 Real-World Evidence Studies: Designs

RWE studies employ retrospective and prospective designs.[3],[4]

Retrospective design studies have predictable short-term data collection period, are dependent on the quality of available clinical records, and are usually low-cost studies. However, such studies have limitations of incomplete or inaccurate data. Examples of such studies include:

Clinical/hospital record/chart reviewAdministrative data/insurance claim dataElectronic medical records (EMRs).

Prospective design studies can include robust patient data set and clinical data not routinely captured, and provide information on long-term data on natural history or disease progression. However, such studies have uncertain timelines, which are dependent on the rate of presentation of eligible patients and are likely to be costly. Examples of such studies include:

RegistriesHealth surveysProspective observational studiesPostauthorization safety study.

 Real-World Evidence Studies: Challenges

There are several challenges in interpretation and in utility of RWE studies.

Validity – Open, unblinded, and nonrandomized designs used in RWE studies raise questions about their validity [2],[3]Quality of data – Most of the sources for RWE studies are data collected to support clinical care and reimbursement. These data are not gathered to support any specific research study. Hence, there are gaps in the accuracy, completeness, and quality of RW data.[2],[3],[6] This is likely to be a big challenge in India, especially in the case of RWE studies based on retrospective hospital databasesConfounding – A small observed effect in an RWE study could be due to confounding factors.[2]

These challenges point to the need for proper planning and conduct of RWE studies.

 Planning of Real-World Evidence Study

Several factors should be considered in planning of RW study, for example, design, conduct, monitoring, quality, ethics, reporting, safety, etc.[3],[6],[7]

The design of an RWE study, for example, registry is based on the following considerations:[3],[7]

Research question – Clinical or public health relevanceResources – Funding, sites, clinicians, and patientsExposures – Measurable exposuresOutcomes – Objective and measurable, appropriate and valid for the research question, detailed recordingData sources – Availability and completenessStudy design – Relevant to the question and consideration of confoundersStudy population – Types of patients, need for comparison group, and selection processSampling – Process of sampling in relation to target population and designStudy size and durationInternal and external validity – Potential biases and concerns about external validity, i.e., generalizability of the results.

As pragmatic trials are conducted in real-world usual care setting, standardized collection of common minimum data set is essential to ensure high-quality data. This would require utilizing special electronic data capture or modifying EMR to gather demographic and disease-specific data.[3]

For long-term prospective studies or retrospective data collection, propensity scoring is effective at reducing bias.[3] Propensity scoring considers classification of the relationship between treatment assignment and baseline characteristics. Factors, which are different between two treatment groups and are associated with treatment preference, are weighted to estimate the probability of any participant in the cohort being assigned a specific treatment. This estimated propensity score is used to match the participants across two treatment groups.[3] For analysis of outcomes, “matchable” participants are compared, and unmatchable participants are excluded. The issues of bias and nonrandomization require following good research practice. The use of checklists, for example, STrengthening the Reporting of OBservational Studies in Epidemiology for general cohort studies, Good ReseArch for Comparative Effectiveness checklist, or PRagmatic Explanatory Continuum Indicator Summary (PRECIS) tool is recommended to ensure that RWE study is valid.

Several problems reported with product registries, for example, delayed start, low accrual rate, low use of product, and missing data need careful consideration while planning.[8]

 Pragmatic Explanatory Continuum Indicator (Precis-2) Tool

A pragmatic RWE study design is expected to meet the PRECIS-2 tool.[9]

Presic-2 tool includes 9 domains of study design, which are:

Eligibility – Similarity between the participants in the trial and those who would receive this intervention if it was part of usual care in practiceRecruitment – Extent of extra effort required to recruit participants over and above what would be needed in the usual care setting to engage with patientsSetting – Difference between the settings of the trial from the usual care settingOrganization – Difference between the resources, clinician/provider expertise, and the organization of care delivery in the intervention arm of the trial from those available in usual careFlexibility (delivery) – Differences in the flexibility of delivery of the intervention and the flexibility anticipated in usual careFlexibility (adherence) – Differences in the flexibility in how participants are monitored and encouraged to adhere to the intervention from the flexibility anticipated in usual careFollow-up – Differences in the intensity of measurement and follow-up of participants in the trial from the typical follow-up in usual carePrimary outcome – Extent of direct relevance of the trial's primary outcome to participantsPrimary analysis – Extent of inclusion of all data in the analysis of the primary outcome.

Each domain is scored using a 5-point Likert scale as follows:

Very explanatoryRather explanatoryEqually pragmatic and explanatoryRather pragmaticVery pragmatic.

The score for each domain is plotted on a PRECIS wheel, in which Score 1 is close to center and Score 5 is away from the center, and the score for each domain is connected to other domains. If the design is RWE or pragmatic, the shape connecting all the domain scores would be away from the center. If the design is explanatory or randomized, the shape would be closer to the center. PRESCIS-2 tool should be used at the stage of planning and design to ascertain whether the design fits the RWE study requirements. If not, the design can be modified to meet RWE objectives.

 Conduct of Real-World Evidence Study

Conduct of RWE study, like any clinical trial, requires review by the ethics committee, and requires informed consent of potential participants for access to identifiable medical records for the purposes of research, whether interventional or noninterventional.[5] A well-trained site team, risk-based monitoring strategy, and medical monitoring are vital in ensuring data integrity. Site monitoring should focus on consent procedures, randomization integrity, safety, and complete follow-up.[6]

 Quality of Real-World Evidence Study

Quality assurance of data, data collection/registry procedures, and computerized systems are essential to provide confidence that the design, conduct, and analysis of the registry could protect against bias (systematic error) and errors in inference, that is, erroneous conclusions drawn from a study.[7] For RWE studies – especially for registries – focus on external validity, internal validity, and analysis, and reporting is essential.

External validity depends on confirming that (1) registry participants were like the target population, (2) selection bias was minimized, and (3) completeness of information on eligible patients was evaluated and described.[6]

Internal validity requires (1) collection of necessary information for relevant key exposures, risk factors, and mitigating or protective factors; (2) collection of specific exposure data to support the main research questions; and (3) data checks for range and consistency checks. In addition, for safety registry studies, a clear and specific approach to ask about complaints or adverse events is a must.

Analysis and reporting entails (1) detailed description of methods, (2) reporting of results reported for all the main objectives, (3) use of accepted analytic techniques used, (4) description of follow-up time to enable assessment of the impact of the observation period on the conclusions, (5) consideration of impact of missing data and confounding factors, (6) clarity of conclusions drawn and implications of study results, and (7) consistency in comparison of results with other relevant researches.

 Real-World Evidence Studies in India

Although there are potential benefits of generating evidence from RW studies for India,[10],[11] there are some unique challenges in conducting such studies – (a) Indian clinical practice pattern and (b) physicians' interest in RWE studies.

Indian clinical practice pattern versus real-world evidence data

In clinical practice, diagnosis and treatment are as per physicians' decision. However, the investigations and follow-up of patients are variables based on the patients' affordability and education about his/her medical condition. Due to this, investigations for assessing effectiveness and safety endpoints are not routinely measured. Often, the clinic or hospital documentation about medical history is not in detail, and information about treatment provided is sometimes inadequate.[12] Some of the published investigator-initiated registries have discussed such limitations in data. In a review of 28 observational studies in rheumatoid arthritis (RA), Handa et al. reported that only seven described the epidemiology of RA, described comorbidities, and extra-articular manifestations; nine described the functioning abilities and quality of life among patients; and 10 provided information on treatments.[13]

In a registry of lymphomas, Nimmagadda et al. reported that survival data of patients appeared to be high possibly because a large proportion of patients were lost to follow-up.[14] In this study, 25% of patients received Physician's interest

Most Indian physicians are busy practitioners with little time or interest in clinical research. Some of them gained experience of clinical research while conducting clinical trials for pharmaceutical company products.

RWE study, for a clinician, is the use of available patients' data from his/her practice for research, without any additional effort. Hence, when a clinician participates in a structured RWE study, for example, registry, she/he is concerned about amount of data collection and documentation. Some of the studies – registry – are long term and last for several years. For an Indian physician, RWE studies appear to be postmarketing or Phase IV clinical trials, as the pharma company RWE studies require collection of effectiveness and safety data in all patients treated with a drug. However, the investigator grant is lower compared to Phase III clinical trial. The sponsor's support and oversight for RWE studies are limited compared to support and oversight for a Phase III RCT for new drug development. There is hardly any monitoring and source data verification. Hence, a physician may wonder about quality of data and relevance of results to his practice and patients. Lack of physicians' commitment is one of the main factors in low recruitment in registries.[8] The sponsors could consider employing a variety of tools – newsletters, training meetings, site audit/retraining visits, physicians' satisfaction/opinion surveys, regular data reports on registry growth to participants, presentations at conferences, and publications – to keep the physicians motivated and engaged.[7]

Myths of real-world evidence studies in India

In India, there is no demand from insurance agencies, patients' groups, or regulators to conduct RWE studies. Hence, such studies are mostly conducted by pharmaceutical companies on a newly launched expensive product. These are prospective observational studies in consenting patients who meet selection criteria and are prescribed a new product for adequate duration. The sample size is based on efficacy endpoints. The sponsor makes efforts to ensure the quality of data by regular monitoring. Some of the pharma-sponsored RWE studies collect data on treatment-related efficacy endpoints in cross-sectional survey manner. Such studies suffer from selection bias as the prescribers are in private clinic/hospital setting, whose patients can afford the cost of treatment and investigations. The results of such studies support premarket efficacy results. If one applies PRECIS-2 criteria, such studies will fall in the category of explanatory studies and not real-world pragmatic studies.

Other types of RWE are investigator-initiated observational studies or disease registries, which describe prevalence and disease characteristics, but often have limited details on patients' profile, comorbidities, treatments, natural history of disease, and long-term outcomes.

Real-world evidence: Scope in India

India has unique opportunity of conducting RWE studies in several interesting areas:

Natural history and long-term complications of communicable, for example, drug-resistant tuberculosis and noncommunicable diseases, for example, heart failureRegistry for orphan/rare disease, for example, thalassemiaHealth Economics and Outcomes Research in public versus private hospitalsIdentification of unmet medical needs relevant to Indian populationComparative effectiveness of brand versus generic and biologic versus biosimilarComparative effectiveness of Indian prescribed dose (usually lower than the Western dose) versus package insert recommended dose.

Obviously, planning and conducting such studies would require participation of diverse stakeholders – government, public and private medical institutions, patients' support groups, nongovernmental organizations, and pharmaceutical industry to support and conduct RWE studies which are relevant to Indian health care.

Financial support and sponsorship


Conflicts of interest

There are no conflicts of interest.


1Food and Drug Administration. Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices. Available from: [Last accessed on 2017 Sep 03].
2Sherman RE, Anderson SA, Dal Pan GJ, Gray GW, Gross T, Hunter NL, et al. Real-world evidence – What is it and what can it tell us? N Engl J Med 2016;375:2293-7.
3Ziemssen T, Hillert J, Butzkueven H. The importance of collecting structured clinical information on multiple sclerosis. BMC Med 2016;14:81.
4Association of the British Pharmaceutical Industry. Demonstrating Value with Real World Data: A Practical Guide. Association of the British Pharmaceutical Industry; 2011. Available from: [Last accessed on 2016 Nov 16].
5Lamberti MJ, Kubick W, Awatin J, McCormick J, Carroll J, Getz K, et al. The use of real-world evidence and data in clinical research and postapproval safety studies. Ther Innov Regul Sci 2018;52:778-83.
6Mentz RJ, Hernandez AF, Berdan LG, Rorick T, O'Brien EC, Ibarra JC, et al. Good clinical practice guidance and pragmatic clinical trials: Balancing the best of both worlds. Circulation 2016;133:872-80.
7Agency for Healthcare Research and Quality Registries for Evaluating Patient Outcomes: A User's Guide. 3rd ed. Agency for Healthcare Research; 2014. Available from: [Last accessed on 2017 Jan 11].
8Alison C. What are the Real-World Evidence Tools and How Can They Support Decision Making? Europeans Medicine Agency; 22 November, 2016. Available from: [Last accessed on 2017 Apr 13].
9Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M, et al. The PRECIS-2 tool: Designing trials that are fit for purpose. BMJ 2015;350:h2147.
10Dang A, Vallish BN. Real world evidence: An Indian perspective. Perspect Clin Res 2016;7:156-60.
11Suvarna VR. Real world evidence (RWE) – Are we (RWE) ready? Perspect Clin Res 2018;9:61-3.
12Rajivlochan M. Clinical audits and the state of record keeping in India. Ann Neurosci 2015;22:197-8.
13Handa R, Rao UR, Lewis JF, Rambhad G, Shiff S, Ghia CJ, et al. Literature review of rheumatoid arthritis in India. Int J Rheum Dis 2016;19:440-51.
14Nimmagadda RB, Digumarti R, Nair R, Bhurani D, Raina V, Aggarwal S, et al. Histopathological pattern of lymphomas and clinical presentation and outcomes of diffuse large B cell lymphoma: A multicenter registry based study from India. Indian J Med Paediatr Oncol 2013;34:299-304.