Home  |  About us  |  Editorial board  |  Ahead of print  | Current issue  |  Archives  |  Submit article  |  Instructions |  Search  |   Subscribe  |  Advertise  |  Contacts  |  Login 
  Users Online: 643Home Print this page Email this page Small font sizeDefault font sizeIncrease font size  

 Table of Contents      
Year : 2019  |  Volume : 10  |  Issue : 2  |  Page : 51-56

Conducting real-world evidence studies in India

Consultant – Clinical Research and Development, Mumbai, Maharashtra, India

Date of Web Publication4-Apr-2019

Correspondence Address:
Dr. Arun Bhatt
303-4, Dheeraj Valley 3/C, Mohan Gokhale Road, Goregaon (East), Mumbai - 400 063, Maharashtra
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/picr.PICR_8_19

Rights and Permissions

Real-world data (RWD) are data relating to patients' health status and/or the delivery of health care routinely collected from a variety of sources. Real-world evidence (RWE) is the clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of RWD. RWE can provide information on diverse areas, for example, natural history and course of disease, effectiveness studies, outcome research, and safety surveillance. India has unique opportunity of conducting RWE studies in several interesting areas, for example, natural history of communicable and noncommunicable diseases and rare disease and Health Economics and Outcomes Research in public versus private hospitals. However, utility of RWE is challenged by diversity of information, large data sets of uncertain quality, and methodological rigor. For India, there are additional challenges in conducting RWE studies – Indian clinical practice pattern and physicians' interest in RWE studies. It is recommended that RWE studies in India should focus on health management issues of relevance to the country India and should be well planned to generate high-quality data.

Keywords: Quality, real-world evidence, registry

How to cite this article:
Bhatt A. Conducting real-world evidence studies in India. Perspect Clin Res 2019;10:51-6

How to cite this URL:
Bhatt A. Conducting real-world evidence studies in India. Perspect Clin Res [serial online] 2019 [cited 2022 Aug 15];10:51-6. Available from: http://www.picronline.org/text.asp?2019/10/2/51/255437

   Introduction Top

Real-world data (RWD) are data relating to patients' health status and/or the delivery of health care routinely collected from a variety of sources.[1] Real-world evidence (RWE) is the clinical evidence regarding the usage and potential benefits or risks of a medical product derived from the analysis of RWD.[1] RWD includes variety of data – electronic health records (EHRs), insurance data, product and disease registries, etc. RWD has potential to complement the knowledge available from conventional randomized clinical trials (RCTs), whose design limitations make it difficult to generalize findings to population at large.[2] RWE can provide information on diverse areas, for example, natural history and course of disease, effectiveness studies, outcome research, and safety surveillance. However, utility of RWE is challenged by diversity of information, large data sets of uncertain quality, and methodological rigor. These issues are relevant to conduct of RWE studies in India.

   Real-World Evidence Study Versus Randomized Clinical Trials Top

RCTs are designed to provide robust evidence of premarket evaluation of efficacy and safety of an investigational product. RCT design focuses on attaining internal validity, which may sometimes compromise generalizability.[2] In contrast, RWE studies are planned to provide data with relevance to clinical practice. Differences between RWE study and RCT [3] are summarized in [Table 1].
Table 1: Differences between randomized clinical trial and real-world evidence studies

Click here to view

   Real-World Evidence Studies: Potential Benefits Top

RWE studies are valuable across product lifecycle and can provide multiple benefits.[2],[3],[4]

  • Prior to product development

    • Natural history of disease – prevalence, incidence, and unmet medical need
    • Current treatment patterns and standard of care
    • Understanding current service structures, especially if they may need to change to deliver the new medicine
    • Generate hypotheses for prospective trials.

  • At or following market authorization

    • Insights into early and/or long-term clinical experience
    • Prospective studies for safety surveillance
    • Evidence to support health outcomes, reduction in resource use, and patients' acceptability
    • Detection of untreated/undiagnosed patients (“unmet need”).

  • After product launch

    • Comparative effectiveness and health outcomes
    • Compliance with product label or treatment guidelines
    • Identification of suboptimal dosing or treatment
    • Support for expansion of indications
    • Characterizing appropriate treatment subgroups
    • Monitoring safety, especially uncommon/rare adverse reactions.

Despite many potential benefits, RWE studies are primarily conducted to meet demands or expectations of reimbursement or insurance agencies and patients and to satisfy drug regulatory agencies.[3] The payers and health-care decision-makers have been increasingly demanding postapproval evidence to support reviews of pricing, reimbursement, licensing for new therapies, and formulation and indication changes.[3] However, according to a recent survey of 30 companies, the objectives of most RWE studies were postapproval safety and commercialization strategies.[5]

   Real-World Evidence Studies: Designs Top

RWE studies employ retrospective and prospective designs.[3],[4]

Retrospective design studies have predictable short-term data collection period, are dependent on the quality of available clinical records, and are usually low-cost studies. However, such studies have limitations of incomplete or inaccurate data. Examples of such studies include:

  • Clinical/hospital record/chart review
  • Administrative data/insurance claim data
  • Electronic medical records (EMRs).

Prospective design studies can include robust patient data set and clinical data not routinely captured, and provide information on long-term data on natural history or disease progression. However, such studies have uncertain timelines, which are dependent on the rate of presentation of eligible patients and are likely to be costly. Examples of such studies include:

  • Registries
  • Health surveys
  • Prospective observational studies
  • Postauthorization safety study.

   Real-World Evidence Studies: Challenges Top

There are several challenges in interpretation and in utility of RWE studies.

  • Validity – Open, unblinded, and nonrandomized designs used in RWE studies raise questions about their validity [2],[3]
  • Quality of data – Most of the sources for RWE studies are data collected to support clinical care and reimbursement. These data are not gathered to support any specific research study. Hence, there are gaps in the accuracy, completeness, and quality of RW data.[2],[3],[6] This is likely to be a big challenge in India, especially in the case of RWE studies based on retrospective hospital databases
  • Confounding – A small observed effect in an RWE study could be due to confounding factors.[2]

These challenges point to the need for proper planning and conduct of RWE studies.

   Planning of Real-World Evidence Study Top

Several factors should be considered in planning of RW study, for example, design, conduct, monitoring, quality, ethics, reporting, safety, etc.[3],[6],[7]

The design of an RWE study, for example, registry is based on the following considerations:[3],[7]

  • Research question – Clinical or public health relevance
  • Resources – Funding, sites, clinicians, and patients
  • Exposures – Measurable exposures
  • Outcomes – Objective and measurable, appropriate and valid for the research question, detailed recording
  • Data sources – Availability and completeness
  • Study design – Relevant to the question and consideration of confounders
  • Study population – Types of patients, need for comparison group, and selection process
  • Sampling – Process of sampling in relation to target population and design
  • Study size and duration
  • Internal and external validity – Potential biases and concerns about external validity, i.e., generalizability of the results.

As pragmatic trials are conducted in real-world usual care setting, standardized collection of common minimum data set is essential to ensure high-quality data. This would require utilizing special electronic data capture or modifying EMR to gather demographic and disease-specific data.[3]

For long-term prospective studies or retrospective data collection, propensity scoring is effective at reducing bias.[3] Propensity scoring considers classification of the relationship between treatment assignment and baseline characteristics. Factors, which are different between two treatment groups and are associated with treatment preference, are weighted to estimate the probability of any participant in the cohort being assigned a specific treatment. This estimated propensity score is used to match the participants across two treatment groups.[3] For analysis of outcomes, “matchable” participants are compared, and unmatchable participants are excluded. The issues of bias and nonrandomization require following good research practice. The use of checklists, for example, STrengthening the Reporting of OBservational Studies in Epidemiology for general cohort studies, Good ReseArch for Comparative Effectiveness checklist, or PRagmatic Explanatory Continuum Indicator Summary (PRECIS) tool is recommended to ensure that RWE study is valid.

Several problems reported with product registries, for example, delayed start, low accrual rate, low use of product, and missing data need careful consideration while planning.[8]

   Pragmatic Explanatory Continuum Indicator (Precis-2) Tool Top

A pragmatic RWE study design is expected to meet the PRECIS-2 tool.[9]

Presic-2 tool includes 9 domains of study design, which are:

  1. Eligibility – Similarity between the participants in the trial and those who would receive this intervention if it was part of usual care in practice
  2. Recruitment – Extent of extra effort required to recruit participants over and above what would be needed in the usual care setting to engage with patients
  3. Setting – Difference between the settings of the trial from the usual care setting
  4. Organization – Difference between the resources, clinician/provider expertise, and the organization of care delivery in the intervention arm of the trial from those available in usual care
  5. Flexibility (delivery) – Differences in the flexibility of delivery of the intervention and the flexibility anticipated in usual care
  6. Flexibility (adherence) – Differences in the flexibility in how participants are monitored and encouraged to adhere to the intervention from the flexibility anticipated in usual care
  7. Follow-up – Differences in the intensity of measurement and follow-up of participants in the trial from the typical follow-up in usual care
  8. Primary outcome – Extent of direct relevance of the trial's primary outcome to participants
  9. Primary analysis – Extent of inclusion of all data in the analysis of the primary outcome.

Each domain is scored using a 5-point Likert scale as follows:

  1. Very explanatory
  2. Rather explanatory
  3. Equally pragmatic and explanatory
  4. Rather pragmatic
  5. Very pragmatic.

The score for each domain is plotted on a PRECIS wheel, in which Score 1 is close to center and Score 5 is away from the center, and the score for each domain is connected to other domains. If the design is RWE or pragmatic, the shape connecting all the domain scores would be away from the center. If the design is explanatory or randomized, the shape would be closer to the center. PRESCIS-2 tool should be used at the stage of planning and design to ascertain whether the design fits the RWE study requirements. If not, the design can be modified to meet RWE objectives.

   Conduct of Real-World Evidence Study Top

Conduct of RWE study, like any clinical trial, requires review by the ethics committee, and requires informed consent of potential participants for access to identifiable medical records for the purposes of research, whether interventional or noninterventional.[5] A well-trained site team, risk-based monitoring strategy, and medical monitoring are vital in ensuring data integrity. Site monitoring should focus on consent procedures, randomization integrity, safety, and complete follow-up.[6]

   Quality of Real-World Evidence Study Top

Quality assurance of data, data collection/registry procedures, and computerized systems are essential to provide confidence that the design, conduct, and analysis of the registry could protect against bias (systematic error) and errors in inference, that is, erroneous conclusions drawn from a study.[7] For RWE studies – especially for registries – focus on external validity, internal validity, and analysis, and reporting is essential.

External validity depends on confirming that (1) registry participants were like the target population, (2) selection bias was minimized, and (3) completeness of information on eligible patients was evaluated and described.[6]

Internal validity requires (1) collection of necessary information for relevant key exposures, risk factors, and mitigating or protective factors; (2) collection of specific exposure data to support the main research questions; and (3) data checks for range and consistency checks. In addition, for safety registry studies, a clear and specific approach to ask about complaints or adverse events is a must.

Analysis and reporting entails (1) detailed description of methods, (2) reporting of results reported for all the main objectives, (3) use of accepted analytic techniques used, (4) description of follow-up time to enable assessment of the impact of the observation period on the conclusions, (5) consideration of impact of missing data and confounding factors, (6) clarity of conclusions drawn and implications of study results, and (7) consistency in comparison of results with other relevant researches.

   Real-World Evidence Studies in India Top

Although there are potential benefits of generating evidence from RW studies for India,[10],[11] there are some unique challenges in conducting such studies – (a) Indian clinical practice pattern and (b) physicians' interest in RWE studies.

Indian clinical practice pattern versus real-world evidence data

In clinical practice, diagnosis and treatment are as per physicians' decision. However, the investigations and follow-up of patients are variables based on the patients' affordability and education about his/her medical condition. Due to this, investigations for assessing effectiveness and safety endpoints are not routinely measured. Often, the clinic or hospital documentation about medical history is not in detail, and information about treatment provided is sometimes inadequate.[12] Some of the published investigator-initiated registries have discussed such limitations in data. In a review of 28 observational studies in rheumatoid arthritis (RA), Handa et al. reported that only seven described the epidemiology of RA, described comorbidities, and extra-articular manifestations; nine described the functioning abilities and quality of life among patients; and 10 provided information on treatments.[13]

In a registry of lymphomas, Nimmagadda et al. reported that survival data of patients appeared to be high possibly because a large proportion of patients were lost to follow-up.[14] In this study, 25% of patients received <6 cycles of standard chemotherapy and 42% received a monoclonal antibody. The authors cite noncompliance, financial factors, and logistic constraints as some of the reasons for inferior outcomes of treatment.

RWE studies of a new product require good documentation of frequency and adequacy of patients' follow-ups, measurable exposures, and outcomes. In a clinical trial – Phase III – the sponsor pays for all expenses and provides investigational and standard therapies. However, in RWE studies – registries, the patients must purchase new expensive drugs and pay for the investigations to measure effectiveness and safety and follow-up visits. This means that conclusions from Indian registries cannot be generalized to all the patients as registries will include the patients who are from higher socioeconomic class and who can afford to pay for the investigations and therapy.

For registries of generic products or biosimilars, there is an additional factor of widespread availability of low-cost generic options, and clinicians' and patients' preference for switching brands to cheaper biosimilar/generic options.

Physician's interest

Most Indian physicians are busy practitioners with little time or interest in clinical research. Some of them gained experience of clinical research while conducting clinical trials for pharmaceutical company products.

RWE study, for a clinician, is the use of available patients' data from his/her practice for research, without any additional effort. Hence, when a clinician participates in a structured RWE study, for example, registry, she/he is concerned about amount of data collection and documentation. Some of the studies – registry – are long term and last for several years. For an Indian physician, RWE studies appear to be postmarketing or Phase IV clinical trials, as the pharma company RWE studies require collection of effectiveness and safety data in all patients treated with a drug. However, the investigator grant is lower compared to Phase III clinical trial. The sponsor's support and oversight for RWE studies are limited compared to support and oversight for a Phase III RCT for new drug development. There is hardly any monitoring and source data verification. Hence, a physician may wonder about quality of data and relevance of results to his practice and patients. Lack of physicians' commitment is one of the main factors in low recruitment in registries.[8] The sponsors could consider employing a variety of tools – newsletters, training meetings, site audit/retraining visits, physicians' satisfaction/opinion surveys, regular data reports on registry growth to participants, presentations at conferences, and publications – to keep the physicians motivated and engaged.[7]

Myths of real-world evidence studies in India

In India, there is no demand from insurance agencies, patients' groups, or regulators to conduct RWE studies. Hence, such studies are mostly conducted by pharmaceutical companies on a newly launched expensive product. These are prospective observational studies in consenting patients who meet selection criteria and are prescribed a new product for adequate duration. The sample size is based on efficacy endpoints. The sponsor makes efforts to ensure the quality of data by regular monitoring. Some of the pharma-sponsored RWE studies collect data on treatment-related efficacy endpoints in cross-sectional survey manner. Such studies suffer from selection bias as the prescribers are in private clinic/hospital setting, whose patients can afford the cost of treatment and investigations. The results of such studies support premarket efficacy results. If one applies PRECIS-2 criteria, such studies will fall in the category of explanatory studies and not real-world pragmatic studies.

Other types of RWE are investigator-initiated observational studies or disease registries, which describe prevalence and disease characteristics, but often have limited details on patients' profile, comorbidities, treatments, natural history of disease, and long-term outcomes.

Real-world evidence: Scope in India

India has unique opportunity of conducting RWE studies in several interesting areas:

  • Natural history and long-term complications of communicable, for example, drug-resistant tuberculosis and noncommunicable diseases, for example, heart failure
  • Registry for orphan/rare disease, for example, thalassemia
  • Health Economics and Outcomes Research in public versus private hospitals
  • Identification of unmet medical needs relevant to Indian population
  • Comparative effectiveness of brand versus generic and biologic versus biosimilar
  • Comparative effectiveness of Indian prescribed dose (usually lower than the Western dose) versus package insert recommended dose.

Obviously, planning and conducting such studies would require participation of diverse stakeholders – government, public and private medical institutions, patients' support groups, nongovernmental organizations, and pharmaceutical industry to support and conduct RWE studies which are relevant to Indian health care.

Financial support and sponsorship


Conflicts of interest

There are no conflicts of interest.

   References Top

Food and Drug Administration. Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices. Available from: https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm513027.pdf. [Last accessed on 2017 Sep 03].  Back to cited text no. 1
Sherman RE, Anderson SA, Dal Pan GJ, Gray GW, Gross T, Hunter NL, et al. Real-world evidence – What is it and what can it tell us? N Engl J Med 2016;375:2293-7.  Back to cited text no. 2
Ziemssen T, Hillert J, Butzkueven H. The importance of collecting structured clinical information on multiple sclerosis. BMC Med 2016;14:81.  Back to cited text no. 3
Association of the British Pharmaceutical Industry. Demonstrating Value with Real World Data: A Practical Guide. Association of the British Pharmaceutical Industry; 2011. Available from: http://www.abpi.org.uk/our-work/library/guidelines/Pages/real-world-data.aspx. [Last accessed on 2016 Nov 16].  Back to cited text no. 4
Lamberti MJ, Kubick W, Awatin J, McCormick J, Carroll J, Getz K, et al. The use of real-world evidence and data in clinical research and postapproval safety studies. Ther Innov Regul Sci 2018;52:778-83.  Back to cited text no. 5
Mentz RJ, Hernandez AF, Berdan LG, Rorick T, O'Brien EC, Ibarra JC, et al. Good clinical practice guidance and pragmatic clinical trials: Balancing the best of both worlds. Circulation 2016;133:872-80.  Back to cited text no. 6
Agency for Healthcare Research and Quality Registries for Evaluating Patient Outcomes: A User's Guide. 3rd ed. Agency for Healthcare Research; 2014. Available from: https://effectivehealthcare.ahrq.gov/topics/registries-guide-3rd-edition/research. [Last accessed on 2017 Jan 11].  Back to cited text no. 7
Alison C. What are the Real-World Evidence Tools and How Can They Support Decision Making? Europeans Medicine Agency; 22 November, 2016. Available from: https://www.ema.europa.eu/documents/presentation. [Last accessed on 2017 Apr 13].  Back to cited text no. 8
Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M, et al. The PRECIS-2 tool: Designing trials that are fit for purpose. BMJ 2015;350:h2147.  Back to cited text no. 9
Dang A, Vallish BN. Real world evidence: An Indian perspective. Perspect Clin Res 2016;7:156-60.  Back to cited text no. 10
[PUBMED]  [Full text]  
Suvarna VR. Real world evidence (RWE) – Are we (RWE) ready? Perspect Clin Res 2018;9:61-3.  Back to cited text no. 11
[PUBMED]  [Full text]  
Rajivlochan M. Clinical audits and the state of record keeping in India. Ann Neurosci 2015;22:197-8.  Back to cited text no. 12
Handa R, Rao UR, Lewis JF, Rambhad G, Shiff S, Ghia CJ, et al. Literature review of rheumatoid arthritis in India. Int J Rheum Dis 2016;19:440-51.  Back to cited text no. 13
Nimmagadda RB, Digumarti R, Nair R, Bhurani D, Raina V, Aggarwal S, et al. Histopathological pattern of lymphomas and clinical presentation and outcomes of diffuse large B cell lymphoma: A multicenter registry based study from India. Indian J Med Paediatr Oncol 2013;34:299-304.  Back to cited text no. 14
[PUBMED]  [Full text]  


  [Table 1]

This article has been cited by
1 Early Outcomes of Surgical Stabilisation of Traumatic Rib Fractures: Single-Center Review With a Real-World Evidence Perspective
Negin Sedaghat,Corinna Chiong,Richard Tjahjono,Jeremy Hsu
Journal of Surgical Research. 2021; 264: 222
[Pubmed] | [DOI]
2 Do weekly surveillance cultures contribute to antibiotic stewardship and correlate with outcome of HSCT in children a multicentre real-world experience of 5 years from Indian subcontinent?
Rakesh Dhanya, Rajat Kumar Agarwal, Dr Stalin Ramprakash, Dr Deepa Trivedi, Dr Vaibhav Shah, Dr Neema Bhat, Dr Mohan Reddy, Sandeep Elizabeth, Dr Aliya Batool, Dr Sadaf Khalid, Dr Lawrence Faulkner
Transplantation and Cellular Therapy. 2021;
[Pubmed] | [DOI]
3 Introduction to real-world evidence studies
Deepa Chodankar
Perspectives in Clinical Research. 2021; 12(3): 171
[Pubmed] | [DOI]
4 Real-world data in COVID-19 pandemic: An essential unmet health-care need
Arun Bhatt
Perspectives in Clinical Research. 2020; 11(3): 103
[Pubmed] | [DOI]


    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

  In this article
    Real-World Evide...
    Real-World Evide...
    Real-World Evide...
    Real-World Evide...
    Planning of Real...
    Pragmatic Explan...
    Conduct of Real-...
    Quality of Real-...
    Real-World Evide...
    Article Tables

 Article Access Statistics
    PDF Downloaded663    
    Comments [Add]    
    Cited by others 4    

Recommend this journal