Media

+

Doctor

Canada

rating information

News stories will be identified by daily reviews of the major media outlet websites using a hand-searching approach.

In order to be eligible for review, the article should:

Be relevant to the management and prevention of disease in the source country, in particular should relate to claims about new treatments, procedures or diagnostic tests
Discuss an intervention that is intended for use on humans
Make explicit or implicit claims of efficacy, lack of efficacy, safety or lack of safety
Be published in the mainstream media of the source country

Reviewers are also asked to provide comments on the stories which may include the use of sensationalist words.

rating instrument

Category: Access Stories

Rating Criteria	Satisfactory	Not Satisfactory
Access to Treatment	Accurately describes the access dilemma. Accurately describes why a treatment is unavailable (i.e: if a drug is not yet approved by Health Canada, not approved by the Common Drug Review, not paid for by a province, and so on)	Is inaccurate in describing the 'access' issue or otherwise telling the reader why a treatment is deemed 'unavailable' to patients.
Evidence	Accurately describes the research/clinical evidence; interprets evidence properly; mentions its strength, whether it has been peer-reviewed or not. Accurate in describing the outcomes	Inaccurately describes the research/clinical evidence; fails to interpret evidence properly or mention its strength. Fails to describe if it has been peer-reviewed or not. Inaccurate in describing the outcomes
Benefits of Treatment	Estimate in both absolute and relative frames, or absolute frame only, or rates with and without treatment or against standard treatment.	No quantitative estimate of benefit Quantitative estimate in relative frame only
Harms of Treatment	Supplies quantitative estimate of harms or side effects or adverse effects.	No quantitative estimate of harms or side effects or adverse effects.
Cost of Treatment	Accurately describes the cost (to the patient or the health care system) of the treatment or compares it to standard treatment. Questions or explores the value of the treatment	Is inaccurate in describing the costs; doesn't question or explore the value of the treatment.
Balance of societal versus individual perspectives	Creates a balanced picture between the individual versus the societal perspective. The article balances a single individual's or a couple of individuals' need for the drug with the reasons why the drug has not been funded (financial consequences on the health care system and/or lack of data on safety/effectiveness).	Judged to be 'imbalanced' if it largely ignores implications for the broader population. Unsatisfactory would be a story where the primary focus is on a single individual's or a couple of individuals' need (or perceived need) for the drug - without balancing concern or even touching on the broader social/policy issues

Category: Diagnostic Test

Rating Criteria	Satisfactory	Not Satisfactory
Availability of Test
Novelty of Test
Diagnostic Options
Disease Mongering
Evidence
Quantification of Diagnostic Accuracy/Benefits
Costs of Testing
Harms of Testing
Sources of Information
Relies on Press Release

Category: Harm Stories

Rating Criteria	Satisfactory	Not Satisfactory
Novelty of harm	Mentions whether or not harm was previously identified or mentions what is added to previous knowledge about the harm.	No mention of whether or not harm has been previously been recognized
Benefit to harm ratio	Tries to balance reporting of both benefits and harms or gives some sense of the ratio between the two.	No mention of whether therapy has more benefits or more harms
Evidence	Where relevant, there is mention of strength of evidence and correct interpretation	No mention of the nature of clinical research that lead to recognition of harm or increased frequency of harm
Quantification of Harms	Even some quantification of harm rates an 'S'.	No quantification of harm or describes it using words such as "minor" or "not serious"
Number of people affected by harm	Even some quantification of number or percent of people affected by the harm is satisfactory	No mention of numbers or percent of people taking treatment expected to be harmed
Stratification of patients with regard to harm	Mentions which groups of patients are most likely to be harmed	No mention of which groups of patients are most likely to suffer harm
Sources of Information	Provides detail on information sources and their potential COI, and reports independent source, or mentions unsuccessful attempt to obtain corroboration	No mention of sources or possible conflicts of interest No attempt at independent corroboration
Treatment options	Mentions alternatives and discusses whether alternatives are more or less harmful.	No mention of alternatives or whether the alternatives are more or less harmful.
Relies on Press Release	No obvious use of text from the press release	Evidence from press release or other news stories that the journalist has relied on a press release as the only information source and used the text in the story

Category: Health Scares

Rating Criteria	Satisfactory	Not Satisfactory
Probable Incidence/Prevalence
Vested Interest in Causing Scare
Independent Comment

Category: Other

Rating Criteria	Satisfactory	Not Satisfactory
Availability of Treatment	Accurate information on availability of treatment in Canada (its approval status and its coverage status in at least one Canadian province)	Does not mention availability of treatment in Canada.
Novelty of Treatment	Accurate information on novelty (or lack of)	Does not mention (or inaccurately represents) if treatment is genuinely new or just a re-formulation of an existing treatment, or another member of a well established drug class
Disease Mongering	No obvious elements of disease-mongering	Frames risk factors (eg BMD cholesterol) as a disease, or No mention (or misrepresentation) of natural history of disease, or Exaggerates prevalence or incidence, or Medicalisation of normal human variation
Treatment Options	Mentions appropriate alternatives and provides comparative information	No mention of alternatives or their comparative performance
Costs of Treatment	Mentions comparative costs and comments on cost-effectiveness	No mention of costs, or downplays cost as an issue Mentions cost only, no comparative information
Evidence	Where relevant, there is mention of strength of evidence and correct interpretation	No mention of the nature of clinical evidence, esp RCTs Mention of the nature of the evidence but interpretation or discussion is inappropriate
Quantification of Benefits of Treatment	Estimate in both absolute and relative frames, or absolute frame only, or rates with and without treatment	No quantitative estimate of benefit Quantitative estimate in relative frame only
Harms of Treatment	Balanced information about harms (frequency or seriousness)	No mention of harms, or discounts potential harms
Sources of Information	Provides detail on information sources and their potential COI, and reports independent source, or mentions unsuccessful attempt to obtain corroboration	No mention of sources or possible conflicts of interest No attempt at independent corroboration
Relies on Press Release	No obvious use of text from the press release	Evidence from press release or other news stories that the journalist has relied on a press release as the only information source and used the text in the story
Quantification of harms of treatment	Even some quantification of harms merits a satisfactory rating.	No quantification of harms is not satisfactory.

Category: Pharmaceutical

Rating Criteria	Satisfactory	Not Satisfactory
Availability of Treatment	Accurate information on availability of treatment in Canada (its approval status and its coverage status in at least one Canadian province)	Does not mention availability of treatment in Canada.
Novelty of Treatment	Accurate information on novelty (or lack of)	Does not mention (or inaccurately represents) if treatment is genuinely new or just a re-formulation of an existing treatment, or another member of a well established drug class
Disease Mongering	No obvious elements of disease-mongering	Frames risk factors (eg BMD cholesterol) as a disease, or No mention (or misrepresentation) of natural history of disease, or Exaggerates prevalence or incidence, or Medicalisation of normal human variation
Treatment Options	Mentions appropriate alternatives and provides comparative information	No mention of alternatives or their comparative performance
Costs of Treatment	Mentions comparative costs and comments on cost-effectiveness	No mention of costs, or downplays cost as an issue Mentions cost only, no comparative information
Evidence	Where relevant, there is mention of strength of evidence and correct interpretation	No mention of the nature of clinical evidence, esp RCTs Mention of the nature of the evidence but interpretation or discussion is inappropriate
Quantification of Benefits of Treatment	Estimate in both absolute and relative frames, or absolute frame only, or rates with and without treatment	No quantitative estimate of benefit Quantitative estimate in relative frame only
Harms of Treatment	Balanced information about harms (frequency or seriousness)	No mention of harms, or discounts potential harms
Sources of Information	Provides detail on information sources and their potential COI, and reports independent source, or mentions unsuccessful attempt to obtain corroboration	No mention of sources or possible conflicts of interest No attempt at independent corroboration
Relies on Press Release	No obvious use of text from the press release	Evidence from press release or other news stories that the journalist has relied on a press release as the only information source and used the text in the story
Quantification of Harms of Treatment	Even some quantification of harms merits a satisfactory rating	No quantification of harms is unsatisfactory.

Category: Prevalence stories

Rating Criteria	Satisfactory	Not Satisfactory
Health consequences	Satisfactory explanation of the health consequences of the "problem," supported by sufficient evidence	Not satisfactory explanation of the health consequences of the "problem" or fails to provide any evidence.
Preventalbe or treatable?	Satisfactory mention of the problem in terms of it being preventable, or treatable? (i.e. why mention it if it is not, it will just lead to unnecessary anxiety and doctor visits.)	Doesn't mention if it is preventable or treatable if it might be.
Availability of preventative measures or treatment	Mentions the available preventative measures or treatments or describes what practices could/should be implemented at a personal, family, school, government, health care provider, etc. level.	No mention of available preventative measures.
Practices encouraging the problem?	Mentions those practices which encourage the problem and how these could be modified.	No mention of practices which encourage the problem or how they could be modified
Harms of intervention	Satisfactory mention of the possible harms of intervention	Not satisfactory in mentioning the possible harms of intervention.
Reliance on press release	Does the article go beyond the press release?	The article does not go beyond the press release.

Category: Surgical Procedure

Rating Criteria	Satisfactory	Not Satisfactory
Novelty of Treatment	Accurate information on novelty (or lack of)	Does not mention (or inaccurately represents) if treatment is genuinely new or just a re-formulation of an existing treatment, or another member of a well established drug class
Availability of Treatment	Accurate information on availability of treatment in source country (both registration and PBS status)	Does not mention availability of treatment in source country
Treatment Options	Mentions appropriate alternatives and provides comparative information	No mention of alternatives or their comparative performance
Disease Mongering	No obvious elements of disease-mongering	Frames risk factors (eg BMD cholesterol) as a disease, or No mention (or misrepresentation) of natural history of disease, or Exaggerates prevalence or incidence, or Medicalisation of normal human variation
Evidence	Where relevant, there is mention of strength of evidence and correct interpretation	No mention of the nature of clinical evidence, esp RCTs Mention of the nature of the evidence but interpretation or discussion is inappropriate
Quantification of Benefits of Treatment	Estimate in both absolute and relative frames, or absolute frame only, or rates with and without treatment	No quantitative estimate of benefit Quantitative estimate in relative frame only
Harms of Treatment	Balanced information about harms (frequency or seriousness)	No mention of harms, or discounts potential harms
Costs of Treatment	Mentions comparative costs and comments on cost-effectiveness	No mention of costs, or downplays cost as an issue Mentions cost only, no comparative information
Sources of Information	Provides detail on information sources and their potential COI, and reports independent source, or mentions unsuccessful attempt to obtain corroboration	No mention of sources or possible conflicts of interest No attempt at independent corroboration
Relies on Press Release	No obvious use of text from the press release	Evidence from press release or other news stories that the journalist has relied on a press release as the only information source and used the text in the story
Quantification of Harms of Treatment	Even some quantification of harm rates an 'S'.	No quantification of harms ' NS'