Quiz: Enhancing Scientific Self-Correction — 18 questions

1. What does the myth of self-correcting science claim about false or flawed findings?

A. They are harmless unless fraud is involved

B. They are automatically corrected by the research system over time

C. They disappear once enough replications are published

D. They are corrected only when journals issue immediate retractions

Answer: (B)

Explanation

The myth is the belief that scientific progress will fix false findings on its own. The material emphasizes that correction is not automatic and requires deliberate action.

2. In the reported loss-of-confidence survey, what proportion of respondents had lost confidence in at least one finding?

A. 28%

B. 56%

C. 74%

D. 44%

Answer: (D)

Explanation

The survey found that 44% of respondents lost confidence in at least one finding. The other percentages refer to reasons for the loss or responsibility for errors.

3. What is a retraction in scientific publishing?

A. A journal notice that withdraws a published record because its validity is affected

B. A private communication between authors and reviewers

C. A minor correction that leaves the article fully citable

D. A method for increasing citation counts after revision

Answer: (A)

Explanation

A retraction withdraws a published record when problems affect its validity. It is more serious than a simple correction.

4. Which outcome can occur when authors self-initiate corrections after an honest error?

A. Their reputation always declines permanently

B. The error is treated as misconduct

C. Their paper becomes immune to further scrutiny

D. Their reputation may remain neutral or improve

Answer: (D)

Explanation

The material notes that self-initiated corrections after honest errors can have neutral or even improved reputation outcomes. Honest error is not the same as misconduct.

5. Which example best fits a careless error?

A. A typo or copy/paste mistake in data entry

B. A theory built on weak training in study design

C. A flawed choice of measurement scale

D. A decision to use a weak sample size

Answer: (A)

Explanation

Careless errors come from inattention or oversight, such as typos, copy/paste mistakes, and coding errors. The other options describe negligent design or analysis problems.

6. What type of error is best illustrated by poor design or weak analysis choices due to inadequate training?

A. Randomization error

B. Retraction error

C. Negligent error

D. Careless error

Answer: (C)

Explanation

Negligent errors are linked to incompetence or poor training and include weak design and analysis choices. Careless errors are slips, not competence problems.

7. What is the main purpose of error detection techniques in scientific reporting?

A. To replace peer review entirely

B. To identify inconsistencies, anomalies, or signs of mistakes

C. To prove that every published result is false

D. To increase the statistical power of a study

Answer: (B)

Explanation

Error detection methods are used to spot inconsistencies, anomalies, and possible mistakes in reports and results. They do not by themselves prove an entire study is wrong.

8. Why can Statcheck miss some problems in statistical reporting?

A. It requires ingredients for recalculation that may be missing or incompatible

B. It only works on qualitative interviews

C. It needs access to raw data in every case

D. It can only detect fraud, not honest mistakes

Answer: (A)

Explanation

Statcheck can miss issues when results are reported in incompatible ways or when necessary ingredients for recalculation are missing. It works from reported summary information, not from every raw dataset.

9. What kind of mismatch does Statcheck primarily flag?

A. A difference between reported statistics, degrees of freedom, and p-values that do not recalculate consistently

B. A difference between raw data and interview notes

C. A disagreement between authors about interpretation

D. A mismatch between sample size and journal impact factor

Answer: (A)

Explanation

Statcheck detects reporting inconsistencies when the statistics, degrees of freedom, and p-values do not jointly make sense. It is a math-consistency checker, not an interpretation tool.

10. What is optimistic rounding in statistical reporting?

A. Rounding all values to the nearest whole number for simplicity

B. Using exact p-values only when they are non-significant

C. Reporting borderline p-values in a way that shifts them toward significance

D. Changing sample sizes after analysis

Answer: (C)

Explanation

Optimistic rounding can turn borderline values such as p ≥ 0.055 into a reported p < 0.05. This can make a result appear more significant than it really is.

11. What does the GRIM test primarily check in reported research results?

A. Whether the experimenters were unaware of group assignments

B. Whether the reported p-value was rounded to three decimal places

C. Whether the random assignment produced equal group sizes

D. Whether a reported mean is compatible with the data’s discrete granularity

Answer: (D)

Explanation

GRIM checks whether a reported mean could actually arise from the underlying discrete response values. It is a granularity plausibility check, not a randomization or blinding check.

12. In a randomized trial, what is the purpose of a randomisation check?

A. To recalculate p-values from the reported test statistics

B. To confirm that the analysts could not see the group labels

C. To verify that reported means match the possible response granularity

D. To test whether baseline distributions look plausible under random assignment

Answer: (D)

Explanation

A randomisation check asks whether the observed baseline values are plausible if participants were assigned at random. It does not examine granularity or statistical-reporting consistency.

13. Which practice best reflects a questionable measurement practice that can weaken construct validity?

A. Creating items ad hoc without clear validity evidence

B. Using a larger sample size to increase precision

C. Reporting all analyses in a transparent appendix

D. Blinding the data collector to hypotheses

Answer: (A)

Explanation

Ad hoc item creation is specifically named as a questionable measurement practice because it can weaken construct validity. The other options are generally good research practices.

14. Why can low statistical power make a study’s findings less reliable?

A. It eliminates the need for careful measurement

B. It mainly detects only large effects, which can exaggerate or miss true effects

C. It guarantees that all real effects will be significant

D. It makes rounding errors less likely in reported statistics

Answer: (B)

Explanation

Low power means the study can reliably detect only larger effects, so results are more vulnerable to exaggeration, instability, or error. It does not guarantee significance or improve measurement quality.

15. Which of the following is an example of a data anomaly that can trigger suspicion?

A. Multiple panels showing implausibly close patterns across groups

B. A clearly stated hypothesis before data collection

C. A figure that includes labels and legends

D. A sample size that is larger than in a previous study

Answer: (A)

Explanation

Implausibly similar patterns across panels or groups are a classic anomaly because they may violate expectations from independent data. The other options are not anomalies by themselves.

16. What happened in the example where integer adjustments were made to F values?

A. The studies were improved through better randomization and blinded assessment

B. The means became compatible with the data’s granularity

C. The effect sizes were reduced because of a higher sample size

D. The results were altered to look statistically significant and the papers were later retracted

Answer: (D)

Explanation

The example describes adding or subtracting integers to F values to make results appear significant, which later led to retractions. It is an instance of self-correction after a data anomaly was uncovered.

17. Which set of actions best matches the global solutions emphasized for improving scientific reliability?

A. Avoiding data visualization to prevent subjective interpretation

B. Relying only on journal prestige to prevent mistakes

C. Checks and balances, data sharing, pre-prints, open peer review, and full audit procedures

D. Increasing the number of positive results and reducing scrutiny

Answer: (C)

Explanation

The global solutions listed include checks and balances, data sharing, pre-prints, open peer review, and full audit procedures. These are intended to make correction and scrutiny easier, not harder.

18. What is one practical good practice for data analysis highlighted as helpful for spotting problems?

A. Plotting the data to surface issues

B. Hiding raw data until publication

C. Skipping review to speed up the workflow

D. Using only summary statistics without inspection

Answer: (A)

Explanation

Plotting the data is specifically named as a practical way to detect problems during analysis. Visual inspection can reveal anomalies that summary statistics may hide.