Reliability and Validity in Research: Types, Examples, How to Test

Reviewed by Prof. Dr. Alexander Holtermann · Updated 16 October 2026 · 5 min read

Reliability means a measure gives consistent results; validity means it measures what it is meant to. A measure can be reliable without being valid.

Key takeaways

Reliability is consistency: the same measure gives the same result under the same conditions. Validity is accuracy: the measure captures the concept it claims to capture.
Reliability is necessary but not sufficient for validity. A bathroom scale that always shows 2 kg too much is reliable but not valid.
Main types of reliability: test-retest, internal consistency (Cronbach’s alpha, usually ≥ .70), inter-rater and parallel forms.
Main types of validity for measures: content, construct (convergent and discriminant) and criterion (concurrent and predictive). For whole studies: internal and external validity.
Qualitative research uses its own criteria, usually trustworthiness after Lincoln and Guba, rather than reliability and validity in the narrow sense.

Reliability vs validity: the core idea

Reliability asks: if you measured again, would you get the same result? Validity asks: are you measuring the right thing at all? Both are classic quality criteria for quantitative research, and examiners expect you to address them in your methods chapter. Many textbooks add objectivity: results should not depend on who administers, scores or interprets the measure.

The classic illustration is a target. Shots tightly clustered but off-center are reliable but not valid. Shots scattered around the center are neither consistently reliable nor precise. Only a tight cluster in the bull’s-eye is both.

	Reliability	Validity
Question	Is the measure consistent?	Does it measure what it should?
Typical evidence	Cronbach’s alpha, test-retest correlation, Cohen’s kappa	Expert review, correlations with related and unrelated measures, prediction of outcomes
Relationship	Necessary for validity	Requires reliability

Types of reliability

Which type you report depends on your instrument and design.

Type	What it checks	How	Rule of thumb
Test-retest	Stability over time	Same people, same measure, two time points; correlate scores	r ≥ .70 for stable traits
Internal consistency	Items of a scale measure the same thing	Cronbach’s alpha or McDonald’s omega	≥ .70 acceptable, ≥ .80 good
Inter-rater	Agreement between coders or observers	Cohen’s kappa, Krippendorff’s alpha, ICC	kappa ≥ .60 substantial
Parallel forms	Equivalence of two versions	Correlate scores on form A and form B	r ≥ .70

Types of validity for measures

Validity of a measure is not a single number but an argument supported by several kinds of evidence.

Face validity: at first glance, the items look as if they measure the concept. Weakest form; useful for participant acceptance.
Content validity: the items cover all relevant aspects of the concept. Evidence: definition from the literature, expert review, a pilot study.
Construct validity: the measure behaves as theory predicts. Convergent: it correlates with measures of related constructs. Discriminant: it correlates weakly with unrelated constructs. Factor analysis also provides evidence.
Criterion validity: the measure agrees with an external criterion. Concurrent: measured at the same time (e.g. a new stress scale correlates with an established one). Predictive: it predicts a later outcome (e.g. an aptitude test predicts first-year grades).

Internal and external validity of a study

Beyond single measures, whole studies are judged on validity.

Internal validity concerns causal conclusions: can you rule out that something other than your independent variable caused the effect? Threats include selection effects, history (events between measurements), maturation, testing effects and attrition. Random assignment is the strongest protection.

External validity concerns generalization: do your results hold for other people, settings and times? Threats include convenience samples, artificial lab settings and very specific populations. Ecological validity is a related term for how well a study reflects real-life conditions.

There is often a trade-off: tightly controlled experiments maximize internal validity but may lose external validity; field studies do the reverse.

Threat	Type	Countermeasure
Groups differ before the intervention	Internal	Random assignment, baseline measurement
Participants drop out unevenly	Internal	Report attrition, intention-to-treat analysis
Only students from one university	External	Name the limitation, compare with population data
Leading question wording	Construct	Neutral wording, pilot testing

How to ensure reliability and validity in your project

Most of the work happens before data collection, not after.

Use established, validated scales wherever possible and cite the validation study. Report the reliability in your own sample too.
If you must create items, define the construct first (operationalization), cover all dimensions and have them reviewed by your supervisor or experts.
Run a pilot study to catch ambiguous wording before the main data collection.
Standardize administration: same instructions, same order, same mode for everyone.
For coding or observation, use a codebook and calculate inter-rater reliability on a subset.
In your methods chapter, report alpha (or omega) for every scale and discuss threats to internal and external validity in your limitations.

What about qualitative research?

Reliability in the sense of identical results on repetition makes little sense for interviews: a second conversation is never the same. Qualitative research therefore uses adapted criteria, most often trustworthiness after Lincoln and Guba (1985): credibility, transferability, dependability and confirmability. Some qualitative researchers still use the words “validity” and “reliability” in a broader sense, and inter-coder agreement is common in qualitative content analysis. Choose one framework, name it, and show how you met it.

Checking your instrument with our tools

The free quality criteria tool walks you through the criteria that apply to your design, and the reliability calculator computes Cronbach’s alpha from your item data. If you run your survey at /umfragen, alpha is calculated for each scale automatically. Departments differ in how much detail they want on quality criteria; your university’s own guidelines take precedence. Upload them to Professor Digital and the checklist is tailored to them.

Frequently asked questions

What is the difference between reliability and validity?

Reliability is about consistency: does the measure give the same result again? Validity is about accuracy: does it measure what it is supposed to measure? A measure can be reliable without being valid, but not valid without being reliable.

What is a good Cronbach’s alpha?

By convention, .70 is acceptable, .80 good and .90 excellent. Values above about .95 can signal redundant items. Alpha also rises with the number of items, so interpret it alongside the scale length.

Can a test be reliable but not valid?

Yes. A scale that always shows 2 kg too much gives consistent but wrong results. Reliability is necessary for validity, but it does not guarantee it.

How do you ensure validity in research?

Use validated instruments, define your constructs carefully, have items reviewed by experts, pilot test, control confounding variables and be transparent about sampling limitations.

Do reliability and validity apply to qualitative research?

Not in the narrow statistical sense. Qualitative research usually uses trustworthiness criteria instead: credibility, transferability, dependability and confirmability.

Sources and further reading

Clark, T., Foster, L., Sloan, L., & Bryman, A. (2021). Bryman’s Social Research Methods (6th ed.). Oxford University Press.
Creswell, J. W., & Creswell, J. D. (2023). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (6th ed.). Sage.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic Inquiry. Sage.

Your university’s examination regulations and style guide always take precedence.