Research methods that appear convincing on paper do not always work as expected when they are applied to real participants, real questionnaires, and real data-collection conditions. Recruitment may be slower than anticipated, survey questions may be misunderstood, interview procedures may take too long, or a proposed measurement process may prove impractical. A pilot study allows researchers to identify these problems before committing substantial resources to the full investigation.
A pilot study is therefore best understood as a preliminary test of the feasibility of a proposed research process. It can help determine whether participants can be recruited, whether instructions and instruments function as intended, whether data can be collected and managed successfully, and whether aspects of the protocol need modification. Its central purpose is preparation and refinement rather than producing a smaller version of the final study's substantive conclusions.
Students often become uncertain about how many participants a pilot study needs, whether pilot participants can enter the final sample, what results should be analysed, and where the pilot should appear in a dissertation. These questions do not have one universal numerical answer because pilot design should follow its specific feasibility objectives. This guide explains the purpose, sample size, design, analysis, and reporting of pilot studies so that students can justify their preliminary work clearly and methodologically.
What Is a Pilot Study in Academic Research?
A pilot study is a small-scale preliminary study conducted before a larger or definitive investigation. Its purpose is to examine whether important elements of the proposed research can operate successfully in practice. These elements may include recruitment, participant eligibility, consent procedures, questionnaires, interview schedules, randomisation processes, data-management systems, intervention delivery, or follow-up procedures.
The key word is feasibility. A pilot does not simply reproduce the main study with fewer participants. Instead, it investigates uncertainties that need to be resolved before the larger research proceeds. A researcher might ask whether eligible participants can be recruited at the required rate, whether respondents understand questionnaire items, whether an interview lasts an acceptable amount of time, or whether the planned data-collection process produces sufficiently complete information.
This purpose fits within the wider logic of research methodology. Epic Essay's guide to dissertation research methods explains that research design, data collection, sampling, and analysis should align with the research questions. Pilot work can help researchers determine whether those planned procedures remain workable when moved from a proposal into practice.
A pilot study primarily asks, "Can the proposed main study be conducted as planned, and what should be changed before it begins?" It should not be treated simply as an underpowered attempt to answer the main research question.
What Is the Main Purpose of a Pilot Study?
The precise purpose depends on the uncertainties surrounding the main research design. In questionnaire research, a pilot may investigate clarity, completion time, response options, missing responses, or administration procedures. In an interview study, it may reveal whether questions generate information relevant to the research objectives and whether the sequence encourages coherent discussion.
In more complex studies, pilot objectives can include recruitment and retention, intervention delivery, adherence, randomisation, data collection, and follow-up. Guidance from the National Center for Complementary and Integrative Health emphasises that feasibility studies should focus primarily on feasibility and acceptability benchmarks rather than using an inadequately powered pilot to test efficacy. This distinction is important beyond clinical research because it illustrates the broader principle that pilot objectives should address whether the proposed research process works.
Typical pilot-study objectives may therefore include:
- Assessing whether the target participants can be recruited successfully
- Checking whether inclusion and exclusion criteria work in practice
- Testing whether questionnaire items and instructions are understandable
- Estimating how long data collection takes
- Examining completion, retention, or follow-up procedures
- Identifying missing-data or data-management problems
- Testing whether equipment, software, or research procedures function as intended
- Assessing whether a proposed intervention or protocol can be delivered consistently
- Identifying practical changes needed before the main study
The pilot should therefore have explicit objectives rather than the vague aim of "testing the research." Clear objectives determine what information must be collected, how many participants may be required, what success criteria should be used, and how the findings will influence the main study.
Pilot Study vs Main Study: What Is the Difference?
A pilot and a main study are connected, but their primary objectives are different. The main study is designed to answer the substantive research questions or test the principal hypotheses. A pilot is designed primarily to determine whether the methods needed to answer those questions are feasible and sufficiently refined.
This distinction affects analysis. A definitive quantitative study may be powered to detect a specified difference or association, whereas a pilot may concentrate on recruitment rates, questionnaire completion, retention, protocol adherence, data completeness, or other feasibility outcomes. Treating a small pilot as though it were the definitive hypothesis test can lead to unstable and misleading conclusions.
The differences are summarised below.
| Feature | Pilot Study | Main Study |
|---|---|---|
| Primary purpose | Assess feasibility and refine research procedures | Answer the substantive research question |
| Typical scale | Smaller and focused on specific uncertainties | Designed for the required substantive analysis |
| Primary outcomes | Feasibility indicators and procedural findings | Research outcomes linked to hypotheses or objectives |
| Statistical emphasis | Often descriptive and precision focused | May involve fully planned inferential analysis |
| Resulting decision | Proceed, modify, reconsider, or stop aspects of the design | Draw conclusions addressing the research questions |
The pilot is therefore valuable because it reduces avoidable uncertainty before the definitive research begins. It can reveal weaknesses at a stage when changes are still possible rather than after the full sample has already been recruited.
When Does a Dissertation Need a Pilot Study?
Not every dissertation automatically requires a pilot. The need depends on the research design, the novelty of the procedures, the instruments being used, the population, and the uncertainties that remain before full data collection. A researcher using a well-established secondary dataset, for example, may have little reason to conduct a participant-based pilot.
A pilot becomes particularly useful when the researcher has developed a new questionnaire, substantially modified an existing instrument, created a new interview schedule, introduced an unfamiliar procedure, or is uncertain whether participants can be recruited and retained. It may also be valuable when the data-collection environment is unfamiliar or when the study involves several practical stages that must operate together.
Students should therefore justify the pilot from a methodological need rather than conducting one simply because they believe every dissertation must include it. Epic Essay's guide to the dissertation methodology chapter emphasises that research choices should be explained in relation to the research questions and design rather than presented as unexplained procedures.
The same principle applies when a pilot is not conducted. If established procedures and validated instruments are being used without meaningful modification, the researcher can explain this context rather than inventing a pilot merely to make the methodology appear more extensive.
What Should You Test During a Pilot Study?
A useful pilot should target the parts of the research process where uncertainty could threaten the main study. Testing everything without defined objectives can produce a large amount of information without clarifying whether the definitive study is feasible. The researcher should therefore decide in advance which processes require evaluation.
For a survey dissertation, the pilot might examine questionnaire length, wording, response options, navigation, technical functionality, missing responses, and completion time. If participants repeatedly skip one item or interpret it differently from what the researcher intended, the pilot has identified a problem that can be addressed before the full survey is launched.
For interview research, attention may shift toward question sequencing, participant comprehension, interview duration, recording procedures, and whether prompts generate evidence relevant to the research objectives. Pilot interviews can also reveal questions that are repetitive, leading, ambiguous, or too broad to generate analytically useful answers.
For experimental or intervention research, the pilot may need to examine recruitment, allocation procedures, adherence, equipment, intervention delivery, follow-up, and data capture. The purpose remains the same: identify practical uncertainty and establish whether the proposed main-study procedures can be implemented reliably.
How Is a Pilot Study Different from Pretesting a Questionnaire?
Pilot testing and questionnaire pretesting are related but should not automatically be treated as identical. Questionnaire pretesting focuses specifically on the instrument. Researchers may examine wording, interpretation, response options, layout, instructions, and whether respondents understand questions as intended.
A pilot study can be broader because it may test the entire research process in miniature. In addition to the questionnaire itself, a survey pilot might test participant recruitment, consent, distribution links, reminders, completion time, data export, coding, and handling of incomplete responses. The distinction is useful when describing exactly what preliminary work was performed.
If the research involves a newly designed questionnaire, instrument quality deserves particular attention. Questions should correspond to the variables and research objectives, response scales should be suitable, and ambiguous or double-barrelled items should be addressed before large-scale administration.
The dissertation should therefore use precise language. If only five students were asked whether questionnaire wording was understandable, describing this as questionnaire pretesting may be more accurate than claiming that the entire research design underwent a comprehensive pilot study.
How Many Participants Should a Pilot Study Have?
There is no universal pilot-study sample size that is correct for every research design. The number should be linked to the pilot's objectives and the precision needed for the feasibility information being collected. A pilot designed to evaluate recruitment or retention has a different sample-size problem from one designed to examine the variability of a continuous measurement or the functioning of a questionnaire.
Methodological guidance increasingly discourages the assumption that a single rule such as 10%, 20 participants, or 30 participants can be applied automatically. Recent discussion in The BMJ on pilot-trial sample sizes emphasises that participant numbers should align with the key pilot objectives and have an explicit rationale. The appropriate number therefore depends on what the researcher needs to learn from the pilot.
A useful justification asks whether the proposed pilot sample can provide adequate information about the relevant feasibility outcome. If the objective is to estimate a recruitment proportion, for example, precision around that estimate may matter. If the objective is primarily to identify obvious problems in questionnaire administration, a different rationale may be appropriate.
Avoid writing that "30 participants are required for every pilot study." Thirty may be defensible in a particular project, but pilot sample size should follow from the objectives rather than from a universal threshold.
Why the Common 10% Pilot Sample Rule Can Be Misleading
Students are sometimes advised to recruit 10% of the intended main sample for a pilot. This can appear convenient because it produces an immediate number. However, the percentage of the final sample does not necessarily correspond to the information required to evaluate feasibility.
Suppose the definitive study needs 100 participants. A 10% rule produces a pilot of 10. If the main study instead needs 1,000 participants, the same rule produces 100 pilot participants. There is no methodological reason why the feasibility question must become ten times more demanding merely because the definitive sample is larger.
A percentage rule can still be used if it is supported by relevant methodological guidance and fits the objectives, but it should not substitute for reasoning. The researcher should explain what the proposed number is expected to reveal and why that information is sufficient to inform the main study.
This principle is similar to main-study sample justification. Epic Essay's guide to sample size justification in research explains that sample numbers should be defended using statistical, methodological, and practical reasoning rather than unsupported convenience.
How to Determine a Defensible Pilot Study Sample Size
The most defensible approach begins with the pilot's primary objective. Once that objective is defined, the researcher can determine what quantity or process needs to be estimated and how much information is necessary to make a decision about the main study. This shifts the discussion from "What is the standard pilot sample?" to "What sample will answer my feasibility question?"
For quantitative pilot studies, confidence-interval approaches can sometimes be used to plan the precision of important feasibility estimates. The National Center for Complementary and Integrative Health notes that sample-size calculations for feasibility studies can be based on the precision with which feasibility benchmarks are estimated. This provides a stronger rationale than selecting a number solely because another dissertation used it.
Different pilot objectives can therefore lead to different sample-size rationales.
| Pilot Objective | Information Needed | Possible Sample Size Rationale |
|---|---|---|
| Assess recruitment feasibility | Recruitment rate and reasons for non-participation | Enough recruitment activity to estimate whether the target rate is achievable |
| Assess participant retention | Completion or follow-up proportion | Precision around an expected retention estimate |
| Test questionnaire administration | Comprehension, missing items, completion time, technical problems | Enough varied participants to expose important instrument and process problems |
| Test an intervention protocol | Delivery, adherence, acceptability, protocol deviations | Enough cases to observe the main operational stages |
| Estimate variability for later planning | Preliminary variance or standard-deviation information | Sample chosen for the required precision, interpreted cautiously |
The table demonstrates why the objective should come before the number. Sample-size reasoning becomes much clearer when the dissertation states exactly what the pilot was expected to establish.
Should You Use a Pilot Study to Calculate the Main Study Effect Size?
Students sometimes conduct a very small pilot, calculate an observed effect size, and enter that value directly into a power analysis for the definitive study. This approach can be problematic because effect estimates from small samples are unstable. An unusually large pilot effect can lead to a main-study sample that is far too small.
The National Center for Complementary and Integrative Health specifically cautions against using highly variable pilot effect-size estimates as the primary basis for future efficacy calculations. When possible, effect-size planning should incorporate stronger evidence such as previous studies, meta-analysis, a scientifically meaningful minimum effect, or other substantive information.
Pilot data can still provide useful information for planning. Estimates of recruitment, retention, measurement variability, completion time, missingness, or procedural feasibility may be valuable depending on the design. The problem arises when an imprecise preliminary treatment effect is treated as though it were a reliable estimate of the true effect.
The distinction should be clear in a dissertation. A pilot can inform main-study planning without being expected to provide a definitive answer to the substantive research hypothesis.
Should Pilot Participants Be Included in the Main Study?
Whether pilot participants can be carried forward into the main analysis depends on the design and whether the pilot changes the research procedures. If questionnaire wording, intervention delivery, eligibility criteria, measurements, or data-collection processes are substantially altered after the pilot, combining the preliminary and definitive data may undermine comparability.
In an external pilot, the pilot is separate from the definitive study and its participants are not automatically included in the main analysis. An internal pilot, used in some formal study designs, may be planned so that eligible pilot data can contribute to the definitive analysis under pre-specified conditions. This is a methodological design decision rather than a convenient choice made after examining the pilot results.
For many student dissertations, the safest principle is transparency. If the pilot led to meaningful changes in the instrument or procedure, the researcher should be cautious about treating pilot observations as though they had been collected under the final protocol. Institutional or supervisory requirements should also be followed.
The dissertation should explicitly state what happened to pilot data. Leaving readers uncertain about whether pilot responses were mixed into the main dataset makes the sampling and data-collection process difficult to evaluate.
What Results Should You Analyse in a Pilot Study?
Pilot analysis should follow the feasibility objectives. If the main question is whether recruitment works, the analysis might report the number approached, eligible, consenting, and completing the procedure. If questionnaire feasibility is being examined, the researcher might summarise missing responses, completion times, misunderstood questions, and participant feedback.
Descriptive statistics are often particularly useful because the objective is to understand whether processes function as intended. Frequencies, proportions, ranges, means, medians, and confidence intervals may be appropriate depending on the outcome. The analytical emphasis should remain on what the pilot was designed to learn.
Formal hypothesis testing of the main study outcome is often inappropriate when the pilot was not designed or powered for that purpose. A non-significant result may simply reflect the small pilot sample, while a significant result can be unstable and exaggerated. Neither should be allowed to determine prematurely whether the substantive hypothesis is worthwhile.
Analyse the pilot according to its feasibility objectives. Do not judge whether the main research question is "true" or "false" from a preliminary study that was never designed to answer it.
How to Use Pilot Results to Improve the Main Study
The value of a pilot lies in what happens after problems are identified. If respondents misunderstand a questionnaire item, the wording may need revision. If recruitment is too slow, additional recruitment channels may be necessary. If interviews consistently exceed the available time, questions may need to be shortened or prioritised.
Researchers can make these decisions more systematic by defining progression criteria in advance. A study might specify acceptable recruitment, retention, completion, or adherence thresholds and determine what action will follow if those thresholds are met or missed. This creates a clearer connection between pilot evidence and the decision to proceed.
A simple framework is shown below.
| Pilot Finding | Possible Interpretation | Potential Main-Study Response |
|---|---|---|
| High questionnaire completion | Administration process appears workable | Retain the procedure |
| One item frequently misunderstood | Question wording is unclear | Revise and recheck the item |
| Recruitment substantially below target | Current recruitment strategy may be infeasible | Add sites, channels, or revise the recruitment plan |
| High follow-up loss | Retention procedure is inadequate | Modify reminders or follow-up procedures |
| Data export produces coding errors | Data-management workflow requires correction | Revise coding and test the export process again |
Not every pilot problem requires abandoning the research. The purpose is to identify weaknesses early enough for them to be corrected. A pilot that exposes an important problem has therefore been useful rather than unsuccessful.
How to Report a Pilot Study in a Dissertation Methodology
The methodology chapter should explain why the pilot was conducted, who participated, how participants were selected, what procedures were tested, what criteria were examined, and how the findings affected the final study. This gives readers a clear account of how the research design was refined before definitive data collection.
A hypothetical questionnaire-based write-up might state: "A pilot study was conducted before the main survey to assess questionnaire clarity, completion time, response-option suitability, and the practical operation of the online data-collection procedure. Participants meeting the intended eligibility criteria completed the draft questionnaire and provided feedback on ambiguous or difficult items. The pilot identified two items requiring wording changes and one response scale requiring clarification. Pilot responses were excluded from the main analysis because the questionnaire was subsequently revised."
The paragraph should be adapted to what actually occurred. If no changes were made, say so. If substantial changes were introduced, identify them sufficiently for readers to understand how the pilot improved the final research procedure.
Epic Essay's dissertation methodology example demonstrates how participant selection, data collection, analysis, and ethics are presented as connected elements of the research design. Pilot reporting should fit naturally within that methodological account rather than appearing as an unexplained sentence stating only that "a pilot study was conducted."
How to Report Pilot Study Results Clearly
Reporting should return directly to the objectives established before the pilot. If the objective concerned recruitment, provide recruitment information. If it concerned questionnaire feasibility, report evidence about completion, missingness, comprehension, or participant feedback. The results should allow the reader to see whether the feasibility question was answered.
Methodological guidance on pilot studies recommends clearly identifying the work as a pilot, explaining how sample size was determined, specifying feasibility criteria, describing analysis, and reporting participant flow and recruitment where relevant. These principles make the preliminary study transparent rather than presenting it as an informal trial run that left no research record.
A useful reporting structure is:
- State the specific purpose of the pilot.
- Describe the participants and recruitment process.
- Explain why the pilot sample size was considered adequate for its objectives.
- Describe the procedures or instruments tested.
- State any pre-specified feasibility or progression criteria.
- Report the relevant descriptive or feasibility findings.
- Identify practical problems or protocol deviations.
- Explain what changes were made before the main study.
- State whether pilot participants or data were included in the definitive analysis.
- Report relevant ethical approval and consent procedures.
This structure provides a clear chain from purpose to evidence to action. Readers can see not only that preliminary testing occurred but also how it strengthened or changed the final methodology.
Where Should a Pilot Study Appear in a Dissertation?
The methodology chapter is usually the natural place to describe a pilot because the pilot forms part of the development and testing of the research procedure. The precise location depends on the dissertation structure. It may appear after the description of the instrument, within data-collection procedures, or in a dedicated pilot-study subsection.
Detailed pilot materials can be placed in an appendix where appropriate. These might include a draft questionnaire, participant feedback form, progression criteria, or supplementary pilot results. The main chapter should still contain enough information for readers to understand what was tested and what changed.
If the pilot generated substantial findings that directly determined the definitive design, some programmes may require more detailed reporting. Students should follow departmental guidance because expectations vary by discipline and degree level.
The central requirement is transparency. The reader should be able to reconstruct how the final research procedure developed from the pilot without searching through unexplained appendices or guessing what modifications were made.
Do Pilot Studies Need Ethical Approval?
A study does not automatically fall outside research ethics requirements merely because it is labelled a pilot. If human participants are recruited, personal information is collected, an intervention is administered, or participants face research-related burdens or risks, the relevant institutional ethics procedures generally need to be considered before the pilot begins.
Researchers should not assume that data collected informally before ethics approval can later be treated as legitimate pilot or main-study data. Institutional requirements differ, so students should follow their university's ethics procedures and supervisory guidance before recruiting participants.
Consent information should also accurately describe the preliminary nature and purpose of the study where appropriate. Participants should understand what they are being asked to do, how their information will be used, and whether their data could contribute to later research.
Ethics therefore belongs in pilot planning rather than being added retrospectively. A well-designed pilot tests procedures while maintaining the same commitment to participant protection and research integrity expected of the larger project.
Common Pilot Study Mistakes Students Should Avoid
One common mistake is describing the pilot merely as "a small version of the main study." This definition misses its methodological purpose. The pilot should investigate specific feasibility uncertainties rather than simply repeat the final analysis on fewer participants.
A second mistake is choosing a sample from an unsupported rule such as "10% is always required." A stronger justification links the number of participants to the objectives, expected information, precision, or methodological guidance relevant to the study.
Another mistake is using an underpowered pilot to decide whether the main hypothesis should be pursued. Failure to obtain statistical significance in a small pilot does not demonstrate that the substantive effect is absent. Similarly, a large observed effect in a small pilot should not automatically be treated as a stable estimate for main-study planning.
Do not report that a pilot "proved the questionnaire is valid" merely because participants completed it successfully. Feasibility, reliability, content validity, construct validity, and other measurement properties are distinct concepts and require evidence appropriate to each claim.
A Step-by-Step Checklist for Planning a Pilot Study
A pilot is most useful when it is planned with the same clarity expected of the definitive study. Begin by identifying uncertainty rather than beginning with a desired sample number. Every later decision should follow from what the pilot needs to establish.
Before conducting the pilot, work through the following sequence:
- Identify the main study and the methodological uncertainty that needs testing.
- Write specific pilot objectives focused on feasibility or procedural refinement.
- Define the participants or data required to address those objectives.
- Choose a pilot sample size based on the information or precision needed.
- Specify recruitment, consent, and data-collection procedures.
- Define feasibility or progression criteria where appropriate.
- Decide which descriptive, qualitative, or other analyses will answer the pilot objectives.
- Obtain required ethical approval before collecting participant data.
- Conduct the pilot and document deviations or practical problems.
- Compare the findings with the pre-specified feasibility objectives.
- Revise the main-study procedures where justified.
- Document what changed and whether pilot data will remain separate from the main analysis.
This process transforms the pilot from an informal trial run into a transparent research stage. It also gives the methodology chapter a clear explanation of why preliminary testing was necessary and how it improved the final design.
Using a Pilot Study to Strengthen Your Dissertation Research Design
A well-designed pilot study provides evidence about whether a proposed research process can work before the full investigation begins. Its value lies in testing feasibility, identifying weaknesses, refining instruments and procedures, and establishing whether recruitment, data collection, and other operational elements are sufficiently workable for the main study.
There is no universal pilot sample size that applies to every dissertation. Participant numbers should follow from the pilot objectives and the amount of information required to evaluate them. The same logic applies to analysis: pilot results should primarily answer feasibility questions rather than being treated as an underpowered test of the definitive research hypothesis.
Strong reporting completes the process. Explain why the pilot was conducted, how its sample was justified, what was tested, what the results showed, what changed, and how pilot data were handled. When those decisions are explicit, the pilot study becomes a defensible methodological tool that strengthens the dissertation rather than an unexplained preliminary exercise.
Comments