By Dalat TESOL
Making research design more accessible for classroom-based and applied linguistics researchers
📌 Introduction
When teachers or graduate students want to test whether a particular teaching method works better than another, they often turn to experimental or quasi-experimental research designs. These designs help answer questions like:
“Does using peer feedback improve writing more than teacher feedback?”
“Will a 4-week vocabulary app intervention enhance EFL learners’ retention?”
“Does AI-assisted writing boost students’ self-efficacy compared to traditional drafting?”
This article introduces:
- What experimental and quasi-experimental designs are
- Key differences between the two
- When to use them in TESOL
- Core components (e.g., control groups, randomization, pre/post-tests)
- Common designs (with diagrams)
- Practical examples in TESOL contexts
- Strengths and limitations
⚖️ 1. What Are Experimental Designs?
An experimental design is a type of quantitative research that manipulates one variable (called the independent variable) to see how it affects another (called the dependent variable), while controlling for other factors.
The hallmark of true experiments is random assignment — participants are randomly placed into groups, which helps rule out pre-existing differences.
🧪 2. What Are Quasi-Experimental Designs?
Quasi-experimental designs are similar to experimental ones, but they lack random assignment. This often happens in classroom-based research where random grouping is not possible.
Instead, researchers work with intact groups (e.g., two existing classes) and compare them before and after an intervention.
🧬 3. Key Terminology and Concepts
| Term | Meaning | Example |
|---|---|---|
| Independent variable (IV) | The variable you manipulate | Type of feedback (peer vs. teacher) |
| Dependent variable (DV) | The outcome you measure | Writing accuracy score |
| Control group | Group that does not receive the treatment | Class with traditional feedback |
| Experimental group | Group that receives the new treatment | Class using peer feedback |
| Random assignment | Randomly placing participants in groups | Using a random number generator |
| Pre-test/Post-test | Test before and after the intervention | Writing test Week 1 and Week 5 |
🔄 4. Common Designs with Diagrams
A. True Experimental Design
Pre-test/Post-test Control Group
bashCopyEditGroup A (Experimental): Pre-test → Treatment → Post-test
Group B (Control): Pre-test → No Treatment → Post-test
✅ Strong internal validity
❌ May not be feasible in real classrooms
B. Quasi-Experimental Design
Non-Randomized Pre-test/Post-test
bashCopyEditClass A (Experimental): Pre-test → Treatment → Post-test
Class B (Control): Pre-test → No Treatment → Post-test
✅ More practical in educational settings
❌ Risk of selection bias
C. One-Group Pre-test/Post-test
bashCopyEditGroup A: Pre-test → Treatment → Post-test
✅ Good for classroom action research
❌ No comparison group — limited conclusions
D. Post-test Only Control Group
yamlCopyEditGroup A: Treatment → Post-test
Group B: No Treatment → Post-test
✅ Useful when pre-test may “prime” learning
❌ No baseline comparison
📚 5. Sample Research Topics in TESOL
| Title | Design | RQ Example |
|---|---|---|
| The effect of vocabulary apps on word retention | Quasi-experimental (2 classes) | Do students using Quizlet outperform those with printed flashcards? |
| Peer feedback vs. teacher feedback in EFL writing | True experimental (random groups) | Which group improves more in writing coherence? |
| AI-assisted writing and learner confidence | One-group pre-post | Does using ChatGPT for planning increase students’ self-efficacy? |
🧠 6. How to Choose a Design
| Ask Yourself… | Suggests |
|---|---|
| Can I randomly assign students to groups? | True experiment |
| Am I working with existing classes? | Quasi-experiment |
| Is there a control group? | Stronger design |
| Can I only study one group? | Pre-post (with caution) |
💡 In TESOL, quasi-experimental designs are most common due to classroom constraints.
⚙️ 7. Steps for Conducting a Study
- Formulate a clear research question
e.g., Does peer feedback improve essay organization more than teacher feedback? - Identify your IV and DV
IV = Type of feedback; DV = Writing rubric score (organization) - Select participants and groups
Try to match groups in ability, age, class level - Design your instruments
Pre/post-tests, scoring rubrics, attitude scales - Conduct the intervention
e.g., 4 writing sessions with peer vs. teacher feedback - Analyze data
Paired/independent t-tests, ANOVA, or regression - Interpret results and write up
Link back to RQ, literature, and pedagogical implications
🔎 8. Common Statistical Tests Used
| Test | When to Use |
|---|---|
| Paired-sample t-test | Compare pre and post in the same group |
| Independent-sample t-test | Compare 2 different groups (e.g., control vs. experimental) |
| ANOVA | Compare 3+ groups or more than one IV |
| ANCOVA | Control for differences in pre-test scores |
| Effect size (Cohen’s d) | Show how much change happened, not just if it was significant |
📉 9. Limitations to Keep in Mind
- Random assignment is often impractical in educational contexts
- Pre-existing differences between groups can affect results
- Hawthorne effect: students change behavior because they know they’re being studied
- Post-test-only designs don’t measure learning gains
- One-group designs cannot rule out external influences (e.g., curriculum, teacher)
📚 10. Further Reading
- Mackey, A., & Gass, S. (2016). Second Language Research: Methodology and Design
- Dornyei, Z. (2007). Research Methods in Applied Linguistics
- Plonsky, L. (2015). Study quality in L2 research: A methodological synthesis and meta-analysis. Language Learning, 65(2), 295–348.
- Shadish, Cook, & Campbell (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference
🌿 Final Thoughts
Experimental and quasi-experimental designs are powerful tools for TESOL researchers and classroom practitioners who want to explore what works — and why.
Start with a clear RQ, be transparent about your design, and remember: even small-scale classroom experiments can lead to meaningful insights.