Writing the questionnaire is the part groups plan for. Validating it is the part that runs over, because it depends on three other people’s diaries and on a pilot run you cannot compress. Validation is also the part a panel examines most closely in Chapter 3, since an unvalidated instrument makes every number in Chapter 4 arguable. Here is what the process actually consists of, and what has to end up on the page.
The three things called validity, and which one you are doing
| Type | The question it answers | How an undergraduate study establishes it |
|---|---|---|
| Face validity | Does the instrument look like it measures the thing? | Read-through by a small group of intended respondents |
| Content validity | Do the items cover the whole construct, and only it? | Expert validation by named validators against your indicators |
| Construct validity | Does the instrument behave the way the theory predicts? | Rarely established from scratch; inherited when you adopt a published scale |
Almost every Philippine undergraduate thesis with a researcher-made instrument establishes content validity by expert review and stops there, and that is acceptable when it is stated honestly. What is not acceptable is claiming construct validity for a questionnaire that has never been factor-analysed. Where construct validity genuinely matters, adopting an existing instrument is the shorter road, and the licence questions that come with it are worked through in the guide to choosing and adapting a standardised scale.
Choosing the validators
Three is the number most Philippine departments ask for. The useful combination is not three people from the same office.
- A content expert. Someone who knows the subject matter: a senior practitioner, a faculty member in the field, a unit head. They catch a missing dimension.
- A research or statistics expert. Someone who reads instruments for a living. They catch double-barrelled items, leading wording and a scale that will not support the test you planned.
- A practitioner from the setting. Someone who works where your respondents work. They catch items that are unanswerable in practice, or that use a term your respondents do not use.
Ask the research coordinator whether your department maintains a list of accredited validators, and whether validators must hold a particular rank. Some do, and discovering that after the sheets are signed costs a fortnight.
What the validation sheet contains
A validation sheet is a short rating form the validator signs. Most Philippine departments issue one; where yours does not, the following is the common shape.
| Criterion | What the validator is judging |
|---|---|
| Clarity of directions and items | Whether a respondent can answer without asking a question |
| Presentation and organisation | Whether the parts follow a sensible order |
| Suitability of items | Whether the items fit the respondents named in the study |
| Adequacy of items per indicator | Whether each dimension has enough items to be measured |
| Attainment of purpose | Whether the instrument answers the specific questions |
| Objectivity | Whether the wording leads the respondent toward an answer |
| Scale and evaluation rating | Whether the response scale suits the items |
Two practical rules. Give each validator the statement of the problem and the operationalisation table alongside the questionnaire, because content validity means coverage of your indicators and they cannot judge coverage without seeing them. And give them a deadline with a date on it; “at your convenience” is how two weeks disappear.
Where departments compute a content validity index, it is the ratio of items the validators judged relevant to the total number of items, and the threshold is set by the department rather than by any universal standard. Ask what yours requires before you compute anything, and report the rule you were given.
Acting on what comes back
Validators return three kinds of comment, and they need different responses.
- Wording. Fix it. Double-barrelled items (“the staff were friendly and efficient”) get split; leading items (“how satisfied were you with the excellent service?”) get neutralised; jargon gets replaced with the term respondents actually use.
- Coverage. If a validator says a dimension is under-measured, add items. If a validator says an item belongs to no dimension, delete it. Both changes alter your operationalisation table, so update it in the same sitting.
- Scope. If a validator says the instrument is measuring something outside your stated variables, that is a Chapter 1 problem, not a wording problem. Take it to your adviser before editing.
Keep a short revision log: item number, comment, what you changed. It takes ten minutes and it answers the defence question “how did you act on the validators’ comments?” with a document rather than a memory.
The pilot test
The pilot exists to produce two things: a reliability coefficient, and the discovery that item 14 is unanswerable.
| Decision | The usual Philippine practice |
|---|---|
| Number of respondents | Commonly 20 to 30, but your department sets it; ask |
| Who they are | People who resemble your respondents in the ways that matter |
| Whether they join the final sample | No. Excluding them is standard and protects you from a fair question |
| What you record | The coefficient, and every question a respondent asked while answering |
| Where it is reported | The validity and reliability section of Chapter 3 |
Run the pilot the way the real administration will run: same medium, same instructions, same time of day if that matters. A pilot administered face to face for an instrument you will later post online tests the wrong thing. Which medium suits your respondents is compared in the piece on Google Forms, Microsoft Forms and paper.
Reading the reliability coefficient
Cronbach’s alpha measures internal consistency: whether the items in one scale move together. Three things about it are worth knowing before you report one.
- It is computed per scale, not per questionnaire. If Part II measures one variable and Part III measures another, that is two coefficients. Running one alpha across both is meaningless and a statistics-minded panellist will say so.
- The threshold is your department’s. Most Philippine departments ask for at least 0.70 for a researcher-made instrument, and many accept 0.60 for a short exploratory scale. This is a departmental convention rather than a universal law, so get yours in writing and report the rule you applied.
- A very high alpha is not a better result. Above roughly 0.95 it usually means several items are asking the same question in different words, which inflates consistency without adding information.
If the coefficient comes back low, do not simply re-run it. Look at the item-total correlations: the item dragging the scale down is usually one that belongs to a different dimension, or one everybody answered the same way. Remove or rewrite it, then re-pilot. Removing items after the pilot changes your operationalisation table and the item numbers in your instrument, so update both. How to run the computation itself is covered in the walkthrough on analysing thesis data in SPSS, and the free alternatives are compared in the piece on SPSS versus Excel.
The validity and reliability paragraph, written out
The researcher-made questionnaire underwent content validation by three experts: a faculty member of the [college], a research and statistics faculty member, and a [practitioner role] from [the setting]. Each validator received the statement of the problem, the operationalisation of variables and the draft instrument, and rated it using the validation form prescribed by the [department]. Comments concerned the clarity of items [numbers] and the adequacy of items under the [indicator] dimension; the items were revised and [number] items were added accordingly.
The revised instrument was pilot tested on [number] [respondent type] who resembled the study respondents but were not included in the final sample. Internal consistency was computed per scale using Cronbach’s alpha, yielding [value] for [variable 1] and [value] for [variable 2]. Both values met the minimum of [threshold] set by the [department] for a researcher-made instrument. The signed validation forms and the final instrument are attached as Appendices [letters].
Two features of that paragraph are worth copying. It names what the validators actually said and what changed, rather than asserting that the instrument “was found valid”. And it reports the threshold it was measured against, which is the question a panel asks next.
What goes in the appendices
- The final instrument, formatted as respondents saw it
- The signed validation forms, one per validator
- The letter requesting validation, if your department requires it on file
- The pilot test reliability output
- The permission letters for the setting and, where applicable, the ethics clearance
Which permissions your study needs, and how long they take, varies sharply by field; the ethics and consent requirements for research on vulnerable populations are set out in the piece on ethics approval and consent. Where the whole validity and reliability section sits among the other Chapter 3 sections is in the guide to writing Chapter 3 section by section, and the matrix the validators need to see alongside the draft is the one built in the piece on operationalising variables. If your instrument is still being written rather than validated, the item-by-item build for an education questionnaire is set out in the questionnaire construction guide.
Start validation three weeks before you need it
Validation fails on calendars far more often than on statistics. Three validators, a revision round, a pilot and a re-pilot cannot be compressed into the week before data collection, and the group that starts late ends up administering an instrument nobody signed. Tesify drafts the instrument and the validity and reliability section together, keeps the item numbers and indicator names consistent after every revision round, and gives you the revision log the panel asks for. The instrument and the validators’ judgements remain yours.
Draft your instrument section free on Tesify
Frequently asked questions
Who can validate my questionnaire?
Most Philippine departments ask for three validators, usually a content expert in the subject, a research or statistics expert, and a practitioner from the setting. Some departments keep a list of accredited validators or require a minimum rank, so ask your research coordinator first.
What Cronbach’s alpha value do I need?
Most Philippine departments set a minimum of 0.70 for a researcher-made instrument, and some accept 0.60 for a short exploratory scale. It is a departmental rule rather than a universal standard; get yours in writing and report the rule you applied.
How many respondents should the pilot test have?
Commonly 20 to 30, though your department sets the number. They must resemble your study respondents and must not be included in the final sample.
Can pilot respondents be part of the final sample?
No. They have already seen the instrument, which changes how they answer it, and including them invites a fair question at the defence.
Do I need to validate a published standardised scale?
You do not re-establish its validity, since that is what publication and prior use supply. You do need to report its published reliability, confirm you are permitted to use it, and compute reliability on your own sample.
What if my reliability coefficient is too low?
Inspect the item-total correlations rather than re-running the same computation. The item pulling the scale down is usually one that belongs to another dimension or one everybody answered identically. Revise or remove it, then pilot again.
Should I compute one alpha for the whole questionnaire?
No. Compute one per scale. A single coefficient across parts measuring different variables does not describe anything.
What is the difference between validity and reliability?
Validity is whether the instrument measures the right thing. Reliability is whether it measures consistently. An instrument can be reliable and still measure the wrong construct, which is why validation comes first.
Do qualitative studies validate their instruments?
Interview guides are usually reviewed by experts in the same way, but reliability is not computed. Qualitative studies establish trustworthiness instead, through member checking, an audit trail and triangulation.
How long does the whole process take?
Plan for three weeks: one for the validators to return the sheets, a few days for revision, and a week for the pilot and the computation. Groups that plan for one week almost always slip.