PSYCHOMETRIC SCALE
A psychometric scale is comprised of items that measure a psychological variable. Psychometric scales are commonly used in employment settings to measure employees and potential candidates. They can be used to measure test-takers' knowledge, personality, or abilities.
BACKGROUND
SCALE DEVELOPMENT
I joined a team of four test developers to develop a psychometric scale. Our team was interested in studying the concept of networking. We were interested in how to measure networking abilities. We called this scale "Networking Know-How".
​
THE PROCESS
The job description includes the tasks, duties, functions, and responsibilities of a position.
TEST PREPARATION
This step involves combing through research databases to find potential a psychological variables and existing scales.
TEST SPECIFICATION
This step involves test developers making decisions about the specifics of the test. These decisions should take into account the use of the test and the context the test is intended for.
ITEM
WRITING
In this step, test developers write items for the measure based on the construct description and domains.
SUBJECT MATTER EXPERTS
Subject matter experts can give feedback on the written items. These experts can be used to rate how well each item assesses the construct.
The ratings should be collected from people who represent the population that will be taking the test in the future.
The test developers make decisions on the test type, length, item response format, and administration.
The number of items constructed should equal take into account the number of anticipated sample participants. This helps to conduct later analyses on items.
Once a psychological variable is chosen, the next step is to define that variable and determine its dimensions.
TEST PREPARATION
01
We decided to create a Networking Know-How scale. We defined it as "the ability to detect and correctly interpret social signals transmitted by other individuals in order to aid in networking for professional and/or organizational development in the workplace." The construct had 5 underlying dimensions.
TEST SPECIFICATION
We determined that the scale would best be used within a work setting. We also decided that the test would measure test-takers' typical performance in networking know-how. Based on these decisions, we determined that the test should be designed to be administered with ease and less training.
02
03
ITEM WRITING
​
We created an initial list of 32-items. Each item was written to represent one of the 5 domains (status detection, general signal detection/interpretation, intelligence detection, personality detection, and professional and organizational development).
SME FEEDBACK
We collected feedback from working professionals on LinkedIn. Since networking expertise was important to us, we restricted eligibility to experts that were working professionals with over 500 LinkedIn connections. 7 SMEs participated in our study.
​
Based on the feedback, we created cutoff scores for the items. We retained 17 items and eliminated 13 items from further consideration.
04
COVID-19 INTERRUPTION
​
*It is important to note that the scale development process took place over the course of a few months, however the process was halted abruptly due to the COVID-19 pandemic. The project was ended after the SME feedback stage.
COMPETENCIES
Decision Making
Writing
Teamwork
Analyzing & Interpreting Data
Critical
Thinking
Attention to Detail
REFLECTION
what was my overall experience?
This experience was beneficial to me because I gained knowledge and skills in item writing. Not only did I learn how to write items in a way that decreased error and increase clarity, but I also learned how to write items in a way that reflected the domains of interest.
Today, the lessons that I learned in item writing are vital to me in my current job where I am responsible for creating exam items and must create items to align with specific competencies.