Q-Assessor Sessions

The Big Picture

Q Methodology has subjects sort a number of statements into a hierarchy of levels of opinion defined along some axis of “most negative” to “most positive.” Precisely how the do this sort is crucial to Q; participants must select certain numbers of statements for each of the different levels along this axis. Most commonly, participants accomplish this via two sequential sorts — first into general tripartite separation into, say “Disagree,” “Uncertain,” and “Agree” categories, with a subsequent second finer-grained ranking. During the initial rough sort, the participants begin to formulate their own distributions, which they finalize in the second sort.

The nuanced complexity of these tasks has traditionally been accomplished by the investigator sitting with the participant to provide directions through each step. This immediacy imposes significant logistic/time/expense costs on Q studies. However, web-based systems offer an obvious potential remedy because they can allow Q studies to be done in automated fashion at a distance.

A number of online Q-sort systems have been developed. We happen to think that Q-Assessor is the most successful of them. Here’s how we do things.

Introducing the Study

Q-Assessor makes it easy for investigators to present background information, inducements, and any other information that will help participants via the email invitations and reminders. Similarly, once a participant clicks on the “participate in the study” link in an email, they are taken in their web browser to the Study’s introductory page, where the investigator can provide even more detail if he likes.


This is a real example of such an introduction, one that was approved by the investigator’s Institutional Review Board. While many studies may not require this level of demonstration of consent from participants, Q-Assessor solicits (in the case of enrolled subjects) in essence a double-layer of informed consent — as the subject must first click the “I want to participate” link in the invitation/reminder email, and then click the “I want to start” button before they can perform the study.

The First Categorization Sort

Once the participant clicks the “Let’s Get Started” button, they are taken to the first rough categorization step.


In this phase, each statement is displayed in the evaluation box on the right side of the page. The participant declares whether they agree, disagree, or are uncertain by clicking on the appropriate button. Each button then moves the statement over to its respective section in the right side of the page. The subject can reassign any statement at any time simply by clicking the “Reassign” button next to the statement, which returns it to the evaluation box so the subject can assess it again. Q-Assessor updates the number of remaining statements so the participant can always monitor her progress through the study.


Once the participant has assigned all of the statements, she can then click the “Done” button to move to the next step, or can back up and reassign as many statements as she wants.

At this point, it is probably worth pointing out the reasons we have elected not to use a “drag-and-drop” interface for this process. We tested the various Javascript libraries that have been developed to implement on-screen effects of this sort, and we found that none of them worked consistently enough across various browsers (which vary considerably in how the implement Javascript).

For simple drag-and-drop tasks, these tools work quite nicely. But when rather large and variable numbers of statements much be managed and subjects need to be constrained to “do the right thing” in a tightly-regulated fashion, we simply found that our more straightforward “click buttons to move” interface was clearer and more reliable. We have found this to be particularly true in the more complicated second phase of the sort process.

The Second Ranking Sort

The second level of the sort begins when the participant clicks the “Done” button of the first phase. This creates the ranking structure for the statements on the left side, and the subject then sequentially moves the statements from the right side back to the left. The participant is forced to pick the top “most agree” statements (in the example shown here, the top two — but this would differ with different structured studies) followed by the bottom “most disagree” statements.


In this first screen-shot, the top agree statement has been assigned. The participant has selected the second most-agree statement — which inverts the appearance of the statement to make its selection absolutely clear. The participant then clicks a “Move Here” button at some point in the structure (here there is only one choice because this is the first step) to move the statement.


In this second screen-shot, all the “most agree” statements have been assigned, which opens up the “most disagree” slots. Here one has been assigned, and a second one has been selected, awaiting assignment.


Once the “most agree” and “most disagree” statements have been assigned, the rest of the ranking structure opens up. Subjects can place all the rest of the statements into the gradations of agree-disagree in any order they choose. Each move is made via the “click to select then click to move” interface — which we have found to be more reliable, deft, and understandable than drag-and-drop, given the complex topography of the sorting grid.


Once the “most agree” and “most disagree” statements have been assigned, the rest of the ranking structure opens up. Participants can place all the rest of the statements into the gradations of agree-disagree in any order they choose. Each move is made via the “click to select then click to move” interface — which we have found to be more reliable, deft, and understandable than drag-and-drop, given the complex topography of the sorting grid.


When all of the statements have been ranked in the sort bin structure on the left, the subject can click the “Done” button to submit the final sort results, or she can click on “Remove” buttons to reassign statements.

At any point in this process, participants can revise their choices simply by clicking the “Remove” buttons by a statement they want to reassign. This places the statement back in the general categories on the right side, and opens up the slot on the left where a new statement can be selected and moved.

Clicking the “Done” button sends the participant’s final sort results back to Q-Assessor, where they are saved into the database. If the investigator has not added any ancillary questions to the Study, the participant receives a simple “thank you” page. But if there are additional questions, the participant is automatically taken to the next “interview” phase.

The Interview

This phase allows investigators to ask participants essentially any question they like. However, at least some of the questions ordinarily ask about why the participant made choices as they did during the sort.


Q-Assessor presents the participant with her choices on the left and the additional questions on the right. The participant can answers as many of those questions as she likes, and when she’s done, she clicks the “Submit Answers” button at the end of the questions.

The Conclusion


When the participant submits the question form back to Q-Assessor, the results are also stuffed into Q-Assessor’s database, and the participant is directed to a simple “thank you” page. Q-Assessor also sends out an automatic thank you email (as configured by the investigator) when enrolled subjects are being studied.

Created: August 04, 2010 17:24
Last updated: September 22, 2010 20:58


Comments

No comments yet.

To comment, you must log in first.