Future Directions for Q-Assessor

The Big Picture

Q-Assessor right now implements all of the standard features required to perform conventional Q-Methodology studies. However, there are several enhancements that might make Q-Assessor more useful. Particularly during this beta-release period, we are keen to receive input on these ideas.


We now have internationalized Q-Assessor. See details here. For historical purposes, we leave our prior discussion of this in place:

Because the content that subjects see during a Q-Assessor Session are all edited by an investigator as she configures the Study, the Statements and ancillary Questions can be written in any language. At this basic level, then, Q-Assessor already supports some degree of internationalization.

However, the instructions and page layout are all hardwired to English and its left-to-right text directionality. Further, all of the administrative pages are all in English.

So the options we are considering are:

  • Make all subject instructions for Q-Assessor sessions editable — defaulting to English but modifiable by an investigator to any other language.
  • Make text directionality a configurable option per Study — defaulting to left-to-right but settable to right-to-left.
  • Revise the entire web site to accommodate multiple languages so that all visible content would be settable to different languages — which would require major assistance from other parties to provide translations for all the administrative language we use — likely an infeasible proposition.

We presume than any investigators or other subscribers will be fluent in English — even if their target subjects are not — so that the most appropriate step to add internationalization to Q-Assessor would be to make a Session effective in other languages, while leaving the basic administrative tasks (configuring the Study etc) to run in English.

What do you think?

Note: we’re still interested in what you think, particularly how useful this feature is to you.

Statement Augmentations

We have now added image-based Q sorts to Q-Assessor’s capabilities. You can play with a demo here and read more about this feature here.

All the regular literature regarding Q-Methodology utilizes simple text statements as the conceptual items that subjects rank. Prior to the computer era when slips of paper with typed words were all that could be used, this limitation was unavoidable. Now, however, a computer-based Q system could augment text with additional types of information:

  • Video clips
  • Sound files
  • Potentially other sorts of attached files

These would presumably be presented to the subjects as clickable icons displayed adjacent to text for each Statement, in the case of video or sound, or thumbnail images that open up a larger image, in the case of images.

Q-Assessor’s Statement system could probably be enhanced to accommodate these additional features, though there are major interface design issues. Most obvious is the fact that these days the size of the screen the participant is trying to do the study on is entirely unpredictable and can vary from a tiny smartphone to a huge monitor. Manipulating the sort elements is hard enough even when they are limited to relatively short texts. It’s quite unclear how images could be displayed while preserving the ability to manipulate the “statement” to which they’re attached. Even if this and other problems could be solved, how important would these features be?

Client/Subscriber Customization

The branding of the Q-Assessor site (look and feel, logos, etc) are distinctly Q-Assessor’s. However, some clients/subscribers may want their studies to appear to originate from their own systems. Corporate clients may particularly prefer this.

There are several levels to which this could be taken:

  • Add an optional client logo to the Session layout that would appear on the initial introductory page.
  • Add an option to replace the Q-Assessor logo on each Session page with a client logo — which would presume that the client logo would have certain proportions or could be modified to more or less the size and shape of the Q-Assessor logo, so the rest of the page layout will fit.
  • Add an option to modify the entire color as well as logo to look like a client site.

The ultimate end of such customization would be for Q-Assessor to license an independent, “private label” installation of its technologies to a client organization. This would enable the site to have a client-specific domain name instead of “q-assessor.com” and would permit all sorts of other customizations — which The Epimetrics Group, LLC, would perform.

Which of these options seem important? Which would you and your organization look for in order to want to use Q-Assessor’s technologies?

Please email us your ideas or post them in the Comments section below. Thank you!

Created: August 04, 2010 18:00
Last updated: December 01, 2014 05:15


Jonathan Sibley — 09 November 2010 - 02:59

If there isn’t a way to import a set of items to create a q-set, that would be helpful.

Some q-sets define a meaning when placed toward each pole (e.g., the Psychotherapy Process Q-set for some items). It would be helpful to have a way to specify this and cycle through the meanings when placed at each pole or in the middle.

If there isn’t a way to clone an existing study and then modify the cloned study, that would also be helpful.


Stan Kaufman — 09 November 2010 - 14:58

Thanks for your suggestions, Jonathan. Adding the ability to import existing statements is definitely possible, though the danger with such a mechanism is that users will fail to use it correctly and end up having to spend more time cleaning up their import than it would have taken to copy&paste each statement into the existing interface one at a time. Q-Assessor has for now adopted the “keep it simple” approach to minimize the chance that users will make errors.

A “clone the study” feature is also eminently doable, though the existing library mechanisms accomplish nearly all of this. If you create a second study, all of the statements and interview questions from your first study are listed in the pages where you manage them for the second study — and you can simply move them in (or not), which takes a very short time. If you’d start with a clone, you’d have to spend as much time going through to remove the unwanted statements and questions as you would adding the ones you want with the current mechanism. Of course, if the two studies were indeed to be absolutely identical (which others have told us is rare), then a clone function would save some time. Otherwise, it’s not clear that it would. Nevertheless, this is a good suggestion and one that would be easy for us to do, so we likely will.

I’m afraid that I don’t understand your “cycle through the meanings” suggestion. Please clarify/expand.

Thanks again for your suggestions and feedback! Much appreciated!

Jonathan Sibley — 16 November 2010 - 14:59

One use of q-methodology, although I believe not mainstream, has people sort from “most characteristic” to “most uncharacteristic” of something they are observing (e.g., a therapy session, a person).

If the characteristic meaning were something like “client appears serious” or “person is hairy”, the researcher might have documented how to interpret the item when placed as highly uncharacteristic “client appears light-hearted or giddy” or “client is bald” to help the sorters distinguish uncharacteristic from neutral (may be true, but not significant).

Perhaps this deviates too much from the sort of use you are thinking of, but I hope it clarifies what I meant.

It might be that a solution would be to allow for a separate, longer description of the item (by hovering, right-click, etc.) which could simply be more detailed, or cover cases such as “when placed as uncharacteristic…” while keeping the actual item description short for easier sorting.

Is this clearer?

Stan Kaufman — 16 November 2010 - 16:26

The categorical “poles” for the initial sort can be set to any phrase. Thus you can have study participants sort the statements into three sets having arbitrary meanings — though in customary Q practice, these always have some meaning along a single continuum.

I gather that you want to assign these three initial sort destinations quite different meanings, so that the problem is communicating to participants what those meanings are. There currently is a limit to the length of the description for these three categories primarily because they need to fit into the layout.

Putting crucial information into a location — like a “tool-tip” that appears only when the user hovers the cursor over another object — is usually a bad idea since it forces people to hunt for that information and remember it from when they do find it until when they use it. If the categories are so unclear that they need a long explanation, I think that they are probably not well chosen and will be less likely to produce useful results.

In contrast to the initial sort pole/category descriptions, the statements themselves can be as long as you like. Most Q studies have used only short one-line statements, but Q-Assessor lets you put in multiple paragraphs with formatting — so this is where you could communicate more obscure meanings if that is what you want. The only penalty for more text is that more scrolling around the layout is required — since there is an inherent limit to the amount of information that can be displayed on screen at any moment.

I hope that this addresses your issue. Please comment further if not, and thanks again for your feedback!

Jonathan Sibley — 16 November 2010 - 18:25

Hi Stan,

Thanks for being so responsive.

I am talking about the definition of items rather than about the labeling of the poles.

As you say, there is a balance between longer item descriptions that might be helpful to the sorter and usability during the sorting.

With a large qset (we may have close to 100 items), having a way to toggle between a shorthand description and a longer description for each item could be handy.

By the way, have you tested the usability of q-assessor with a large qset? I haven’t had a chance yet.

ValĂ©rie Dory — 24 September 2012 - 15:14

Hi Stan,
I contacted you probably a year ago about the issue of language. I’ve conducted my first study without Q assessor but for my next one, I’d really like to do it online for feasibility reasons. Any progress on the internationalization issue?

Stan Kaufman — 24 September 2012 - 16:20

Internationalization appears to be an important feature, as most interest in Q-Assessor has been coming from Europe lately. The next version of Q-Assessor will implement a more fully-implemented form of language support — beyond the current ability to phrase the study itself in an arbitrary language. I don’t have a predicted time frame by which that will be done, though.

To comment, you must log in first.