About This Blog

Here are our occasional discussions of new developments at Q-Assessor, current ideas, technical issues, and other topics of interest.

Blog Posts

Doing Q "the R Way"

One unfortunate result of online Q systems like Q-Assessor — and one that is frequently criticized by traditional Q practitioners on the Q Listserv — is that investigators can rapidly accumulate huge data sets. Instead of collecting perhaps a few dozen qsorts via in-person, on-paper administration, online data collection can produce hundreds and thousands of qsorts.

This is usually dismissed as “doing R and not Q.” The conceptual problem is that Q seeks to find relevant types of perspective in a population, not the prevalence of such types. All that is required for this, according to Q experts, is a relatively small number of responses, whereas a large number introduces noise and reduces the meaning of any analysis.

The practical problem with this is that the factor analysis algorithms that Q uses — notably those implemented in the canonical Fortran program called QMETHOD and its subsequent PC wrapper PQMethod — are not designed for efficiency since they don’t expect such huge data sets. Q-Assessor runs a direct implementation, within its development environment, of this original code. Trying to crunch through huge data sets with these inefficient implementations on a shared server basically exhausts available resources and causes processes to hang and eventually crash. Not acceptable.

...Read more

New Payment Options

In response to subscriber requests, we’ve increased the payment options for subscriptions to Q-Assessor. These options are described in greater detail on the subscriptions page. The two new options are these:

  • Paymode-X: This is a payment processor used by many larger institutions like universities and corporations. It imposes no fees on us either, so we welcome this form of payment. If your institution is picking up your subscription cost, they likely will insist on this mechanism. The information we provide on the invoice is sufficient for your finance people to make the payment to us via Paymode-X.
  • PayPal: This payment processor has a global reach and allows subscribers to pay via their PayPal accounts, if they have one, or via credit cards, if they don’t. Due to the low volume of subscriptions that want to pay this way, we have not implemented any automated PayPal functions like a “Pay With PayPal” button. Instead, if you want to pay this way, we will send you an invoice via PayPal that has links by which you can then log into PayPal and send the payment. This works well, though PayPal extracts a usurious percentage rate for “service fees” particularly if there is a currency exchange. If you are in a hurry and are unwilling or unable to pay via one of the above alternatives, then we will work with you to use PayPal.

You select your preferred payment option when requesting a subscription. If there are questions or issues regarding handling payments, we will of course discuss with you via email.

Note: because of problems we have had with bank wire transfers, this payment option is no longer offered.

Making Sense of Your Results

So you’ve successfully completed your study — all your participants have submitted their sorts and completed the post-sort interviews. You’ve munged your raw data using Q-Assessor’s online tools or you’ve downloaded your data and churned it with PQMethod or R. You’re now staring at some number of factors with reported eigenvalues and variances and whatnot. What does this all mean and how do you use it?

Well of course this is where the majesty or genius or mystery or bollocks (depending on your disposition) of Q stands forth. You can (and should) resort to the many obsessively detailed books and discussions out there where the vagaries of Q analysis and interpretation are scrutinized in densely-packed jargon. Since you’re doing a Q study, you will presumably want to report your results in similar fashion.

However en route, you will probably benefit if you spend a bit of time stepping back and thinking about your study from the simple first principles of Q. Recall that Q identifies types of people and their perspectives from the cross-correlations amongst their sorts. Each factor in your results thus represents a type of person and what they think. The task is to understand the nature of that person type by looking at two things: 1) the meaning of the statements that most define that type; and 2) the self-expressed features of people of that type as revealed in their answers to the post-sort interview. Once you understand the type, you can then provide it with an explanatory label that provides a useful shorthand as you compare that type to the others you discover and then interpret them all within the semantic arena of your study.

...Read more

Q-Assessor Now Provides Export to PQMethod

From its beginning, Q-Assessor has utilized the centroid factor extraction method used in PQMethod’s FORTRAN public domain source code. Recent Q-Assessor users have reported the need to use some of PQMethod’s other tools — notably the PCA method. Although Q-Assessor was intended to provide an end-to-end solution where this wouldn’t be necessary, we have realized the value in providing this option.

Now you will find in the “General Configuration” section of your study two links to export your statements and sort data in PQMethod-compatible files. (Note: these links only appear when you actually have something to export.) These links also are on the Statements and Responses pages, respectively.

Clicking these links will bring up a dialog in your browser allowing you to save these files to your local computer. The files will be named thus: SIDxxx.sta and SIDxxx.dat, where xxx is your study’s ID number used by Q-Assessor, and .sta is the PQMethod suffix for the statements file and .dat is the PQMethod suffix for the sorts file. (Note: if you’re using Safari on a Mac, the browser will forceably add a .txt suffix — producing SIDxxx.sta.txt and SIDxxx.dat.txt — in a Microsoftian display of unwanted “helpfulness.” You will have to manually remove the .txt suffix from the file names before PQMethod will be able to use these files.)

...Read more

Q-Assessor Now Supports Images In Statements

A recurring if infrequent request has been for Q-Assessor to allow investigators to place images within statements so that participants can sort graphical elements, not just text. Because of the variable and always too-limited screen real estate, this is a challenge. To be visible, images need to be large, but to fit numerous ones in a single view, they need to be small.

Q-Assessor now has what we think is a useful solution for this. Thumbnail versions of each image are placed in the statements when they’re being dragged around the screen, but large versions of each image show in a popup window when the participant hovers her cursor over the thumbnail — or taps the thumbnail on a tablet computer or smartphone. This allows the participant to quickly and intuitively switch between minimum and maximum views of each image.

During construction of the statements, an investigator uploads into Q-Assessor images for the statements. Q-Assessor automatically creates the necessary sized versions of the image and handles the popup magic when the study is deployed.

...Read more

Q-Assessor Now Supports Specifying Number of Extracted Factors

One feature some Q-Assessor users have requested has been the ability to specify how many centroid factors are extracted from the initial correlation matrix. We resisted adding this step, as there is no support for it in Brown’s book (cf the Factor Analysis section of chapter 4) and from a mathematical viewpoint, we can’t understand why this “feature” was added to PQMethod. Nevertheless, because this is a request that doesn’t seem to go away, we have added this capability.

The default number of extracted factors when you first analyze your data is the nominal seven, which is the “magic number” (per Brown) that is the maximum number of plausibly significant factors in a data set. A single popup menu option then lets you recalculate your results restricting the analysis to 1-7 factors. Thus you can restrict the search to a smaller number of factors and then rotate them as you like. Some users find significant value in this flexibility.

We remain skeptical of the value of this step, however, for this reason. The basic idea behind Q is that the cross-correlations amongst the participants’ sorts encode patterns of opinion/belief/understanding that the factor analysis process reveals. The number of statistically-significant factors thus relies within the data — not the sieve used to examine the data. If there are five significant factors, looking for seven will identify the five and show that there are two additional factors that are insignificant and can be ignored. If however the algorithms are restricted arbitrarily to look only for, say, three factors, then important information is lost. This is not part of how Brown explains this should be done, so we can only speculate why the FORTRAN code underlying PQMethod included it.

...Read more

Free "Test Drive" Use of Q-Assessor Now Available In Addition to Subscriptions

Because of the gratifying interest in Q-Assessor over the past year, we have reworked our policies regarding access to the system:

  • We have resumed open registration to the site.
  • New registered users can configure and conduct one test study using all of Q-Assessor’s features, limited only by the number of concourse statements, interview questions, and allowed responses.
  • We have improved the subscription request system, so that a simple form — inquiring the number of months desired, the projected start date, and a billing address — generates an invoice and signals to us of your interest.
  • A paid subscriber can create and conduct any number of studies during the subscription period, plus gain access to older, completed studies.
  • All users who tried out Q-Assessor during the prior beta-test phase can access (when logged in) their studies again, albeit in “read-only” mode. Thus they can retrieve their data and analyses if they like, just not restart those studies — unless they subscribe.
  • Subscriptions are denominated in one-month intervals. The cost is currently $500 USD per investigator per month.
  • Q-Assessor can accommodate teams collaborating on a study once one team member is the designated subscriber for the group. All group members can develop and deploy group studies, though they cannot run their own personal studies without a personal subscription.
  • Payment options at this point are bank checks drawn on US funds and bank wire transfers. We do not as yet accept other payment processors like PayPal and credit cards, but at whatever time subscription volume is significant to offset the substantial fees of those processors, we will reconsider.

We think that these policies will simplify and improve use of Q-Assessor. We’re always glad for your feedback.

You can read more about the subscription system and policies here,

Q-Asssessor Now Fully Internationalized

Developed primarily for use in the U.S., Q-Assessor previously hard-wired the English language in operational texts like sort instructions, button labels, dialog window texts, email links, and the like. Investigators could author their statements, categorical poles, and interview questions in their language of choice, but all these other elements persisted in English. Clearly this was an unsatisfactory situation.

Q-Assessor now supports full internationalization of a study in any of — at present — 22 other languages besides English. With a single configuration, investigators can match all of Q-Assessor’s own texts to the language they use in the content of the study.

We will continue to add more languages as demand for them arises. We also welcome any suggested edits to our translated texts from native speakers who find issues with our initial translations — which were generated by Google’s translation service.

Limited Availability to Serious Investigators

Because of recurring inquiries, we’ve decided to make Q-Assessor accessible to serious investigators doing projects with real budgets. The cost per month per user is $500 USD. If you are interested, please email us with details about your project and desired use. Others should investigate the various free tools out there.

Extended Beta Test Phase Drawing to Close

After a couple years experimenting with Q-Assessor, listening to feedback from users, and evaluating the opportunities for Q in various applications, we’re closing down the beta test phase. Thus new sign-ups are no longer being accepted.

It appears to us that Q-Assessor’s technologies have most potential in bespoke vertical applications and not in a general-purpose “online Q system” as Q-Assessor is currently configured. There just isn’t a sustainable market amongst the small, primarily academic Q practitioners out there. Q simply appears to be a too-little used methodology to support a web system like Q-Assessor. We have identified several specific applications where the Q technique might be quite valuable without invoking all the baggage that a specific identification with “Q” involves. We hope that Q-Assessor’s DNA will live on in other forms.

We will provide current users until 1 June 2013 to wind their current studies to conclusion and download their data. We are not supporting any new studies.

...Read more