OPW INTERVIEW — July 13, 2005 — Eric Straus of Cupid.com thinks matching based on personality profiling is B.S. Online dating upstart, True.com, would beg to differ. Mark Brooks interviewed True.com's Chief Psychologist, Dr James Houran, to get a second opinion on personality profiling for online dating.
Tell me about your background Jim?
I’m first and foremost a researcher, but I worked as a mental health counselor with both adults and children in a hospital setting for 6 years. That experience motivated me to complete my masters in clinical psychology. My focus was how attitudes influenced behaviors, as well as the interplay among imagination, cognition, and personality. After my M,A., I was hired as faculty at the SIU School of Medicine’s Department of Psychiatry There, my principal clinical and research interests involved advanced psychological testing, gender differences, mental health and wellness, as well as relationship quality. I worked with couples as well as families in both platonic and romantic contexts. I completed my Ph.D. became known for my work on the validity of psychological testing. Unfortunately, many test methodologies used today are outdated. As a result, much of what we think we know in the social sciences is probably skewed or downright wrong.
Why did you join True?
True.com gave me opportunity to do what I was already doing – a combination of research, clinical work, and public education – but on a scale that psychologists can only dream of. Usually in the academic community we worry about funding and finding large samples of research participants. Now I can conduct state-of-the-art online testing where funding is not a problem and which can make a difference in people’s lives on a topic that touches us all. So, I could not imagine a better job. Also, I’m able to help lead and pioneer the next era of online testing technology. Very exciting!
How can you be sure True.com’s tests are true and correct?
Luckily we don’t have to rely on opinion. There are professional standards for testing, which are outlined in a manual authored by the American Educational Research Association, the American Psychological Association, along with the National Council on Measurement and Education. One key element in test creation is in providing the test rationale. It’s also important to create and validate tests in the context they are to be used, that is, online tests can’t be created and tested offline as a pencil and paper measure and then transferred online. True.com uses Item Response Theory based measures. This is the same approach used by the GRE, MCAT and LSAT. We’ve known since 1960 that it’s superior to Classical Test Theory methods. All good tests have a test manual to show their basis, construction and validation… and preferably the research should be published in a peer-reviewed journal or at least independently audited by an expert in modern psychometrics. To my knowledge True.com is the only company that has done this. And, that angers me, because I hear people on TV saying, “Trust me, I have a test based in science, trust me.” Ask to see that science and you don’t get a response back.
Eric Straus, the CEO of Cupid, says personality profiling for online dating context is B.S. What do you think?
I would agree to some extent. The public should be skeptical! Public exposure to professional testing is limited. The public is more familiar with fun little quizzes in Cosmo or on entertainment websites. These tests are fun diversions but they are not the real thing. But, can we identify the variables that are associated with long-term compatibility? Yes, we can. Can we measure these variables in a reliable and valid way? Yes, we can. Can we use and apply these variables? Yes, we can. Published research has already shown this. However, what we’ve learned from advanced statistics is that things we thought we knew often turn out to be skewed or wrong. For example, eHarmony’s test says ‘birds of a feather flock together.’ The more similar a couple is the happier they will be in the relationship. That’s incorrect. “Similarity” is a relationship principle that academics have long known from research to be oversimplified.
I can’t make sense of eHarmony. They claim their test is based on an impressive study of 5,000 married couples. I can’t find that study anywhere. Last February, eHarmony published a general description of their services in a psychology magazine. Several ‘Letters to the Editor’ came in from professionals criticizing the company for unsubstantiated claims of scientific testing. eHarmony has yet to show the public anything of substance. The only attempt I’ve ever seen is a paper presented a year ago at a psychology conference. This paper reported research on two sets of married couples. One sample of marrieds met on eHarmony, while the other sample met in the real world. The conclusion was the eHarmony couples had higher levels of relationship satisfaction. As we reviewed their research it was apparent that the study and its conclusions were wrong. A rebuttal was published in the North American Journal of Psychology. This rebuttal showed that eHarmony’s own data contradicted the idea that similarity is the best predictor of relationship satisfaction. The media is also now starting to question the legitimacy of eHarmony’s testing. Test manuals or any proof of testing research are also absent from Perfectmatch, Tickle, and Yahoo Personals. Personality profiling tests can indeed be harmful if they are not based in legitimate research. Many people take these test results to heart and they influence life-changing decisions. If a company says they use tests based in science they have a legal and moral responsibility to consumers need to back that claim up.
How can a company give out enough information on these tests to win confidence, without giving away their secrets?
Independent auditors who are established experts in modern test construction and validation can review the tests and certify that they meet professional testing standards. Data can be reported to the public without divulging the specifics of matching algorithms. Take drug companies, for example, you don’t see the specific recipe for the chemical makeup of drugs or detailed explanations of how those drugs work exactly. Yet, drug companies routinely sponsor and publish research on their efficacy of their products.
So what can you tell me about True.com’s matching algorithm?
True matches people based on research concerning similar as well as complementary relationship variables. We take into account factors well beyond mere personality. Some of those variables include lifestyle preferences, world-views, sense of humor, social life, money management, and readiness to commit. Companies are free to use their preferred methods to pair individuals. Perfect Match uses a test derived from the popular Myers Briggs Type Indicator. We use what’s known as the ‘big five model.’
eHarmony and Perfectmatch make all their users take their profiling tests. Why doesn’t True.com?
Not all users want to take a compatibility test for many reasons. Users may be skeptical or just not want to spend the time. If people are antagonistic about being forced to take a test they won’t give accurate responses. Compatibility testing implies long-term relationships, and True.com realizes that not everyone is looking for that exclusively. Our research indicates that the bulk of the online daters do not want exclusively short-term or long-term relationships. We’re offer tools to help them find the relationship that is right for them. Our vision is to reduce the divorce rate by pairing people up for good relationships and marriages, as well as by preventing bad marriages in the first place by not prematurely pushing people down a path they’re not comfortable with.
How is True.com planning to help move the industry forward over the next 6 months?
You’ll see more of a push towards educating people on the power and value of good science-based products. True.com is committed to the safety and security of all our members and to give them the right tools so they can find the right person. We’re also working towards answering the second half of the problem. Once we pair people up… then what? We’re developing even more tools like supplemental tests and relationship guides to help people maintain and nurture relationships. Basically you’ll see more of a hand-holding approach. We’re uniquely positioned to help people with other aspects of their life. We haven’t painted ourselves into a corner to be simply an online dating site. Our tagline is “Live, Love, Learn”’ for a reason. Love is one element of a person’s life. We’ll also branch out to help people with self-growth and platonic relationships. So, you’re going to see us increase in scope.
