Researchers Claim USB Security Is Broken
Oregon Attorney General Filed Lawsuit Against Makers Of 5-Hour Energy Drink

Dating Service Admitted Performing Online Experiments Using Its Customers Without Notice And Consent

In a wide-ranging and arrogant blog post to promote his new book, Christian Rudder, the co-founder of the OKCupid dating website, described several experiments the site performed on its customers:

"... chose to celebrate the app’s release by removing all the pictures from OkCupid on launch day. “Love Is Blind Day” on OkCupid—January 15, 2013... But by comparing Love Is Blind Day to a normal Tuesday, we learned some very interesting things. In those 7 hours without photos: people responded to first messages 44% more often; conversations went deeper; contact details were exchanged more quickly; in short, OKCupid worked better..."

In another experiment, the OKCupid site changed its display parameters telling some users with poor matched that the matches were excellent and the reverse:

"... the “match percentage” we calculate for users is very good at predicting relationships. It correlates with message success, conversation length, whether people actually exchange contact information, and so on... To test this, we took pairs of bad matches (actual 30% match) and told them they were exceptionally good for each other (displaying a 90% match.)† Not surprisingly, the users sent more first messages when we said they were compatible..."

In effect, OKCupid lied to its users about their compatibility matches. You can view the message OKCupid sent to users after the test:

"Because of a diagnostic test, your match percentage with XXX was misstated as 31%. It is really 91%. We wanted to let you know."

Diagnostic test? That explanation doesn't sound entirely accurate. It sounds like some type of error-checking routine, and not a true admission or notification of an intentional marketing test. Were customers offered refunds for "misstated" compatibility matches? If I were an OKCupid customer, I'd demand a refund as the service didn't seem to deliver what was promised.

Rudder's blog post provides plenty of statistics about what the company learned from its live tests with customers. Rudder's blog post gave the impression that the ends justify the means -- that the wealth of data the company collected justified the test approach. Rudder also defended Facebook, after that social networking site had been criticized for performing experiments on its members without notice nor explicit consent:

"We noticed recently that people didn’t like it when Facebook “experimented” with their news feed. Even the FTC is getting involved. But guess what, everybody: if you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work."

All websites? For sure, at least Facebook and OKCupid.

I am sure that OKCupid, like Facebook, protected itself with the appropriate legal language in its terms of use and privacy policy. The broader issue I see is one of respect. Some social networking websites, or at least their managing executives, seem to no longer view it as appropriate, respectful, or necessary to notify users affected by online experiments and tests.

I am no prude. I fully expect websites to explore and implement new services, content, and functionality. How one does it matters. The ends do not justify the means.

During the last 20 years, as a usability professional I have built dozens of websites in a variety of industries: telecommunications, petroleum, travel, banking, insurance, higher education, food, consumer packaged goods, and more. In all instances, we used a variety of standard, proven test methods to collect users' opinions and reactions to proposed website features and functionality. Usually, we started by asking users -- customers and prospective customers -- what they wanted in the site that they couldn't get today. Many users will tell you. Man users are happy to tell you.

Frankly, it makes sense -- time wise and financially -- to build features that users want. No matter how curious OKCupid executives may be, I highly doubt that the site's users wanted the service to lie to them about compatibility matches.

After compiling a list of requested features (e.g., content and/or functionality), we tested implementation approaches... not on the live site, but in usability sessions with mockups or with prototypes. That approach builds users' trust. Many users appreciated the opportunity to view and comment on new features before those features are added to the live site.

In other cases, we used focus group sessions to uncover users' needs and to explore their reactions and attitudes. We often used rigorous questionnaires (sometimes in combination with other test methods), so that we could analyze the results later. In some instances, we included survey forms with the live site.

My point: we never adjusted the live site's core functions and contents without notice. We didn't add new features to live sites until after all testing was finished, the new features were built, and all "bugs" or code glitches were fixed. Anyone experienced with website development knows that it takes time to get the bugs out. When you add new features, they often affect, or break, something else -- unintended consequences.

Users' trust and reliability are critical. Frankly, we trusted uses to ask them what they wanted. We trusted users enough to inform them of tests. We respected users enough to compensate test participants for their time. We respected users enough to acknowledge that some have a right to not participate in tests.

After reading Mr. Rudder's blog post, I began to wonder how trustworthy the OKCupid site really is. The good: OKCupid executives are curious, want to continually improve their site, and act quickly. The not-so-good: curiosity and acting quickly aren't enough. Users rely on the live site to to operate as advertised and promised. Deviations from that with unannounced tests that users can't opt out of, erode users' confidence and trust.

All of the tests Rudder described could have been performed with standard testing methods, some of which I have described above; without directly changing the live site. Maybe the OKCupid executives aren't aware of or wanted to skip the costs and time of traditional testing methods. Maybe speed is their primary goal. In their rush to improve things, Mr. Rudder and his executive team seem comfortable to unnecessarily risk consumers' trust and respect.

If this is the current state of social networking sites, then the industry has fallen. It has moved beyond simply collecting, archiving, and analyzing massive amounts of consumers' personal information for advertising revenues. It also operates arrogantly: making any changes they please to live sites, while ignoring users' trust nor respect. That's not something I look for in a site. Nor will I buy Mr. Rudder's book.

What are your opinions of OKCupid's tests?

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Chanson de Roland

It is a new day, where privacy and the human dignity that goes with it are of yesterday and where the law does not grant users any proprietary rights in their own information. So today there is nothing to make what OKCupid did either immoral or illegal, at least not illegal if it included the requisite disclaimers in its Terms of Service, which nobody reads and which most could not understand as to its meaning and its legal effect even if they did read it.

So welcome to the new day, where we share and, indeed, must share everything and where the private self may extend no further than one’s inner thoughts, if even that far, and where the public self, which really isn’t all and who we are, is the only self that is permitted and is the only self that one really has, and that self belongs to everyone but you. So you don’t exist any more a distinct and precious individual but only as node of information for the profit and benefit of others and subject to their control.

Rich F

I am a user who believes that what they've done is unethical. And I find their cavalier and unapologetic stance on this disturbing.

I first became aware of their questionable ethics a couple years ago on April Fool's Day. They added a new option which appeared when a user logged on. If a user clicked on it, it appeared as if you were viewing the inbox of one of your contacts, so you could see all the messages they were receiving from other users. They fabricated those messages, and they were quite offensive. For example, it appeared that one of the women I had been communicating with had been exchanging nude photos with some guy, and he was commenting on them. There were many messages of this nature to be viewed. Even if it had been clear that this was an April Fool's Day prank (which is wasn't -- they made it look very real), this still would have been an unprofessional and offensive thing to do. The fact that people could have and probably did take this seriously means that potential relationships could have been destroyed.

Now they are at it again, totally insensitive to the effect they have on people's lives. This is not a matter of whether they are allowed to be doing research, this is about what that research entails. It's unethical to screw with people's lives, lead them to believe that someone is a good match when they are not, that they are a bad match when they could be a good one, to change the content of what a user has put in his or her profile, to read their personal messages for the sake of their so-called research.

Yes, it is a new day, but that doesn't mean this type of thing is not immoral. And it's all the more reason why people have to stand up against it.

George

When social networking websites act like this (e.g., OKCupid and Facebook), it's abusive behavior since we consumers provide the content (e.g., personal information, likes, messages, photos, video, etc.) that make their social websites operate... content these social networking websites NEED to operate. A reader shared this article:

The Problem With OKCupid Is The Problem With The Social Web
http://kottke.org/14/08/the-problem-with-okcupid-is-the-problem-with-the-social-web

And, these social sites operate, thanks to their users' information, quite profitably. Facebook's worldwide revenues are over $7 billion, almost as large as the National Football League.

We are truly starting to see the dark side of the Web.

George
Editor
http://ivebeenmugged.typepad.com

Rich F

I stopped using Facebook years ago as soon as I detected what they were up to. For reasons like the ones we are discussing, I do not use any social networking sites other than dating sites and I'm just about ready to dump this particular one. Let's see... dishonest, arrogant, insensitive, immature, and unprofessional -- I wouldn't take that from a date and I'm not going to take it from a dating site either.

George

Rich F:

Thank you very much for sharing your experiences and insight. I agree with you. The behavior was unethical. The unapologetic and arrogant attitude was definitely disturbing.

George
Editor
http://ivebeenmugged.typepad.com

BobbyG

people seem to be forgetting the OkCupid IS FREE TO USE and if you choose to sign up you do so accepting THEIR TERMS AND CONDITIONS. nobody is forcing you to use this service at all. and yes, some of the methods are questionable, but they are not illegal and the intentions were probably good ones and THESE EXPERIMENTS WERE PERFORMED ON USERS THAT VOLUNTARILY SIGNED UP FOR A FREE SERVICE. I haven't read their T&O but their's probably something in there that gives them permission to do this kind of thing. People who get very upset about this kind of thing should completely read any terms and conditions that they accept and understand. AND STOP COMPLAINING when something free doesn't do things the way you like.

Steve

Let's be clear. If the service is free, YOU are the product.

George

BobbyG and Steve:

Thanks for sharing your opinions. Some reactions to your comments:

Free? There are paid features, as the OKC Legal Terms state:
https://www.okcupid.com/legal/terms

Last I checked, there is no law preventing customers of a free service from commenting or voicing their opinions about that service. Open-minded business executives would welcome such feedback to improve their service; not blame the customers that keep them in business.

As commenters on the OKC blog post said, the unannounced changes screwed with people's lives. And, ethical research matters. There are several professional research guidelines OKC could have followed. You might read:

What Is Ethics in Research And Why Is It Important?
http://www.niehs.nih.gov/research/resources/bioethics/whatis/

OkCupid Experiment May Violate FTC Rules on Deceptive Practices
http://www.reuters.com/article/2014/07/30/us-internet-okcupid-idUSKBN0FZ03K20140730

The article mentioned Rudder saying that OKC had received only 10 complaints. Huh? There seem to be lots of complaints in the comments thread on his blog post.

George
Editor
http://ivebeenmugged.typepad.com

Ari

I think it's one thing for websites with large volumes of data (like OKC and Facebook) to conduct social experiments, and quite another for individuals to do so using their profiles. This blogger essentially came to the same conclusion as OKC that photos matter much, much more than the write-up of your profile. http://middleagedteen.blogspot.com/2013/07/a-social-experiment.html

Vinny

I have mixed feelings on these 'experiments'.

On the one hand, the "love is blind" experiment is relatively harmless, other than to show that, regardless of what people may say, both men and women use vision as the first "filtering" criteria of mate selection. Or, as I've often said to my friends, "you're not going to sleep with someone who disgusts you". That is, unless you have zero standards or no self-respect.

On the other hand, I find other "experiments", such as changing the percentages, to be akin to lying. That type of behavior steps over a line IMO. Lying to your customers is simply unacceptable.

The comments to this entry are closed.