Facebook Announced A New Feature Where It Listens And Identifies Music And TV Content

Before and during the Memorial Day holiday, I was busy with work and family events. Perhaps, you were busy too and missed this. Just before the holiday, Facebook announced a new, optional feature where it will listen and identify whatever you are listening to while typing status messages.

If you have used mobile apps like Shazam, then you know how technology can easily identify the name and artist of music. Facebook wants to take the technology further by identifying the background content (e.g., music you are listening to, movies or television show you are watching) while posting messages to Facebook. Facebook's announcement pitched the new feature as:

"You may have seen a friend post a photo after a tough workout with a “feeling proud” icon. Or you’ve seen your friend check in at a coffee shop “drinking an iced coffee.” In the last year, people shared more than 5 billion status updates... we’re making those conversations quicker and easier by introducing a new way to share and discover music, TV and movies. When writing a status update – if you choose to turn the feature on – you’ll have the option to use your phone’s microphone to identify what song is playing or what show or movie is on TV."

Unlike the Shazzam app (which the user initiates), once you turn on the "Identify TV and Music" feature, it will operate quietly and identify whatever is playing in the background when you post messages:

"If you leave the feature on, you will see the audio icon move and attempt to detect a match when you’re writing a status update. No sound is stored and you’ll always get to choose whether you post to your friends... if you choose to turn this feature on, it will only use your microphone (for 15 seconds) when you’re actually writing a status update to try and match music and TV... when you write a status update, the app converts any sound into an audio fingerprint on your phone. This fingerprint is sent to our servers to try and match it against our database of audio and TV fingerprints. By design, we do not store fingerprints from your device for any amount of time."

It's important to read Facebook's words closely. It says it won't store the music, TV show, or movie you are watching or listening to. It does store the status message you authorize about the background content. That means, the feature will record the name or title of the show/music, the artists, along with the date, time, your geolocation (e.g., GPS) data, and probably other relevant metadata. It needs these metadata elements to create a status message for you to post to your Timeline.

Based upon its matching algorithm, the message includes an excerpt of the music or show, since Facebook assumes that your friends may want to purchase the music or video item. In this way, Facebook can sell more advertising to its corporate sponsors; where once again Facebook members are the product. The feature allows Facebook to analyze its members' actions and build a a more robust activity profile. For example, people with certain demographic characteristics (e.g., age, sex, students, rural residents, etc.) or in certain locations, listen to XYZ music and/or watch a certain genre of television shows while posting status messages. And, Facebook can associate certain moods or feelings in your posts to the moods or feelings in the background content (e.g., music, movie, or TV shows).

The Naked Security blog by Sophos reported:

"When it initially announced the eavesdroppish new service, Facebook didn't say anything about listening in on background noise, including private conversations. But this week, Facebook's security head honcho, Gregg Stefancik, filled in that gap. Stefancik, head of security infrastructure for the very-data-rich, o-so-good-at-data-mining social network, explicitly told journalists that the new audio feature does not snoop on users and does not record conversations... The raw audio never leaves the phone, Stefancik said, while the data about the match is only stored if a user opts to post it:.. The app can't identify background noise and conversation before the feature is enabled."

I guess that this new feature will be a benefit to consumers who want to share easily, quickly, and automatically without having to do anything. You literally won't need to lift a finger. It seems wise for consumers to give a new feature like this a lot of thought and consideration before turning it on. Why? The background content (via the authorized status messages) will be associated with your profile.

Maybe, the background content is the television you've left on because you're home alone, not really watching it, and want some noise in your home. Maybe you are simply in the same room with a family member or friend who is watching TV, movies, or listening to music. Their selections identify their choices, not necessarily yours. Maybe you are in a shopping mall and muzak is playing in the background. Maybe the music playing is from an advertisement on television. Maybe Facebook's matching algorithm was incorrect.

My point: the background content may have nothing to do with your profile, but it gets recorded and associated with your profile anyway. The background content may be items you'd ever select nor buy, but Facebook would assume so. Then, who is right? Who knows more about you and your habits: you or Facebook?

I see this new feature as extremely invasive and problematic. I know my profile better than any social networking service, and remaining in control is important to me. Facebook addressed the issue of control in its announcement:

"... this feature is completely optional. If you don’t turn it on, we won’t use your microphone to try and match TV or music when you write a status update. If you do choose to turn it on and later decide it’s not for you, you can easily turn it off at any time."

This implies, if you want to delete any background content from your Timeline, then you would do so consistent with the capabilities and limitations of the current Timeline system. Does a user really have effective control? I don't see how any consumer can verify that Facebook uses the new feature to comply with its promises (e.g., don't record conversations, 15 seconds, identify only TV/music, etc.). The announcement did not specify how accurate the feature is. If it incorrectly identifies some background content, and you authorize that status message then an error has been introduced to your profile. Facebook member may not know the background content identified.

I'd like to see Facebook explain more about its matching algorithm. How accurate is it? Does it match any song or music? Does that include music in TV advertisements? If so, then, the matching algorithm could identify what commercials you have viewed. What about radio? The announcement didn't say anything about radio. People listen to traditional radio and satellite radio. What matching is done then?

This technology confirms what a lot of people have been worried about with surveillance by government spy agencies: the ability to remotely control the microphone in your smart phone or mobile device, and monitor what you are doing, listening to, and watching. Since Facebook already records and archives everything (including deletions) you type into the status message box, the two features combined provide the social networking site with very strong capabilities to determine what you are  thinking, feeling, and considering -- not just what you typed in the status message. That is very strong personal content.

It's also very creepy stuff, in my opinion. Spy agencies must be looking at this and wondering: if Facebook can do this, we should be able to do this, too. If I operated a Web design service that was a front for a spy agency, I'd want to use an app like this.

I wouldn't want any mobile device in my pocket running an app like this. Nor would I want to be around people using an app like this; especially in business meetings. Yes, this upcoming Facebook feature reminds me a lot of Google Glass. Very invasive for people who value their privacy.

What's your opinion of the upcoming Facebook feature? Is this more or less invasive than government spy programs?