Sunday, June 20, 2010

Smile/Frown/Cry : Technology is watching you !

This is supposed to be old news, but computer scientists have developed software that can guess what you are thinking by looking at your face. They do this by mapping your face onto a mesh or grid, and "evaluating the angle" for, the curl of your lip, the arching of your eyebrow, and sucking in of your cheeks.

Apparently, they are able to detect happiness, disgust, fear, anger, surprise and sadness, and that too with 85% accuracy. A computer vision expert in Amsterdam used software developed at the University of Illinois at Urbana-Champaign to design emotion recognition technology.

It seems they were able to analyze the Mona Lisa, and decided that she was 83 percent happy, 9 percent disgusted, 6 percent fearful, and 2 percent angry. While one could feel fearful and angry based on who the painter is and how well connected he is, I cannot figure out why she would feel disgusted. Unless of course she was thinking of something else while doing that beatific smile. You wouldn't know all this staring at the stuff in the Louvre, but she certainly comes across now as a spirited type.

Of course , they even analysed President George W Bush, giving an Iraq speech, and detected surprise , fear and sadness, which could have been a surprise to the late Saddam Hussain, and maybe , Cheney.

Michael Jackson was analysed from his mugshot emerging from his trial, and turns out he has 33% fearful, and angry and disgusted. I wish there was someone in his home who worried about that. Maybe things would have been different for him later on.

This kind of technology is supposed to be very useful, in detecting, say, when a car driver is feeling sleepy, and the software , say, installed somewhere in the car would immediately ring and alarm that says, Wake up, Stop, "let your wife drive", or whatever.

I have been wondering how we could use this technology here. We already have a technology to detect if a person is lying. Maybe some IT image processing analysis research type could patch the two together. That would be so useful.


You could analyse the Paksitani government spokesperson when he says, they have no clue on the Mumbai terrorist organizers. Every look and dither of an eyebrow would be caught, a quivering moustache would spill the beans, and the angle of the chin to the shoulder would tell other secrets.

You could analyse the Mumbai Municipal corporators, and officials, when they give details about expenditures on road repairs and water supply. Possibly a detection of fear and fibbing amidst the visible bravado, given the various folks breathing down their necks, bent at a secret angle.

Police fellows and honchos, could be analysed when accompanying their Minister bosses and you would be able to possibly get the truth on who is pressured by whom, who is lying and who is disgusted.

The applications of this technology to New Delhi and Parliament are too numerous. I am not too sure if the MP's would go for it. Facial analysis during secret ballot voting might indicate guys who are more than 60% fearful. Those whose faces indicate 80% wild happiness and 20 % complacency are probably those benefitting most in the horse trading. An analysis of the PM's face , while listening to a street-fight-like speech from Laloo Yadav, would possible reveal secrets about Yadav himself, given the general propensity for deals.

The judge in the Mumbai terror trials, would have inside information on the stuff ticking in Ajmal Kasab's brain. Bosses in companies would be surprised to know what their immediate juniors are thinking about them, while appearing to be their confidantes. Lalit Modi, BCCI and all the connected political folks would be totally exposed, and if someone could do mass analysis of public faces, they would find 90% disgust being the dominant emotion, the remaining 10% trust attributable solely to Sachin Tendulkar's place in the scheme of things...

Of course, since we are so good at defying the existing rules, and creating and finding loopholes, we will soon have folks in power, who talk like robots, not betraying any emotion, lest their secrets be found out. Say, two actors who can't stand each other putting on an act of great friendship, won't be able to fool anyone. All the fun will go out of watching the so called hitherto entertaining speeches in Parliament, with people calling each other the vilest of names with the straightest face.

On a personal level, I doubt if this technology will ever be installable. Folks meeting each other for the first time would have an interesting time analysing what the contraption says, vis-a-vis their own take. I wonder how it would feel to know that someone who is smiling so wonderfully at you is actually disgusted, and even angry with you.

And it wont be long before someone cracks this technology, and publishes details on what facial expressions to assume to mislead the contraption. I can see webpages dedicated to this, TV programs detailing the various technology versions, Microsoft coming up with OS's with this feature built in, and issuing patches after patches after some guy from Bulgaria hacks, and folks swearing by Linux saying, "this tells it like it is, no hacking possible" .

At the back of all this , will possibly be some entrepreneur related to someone in power, who gets the contract for the stuff, markets the contraptions installed across the country to sense these analyses, who will mint a fortune, and probably need no facial analysis as he goes guffawing 100%, on his way to the bank.

It seems this emotion recognition technology is being encouraged by companies marketing things, because they want to know what the consumers are thinking when they set their eyes on the merchandise etc.

And so , we humans, with a wonderful brain, amazingly designed, not yet replicable anywhere by man, will slowly become programable robots, in the effort to fool this emotion recognition technology.

The circle will be complete. Humans ---> Thinking machines ---->Unthinking humans.

Maybe 20 years later , they will design something to bring back behaviour with emotions.

I wonder if they will have any algorithm to detect "trust" , "love" , and "innocent laughter" on a face.....?


  1. Wow..that was a wonderful if frightening thought..That day is not far for sure..I saw a vedio recently of an age later when we will all have TV as a face and cell phones stuck to the ear. The phone and the TV fall in love but can't come near as they have this disturbance that is too over powering..So in the end , the same communication devices which were invented to bring us closer..are the same devices that prevents us coming closer...
    Which is actually the case, now..Because of Fb AND Twitter we forget to go visit our close friends instead prefer to be in touch virtually...

  2. 'It seems they were able to analyze the Mona Lisa, and decided that she was 83 percent happy, 9 percent disgusted, 6 percent fearful, and 2 percent angry.'...
    I do not understand that. How can she be happy and fearful and disgusted at the same time? Does it mean that the phenomenon of fragmented personality is as old as human civilization?

  3. When technology becomes intrusive - 99% hate, sadness, fear and disgust, 1% thinking why not 100%! Good questions you've raised there, in your post!

  4. If we could have the technology installed on your computers in such a way that we be able to read the expressions of anybody reading our blog!!!

    I think this might be used without informing the people that it is being used to catch the expressions...

  5. Don't you think even without this technology we are already able to spot it when politicians and governement officials lie to us bare facedly even without a twitch of their eyes? And in any case most of them have just one emotion - GREED!

  6. Loved this post! I wonder when we're all going to learn to look into each others' eyes and talk lovingly and honestly to each other without deceit and ulterior motives.

  7. Ever since algorithms have taken centre stage in our lives, the screens have had life and we have begun talking with our fingers....

    life has a different take ! With hard disks and soft ware ! And ofcourse, virus and anti virus ! Trust love to be driven by algorithms is indeed some prospect

    Super post. As usual

  8. You could check out some episodes of .

    Check out one of the full episodes.

  9. good one. Is there a way of evaluating MIL/DIL emotions for each other?i may need it sometime in the future as long as my DIl does not get her hands on it.

  10. sindhu Yes, these days its FB and twitter, and god knows what else we will blindly ape the west with...

    aativas I think fragmented personalities were always there. We just react differently now to them..

    RGB Thank you..

    IHM Would be interesting to match the analysis with the actual comments !

    USHA You took the words right out of my mouth :-))

    starry Thank you

    kavi Thank you...

    HHG LOL !