Image-based Personality Profiling
Conscious vs Unconscious Mind
In the past year, I have been thinking about the possibilities to apply Deep Learning in order to gain more insights about the observed persons. When speaking with a person, it is worthwhile to observe if the face is telling the same story as words. Sometimes the facial expression diverges from statements. As we have been taught to rely on statements, sometimes our rational mind suppresses the evolutionary instincts that have been stored in our unconscious mind to raise attention in case of a hostile environment.
Our modern skulls house a stone age brain.
Sometimes, during the video conference calls, I try to mute the sound for a brief period and try to figure out what the face is telling. Unfortunately, in some cases, I got the sense that the spoken words were deceiving the intuition. In order to apply machine learning for the tasks that are easy to interpret intuitively and hard to explain rationally, I have been looking for the existing studies and datasets to train the neural network.
In 2018, a paper Deep neural networks are more accurate than humans at detecting sexual orientation from facial images attracted a lot of critiques. Although it has been published by a credible journal, I believe that unconditionally relying on a single study might harm many people, especially if the profile serves as a basis for potential life-changing decisions, such as legal case outcomes. Personally, while trying to figure out how to train a reliable model to be applied in the field of profiling people, I am walking on the uncharted territory of social sciences where the hypotheses are proven with natural experiments instead of laboratory experiments as in the case of natural sciences and the sloppy formulation of the natural experiment might lead to harmful perception bias. Up until now, I have mainly dealt with laboratory experiments that are much safer in this respect; you set up the environment where the hypothesis is proven by reproducible outcomes.
Democracy at Stake
The insights, available from the data, collected from social network platforms, however, have been proven to be very accurate. In March 2018, multiple media outlets broke the news of Cambridge Analytica’s business practices. The New York Times and The Observer reported that the company had acquired and used personal data about Facebook users. Shortly afterwards, Channel 4 News aired undercover investigative videos showing Nix boasting about using prostitutes, bribery sting operations, and honey traps to discredit politicians on whom it conducted opposition research, and saying that the company “ran all of (Donald Trump’s) digital campaign”.
Recently, I have watched Patrick Fagan’s talk Decoding Clothing. As there has been claimed by a number of psychologists so far, there are many subconscious factors that might impact a person’s decision. We are cognitive misers; we have very limited attention spans, we have limited cognitive energy, limited time, limited physical effort, limited motivation to put conscious effort into decisions. Therefore, with designed environmental stimulations, a decision can be manipulated without a person being aware of it. As democracy is founded on the idea of free will, the exploitation of the personality insights can affect the votes and consequently, those who have access to the data can impact the world order.
Where thought conflicts with emotion, the latter is designed by the neural circuitry in our brains to win. — Carter and Frith
Face Analysis
Thin-slicing is a term used in psychology and philosophy to describe the ability to find patterns in events based only on “thin slices”, or narrow windows, of experience. The term means making very quick inferences about the state, characteristics or details of an individual or situation with minimal amounts of information. Brief judgments based on thin-slicing are similar to those judgments based on much more information. Judgments based on thin-slicing can be as accurate, or even more accurate, than judgments based on much more information.
Divorce Prediction
In 1998, a study Predicting Marital Happiness and Stability from Newlywed Interactions with 130 newlywed couples was designed to explore marital interaction processes that are predictive of divorce or marital stability. Divorce and stability were predicted with 83% accuracy and satisfaction with 80% accuracy.
By observing 15 minutes of newlywed couples' interactions, it was possible to predict the divorce with 83% accuracy.
Face Shape Classification with Convolutional Neural Networks
In order to use a model to predict any reliable psychological feature, I have been looking for credible studies and datasets that could be used to train the model. There have been a lot of articles published about character assessment from face shape but I haven’t found any credible source except How Do Faces Shape First Impressions?, published by Benedict Jones, a Professor in the Institute of Neuroscience and Psychology at the University of Glasgow that runs the Face Research Lab with Dr. Lisa DeBruine.
On Kaggle, a dataset comprised of a total of 5000 images of female celebrities has been published. Using fastai library, I tried to train ResNet34 to predict the face shapes.
With ResNet34, I achieved an accuracy of 27.5%. The accuracy might be improved further but in the scope of this experiment, I just wanted to test the potential for classifying face shape.
Below is the confusion matrix that shows how many images from the test set were classified correctly and how were the remaining misclassified.
Where does the neural network look for cues?
In order to check which parts of the image does the network look for cues, I implemented the logic to overlay the image with a heatmap. On the left, I used my image as an example. As in most cases, we judge people unconsciously, in case of having a reliable dataset, by observing the neural network decision cues, we could get more insights about our personal triggers. For organizations such as Facebook where millions of pictures are published daily, it is possible to establish a data monopoly and knowing more about ourselves than we know. As stated by Mr. Fagan, Facebook knows us better than our spouse and by establishing the cooperation between psychologists and computer scientists to analyze the available data, it is hard to find any other organization that has a greater potential to manipulate the world order.
The jupyter notebook is available here
