Determination of emotions from photographs. Microsoft will help determine not only the age and gender of a person from a photo, but also his emotions Methods for determining emotions from photographs

Instruction. I have a very interesting game for you. You know that all people are sometimes happy, sometimes surprised or angry. Now I'm going to show you some pictures that show a person with some kind of emotion on his face (for example, joy, grief, fear, surprise, contempt, disgust, shame, interest, anger). Here is a list of those emotions. Try to determine which emotion is reflected in each photo.

The experimenter alternately shows 9 photographs and writes down the answer of the child on the form next to the number of the photograph.

Form for registration of answers

622 Application

The number of correct answers is taken into account, according to which the formed ability to determine the sign of emotion by facial expression is judged.

Definition of emotions by pantomime

Instruction. And now I will show you pictures, which also depict a person with some kind of experience. Try to determine what emotion the person has in each picture.

The subject is shown 8 pictures in turn and writes down his answers on the form.

Form for registration of answers

The number of correct answers is taken into account and they are used to judge the formed ability to determine the sign of emotion by pantomime.

The study of the perception of the expressive side of the picture

The technique is taken from the article: Repina T. A. Perception of the expressive side of the drawing by preschoolers and its influence on the attitude of children to the hero of the book // Questions of Psychology. - 1960. - No. 5.

Study preparation. Pick up 10-14 reproductions of paintings in which the emotional content is conveyed by showing facial expressions and gestures, postures of characters, the relationship of characters with each other or their relationship to surrounding objects.

Conducting research. The experiment is carried out individually with children 3-6 years old. The child is shown pictures sequentially and asked to talk about them. If he finds it difficult to describe, ask leading questions (prepare in advance) to find out how he understands the plot of the image, the emotional states of the characters.

Data processing. The number of correct answers is counted when the children correctly understood the plot and the emotional state of the characters. Counting is carried out for each picture. The results are drawn up in a table by age: 3-4 years, 4-5 years, 5-6 years, 6-7 years.

They find out which emotional states depicted in the pictures are correctly perceived by children and at what age. How they perceive the facial expressions of joy and anger, the expression of sadness and sadness, with what emotional experience they associate the bright colors of the picture, how they understand the emotional content conveyed through the posture and gestures of the hero.

The study of the emotional manifestations of children when playing plot scenes

The technique is taken from the book: Emotional development of a preschooler / Ed. A. D. Ko-sheleva. - M., 1985. - S. 100-102.

Study preparation. Choose situations from the life of children that are close and understandable to them, for example:

Addendum 623

1. A sick mother lies in bed, the eldest daughter (preparatory group) brings her brother (nursery group).

2. During lunch in a group, a boy accidentally spills soup, all the children jump up and laugh; the boy is frightened, the teacher strictly explains that one must be more careful and that there is absolutely nothing to laugh at here.

3. The boy lost his mittens, and during the walk his hands were very cold, but he does not want to show others that he is very cold.

4. The girl was not accepted into the game, she went to the corner of the room, lowered her head low and is silent, she is about to cry.

5. A boy (girl) is happy for his friend (girlfriend), whose drawing turned out to be the best in the group.

Prepare 2-3 children to dramatize these stories. Conducting research. The study is conducted with children 4-7 years old. First episode. Children prepared in advance act out a scene in front of the group, then the experimenter asks the children how the characters of this scene feel. Second series. The experimenter describes the situation and offers to portray it.

I situation - show the sad, suffering face of the mother, the capricious crying boy and the sympathetic face of the girl.

II situation - show the strict face of the teacher, the laughing and then embarrassed children, the frightened face of the boy.

III situation - how the boy does not want to show that he is cold.

IV situation - show offended girls.

V situation - show genuine joy for another. If the children are not expressive enough or incorrectly portray the feelings and emotions of the characters, the experimenter again describes the situations and tells in detail what each of the characters is experiencing.

Data processing. They analyze how children embody the emotional states of the characters in the scenes. They make a conclusion about the expressiveness and richness of expressive-mimic means of communication and about the development of the ability to empathize with other people. The data is drawn up in a table (Table 6).

Instruction. I have a very interesting game for you. You know that all people are sometimes happy, sometimes surprised or angry. Now I'm going to show you some pictures that show a person with some kind of emotion on his face (for example, joy, grief, fear, surprise, contempt, disgust, shame, interest, anger). Here is a list of those emotions. Try to determine which emotion is reflected in each photo.

The experimenter alternately shows 9 photographs and writes down the answer of the child on the form next to the number of the photograph.

Form for registration of answers

622 Application

The number of correct answers is taken into account, according to which the formed ability to determine the sign of emotion by facial expression is judged.

Definition of emotions by pantomime

Instruction. And now I will show you pictures, which also depict a person with some kind of experience. Try to determine what emotion the person has in each picture.

The subject is shown 8 pictures in turn and writes down his answers on the form.

Form for registration of answers

The number of correct answers is taken into account and they are used to judge the formed ability to determine the sign of emotion by pantomime.

The study of the perception of the expressive side of the picture

The technique is taken from the article: Repina T. A. Perception of the expressive side of the drawing by preschoolers and its influence on the attitude of children to the hero of the book // Questions of Psychology. - 1960. - No. 5.

Study preparation. Pick up 10-14 reproductions of paintings in which the emotional content is conveyed by showing facial expressions and gestures, postures of characters, the relationship of characters with each other or their relationship to surrounding objects.

Conducting research. The experiment is carried out individually with children 3-6 years old. The child is shown pictures sequentially and asked to talk about them. If he finds it difficult to describe, ask leading questions (prepare in advance) to find out how he understands the plot of the image, the emotional states of the characters.

Data processing. The number of correct answers is counted when the children correctly understood the plot and the emotional state of the characters. Counting is carried out for each picture. The results are drawn up in a table by age: 3-4 years, 4-5 years, 5-6 years, 6-7 years.

They find out which emotional states depicted in the pictures are correctly perceived by children and at what age. How they perceive the facial expressions of joy and anger, the expression of sadness and sadness, with what emotional experience they associate the bright colors of the picture, how they understand the emotional content conveyed through the posture and gestures of the hero.

The study of the emotional manifestations of children when playing plot scenes

The technique is taken from the book: Emotional development of a preschooler / Ed. A. D. Ko-sheleva. - M., 1985. - S. 100-102.

Study preparation. Choose situations from the life of children that are close and understandable to them, for example:

Addendum 623

1. A sick mother lies in bed, the eldest daughter (preparatory group) brings her brother (nursery group).

2. During lunch in a group, a boy accidentally spills soup, all the children jump up and laugh; the boy is frightened, the teacher strictly explains that one must be more careful and that there is absolutely nothing to laugh at here.

3. The boy lost his mittens, and during the walk his hands were very cold, but he does not want to show others that he is very cold.

4. The girl was not accepted into the game, she went to the corner of the room, lowered her head low and is silent, she is about to cry.

5. A boy (girl) is happy for his friend (girlfriend), whose drawing turned out to be the best in the group.

Prepare 2-3 children to dramatize these stories. Conducting research. The study is conducted with children 4-7 years old. First episode. Children prepared in advance act out a scene in front of the group, then the experimenter asks the children how the characters of this scene feel. Second series. The experimenter describes the situation and offers to portray it.

I situation - show the sad, suffering face of the mother, the capricious crying boy and the sympathetic face of the girl.

II situation - show the strict face of the teacher, the laughing and then embarrassed children, the frightened face of the boy.

III situation - how the boy does not want to show that he is cold.

IV situation - show offended girls.

V situation - show genuine joy for another. If the children are not expressive enough or incorrectly portray the feelings and emotions of the characters, the experimenter again describes the situations and tells in detail what each of the characters is experiencing.

Data processing. They analyze how children embody the emotional states of the characters in the scenes. They make a conclusion about the expressiveness and richness of expressive-mimic means of communication and about the development of the ability to empathize with other people. The data is drawn up in a table (Table 6).

Stimulus material: portrait pictures (photographs) depicting children and adults who have a pronounced emotional state (joy, fear, anger, grief); plot pictures depicting positive and negative actions of children and adults.
Order of conduct. Two series of experiments are carried out individually with children of preschool and primary school age.
First episode. The child is consistently shown portrait pictures depicting children and adults and asked questions: Who is shown in the picture? What is he doing? How does he feel? How did you guess about it? Describe what you see in the picture?
Second series. The child is consistently shown plot pictures and asked questions: What do children (adults) do? How do they do it (friendly, quarreling, not paying attention to each other, etc.)? How did you guess? Who is good and who is bad? How did you guess?
Processing and interpretation of results. The number of correct answers in different age groups is counted separately for each series and for each picture. It is revealed whether it is possible for children to understand the emotional states of adults and peers, by what signs they are guided, who they understand better - an adult or a peer. Determine the dependence of these indicators on the age of children.

Methodology "Identification of emotions from photographs"

Stimulus material: a set of 9 photographs depicting a person whose face reflects one or another emotion, and a list of emotions.
Order of conduct. Instructions: “I have a very interesting task. You know that all people are sometimes happy, sometimes surprised or angry. Now I'm going to show you some photographs that show a person in a certain emotional state (joy, grief, fear, surprise, contempt, disgust, shame, interest, anger). Here is a list of those emotions. Try to determine which emotion is reflected in each photo.
The experimenter alternately shows 9 photographs and writes down the child's answer on the form next to the photo number (Table 3).

Table 3
Form for registration of answers

Processing and interpretation of results. The number of correct answers is taken into account, according to which the formed ability to determine a person's emotion by facial expression is judged. 50% or less correct answers indicate a very low level of ability to distinguish emotions.

Methodology "Personal aggressiveness and conflict" (E.P. Ilyina, P.A. Kovaleva)

1. Scales of touchiness and vindictiveness
Purpose: to determine a person's tendency to hostility, manifested through resentment and vindictiveness.
Stimulus material: questionnaire test consisting of 20 statements proposed below.
Order of conduct. Instruction: “You are offered a series of statements. If you agree with them, put a plus sign (+) next to the corresponding statement number, if you disagree, put a minus sign (-). Suggested statements
1. I am most often not rewarded for my deeds.
2. If someone treats me unfairly, then inwardly I wish the offender all sorts of misfortunes.
3. Situations almost always work out better for others than for me.
4. I think that the slogan from the cartoon "Tooth for a tooth, a tail for a tail" is true.
5. I often take offense at the comments of others, even if I understand that they are fair.
6. Often I imagine the punishments that could fall on my offenders.
7. I am offended by the lack of attention from others ..
8. No insult should go unpunished.
9. I am always offended if I am not among those awarded for the cause in which I participated.
10. I do not calm down until I take revenge on the offender.
11. I have a feeling that I am often treated unfairly,
12. I believe that kindness is better than revenge.
13. I am not offended by the jokes of friends, even if they are very angry.
14. I like people who do not remember evil.
15. It doesn’t bother me if my name is not mentioned when praising for a common work.
16. I believe that evil can be repaid with good, and I do so.
17. I feel sorry for overly touchy people.
18. I have a negative attitude towards vengeful people.
19. I do not see anything offensive in what they tell me about my shortcomings.
20. Is the opinion fair: if you are hit on one cheek, then you have to substitute the other?
Processing and interpretation of results. Touchiness scale: for each agreement with the statement on points 1,
3, 5, 7, 9, 11 and for each disagreement on points 13, 15, 17, 19 one point is awarded.
Vindictive scale: for each agreement with the statement on points 2, 4, 6, 8, 10 and for each disagreement on points 12, 14, 16, 18, 20, one point is awarded. The scores for each scale are then calculated.
The greater the total score, the more the subject is prone to touchiness and vindictiveness (range from 0 to 10 points), and in their total amount - to hostility (range from 0 to 20 points),
2. Scale of temper
Purpose: diagnosis of the level of emotional excitability by self-assessment.
Stimulus material: questionnaire test consisting of 10 statements proposed below.
Order of conduct. Instruction: “Read the statements proposed to you, if you agree with them, put a plus sign (+), if you disagree, a minus sign (~).”
Suggested statements
1. I get easily irritated but calm down quickly.
2. If someone pisses me off, I don't pay attention to it.
3. I can hide my irritability.
4. I always react calmly to criticism, even if it seems unfair to me.
5. I really don't like being mocked.
6. I never have outbursts of anger.
7. I am outraged when people push me on the street or in transport.
8. In a conflict situation, I behave calmly.
9. I do not know how to restrain myself when I am undeservedly reproached.
10. It's usually hard to piss me off.
Processing and interpretation of results. If you agree on points 1, 3, 5, 7,9 and disagree on points 2, 4, 6, 8, 10, one point is awarded. The total scores are calculated. If the respondent scores 8-10 points, he has high emotional excitability, if he scores 4 points or less, he has low emotional excitability. Points from 5 to 7 characterize the average level of emotional excitability.

Picture Method for Determining Empathy in Primary School Students

Stimulus material: a set of pictures depicting various life situations.
Order of conduct. The child is shown several pictures in which the artist depicted various situations from the lives of children and adults. Show the first picture and indicate the silhouette in the lower right corner. Instruction: “Look, you are here. Tell me, please, what will you do in this situation? The child's answer is recorded in the protocol of the survey (Table 4) under the number of the picture. In the "Note" column, emotional manifestations are recorded when answering questions.

Table 4
Poll Protocol

School anxiety test (F. Phillips)

Stimulus material: a test consisting of 58 questions that can be read to students, or can be offered in writing.
Order of conduct. Instructions: “Now you will be given a questionnaire, which consists of questions about how you feel at school. Try to answer sincerely and truthfully. Don't think too long about the answers. When answering a question, write down its number and answer: put a plus sign (H-) if you agree with it, or a minus sign (-) if you disagree.
Questions
1. Do you find it difficult to keep up with the whole class?
2. Do you get nervous when the teacher says that he is going to test how well you know the material?
3. Do you find it difficult to work in the classroom the way the teacher wants?
4. Do you sometimes dream that the teacher is furious because you do not know the lesson?
5. Has anyone in your class ever hit you?
6. Do you often wish your teacher would take your time explaining new material until you understand what he is saying?
7. Do you worry a lot when answering or completing a task?
8. Does it happen to you that you are afraid to speak up in class because you are afraid to make a stupid mistake?
9. Do your knees tremble when you are called to answer?
10. Do your classmates often laugh at you when you play different games?
11. Do you ever get a lower grade than you expected?
12. Are you worried about the question of whether you will be left for a second year?
13. Do you try to avoid games in which choices are made because you are usually not chosen?
14. Does it happen at times that you are trembling all over when you are called to answer?
15. Do you often get the feeling that none of your classmates want to do what you want?
16. Do you worry a lot before starting a task?
17. Is it difficult for you to get the grades your parents expect from you?
18. Are you afraid at times that you will feel sick in class?
19. Will your classmates laugh at you if you make a mistake when answering?
20. Do you look like your classmates?
21. After completing a task, do you worry about how well you did it?
22. When you work in a class, are you sure that you will remember everything well?
23. Do you sometimes dream that you are at school and cannot answer the teacher's question?
24. Is it true that most guys are friendly to you?
25. Do you work harder if you know that your work will be compared in class with your classmates?
26. Do you often dream about being less worried when you are asked?
27. Are you afraid at times to get into an argument?
28. Do you feel that your heart begins to beat strongly when the teacher says that he is going to test your readiness for the lesson?
29. When you get good grades, do any of your friends think that you want to curry favor?
30. Do you feel good with those of your classmates whom the guys treat with special attention?
31. Does it happen that some guys in the class say something that hurts you?
32. Do you think that those students who do not cope with their studies lose the favor of others?
33. Can you say that most of your classmates do not pay attention to you?
34. Are you often afraid to look ridiculous?
35. Are you satisfied with the way teachers treat you?
36. Does your mother help in organizing evenings, like other mothers?
37. Have you ever worried about what others think of you?
38. Do you hope to study better in the future than before?
39. Do you think that you dress for school as well as your classmates?
40. When answering a lesson, do you often think about what other people think about you at that time?
41. Do bright students have any special rights that other children in the class do not have?
42. Do some of your classmates get angry when you manage to be better than them?
43. Are you satisfied with the way your classmates treat you?
44. Do you feel good when you are alone with a teacher?
45. Do your classmates sometimes make fun of your appearance and behavior?
46. ​​Do you think that you worry about your school affairs more than other guys?
47. If you can't answer when asked, do you feel like you're about to burst into tears?
48. When you lie in bed in the evening, do you sometimes think with anxiety about what will happen at school tomorrow?
49. When working on a difficult task, do you sometimes feel that you have completely forgotten things that you knew well before?
50. Does your hand tremble slightly when you are working on a task?
51. Do you feel nervous when the teacher is about to give the class an assignment?
52. Are you afraid of testing your knowledge at school?1
53. When the teacher is about to give the class a task, do you feel fear that you will not cope with it?
54. Have you ever dreamed that your classmates can do things that you cannot?
55. When the teacher explains the material, do you think that your classmates understand it better than you?
56. On your way to school, do you worry that the teacher might give the class a test paper?
57. When you complete a task, do you usually feel that you are doing it poorly?
58. Does your hand tremble slightly when the teacher asks you to complete the task on the blackboard in front of the whole class?
Processing and interpretation of results. When processing the results, questions are selected, the answers to which do not match the test key.

The key to the questions "+" - Yes, "-" - No.

results

1) The number of mismatches (“+” - Yes, “-” - No) for each factor (the absolute number of mismatches, in percent:<50%; >50%; > 75%):
a) the total number of mismatches for the entire test. If it is more than 50% of the total number of test questions, we can talk about increased anxiety of the child, if more than 75% - about high anxiety;
b) the number of matches for each of the eight anxiety factors identified in the test is multiplied by the coefficient (As). The general internal emotional state of the schoolchild is analyzed, which is largely determined by the presence of certain disturbing factors and their number.
for each respondent.
2) Representation of these data in the form of individual diagrams.
Number of mismatches for each dimension for the entire class: absolute value -<50%; > 50%; > 75%.
3) Representation of this data in the form of a diagram.
4) The number of students who have mismatches for a certain factor<50% и >75% (for all factors).
5) Presentation of comparative results in repeated measurements.
6) Complete information about each student (according to test results).

1. General anxiety at school - the emotional state of the child associated with various forms of his inclusion in the life of the school.
2. Experience of social stress - the emotional state of the child, against which his social contacts develop (primarily with peers).
3. Frustration of the need to achieve success - an unfavorable mental background that does not allow the child to develop his needs for success, achieving a high result, etc.
4. Fear of self-expression - negative emotional experiences of situations associated with the need for self-disclosure, presenting oneself to others, demonstrating one's capabilities.
5. Fear of a verification situation - a negative attitude and anxiety before a verification (especially public) of knowledge, achievements, and opportunities.
6. Fear of not meeting the expectations of others - orientation to the significance of others in assessing their results, actions and thoughts. Anxiety about grades.
7. Low physiological resistance to stress - features of the psychophysiological organization that reduce the child's adaptability to situations of a stressful nature, increase the likelihood of an inadequate, destructive response to an alarming environmental factor.
8. Problems and fears in relations with teachers - a general negative emotional background of relations with adults at school, which reduces the success of a child's education.

Composition-competition "Who laughs - he gets smarter"

Purpose: to identify the characteristics of children's behavior. This target is disguised. Children should be told about their behavior briefly and with humor. Not everyone can do it, but it's worth trying. Firstly, it is interesting and fun, and secondly, hidden or unconscious character traits can break out.
Order of conduct. Before the start of the competition, the teacher quotes excerpts from the essays of the third graders, for example: “To see what kind of teeth the cat has, I tried to open its mouth. The teeth are good. My hand has not healed so far." Prizes are awarded for the best essay, which children evaluate in terms of the truthfulness of what was said, humor, originality of the described situation.
You can set a specific situation. The victory is not awarded if the children find inaccuracies.
Processing of results. The teacher, conducting this competition, receives information about: 1) how the child evaluates his actions; 2) how his classmates evaluate him; 3) whether there are differences in self-assessment and assessments of others.
At the same time, data on strong and weak qualities are accumulated, and children who require behavior correction are identified.

Today, more and more aspects of our lives are subjected to automation and mathematical processing, one of which is the main part of our memory - photographs that capture us, our friends, memorable moments and much more. Not so long ago, Microsoft has already taken the first and rather interesting step in photo processing - an algorithm was developed that made it possible to determine the age of captured people (). But time does not stand still, and today we will talk about a new algorithm proposed by Microsoft - emotion recognition from a photo.

Subconsciously, we all know how to determine whether a person in a photo is laughing or sad, but is it possible to measure these feelings? It turns out you can! And let's immediately see what this dimension is.

To get started, go to the project page https://www.projectoxford.ai/demo/emotion#detection .

And we immediately see an example of how the algorithm works. Let's try to figure out what it is, and then we will conduct some experiments.

So, on the left there is an analyzed image, in which faces are highlighted. It immediately becomes clear that the algorithm is able to select all the faces in the photographs and analyze each one.

On the right (and when hovering over the frames on the faces) there is an area containing a bunch of incomprehensible text (the indicated text is the result of the algorithm in a special format), let's break it down.

In the first line, the inscription "detection result" - "analysis results".

"FaceRectangle" :(

Left: 488

Top: 263

Width: 148

"Height": 148

"scores" :(

"Anger": 9.075572e-13,

"Contempt": 7.048959e-9,

"Disgust": 1.02152783e-11,

"Fear": 1.778957e-14,

"Happiness": 0.9999999,

"Neutral": 1.31694478e-7,

"Sadness": 6.04054263e-12,

Surprise: 3.92249462e-11

What does all this mean?

The "FaceRectangle" fragment contains the coordinates of the corners of the frame of each of the faces, so that it is clear which person's emotions we will look at.

The "Scores" fragment presents the results themselves. Because the algorithm has so far been provided in trial mode, the results have not been processed in a “human” form and are presented in a “raw form”. But do not be afraid - everything is very simple! There is a list of eight emotions:

Anger - Anger

Context - Contempt

Disgust - Disgust

Fear- Fear

Happiness - Happiness

Neutral - Neutral

Sadness - Sadness

Surprise - surprise

And the contribution of each emotion to the general emotional state of a person in the photo is indicated from 0 to 1 - the larger the number, the more significant the emotion is.

In the considered example, we see that the first person in the photo has the most pronounced emotion - happiness. Do not be afraid of incomprehensible values ​​like "3.92249462e-11" - this is a mathematical form of writing a number, where "e-11" indicates how many characters to the left the comma should be moved, i.e. in the example, the number "0.0000000000392249462" will be obtained, which is insanely small.

Now you can fantasize about applying the algorithm for recognizing emotions from a photo. For example, I see a photo album on a social network, which, depending on the selected user status (sadness, joy or boredom), automatically rearranges the photos to suit the mood. Or another example, you are going to look at pictures on the network, the camera of your device takes pictures of you and the mood is determined by the specified algorithm, and then images are offered for viewing that correspond to the emotional state, for example, if you are sad, then the images will be colorful and positive.

I'm sure the algorithm will find many uses. Moreover, Microsoft has provided an API for accessing the algorithm, which means that anyone can use it and provide processing results in a variety of ways.

And finally, I played a little with automatic emotion recognition and I want to show what came of it.

The first photo I checked was, of course, the famous terminator Mr. Equanimity. And the algorithm supported me - 0.89 neutrality.

Why not try something fun? And who is the best at grimacing and having fun - Musk's first thought. And unanimously - a face full of "happiness".

Since the films have gone, we must try the hobbit. Here I myself can not determine the whole gamut of emotions on the face and the algorithm agrees with me. However, at that moment I did not go through an interesting thing - if you hover over the frame on the photo, a window with the results appears, a nice little thing, especially when there are a lot of faces.

And again, Jim Carrey, well, it seems to me that he has very vivid emotions - and we look at surprise. It seems I was not mistaken.

The photo shows a young man whose emotional state we have identified as angry.

Let's complicate the task and give the algorithm pre-prepared photos of emotions. Faces are certainly not quite natural for a clear transfer of state, but everything was determined with very high accuracy.

Well, we're a little shy. Not a photo, but a portrait drawn by an artist from our website () and I think this is also a good result!

Of course, I could not resist and tried to recognize the feelings of Cheburashka and was left with nothing - the algorithm refused to process the cartoon character.

Summing up the review of the algorithm for recognizing emotions from photos, implemented by Microsoft, I would like to note the very good results of its work and a large number of accurate recognitions. For a demo start, just a great result! We will wait for new interesting solutions.