Deep neural network trained to detect eye contact from facial image
This repository provides a deep convolutional neural network model trained to detect moments of eye contact in egocentric view. The model was trained on over 4 millions of facial images of > 100 young individuals during natural social interactions, and achives an accuracy comaprable to that of trained clinical human annotators.
python demo.py
python demo.py --video yourvideofile.avi
Try this if you don't want to use dlib's face and instead test with pre-detected faces.
Comment out the first line of demo.py "import dlib" if you didn't install dlib.
python demo.py --video demo_video.avi --face demo_face_detections.txt
Demo video has been downloaded from here. I used this face detector to generate the face detection file.
--face
: Path to pre-processed face detection file of format [frame#, min_x, min_y, max_x, max_y]. If not specified, dlib's face detector will be used.-save_vis
: Saves the output as an avi video file.-save_text
: Saves the output as a text file (Format: [frame#, eye_contact_score]).-display_off
: Turn off display window.Please cite this paper in any publications that make use of this software.
@article{chong2020,
title={Detection of eye contact with deep neural networks is as accurate as human experts},
url={osf.io/5a6m7},
DOI={10.31219/osf.io/5a6m7},
publisher={OSF Preprints},
author={Chong, Eunji and Clark-Whitney, Elysha and Southerland, Audrey and Stubbs, Elizabeth and Miller, Chanel and Ajodan, Eliana L and Silverman, Melanie R and Lord, Catherine and Rozga, Agata and Jones, Rebecca M and et al.},
year={2020}
}
Link to the paper: here