Stealing Ur Feelings Save
Winner of Mozilla's $50,000 prize for AI
stealing ur feelings
:compass: Table of contents
- What is Stealing Ur Feelings?
- Press and recognition
- Festival and exhibiton history
- Tech notes
- Archive
- Changelog
:question: What is Stealing Ur Feelings?
https://stealingurfeelin.gs :arrow_left: :arrow_left: :arrow_left:
Meet the new AI that knows you better than you know yourself.
Stealing Ur Feelings is a deep learning-powered AR experience which analyzes your facial reactions to reveal the dangers of Big Tech's emotional surveillance programs. Using the AI techniques described in corporate patents, Stealing Ur Feelings learns your deepest secrets just by analyzing your face.
It won the 2020 Webby Award for Best Immersive Documentary.
The project premiered on the internet on September 23, 2019.
It world premiered at the 2019 Tribeca Film Festival and was exhibited at the Tate Modern. It was profiled by the Museum of the Moving Image, Scientific American, Engadget, CBC News, and many more; it made the front page of Hacker News; MIT selected it for inclusion in DocuBase, a curated database of people and projects transforming documentary in the digital age.
Stealing Ur Feelings was somewhat notoriously plagiarized by the Financial Times; this event was chronicled in an article for The American Prospect.
The project began life as an application for Mozilla's 2018 awards for art and advocacy exploring artificial intelligence.
:newspaper: Press and recognition
Let's Go with Sabrina Marandola on CBC Radio One / .mp3
:movie_camera: Festival and exhibition history
2019 Camden International Film Festival
2019 Montreal International Documentary Festival
2019 Open City Documentary Festival
Tate Exchange at the Tate Modern
The Glass Room SF presented by Tactical Tech
:eyes: Tech notes
-
The code is a mess. It required a lot of trial and error over many months just to figure out how to tell the story. Consequently, the version of Stealing Ur Feelings which premiered at the Tribeca Film Festival was programmed over 6 frantic weeks between March and April 2019. That version was so unoptimized that it simply wouldn't run on average hardware. Optimization began in June 2019, and there wasn't enough time to improve most of the earlier bad decisions. The indecipherable comments have been preserved.
-
Different versions of the project have used different computer vision engines. Journalists often asked what emotion recognition models we used; the answer changed depending on the day they viewed the project. We built an excessively complicated custom engine on top of TensorFlow that wound up getting scrapped. At one point we managed to compile Dlib for the browser via the WebAssembly toolchain, and we got its multithreaded facial landmark detector working with HTML5 canvas by mapping native pthreads to WebWorkers. You can see that experiment here. The version of Stealing Ur Feelings published in September 2019 uses face-api.js.
-
Stealing Ur Feelings uses a custom system for frame-accurate video synchronization. It's based on embedding encoded information in the overscan area of the video, a la datacasting. The purely arithmetic approaches to frame-accurate sync are interesting, but the effects of floating point approximation error means that none of them work at the level of precision we required.
-
On Chrome, Stealing Ur Feelings deploys a zero-copy hack using OffscreenCanvas to run the computer vision engine on a separate thread. During development, Firefox's implementation of OffscreenCanvas was incomplete. The result is that Stealing Ur Feelings runs much smoother on Chrome. (Another result: there have been OffscreenCanvas regressions in Chrome stable releases which have caused Stealing Ur Feelings to segfault the browser.)
-
There's a secret physical installation mode. At the Tribeca Film Festival premiere, Stealing Ur Feelings was installed under a big television hanging from the ceiling which broadcast the AI-extracted "secrets" of whoever was experiencing the project at that moment. The client and server to support this functionality are located in the /installation directory. For the premiere, we tuned the political bias algorithm to classify absolutely everyone as a far-right conservative Republican. It was fun to watch journalists freak out when their secret political affiliation was revealed on the jumbotron.
:floppy_disk: Archive
- Source code
- Interactive tech demo v2.0
- Old interactive tech demo
- Wireframe mockups
- Initial funding concept
- Full application
- Slides from the 10/24/2018 Pecha Kucha talk at London's Royal Society of Arts
:zap: Changelog
06/01/2021 The American Prospect article is published, detailing plagiarism by The Financial Times
05/26/2021 We're featured in Slanted Magazine #37, the AI issue
05/19/2020 We won the 2020 Webby Award for Best Immersive Documentary
01/31/2020 Named one of the Top 50 XR Experiences of 2019 by Forbes
01/15/2020 Feature in El Pais
11/22/2019 Featured on Let's Go with Sabrina Marandola on CBC Radio One
11/18/2019 Profiled by CBC News
11/12/2019 Featured on Report on Italy's national public broadcaster Rai3
11/03/2019 Feature in It's Nice That
10/21/2019 We're coming to Montreal International Documentary Festival and The Glass Room SF
09/23/2019 Internet premiere
09/14/2019 We're coming to Tate Exchange at the Tate Modern
08/19/2019 We're coming to Camden International Film Festival in Maine
08/14/2019 MIT selected us for inclusion in DocuBase
08/10/2019 We're coming to Open City Documentary Festival in London
05/07/2019 Profiles by the Museum of the Moving Image and Engadget
03/06/2019 We've been selected to premiere at the 2019 Tribeca Film Festival!
01/02/2019 Interactive tech demo v2.0 is live!
10/24/2018 We won
07/12/2018 Submitted full application to Mozilla
07/08/2018 The interactive tech demo is live!
07/06/2018 Registered domain name: stealingurfeelin.gs
06/29/2018 Original funding concept accepted by Mozilla - we've been invited to submit a full application!
Open Source Agenda Badge
