A simple GUI tool for generating adversarial poses of objects.
This is the companion tool to the paper:
Michael A. Alcorn, Qi Li, Zhitao Gong, Chengfei Wang, Long Mai, Wei-Shinn Ku, and Anh Nguyen. Strike (with) a pose: Neural networks are easily fooled by strange poses of familiar objects. Conference on Computer Vision and Pattern Recognition (CVPR). 2019.
Code to run experiments like those described in the paper can be found in the paper_code
directory. The tool allows you to generate adversarial poses of objects with a graphical user interface. Please note that the included jeep object does not meet the realism standards set in the paper. Unfortunately, the school bus object shown in the GIF is proprietary and cannot be distributed with the tool. A browser port of the tool (created by Zhitao Gong) can be found here.
If you use this tool for your own research, please cite:
@article{alcorn-2019-strike-with-a-pose,
Author = {Alcorn, Michael A. and Li, Qi and Gong, Zhitao and Wang, Chengfei and Mai, Long and Ku, Wei-Shinn and Nguyen, Anh},
Title = {{Strike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects}},
Journal = {Conference on Computer Vision and Pattern Recognition (CVPR)},
Year = {2019}
}
glx-utils
on Fedora or mesa-utils
on Ubuntu):glxinfo | grep "OpenGL version"
/Applications/Python\ 3.x/Install\ Certificates.command
where x
is the minor version of your particular Python 3 install. If you used the above link to install Python, the file will be at:
/Applications/Python\ 3.6/Install\ Certificates.command
Note: the tool takes a little while to start the first time it's run because it has to download the neural network.
In the terminal, enter the following commands:
# Clone the strike-with-a-pose repository.
git clone https://github.com/airalcorn2/strike-with-a-pose.git
# Move to the strike-with-a-pose directory.
cd strike-with-a-pose
# Install strike-with-a-pose.
pip3 install .
# Run strike-with-a-pose.
strike-with-a-pose
You can also run the tool (after installing) by starting Python and entering the following:
from strike_with_a_pose import app
app.run_gui()
install.command
in the strike-with-a-pose-master/
directory.strike-with-a-pose.command
in the strike-with-a-pose-master/run/
directory.install.bat
in the strike-with-a-pose-master\
directory.strike-with-a-pose.bat
in the strike-with-a-pose-master\run\
directory.Users can test their own objects and backgrounds in Strike (With) A Pose by:
scene_files/
directory.BACKGROUND_F
, OBJ_F
, and MTL_F
variables in settings.py
accordingly.strike-with-a-pose/
directory:PYTHONPATH=strike_with_a_pose python3 -m strike_with_a_pose.app
Users can experiment with different machine learning models in Strike (With) A Pose by:
get_gui_comps
, init_scene_comps
, predict
, render
, and clear
functions (e.g., image_classifier.py
, object_detector.py
, image_captioner.py
, and class_activation_mapper.py
[with major contributions by Qi Li]).MODEL
variable in settings.py
accordingly.strike-with-a-pose/
directory:PYTHONPATH=strike_with_a_pose python3 -m strike_with_a_pose.app
To use the image captioner model, first download and install the COCO API:
git clone https://github.com/pdollar/coco.git
cd coco/PythonAPI/
make
python3 setup.py build
python3 setup.py install
"The Elephant in the Room"-like (Rosenfeld et al., 2018) examples:
"Learning Deep Features for Discriminative Localization"-like (Zhou et al., 2016) examples.
Press L to toggle Live mode. When on, the machine learning model will continuously generate predictions. Note, Live mode can cause considerable lag if you do not have each of (1) a powerful GPU, (2) CUDA installed, and (3) a PyTorch version with CUDA support installed.
Press X to toggle the object's teXture, which is useful for making sure the directional light is properly interacting with your object. If the light looks funny, swapping/negating the vertex normal coordinates can usually fix it. See the fix_normals.py
script for an example.
Press F to toggle back-Face culling, which is necessary when rendering certain models (like some found in ShapeNet).
Press I to bring up the Individual component selector. This feature allows you to display individual object components (as defined by each newmtl
in the .mtl
file) by themselves.
Press C to Capture a screenshot of the current render. Screenshots are saved in the directory where the tool is started.