Code for running real vs fake experiments on Amazon Mechanical Turk
Run "real vs fake" experiments on Amazon Mechanical Turk (AMT).
Runs a series "real vs fake" trials. Each trial pits a real image against a "fake" image generated by an algorithm.
Python
opt.which_algs_paths
). Must also contain a subfolder for the real images (path: opt['gt_path']
). Images should be named "0.jpg", "1.jpg", etc, in consecutive order up to some total number of images N (or they can be named differently, but you will have to specify a lambda function in opt['filename']
).opt
in getOpts
function.python mk_expt.py -n EXPT_NAME
to generate data csv and index.html for Turk.python process_csv.py -f CSV_FILENAME --N_imgs NUMBER_IMAGES --N_practice NUMBER_PRACTICE
. This will compute and run bootstrap statistics.opt['ut_id']
)opt['which_algs_paths']
, then each HIT tests all algorithms randomly i.i.d. from this list.opt['paired']
is true, then "fake/n.jpg" will be pitted against "real/n.jpg"; if false, "fake/n.jpg" will be pitted against "real/m.jpg", for random n and mgetDefaultOpts()
for documentation on more featuresThis tool was initially developed for Colorful Image Colorization in Matlab (see this branch). This master branch has been translated into Python. Feel free to use this bibtex to cite.