Anki Vector AI++
Anki Vector - The Home Robot With Interactive AI Technology.
Well, I bought this little guy at 10 Feb 2019, if you want a robot pet, and you want to do some AI programming on it, then I highly recommend you to get Anki Vector.
I build this project to share my codes and docs.
This program is to enable Vector to detect objects with its camera, and tell us what it found.
We take a photo from Vector's camera, then post to Google Vision Service, then Google Vision Service returns the object detection result, finally, we turn all the label text into a sentence and send to Vector so that Vector can say it out loud.
Here are some demo videos:
Well, let's see how to do it.
export GOOGLE_APPLICATION_CREDENTIALS="/Workspace/Vector-vision-62d48ad8da6e.json"
python3 object_detection.py
.enable_camera_feed=True
, because we need the anki_vector.camera API.robot = anki_vector.Robot(anki_vector.util.parse_command_args().serial, enable_camera_feed=True)
def show_camera():
print('Show camera')
robot.camera.init_camera_feed()
robot.vision.enable_display_camera_feed_on_face(True)
and close the camera after the detection.
def close_camera():
print('Close camera')
robot.vision.enable_display_camera_feed_on_face(False)
robot.camera.close_camera_feed()
def save_image(file_name):
print('Save image')
robot.camera.latest_image.save(file_name, 'JPEG')
def detect_labels(path):
print('Detect labels, image = {}'.format(path))
# Instantiates a client
# [START vision_python_migration_client]
client = vision.ImageAnnotatorClient()
# [END vision_python_migration_client]
# Loads the image into memory
with io.open(path, 'rb') as image_file:
content = image_file.read()
image = types.Image(content=content)
# Performs label detection on the image file
response = client.label_detection(image=image)
labels = response.label_annotations
res_list = []
for label in labels:
if label.score > 0.5:
res_list.append(label.description)
print('Labels: {}'.format(labels))
return ', or '.join(res_list)
def robot_say(text):
print('Say {}'.format(text))
robot.say_text(text)
def analyze():
stand_by()
show_camera()
robot_say('My lord, I found something interesting. Give me 5 seconds.')
time.sleep(5)
robot_say('Prepare to take a photo')
robot_say('3')
time.sleep(1)
robot_say('2')
time.sleep(1)
robot_say('1')
robot_say('Cheers')
save_image(image_file)
show_image(image_file)
time.sleep(1)
robot_say('Start to analyze the object')
text = detect_labels(image_file)
show_image(image_file)
robot_say('Might be {}'.format(text))
close_camera()
robot_say('Over, goodbye!')
def main():
while True:
connect_robot()
try:
analyze()
except Exception as e:
print('Analyze Exception: {}', e)
disconnect_robot()
time.sleep(random.randint(30, 60 * 5))
2019-02-17 21:55:42,113 anki_vector.robot.Robot WARNING No serial number provided. Automatically selecting 009050ae
Connect to Vector...
2019-02-17 21:55:42,116 anki_vector.connection.Connection INFO Connecting to 192.168.1.230:443 for Vector-M2K2 using /Users/gaolu.li/.anki_vector/Vector-M2K2-009050ae.cert
2019-02-17 21:55:42,706 anki_vector.connection.Connection INFO control_granted_response {
}
Show camera
Say My lord, I found something interesting. Give me 5 seconds.
Say Prepare to take a photo
Say 3
Say 2
Say 1
Say Cheers
Save image
Show image = /Workspace/labs/Anki-Vector-AI/resources/latest.jpg
Display image on Vector's face...
Say Start to analyze the object
Detect labels, image = /Workspace/labs/Anki-Vector-AI/resources/latest.jpg
Labels: [mid: "/m/08dz3q"
description: "Auto part"
score: 0.6821197867393494
topicality: 0.6821197867393494
]
Show image = /Workspace/labs/Anki-Vector-AI/resources/latest.jpg
Display image on Vector's face...
Say Might be Auto part
Close camera
Say Over, goodbye!
2019-02-17 21:56:12,460 anki_vector.connection.Connection INFO control_lost_event {
}
2019-02-17 21:56:12,460 anki_vector.robot.Robot WARNING say_text cancelled because behavior control was lost
2019-02-17 21:56:12,461 anki_vector.util.VisionComponent INFO Delaying disable_all_vision_modes until behavior control is granted
2019-02-17 21:56:12,707 anki_vector.connection.Connection INFO control_granted_response {
}
Vector disconnected
You can find the latest photo that Vector uses to detention in resources/latest.jpg
.
This program is to enable Vector to place shoes for us. Vector will place our shoes when we're not at home, so we can leave home without worry about the shoes, especially when we're in a hurry.
This program is in research. I'll share the plan, the design, the docs, the codes here. I highly recommend you make an issue on GitHub so we can talk about it further if you're interesting, any help is welcome!
The design proposal is in this Google doc. https://docs.google.com/document/d/10TQEdbIdcvCW8gNAUvVVSe1YxzFxsQExP_X33M3_Aos/edit?usp=sharing
Here is a draft demo video I made to give you guys a sense of the program: