VU Amsterdam - CLTL - Robot Framework

Repository for Robot Applications created as part of the Computational Lexicology & Terminology Lab (CLTL) at the Vrije Universiteit, Amsterdam.

Features

• A framework for creating interactive Robot Applications using Python, to enable:
  • Human-Robot conversation using Speech-to-Text and Text-to-Speech
  • Recognising friends by face and learning about them and the world through conversation
  • Recognising and positioning the people and objects in its enviroment.
• Natural Language Understanding through Syntax Trees (Grammars)
• Knowledge Representation of all learned facts through a RDF Graph: the robot's Brain!
• Curiosity based on Knowledge Gaps and Conflicts resulting from learned facts
• Realtime visualisation in web browser

Getting started

Check out our WIKI for information on how it works and how to set up.

Check out our API Reference and Sample Applications!

Prerequisites

• NAOqi Python SDK (version 2.5.10) from the Softbank robotics download page
• Python 2.7.10
• An application on Wolfram Alpha with API access
• A project on the Google Cloud Platform supporting Text-To-Speech and Speech-To-Text APIs
• Docker Engine
• OpenFace
• Pepper's sister-project: pepper_tensorflow, which includes the Tensorflow services within Python 3.6.
• GraphDB with a repository named leolani

Detailed instructions on how to set up.

More information

More information on the Make Robots Talk project at CLTL can be found on its website

Citation

Please cite (at least one of) the following papers if you use this software in your research:

Leolani: a reference machine with a theory of mind for social communication

@inproceedings{vossen2018leolani,
      title={Leolani: a reference machine with a theory of mind for social communication},
      author={Vossen, Piek and Baez, Selene and Bajc̆eti{\'c}, Lenka and Kraaijeveld, Bram},
      booktitle={International conference on text, speech, and dialogue},
      pages={15--25},
      year={2018},
      organization={Springer},
}

A communicative robot to learn about us and the world

@inproceedings{vossen2019communicative,
      title={A communicative robot to learn about us and the world},
      author={Vossen, Piek and Santamaria, Selene Baez and Bajc̆eti{\'c}, Lenka and Ba{\v{s}}i{\'c}, Suzana and Kraaijeveld, Bram},
      booktitle={2019 Annual International Conference on Computational Linguistics and Intellectual Technologies, Dialogue 2019},
      pages={728--743},
      year={2019}
}

Leolani: A robot that communicates and learns about the shared world

@inproceedings{vossen2019leolani,
      title={Leolani: A robot that communicates and learns about the shared world},
      author={Vossen, Piek and Baez, Selene and Baj{\v{c}}eti{\'c}, Lenka and Ba{\v{s}}i{\'c}, Suzana and Kraaijeveld, Bram},
      booktitle={2019 ISWC Satellite Tracks (Posters and Demonstrations, Industry, and Outrageous Ideas), ISWC 2019-Satellites},
      pages={181--184},
      year={2019},
      organization={CEUR-WS}
}

Modelling context awareness for a situated semantic agent

@inproceedings{vossen2019modelling,
      title={Modelling context awareness for a situated semantic agent},
      author={Vossen, Piek and Baj{\v{c}}eti{\'c}, Lenka and Baez, Selene and Ba{\v{s}}i{\'c}, Suzana and Kraaijeveld, Bram},
      booktitle={International and Interdisciplinary Conference on Modeling and Using Context},
      pages={238--252},
      year={2019},
      organization={Springer}
}

License

Distributed under the MIT License. See LICENSE for more information.

Authors

• Bram Kraaijeveld
• Selene Báez Santamaría
• Lenka Bajc̆etic
• Piek Vossen