This phis project aims to develop a service and assistive robot for personal domestic applications, using a standard platform as Softbank Pepper Robot. To achieve this, we explore, develop, and implement artificial intelligence and machine learning algorithms in order to provide the robot with basic social skills. Some of them are human-robot interaction, mapping and autonomous navigation in dynamic and static environments, computer vision for object and people recognition, object manipulation, natural language processing, among others. This project is part of the SinfonIA alliance, made up of the Artificial Intelligence Competence Center of Bancolombia, Universidad de los Andes, Universidad Santo Tomás, and Universidad del Magdalena.

Autonomous Navigation

This tool is responsible for equipping Opera and Nova with sufficient skills to allow its sensors to map, locate and navigate autonomously through various environments in a robust manner. Our developments are under the ROS framework, and we use Python and C ++ as the main programming languages.

• David Santiago Ortiz Almanza (Electronic Engineering and Systems Engineering, Universidad de los Andes).
• Daniel Villar González (Electronic Engineering and Computer and systems engineering, Universidad de los Andes).
• David Santiago Vargas Prada (Electronic Engineering, Universidad de los Andes).

Speech Recognition

This tool enables Nova and Opera to have conversations with their environment and allows the extraction of information to perform assistance tasks. This process consists in obtaining the voice through their microphones and in making a speech-to-text transcription. This input is transformed using natural language processing (NLP). This allows that the dialogue established with the robot is responded to appropriately. During this process, various models are used to determine the relationship between audio signals and the different phonemes of natural language. These models match sounds which sentences, and, in this way, the robot can assimilate the conversation and continue with it.

• Susana Marcela Chavez Leyton (Electronic Engineering, Universidad de los Andes).
• Sergio Andres Guillen Fonseca (Computer and systems engineering, Universidad de los Andes).
• Juan Esteban Padilla Torres (Electronic Engineering, Universidad de los Andes).

Manipulation

This tool allows Opera and Nova to use their arms to manipulate objects and interact with them. The robot is supported on Move it, a software that allows to control the robot’s joint and movement paths. First, it goes through a process of recognition of initial coordinates – starting states-  and will define the coordinates (x, y, z) of each joint, as well as the angle with respect to each axis. Second, the final coordinate of each set must be defined -goal states-. Move it will allow us to plan a valid trajectory based on the mechanical limitations of the robot. In this way, it is possible to use the robot’s manipulators for various actions such as greeting, carrying objects or gesturing.

• Carlos Felipe Torres Usma (Mechanical Engineering, Universidad de los Andes).
• Luccas Rojas Becerra (Mechanical Engineering and Computer and Systems Engineering, Universidad de los Andes).

Perception

This tool enables Nova and Opera to detect objects and faces. It obtains information from the environment, and through inferences and relationships that objects have in the space, it provides support to other tools. This tool offers a more personalized experience by identifying particular things about each user, making a much warmer and more natural contact.

• Santiago Rodríguez Ávila (Electronic Engineering, Universidad de los Andes).
• Daniela Andrea Ruiz López (Biomedical engineering, Universidad de los Andes).
• Omar Esteban Vargas Salamanca (Computer and Systems Engineering, Universidad de los Andes).