An Introduction to Cognitive Robotics

(alpha version)

David Vernon

davidvernon.eu

Course Description  |  Learning Objectives  |  Content  |  Lecture Notes  |  Course Textbook  |  Recommended Reading |  Software

Course Description

The course covers both the essentials of classical robotics (mobile robots and robot arms for manipulation) and the principles of cognition (cognitive architectures, learning and development, prospection, memory, knowledge representation, internal simulation, and meta-cognition). It brings these components together by working through some recent advances in robotics for everyday activities, and by including practical and detailed material based on the CRAM (Cognitive Robot Abstract Machine) cognitive architecture, incorporating the KnowRob knowledge base, building on ROS (Robot Operating System) and exploiting functional, object-oriented, and logic programming to reason about and execute under-specified tasks in everyday activities.

This course emphasizes both theory and practice and makes use of physical robots and robot simulators for visual sensing and actuation.

Support for the preparation of this course was provided by grant from the IEEE Robotics and Automation Society under the program Creation of Educational Material in Robotics and Automation (CEMRA) 2020

Learning Objectives

After completing this course, students should be able to:

  1. Apply their knowledge of machine vision and robot kinematics to create computer programs that control mobile robots and robot arms, enabling the robots to recognize and manipulate objects and navigate their environments.
  2. Explain how a robot can be designed to exhibit cognitive goal-directed behaviour through the integration of computer models of vision, reasoning, learning, prospection, and social interaction.
  3. Use computer programs that realize limited instances of these faculties.

Course Content

Overview of Cognitive Robotics (Lectures 1 - 2)

  • The emerging discipline of cognitive robotics.
  • What does it mean to be cognitive?
  • Why is cognition useful in robotics?
  • How do cognitive robots work?
  • Goals of the course.
  • Learning outcomes.
  • Syllabus and lecture schedule.
  • Course operation.
  • Industrial requirements for cognitive robots.
  • Installation of software development environment for exercises and assignments.

Mobile Robotics (Lectures 3 - 6)

  • Types of mobile robots.
  • Wheeled locomotion.
  • Kinematics of a two-wheel differential drive robot.
  • Relative, absolute, and combined position estimation.
  • Odometry-based position estimation.
  • Closed-loop control.
  • Go-to-position problem.
  • Divide and conquer controller.
  • MIMO controller.
  • Introduction to ROS (Robot Operating System).
  • Using the ROS Turtlebot simulator.
  • Writing ROS software.
  • Path planning.
  • The search problem in AI.
  • Path planning as a search problem.
  • Breadth-First Search (BFS): The Wavefront Algorithm.
  • Other search strategies.
  • Robot control architectures.

Robot Arms (Lectures 7 - 10)

  • Robot programming.
  • Description of object pose with homogenous transformations.
  • Robot programming by frame-based task specification.
  • Example robot programming using task specification.
  • Forward and inverse kinematics.
  • The LynxMotion AL5D arm.

Robot Vision (Lectures 11 - 14)

  • Computer vision.
  • Optics and sensors.
  • Image acquisition and image representation.
  • Image processing.
  • Introduction to OpenCV.
  • Segmentation.
  • Region-based approaches: binary thresholding, colour segmentation, connected component analysis, graph cuts.
  • Boundary-based approaches: edge detection, boundary representations. I
  • mage Analysis. Inspection, location, identification.
  • Feature extraction.
  • Classification: nearest neighbour classifier / minimum distance classifier; linear classifier; (naive) maximum likelihood classifier / naive Bayes classifier.
  • Homogeneous coordinates and transformations.
  • Perspective transformation.
  • Camera model and inverse perspective transformation.
  • Camera calibration.

Artificial Cognitive Systems (Lectures 15 - 16)

  • The nature of cognition.
  • Paradigms of cognitive science.
  • Learning and development.
  • Memory and prospection.
  • Knowledge and representation.
  • Social cognition.

Robot Cognitive Architectures (Lectures 17 - 18)

  • Role and requirements.
  • Core cognitive abilities.
  • Example cognitive architectures: Soar, Clarion, ISAC.
  • The Common Model of Cognition.

The CRAM Cognitive Architecture (Lectures 19 - 28)

  • Overview of the CRAM (Cognitive Robot Abstract Machine) cognitive architecture.
  • Essentials of Common Lisp.
  • Emacs.
  • The CRAM plan language.
  • Description of object pose with quaternions in ROS.
  • CRAM and ROS Turtlebot simulator.
  • Fetch-and-place CRAM plan for the PR2 robot using Bullet Real-time Physics Simulation

Lecture Notes

Lecture 1. Overview of cognitive robotics  
Lecture 2. Robot vision I  
Lecture 3. Robot vision II
Lecture 4. Robot vision III  
Lecture 5. Robot vision IV  
Lecture 6. Mobile robotics I  
Lecture 7. Mobile robotics II  
Lecture 8. Mobile robotics III  
Lecture 9. Learning from demonstration 
Lecture 10. Robot arms I  
Lecture 11. Robot arms II  
Lecture 12. Robot arms III  
Lecture 13. Robot Cognitive Architectures I 
Lecture 14. Robot Cognitive Architectures II  

Course Textbook

In preparation.

Recommended Reading

Beetz, M., Mösenlechner, L., and Tenorth, M. (2010). "CRAM - A Cognitive Robot Abstract Machine for Everyday Manipulation in Human Environments", IEEE/RSJ International Conference on Intelligent Robots and Systems, 1012-1017.

Corke, P. (2016). Robotics, Vision and Control, 2nd Edition, Springer.

Paul, R. (1981). Robot Manipulators: Mathematics, Programming, and Control. MIT Press.

Vernon, D. (1991). Machine Vision: Automated Visual Inspection and Robot Vision, Prentice-Hall.

D. Vernon, Artificial Cognitive Systems, MIT Press, 2014.

Vernon, D. and Vincze, M. "Industrial Priorities for Cognitive Robotics", Proceedings of EUCognition 2016, Cognitive Robot Architectures, European Society for Cognitive Systems, Vienna, 8-9 December, 2016, R. Chrisley. V. C. Müller, Y. Sandamirskaya. M. Vincze (eds.), CEUR-WS Vol-1855, ISSN 1613-0073, pp. 6-9.

Software Development Environment

Lecture 2 has detailed instructions for installing the software required for the various exercises in the course.

David Vernon's Personal Website