Cognitive Robotics

David Vernon
Carnegie Mellon University Africa in Rwanda

Course Description  |  Learning Objectives  |  Content  |  Lecture Notes  |  Course Textbook  |  Recommended Reading |  Software

Course Description

Cognitive robotics is an emerging discipline that draws on robotics, artificial intelligence, and cognitive science. A cognitive ability is useful in robotics because:

  • It allows the robot to work autonomously in challenging environments, adapting to changes and unforeseen situations, and anticipating outcomes when selecting the actions it will perform.
  • It fosters interaction with people. Humans have a strong preference for interaction with other cognitive agents so being able to exhibit a capacity for cognition encourages human robot interaction. Conversely, a cognitive ability provides the robot with the ability to infer the goals and intentions of the person it is interacting with and thereby allows it to do so in safe and helpful manner.
This course covers the key elements of traditional robotics, building on these to introduce the essentials of cognitive robotics. It emphasizes both theory and practice and makes use of physical robots, both a mobile robot and a manipulator arm, robot simulators, and robot vision.

Back to Top

Learning Objectives

After completing this course, students should be able to:

  1. Apply their knowledge of machine vision and robot kinematics to create computer programs that control mobile robots and robot arms, enabling the robots to recognize and manipulate objects and navigate their environments.
  2. Explain how a robot can be designed to exhibit cognitive goal-directed behaviour through the integration of computer models of visual attention, reasoning, learning, prospection, and social interaction.
  3. Use computer programs that realize limited instances of these faculties.

Back to Top

Course Content

Overview of cognitive robotics

  • Goals of the course, syllabus, lecture schedule, and course operation.
  • Artificial cognitive systems. AI and cognition in robotics.
  • Industrial requirements for cognitive robots.
  • Software development tools for assignments.

Robot vision

  • Computer vision.
  • Optics and sensors.
  • Image acquisition and image representation.
  • Image processing.
  • Introduction to OpenCV.
  • Segmentation.
  • Region-based approaches: binary thresholding, colour segmentation, connected component analysis, graph cuts.
  • Boundary-based approaches: edge detection, boundary representations.
  • Image Analysis.
  • Inspection, location, identification.
  • Feature extraction.
  • Classification: nearest neighbour classifier / minimum distance classifier; linear classifier; (naive) maximum likelihood classifier / naive bayes classifier.
  • Homogeneous coordinates and transformations.
  • Perspective transformation.
  • Camera model and inverse perspective transformation.
  • Camera calibration.

Mobile Robotics

  • Types of mobile robots.
  • Wheeled locomotion; kinematics of a two-wheel differential drive robot.
  • The position estimation problem: relative position estimation, odometry-based navigation, absolute position estimation, combined position estimation.
  • Closed-Loop Control.
  • Go-to-position problem.
  • Divide and conquer controller.
  • MIMO controller.
  • Cozmo.
  • Path planning: the search problem in AI; path planning as a search problem.
  • Breadth-First Search (BFS): The Wavefront Algorithm.
  • Depth-First Search (DFS).
  • Heuristic Search.
  • Greedy Search.
  • A* Search.
  • Robot control architectures.

Learning from Demonstration

  • Types of learning from demonstration.
  • Predictive sequence learning

Robot Arms

  • Robot programming.
  • Description of object pose with homogenous transformations.
  • Robot programming by frame-based task specification.
  • Example robot programming using task specification
  • Forward and inverse kinematics.
  • The LynxMotion AL5D arm.

Robot Cognitive Architectures

  • Role and requirements.
  • Example cognitive architectures.
  • ISAC.
  • The Standard Model.
  • CRAM: Cognitive Robot Abstract Machine.
  • CRAM Plan Language (CPL).
  • KnowRob knowledge processing and reasoning.
  • CRAM PR2 simulation.

Lecture Notes

Lecture 1. Overview of cognitive robotics  
Lecture 2. Robot vision I  
Lecture 3. Robot vision II
Lecture 4. Robot vision III  
Lecture 5. Robot vision IV  
Lecture 6. Mobile robotics I  
Lecture 7. Mobile robotics II  
Lecture 8. Mobile robotics III  
Lecture 9. Learning from demonstration 
Lecture 10. Robot arms I  
Lecture 11. Robot arms II  
Lecture 12. Robot arms III  
Lecture 13. Robot Cognitive Architectures I 
Lecture 14. Robot Cognitive Architectures II  

Back to Top

Course Textbook

D. Vernon, Artificial Cognitive Systems, MIT Press, 2014.

Back to Top

Recommended Reading

Argall, B. D., Chernova, S., Veloso, M., and Browning, B. (2009). "A survey of robot learning from demonstration". Robotics and Autonomous Systems, 57:469-483, 2009.

Beetz, M., Mösenlechner, L., and Tenorth, M. (2010). "CRAM - A Cognitive Robot Abstract Machine for Everyday Manipulation in Human Environments", IEEE/RSJ International Conference on Intelligent Robots and Systems, 1012-1017.

Billard, A., Calinon, S., Dillmann, R. and Schaal, S. (2008). "Robot programming by demonstration". In Springer Handbook of Robotics, pages 1371-1394.

Billing, E., Hellström, T., and Janlert, L-E. (2011). "Predictive Learning from Demonstration", in ICAART 2010, CCIS 129, Filipe, J., Fred, A., and Sharp, B. (Eds.), pp. 186-200.

Kragic, D. and Vincze, M. (2010). "Vision for Robotics", Foundation and Trends in Robotics, Vol 1, No 1, pp 1-78.

Paul, R. (1981). Robot Manipulators: Mathematics, Programming, and Control. MIT Press.

Vernon, D. (1991). Machine Vision: Automated Visual Inspection and Robot Vision, Prentice-Hall, 1991.

Vernon, D. (2014). Artificial Cognitive Systems - A Primer, MIT Press, 2014.

D. Vernon and M. Vincze, "Industrial Priorities for Cognitive Robotics", Proceedings of EUCognition 2016, Cognitive Robot Architectures, European Society for Cognitive Systems, Vienna, 8-9 December, 2016, R. Chrisley. V. C. Müller, Y. Sandamirskaya. M. Vincze (eds.), CEUR-WS Vol-1855, ISSN 1613-0073, pp. 6-9.

Back to Top

Software Development Environment

Click here for a step-by-step guide to downloading, installing, and using the software required to run examples and complete the assignments.

David Vernon's Personal Website