Difference between revisions of "Cognitive Robotics Lecture Plan"
From David Vernon's Wiki
| Line 11: | Line 11: | ||
! Homework exercises | ! Homework exercises | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | Tues. | + | | Tues. 17 Jan. |
| 1 | | 1 | ||
| Cognitive robotics | | Cognitive robotics | ||
| Line 20: | Line 20: | ||
| Installation of software tools. | | Installation of software tools. | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | Thurs. | + | | Thurs. 19 Jan. |
| 2 | | 2 | ||
| Robot vision I | | Robot vision I | ||
| Line 29: | Line 29: | ||
|Image acquisition and image processing using OpenCV | |Image acquisition and image processing using OpenCV | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 24 Jan. |
| 3 | | 3 | ||
| Robot vision II | | Robot vision II | ||
| Line 38: | Line 38: | ||
| Hough transforms and colour segmentation using OpenCV | | Hough transforms and colour segmentation using OpenCV | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Thurs. 26 Jan. |
| 4 | | 4 | ||
| Robot vision III | | Robot vision III | ||
| Line 47: | Line 47: | ||
| Face detection and object recognition using OpenCV | | Face detection and object recognition using OpenCV | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 31 Jan. |
| 5 | | 5 | ||
|Robot vision IV | |Robot vision IV | ||
| Line 56: | Line 56: | ||
|Camera calibration | |Camera calibration | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Thurs. 2 Feb. |
| 6 | | 6 | ||
|Robot vision V | |Robot vision V | ||
| Line 65: | Line 65: | ||
|Analysis of point cloud data from RGB-D camera | |Analysis of point cloud data from RGB-D camera | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 7 Feb. |
| 7 | | 7 | ||
|Robot vision VI | |Robot vision VI | ||
| Line 74: | Line 74: | ||
|Implementation of a saliency function for covert attention | |Implementation of a saliency function for covert attention | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Thurs. 9 Feb. |
| 8 | | 8 | ||
|Mobile robots I | |Mobile robots I | ||
| Line 83: | Line 83: | ||
|Cozmo locomotion. | |Cozmo locomotion. | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 14 Feb. |
| 9 | | 9 | ||
|Mobile robots II | |Mobile robots II | ||
| Line 92: | Line 92: | ||
|Cozmo landmark recognition. | |Cozmo landmark recognition. | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Thurs. 16 Feb. |
| 10 | | 10 | ||
|Mobile robots III | |Mobile robots III | ||
| Line 101: | Line 101: | ||
|Cozmo landmark recognition | |Cozmo landmark recognition | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 21 Feb. |
| 11 | | 11 | ||
|Mobile robots IV | |Mobile robots IV | ||
| Line 110: | Line 110: | ||
|Cozmo object recognition | |Cozmo object recognition | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 23 Feb. |
| 12 | | 12 | ||
|Mobile robots V | |Mobile robots V | ||
| Line 119: | Line 119: | ||
|Cozmo navigation | |Cozmo navigation | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 18 Feb. |
| 13 | | 13 | ||
|Robot arms I | |Robot arms I | ||
| Line 128: | Line 128: | ||
|Move end-effector along various paths in joint space | |Move end-effector along various paths in joint space | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Thurs. 2 Mar. |
| 14 | | 14 | ||
|Robot arms II | |Robot arms II | ||
| Line 137: | Line 137: | ||
|Move end-effector along various paths in Cartesian frame of reference | |Move end-effector along various paths in Cartesian frame of reference | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 7 Mar. |
| 15 | | 15 | ||
|Robot arms III | |Robot arms III | ||
| Line 146: | Line 146: | ||
|Compute the pose of a light cube | |Compute the pose of a light cube | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Thurs. 9 Mar. |
| 16 | | 16 | ||
|Robot arms IV | |Robot arms IV | ||
| Line 155: | Line 155: | ||
|Implement a program to move light cube from one position/pose to another position/pose | |Implement a program to move light cube from one position/pose to another position/pose | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 14 Mar. |
| 17 | | 17 | ||
|Cognitive architectures I | |Cognitive architectures I | ||
| Line 164: | Line 164: | ||
|Group discussion on which cognitive architectures are suitable for cognitive robotics | |Group discussion on which cognitive architectures are suitable for cognitive robotics | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Thurs. 16 Mar. |
| 18 | | 18 | ||
|Cognitive architectures II | |Cognitive architectures II | ||
| Line 173: | Line 173: | ||
|CRAM test programs | |CRAM test programs | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 21 Mar. |
| 19 | | 19 | ||
|Cognitive architectures III | |Cognitive architectures III | ||
| Line 182: | Line 182: | ||
|OpenEASE test programs | |OpenEASE test programs | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Thurs. 23 Mar. |
| 20 | | 20 | ||
|Learning and development I | |Learning and development I | ||
| Line 191: | Line 191: | ||
|Hebbian learning | |Hebbian learning | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 28 Mar. |
| 21 | | 21 | ||
|Learning and development II | |Learning and development II | ||
| Line 200: | Line 200: | ||
|PSL test programs | |PSL test programs | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Thurs. 30 Mar. |
| 22 | | 22 | ||
|Learning and development III | |Learning and development III | ||
| Line 209: | Line 209: | ||
|PSL test programs | |PSL test programs | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 4 Apr. |
| 23 | | 23 | ||
|Learning and development IV | |Learning and development IV | ||
| Line 218: | Line 218: | ||
| | | | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Thurs. 6 Apr. |
| 24 | | 24 | ||
|Memory and Prospection | |Memory and Prospection | ||
| Line 227: | Line 227: | ||
|Implementation of episodic memory on Cozmo | |Implementation of episodic memory on Cozmo | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 18 Apr. |
| 25 | | 25 | ||
|Internal simulation I | |Internal simulation I | ||
| Line 236: | Line 236: | ||
|PSL test programs | |PSL test programs | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Thurs. 20 Apr. |
| 26 | | 26 | ||
|Internal simulation II | |Internal simulation II | ||
| Line 245: | Line 245: | ||
|HAMMER tutorial using the ICL library | |HAMMER tutorial using the ICL library | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Tues. 25 Apr. |
| 27 | | 27 | ||
|Social interaction I | |Social interaction I | ||
| Line 254: | Line 254: | ||
|Perspective taking using the ICL library. | |Perspective taking using the ICL library. | ||
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| − | | | + | | Thurs. 27 Apr. |
| 28 | | 28 | ||
|Social interaction II | |Social interaction II | ||
| Line 262: | Line 262: | ||
|Vernon (2014), Chapter 9. | |Vernon (2014), Chapter 9. | ||
|Perspective taking using the ICL library | |Perspective taking using the ICL library | ||
| + | |- style="vertical-align: top;" | ||
| + | | Tues. 2 May | ||
| + | | 29 | ||
| + | | | ||
| + | | | ||
| + | | | ||
| + | | | ||
| + | | | ||
| + | | | ||
| + | |- style="vertical-align: top;" | ||
| + | | Thurs. 5 May | ||
| + | | 30 | ||
| + | | | ||
| + | | | ||
| + | | | ||
| + | | | ||
| + | | | ||
| + | | | ||
|} | |} | ||
</small> | </small> | ||
Revision as of 02:10, 27 December 2016
| Date | Lecture | Topic | Material covered | Required hardware | Required software | Pre-class reading | Homework exercises |
|---|---|---|---|---|---|---|---|
| Tues. 17 Jan. | 1 | Cognitive robotics | Introduction to AI and cognition in robotics. Industrial requirements. Artificial cognitive systems. Cognitivist, emergent, and hybrid paradigms in cognitive science. Autonomy. Introduction to OpenCV and course software and hardware. | None | None | Vernon (2014), Chapters 1, 2, and 4. | Installation of software tools. |
| Thurs. 19 Jan. | 2 | Robot vision I | Optics, sensors, and image formation. Image acquisition. Image filtering. Edge detection. Introduction to OpenCV. | USB camera | OpenCV | Kragic and Vincze (2010). Szeliski (2010), Sections 1.1, 1.2, 2.3, 3.2, 4.2. Vernon (1991), Sections 2.1, 2.1 | Image acquisition and image processing using OpenCV |
| Tues. 24 Jan. | 3 | Robot vision II | Segmentation. Hough transform: line, circle, and generalized transform; extension to codeword features. Colour-based segmentation. | USB camera | OpenCV | Szeliski (2010), Sections 3.1.2, 3.3.4, 4.3.2. Vernon (1991), Section 3.1, 3.2, 3.3, 4.2.1, 4.2.2, 5.3, 6.4. | Hough transforms and colour segmentation using OpenCV |
| Thurs. 26 Jan. | 4 | Robot vision III | Object recognition. Interest point operators. Gradient orientation histogram - SIFT descriptor. Colour histogram intersection. Haar features, boosting, face detection. | USB camera | OpenCV, Vienna University of Technology BLORT Library | Szeliski (2010), Sections 4.1.2, 4.1.3, 4.1.4, 4.1.5, 14.1.1. | Face detection and object recognition using OpenCV |
| Tues. 31 Jan. | 5 | Robot vision IV | Homogeneous coordinates and transformations. Perspective transformation. Camera model and inverse perspective transformation. Stereo vision. Epipolar geometry. Structured light & RGB-D cameras. | USB camera | OpenCV | Szeliski (2010), Sections 2.1, 11.1, 11.2, 11.3. Vernon (1991), Section 8.6, 9.4.2. | Camera calibration |
| Thurs. 2 Feb. | 6 | Robot vision V | Plane pop-out. RANSAC. Differential geometry. Surface normals and Gaussian sphere. Point clouds. 3D descriptors. | Kinect RGB-D sensor | Technische Universität Wien RGB-D Segmentation Library and V4R Library | Szeliski (2010), Sections 12.4. Point Cloud Library tutorial. | Analysis of point cloud data from RGB-D camera |
| Tues. 7 Feb. | 7 | Robot vision VI | Visual attention. Spatial & selective attention. Saliency functions. Selective Tuning. Overt attention. Inhibition of return. Habituation. Top-down attention. | USB camera | CINDY cognitive architecture | Implementation of a saliency function for covert attention | |
| Thurs. 9 Feb. | 8 | Mobile robots I | Differential drive locomotion. Forward and inverse kinematics. Holonomic and non-holonomic constraints. Cozmo mobile robot. | Anki Cozmo mobile robot | Anki Cozmo SDK, OpenCV | Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | Cozmo locomotion. |
| Tues. 14 Feb. | 9 | Mobile robots II | Relative and absolute position estimation. Odometry. | Anki Cozmo mobile robot | Anki Cozmo SDK, OpenCV | Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | Cozmo landmark recognition. |
| Thurs. 16 Feb. | 10 | Mobile robots III | Map representation. Probabilistic map-based localization. Landmark-based localization. | Anki Cozmo mobile robot | Anki Cozmo SDK, OpenCV | Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | Cozmo landmark recognition |
| Tues. 21 Feb. | 11 | Mobile robots IV | SLAM: simultaneous localization and mapping. Extended Kalman Filter (EKF) SLAM. Visual SLAM. Particle filter SLAM. | Anki Cozmo mobile robot | Anki Cozmo SDK, OpenCV | Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | Cozmo object recognition |
| Tues. 23 Feb. | 12 | Mobile robots V | Graph search path planning. Potential field path planning. Navigation. Obstacle avoidance. Object search. | Anki Cozmo mobile robot | Anki Cozmo SDK, OpenCV | Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | Cozmo navigation |
| Tues. 18 Feb. | 13 | Robot arms I | Homogeneous transformations. Frame-based pose specification. Denavit-Hartenberg specifications. Robot kinematics. | Lynxmotion 5DoF arm, Arduino interface | Arduino sketch programs for Lynxmotion | Paul (1981), Chapters 1 & 2. | Move end-effector along various paths in joint space |
| Thurs. 2 Mar. | 14 | Robot arms II | Analytic inverse kinematics. Iterative approaches. Kinematic structure learning. Kinematics structure correspondences. | Lynxmotion 5DoF arm, Arduino interface | Arduino sketch programs for Lynxmotion | Paul (1981), Chapter 3. | Move end-effector along various paths in Cartesian frame of reference |
| Tues. 7 Mar. | 15 | Robot arms III | Robot manipulation. Frame-based task specification. Vision-based pose estimation. | Lynxmotion 5DoF arm, Arduino interface | Arduino sketch programs for Lynxmotion | Vernon (1991), Sections 8.1-8.4. | Compute the pose of a light cube |
| Thurs. 9 Mar. | 16 | Robot arms IV | Programming by demonstration. Language-based programming. | Lynxmotion 5DoF arm, Arduino interface | Arduino sketch programs for Lynxmotion | Vernon (1991), Sections 8.1-8.4 | Implement a program to move light cube from one position/pose to another position/pose |
| Tues. 14 Mar. | 17 | Cognitive architectures I | Role and requirements; cognitive architecture schemas; example cognitive architectures including Soar, ACT-R, Clarion, LIDA, and ISAC. The Standard Model. | Vernon (2014) Chapter 3. Chella et al. (2013). Scheutz et al. (2013). Vernon et al. (2016). | Group discussion on which cognitive architectures are suitable for cognitive robotics | ||
| Thurs. 16 Mar. | 18 | Cognitive architectures II | CRAM: Cognitive Robot Abstract Machine. CRAM Plan Language (CPL). KnowRob knowledge processing and reasoning | CRAM | Beetz et al. (2010) | CRAM test programs | |
| Tues. 21 Mar. | 19 | Cognitive architectures III | Knowledge representation, processing, and reasoning. | KnowRob and OpenEASE | Beetz et al. (2015) | OpenEASE test programs | |
| Thurs. 23 Mar. | 20 | Learning and development I | Supervised, unsupervised, and reinforcement learning. Hebbian learning. | MaxHebb library | Harmon and Harmon (1997) | Hebbian learning | |
| Tues. 28 Mar. | 21 | Learning and development II | Predictive sequence learning (PSL). | Anki Cozmo mobile robot | Anki Cozmo SDK, PSL library | Sun and Giles (2001). Billing et al. (2011, 2016). | PSL test programs |
| Thurs. 30 Mar. | 22 | Learning and development III | Learning from demonstration | Anki Cozmo mobile robot | Anki Cozmo SDK, PSL library | Vernon (2014), Chapters 6 & 8. Billard et al. (2008). Argall (2009). | PSL test programs |
| Tues. 4 Apr. | 23 | Learning and development IV | Cognitive development in humans and robots. Value systems for developmental and cognitive robots. | Vernon (2014), Chapters 6 & 9. Lungarella et al. (2003). Asada et al. (2009). Cangelosi and Schlesinger (2015), Chapters 1 & 2. Merrick (2016). Vernon et al. (2016). | |||
| Thurs. 6 Apr. | 24 | Memory and Prospection | Declarative vs. procedural memory. Semantic memory. Episodic memory | Anki Cozmo mobile robot | Anki Cozmo SDK, CINDY library, OpenCV | Vernon (2014), Chapter 7. | Implementation of episodic memory on Cozmo |
| Tues. 18 Apr. | 25 | Internal simulation I | Episodic future thinking. Forward and inverse models. Internal simulation hypothesis, Internal simulation with PSL | Anki Cozmo mobile robot | Anki Cozmo SDK, PSL library | Vernon (2014), Chapter 8. Billing et al. (2016). | PSL test programs |
| Thurs. 20 Apr. | 26 | Internal simulation II | HAMMER cognitive architecture | Boost, Imperial College London HAMMER library | Demiris and Khadhouri (2006). Sarabia et al. (2011). | HAMMER tutorial using the ICL library | |
| Tues. 25 Apr. | 27 | Social interaction I | Joint action. Joint attention. Shared intention. Shared goals. Perspective taking. Theory of mind. | Kinect RGB-D sensor | Ubuntu 14.04, ROS, Imperial College London Perspective Taking library | Vernon (2014), Chapter 9. Fisher and Demiris 2016. | Perspective taking using the ICL library. |
| Thurs. 27 Apr. | 28 | Social interaction II | Action and intention recognition. Embodied cognition. Humanoid robotics. | Kinect RGB-D sensor | Ubuntu 14.04, ROS, Imperial College London Perspective Taking library. | Vernon (2014), Chapter 9. | Perspective taking using the ICL library |
| Tues. 2 May | 29 | ||||||
| Thurs. 5 May | 30 |
Back to Cognitive Robotics
