Difference between revisions of "Cognitive Robotics Lecture Schedule"
From David Vernon's Wiki
| (26 intermediate revisions by the same user not shown) | |||
| Line 5: | Line 5: | ||
! scope="col" style="width: 3%;" | Lecture | ! scope="col" style="width: 3%;" | Lecture | ||
! scope="col" style="width: 15%;" |Topic | ! scope="col" style="width: 15%;" |Topic | ||
| − | ! scope="col" style="width: | + | ! scope="col" style="width: 25%;" | Material covered |
! scope="col" style="width: 10%;" | Required hardware | ! scope="col" style="width: 10%;" | Required hardware | ||
! scope="col" style="width: 9%;" | Required software | ! scope="col" style="width: 9%;" | Required software | ||
| − | ! scope="col" style="width: | + | ! scope="col" style="width: 20%;" | Reading |
| − | ! scope="col" style="width: | + | ! scope="col" style="width: 20%;" | Homework exercises |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| Tues. 17 Jan. | | Tues. 17 Jan. | ||
| Line 36: | Line 36: | ||
| OpenCV | | OpenCV | ||
| [http://www.vernon.eu/04-801/04-801_Cognitive_Robotics_03_Robot_Vision_II.pdf Lecture 3 Slides]. Vernon (1991), Sections 5.2, 6.4. | | [http://www.vernon.eu/04-801/04-801_Cognitive_Robotics_03_Robot_Vision_II.pdf Lecture 3 Slides]. Vernon (1991), Sections 5.2, 6.4. | ||
| − | | Assignment 1: Object detection, localization, and pose estimation using the Hough transform | + | | [http://www.vernon.eu/04-801/04-801_Assignment_1.pdf Assignment 1: Object detection, localization, and pose estimation using the Hough transform] |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| Thurs. 26 Jan. | | Thurs. 26 Jan. | ||
| Line 44: | Line 44: | ||
| USB camera | | USB camera | ||
| OpenCV <!--, Vienna University of Technology BLORT Library --> | | OpenCV <!--, Vienna University of Technology BLORT Library --> | ||
| − | | [http://www.vernon.eu/04-801/04- | + | | [http://www.vernon.eu/04-801/04-801_Cognitive_Robotics_04_Robot_Vision_III.pdf Lecture 4 Slides]. Hanbury 2002. Swain and Ballard 1991. <!-- Szeliski (2010), Sections 4.1.2, 4.1.3, 4.1.4, 4.1.5, 14.1.1. --> |
| − | | Face detection and object segmentation using OpenCV | + | |<!-- Face detection and object segmentation using OpenCV --> |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| Tues. 31 Jan. | | Tues. 31 Jan. | ||
| 5 | | 5 | ||
|Robot vision IV | |Robot vision IV | ||
| − | |Homogeneous coordinates and transformations. Perspective transformation. Camera model and inverse perspective transformation. Stereo vision. Epipolar geometry. Structured light & RGB-D cameras. | + | |Homogeneous coordinates and transformations. Perspective transformation. Camera model and inverse perspective transformation. <!-- Stereo vision. Epipolar geometry. Structured light & RGB-D cameras. --> |
|USB camera | |USB camera | ||
|OpenCV | |OpenCV | ||
| − | | | + | |[http://vernon.eu/04-801/04-801_Cognitive_Robotics_05_Robot_Vision_IV.pdf Lecture 5 Slides]. [http://vernon.eu/publications/91_Vernon_Machine_Vision.pdf Vernon (1991)], Section 8.6, 9.4.2. [http://vernon.eu/publications/95_Dawson-Howe_Vernon_IJIST.pdf Dawson-Howe and Vernon (1995)]. [http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html OpenCV documentation on camera calibration]. |
| − | |Camera calibration | + | |<!-- Camera calibration --> |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| Thurs. 2 Feb. | | Thurs. 2 Feb. | ||
| 6 | | 6 | ||
|Mobile robots I | |Mobile robots I | ||
| − | | | + | |Types of mobile robots. The challenge of robot navigation. Wheeled locomotion. Kinematics of a two-wheel differential drive robot. Inverse kinematics. |
| − | |Anki Cozmo mobile robot | + | |<!-- Anki Cozmo mobile robot --> |
| − | |Anki Cozmo SDK, OpenCV | + | |<!-- Anki Cozmo SDK, OpenCV --> |
| − | |Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | + | |[http://vernon.eu/04-801/04-801_Cognitive_Robotics_06_Mobile_Robots_I.pdf Lecture 6 Slides]. [http://www.vernon.eu/04-801/Vernon_2009.pdf Vernon (2009)]. <!-- Python tutorial. Cozmo SDK API. OpenCV Python tutorial. --> |
| − | |Cozmo locomotion. | + | |<!-- Cozmo locomotion. --> |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| Tues. 7 Feb. | | Tues. 7 Feb. | ||
| 7 | | 7 | ||
|Mobile robots II | |Mobile robots II | ||
| − | |Relative | + | |The position estimation problem. Relative position estimation. Odometry-based navigation. Absolute position estimation. Combined position estimation. |
| − | |Anki Cozmo mobile robot | + | |<!-- Anki Cozmo mobile robot --> |
| − | |Anki Cozmo SDK, OpenCV | + | |<!-- Anki Cozmo SDK, OpenCV --> |
| − | |Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | + | |[http://vernon.eu/04-801/04-801_Cognitive_Robotics_07_Mobile_Robots_II.pdf Lecture 7 Slides]. <!-- Python tutorial. Cozmo SDK API. OpenCV Python tutorial. --> |
| − | |Cozmo landmark recognition. | + | |<!-- Cozmo landmark recognition. --> |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| Thurs. 9 Feb. | | Thurs. 9 Feb. | ||
| 8 | | 8 | ||
|Mobile robots III | |Mobile robots III | ||
| − | |Map representation. Probabilistic map-based localization. Landmark-based localization. | + | |<!-- Map representation. Probabilistic map-based localization. Landmark-based localization --> Closed-Loop Control. Go-to-position problem. Divide and conquer controller. MIMO controller. Cozmo Robot. Python Cozmo SDK. |
|Anki Cozmo mobile robot | |Anki Cozmo mobile robot | ||
| − | |Anki Cozmo SDK | + | |Anki Cozmo SDK |
| − | |Python tutorial. Cozmo SDK API. | + | |[http://vernon.eu/04-801/04-801_Cognitive_Robotics_08_Mobile_Robots_III.pdf Lecture 8 Slides]. Python tutorial. Cozmo SDK API. |
| − | |Cozmo landmark recognition | + | |<!-- Cozmo landmark recognition --> |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| Tues. 14 Feb. | | Tues. 14 Feb. | ||
| 9 | | 9 | ||
|Mobile robots IV | |Mobile robots IV | ||
| − | |SLAM: simultaneous localization and mapping. Extended Kalman Filter (EKF) SLAM. Visual SLAM. Particle filter SLAM. | + | |<!-- SLAM: simultaneous localization and mapping. Extended Kalman Filter (EKF) SLAM. Visual SLAM. Particle filter SLAM. --> Path planning. The search problem in AI. Path planning as a search problem. Breadth-First Search (BFS): The Wavefront Algorithm. Depth-First Search (DFS). Heuristic Search. Greedy Search. A* Search. |
|Anki Cozmo mobile robot | |Anki Cozmo mobile robot | ||
| − | |Anki Cozmo SDK | + | |Anki Cozmo SDK |
| − | |Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | + | |<!-- Python tutorial. Cozmo SDK API. OpenCV Python tutorial. --> [http://vernon.eu/04-801/04-801_Cognitive_Robotics_09_Mobile_Robots_IV.pdf Lecture 9 Slides]. |
| − | |Cozmo object recognition | + | |<!-- Cozmo object recognition --> [http://www.vernon.eu/04-801/04-801_Assignment_2.pdf Assignment 2: Mobile robot locomotion] |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| Tues. 16 Feb. | | Tues. 16 Feb. | ||
| 10 | | 10 | ||
|Mobile robots V | |Mobile robots V | ||
| − | |Graph search path planning. Potential field path planning. Navigation. Obstacle avoidance. Object search. | + | |<!-- Graph search path planning. Potential field path planning. Navigation. Obstacle avoidance. Object search. -->The problem of autonomous navigation. Defining the system. The challenge of uncertainty. Architectures for autonomy. The Sense-Plan-Act architecture. The Reactive architecture. The Hybrid architecture. |
| − | |Anki Cozmo mobile robot | + | |<!-- Anki Cozmo mobile robot --> |
| − | |Anki Cozmo SDK, OpenCV | + | |<!-- Anki Cozmo SDK, OpenCV --> |
| − | |Python tutorial. Cozmo SDK API. OpenCV Python tutorial. | + | |<!-- Python tutorial. Cozmo SDK API. OpenCV Python tutorial. --> [http://vernon.eu/04-801/04-801_Cognitive_Robotics_10_Mobile_Robots_V.pdf Lecture 10 Slides]. |
| − | |Cozmo navigation | + | |<!-- Cozmo navigation --> |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| Tues. 21 Feb. | | Tues. 21 Feb. | ||
| 11 | | 11 | ||
|Robot arms I | |Robot arms I | ||
| − | | | + | |Robot programming. Description of object pose with homogenous transformations. Robot programming by frame-based task specification. |
| − | |Lynxmotion 5DoF arm, Arduino interface | + | |<!-- Lynxmotion 5DoF arm, Arduino interface --> |
| − | |Arduino sketch programs for Lynxmotion | + | |<!-- Arduino sketch programs for Lynxmotion--> |
| − | |Paul (1981), Chapters 1 & 2. | + | |[http://vernon.eu/04-801/04-801_Cognitive_Robotics_11_Robot_Arms_I.pdf Lecture 11 Slides]. Paul (1981), Chapters 1 & 2. Vernon (1991), Sections 8.1-8.4. |
| − | |Move end-effector along various paths in joint space | + | |<!-- Move end-effector along various paths in joint space --> |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| Thurs. 23 Feb. | | Thurs. 23 Feb. | ||
| 12 | | 12 | ||
|Robot arms II | |Robot arms II | ||
| − | |Analytic inverse kinematics. Iterative approaches. Kinematic structure learning. Kinematics structure correspondences. | + | |Robot programming by frame-based task specification <!-- Analytic inverse kinematics. Iterative approaches. Kinematic structure learning. Kinematics structure correspondences. --> |
| − | |Lynxmotion 5DoF arm, Arduino interface | + | |<!-- Lynxmotion 5DoF arm, Arduino interface --> |
| − | |Arduino sketch programs for Lynxmotion | + | |<!-- Arduino sketch programs for Lynxmotion --> |
| − | |Paul (1981), Chapter 3. | + | |[http://vernon.eu/04-801/04-801_Cognitive_Robotics_12_Robot_Arms_II.pdf Lecture 12 Slides]. Paul (1981), Chapters 1 & 2. Vernon (1991), Sections 8.1-8.4.<!-- Paul (1981), Chapter 3. --> |
| − | |Move end-effector along various paths in Cartesian frame of reference | + | |<!-- Move end-effector along various paths in Cartesian frame of reference --> |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| Tues. 28 Feb. | | Tues. 28 Feb. | ||
| 13 | | 13 | ||
|Robot arms III | |Robot arms III | ||
| − | | | + | |<!-- Vision-based pose estimation. --> Denavit-Hartenberg representation. Forward kinematics of a manipulator. |
|Lynxmotion 5DoF arm, Arduino interface | |Lynxmotion 5DoF arm, Arduino interface | ||
|Arduino sketch programs for Lynxmotion | |Arduino sketch programs for Lynxmotion | ||
| − | | | + | |[http://vernon.eu/04-801/04-801_Cognitive_Robotics_13_Robot_Arms_III.pdf Lecture 13 Slides]. |
| − | |Compute the pose of a light cube | + | |<!-- Compute the pose of a light cube --> |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| Thurs. 2 Mar. | | Thurs. 2 Mar. | ||
| 14 | | 14 | ||
| Robot arms IV | | Robot arms IV | ||
| − | | Programming by demonstration. Language-based programming. | + | | <!-- Programming by demonstration. Language-based programming. --> Inverse kinematics. From forward and inverse kinematics to internal simulation. Hesslow's simulation hypothesis. HAMMER. |
| Lynxmotion 5DoF arm, Arduino interface | | Lynxmotion 5DoF arm, Arduino interface | ||
| − | | Arduino | + | | [http://www.vernon.eu/downloads/arduino-1.8.1-windows.exe Arduino 1.8.1 for Windows] |
| − | | | + | [http://www.vernon.eu/downloads/inverse_kinematics_demo.ino Arduino inverse kinematics demo sketch] |
| − | | Implement a program to move light cube from one position/pose to another position/pose | + | [http://www.vernon.eu/downloads/CDM21224_Setup.exe CDM21224 COM port on USB.exe] |
| + | | [http://vernon.eu/04-801/04-801_Cognitive_Robotics_14_Robot_Arms_IV.pdf Lecture 14 Slides]. [http://vernon.eu/04-801/AbuQassem_et_al_2010.pdf Abu Qassem et al (2010)]. [http://vernon.eu/04-801/Gan_et_al_2005.pdf Gan et al (2005)]. | ||
| + | | <!-- Implement a program to move light cube from one position/pose to another position/pose --> [http://www.vernon.eu/04-801/04-801_Assignment_3.pdf Assignment 3: Robot Manipulation] | ||
|} | |} | ||
| Line 148: | Line 150: | ||
! scope="col" style="width: 3%;" | Lecture | ! scope="col" style="width: 3%;" | Lecture | ||
! scope="col" style="width: 15%;" |Topic | ! scope="col" style="width: 15%;" |Topic | ||
| − | ! scope="col" style="width: | + | ! scope="col" style="width: 25%;" | Material covered |
! scope="col" style="width: 10%;" | Required hardware | ! scope="col" style="width: 10%;" | Required hardware | ||
! scope="col" style="width: 9%;" | Required software | ! scope="col" style="width: 9%;" | Required software | ||
| − | ! scope="col" style="width: | + | ! scope="col" style="width: 20%;" | Reading |
| − | ! scope="col" style="width: | + | ! scope="col" style="width: 20%;" | Homework exercises |
|- style="vertical-align: top;" | |- style="vertical-align: top;" | ||
| TBD | | TBD | ||
Latest revision as of 05:17, 3 March 2017
Mini 1: Cognitive Robotics: Foundations
| Date | Lecture | Topic | Material covered | Required hardware | Required software | Reading | Homework exercises |
|---|---|---|---|---|---|---|---|
| Tues. 17 Jan. | 1 | Introduction | Motivation. Goals of the course. Syllabus and lecture schedule. Course operation. Industrial requirements for cognitive robots. Artificial cognitive systems. Cognitivist, emergent, and hybrid paradigms in cognitive science. Autonomy. AI and cognition in robotics. Software development tools for assignments. | None | None | Lecture 1 Slides. Vernon (2014), Chapters 1, 2, and 4. | Install software tools and run example assignment0 programs. |
| Thurs. 19 Jan. | 2 | Robot vision I | Computer vision. Optics, sensors, and image formation. Image acquisition. Fundamentals of image processing. Segmentation and edge detection. Introduction to OpenCV. | USB camera | OpenCV | Lecture 2 Slides. Vernon (1991), Sections 2.2.1, 2.2.2, 3.1, 4.1.4, 4.2.1, 5.1, 5.3.1. | Image acquisition and image processing using OpenCV |
| Tues. 24 Jan. | 3 | Robot vision II | Hough transform: line, circle, and generalized transform; extension to codeword features. | USB camera | OpenCV | Lecture 3 Slides. Vernon (1991), Sections 5.2, 6.4. | Assignment 1: Object detection, localization, and pose estimation using the Hough transform |
| Thurs. 26 Jan. | 4 | Robot vision III | Colour segmentation. Colour histogram intersection and back-projection. | USB camera | OpenCV | Lecture 4 Slides. Hanbury 2002. Swain and Ballard 1991. | |
| Tues. 31 Jan. | 5 | Robot vision IV | Homogeneous coordinates and transformations. Perspective transformation. Camera model and inverse perspective transformation. | USB camera | OpenCV | Lecture 5 Slides. Vernon (1991), Section 8.6, 9.4.2. Dawson-Howe and Vernon (1995). OpenCV documentation on camera calibration. | |
| Thurs. 2 Feb. | 6 | Mobile robots I | Types of mobile robots. The challenge of robot navigation. Wheeled locomotion. Kinematics of a two-wheel differential drive robot. Inverse kinematics. | Lecture 6 Slides. Vernon (2009). | |||
| Tues. 7 Feb. | 7 | Mobile robots II | The position estimation problem. Relative position estimation. Odometry-based navigation. Absolute position estimation. Combined position estimation. | Lecture 7 Slides. | |||
| Thurs. 9 Feb. | 8 | Mobile robots III | Closed-Loop Control. Go-to-position problem. Divide and conquer controller. MIMO controller. Cozmo Robot. Python Cozmo SDK. | Anki Cozmo mobile robot | Anki Cozmo SDK | Lecture 8 Slides. Python tutorial. Cozmo SDK API. | |
| Tues. 14 Feb. | 9 | Mobile robots IV | Path planning. The search problem in AI. Path planning as a search problem. Breadth-First Search (BFS): The Wavefront Algorithm. Depth-First Search (DFS). Heuristic Search. Greedy Search. A* Search. | Anki Cozmo mobile robot | Anki Cozmo SDK | Lecture 9 Slides. | Assignment 2: Mobile robot locomotion |
| Tues. 16 Feb. | 10 | Mobile robots V | The problem of autonomous navigation. Defining the system. The challenge of uncertainty. Architectures for autonomy. The Sense-Plan-Act architecture. The Reactive architecture. The Hybrid architecture. | Lecture 10 Slides. | |||
| Tues. 21 Feb. | 11 | Robot arms I | Robot programming. Description of object pose with homogenous transformations. Robot programming by frame-based task specification. | Lecture 11 Slides. Paul (1981), Chapters 1 & 2. Vernon (1991), Sections 8.1-8.4. | |||
| Thurs. 23 Feb. | 12 | Robot arms II | Robot programming by frame-based task specification | Lecture 12 Slides. Paul (1981), Chapters 1 & 2. Vernon (1991), Sections 8.1-8.4. | |||
| Tues. 28 Feb. | 13 | Robot arms III | Denavit-Hartenberg representation. Forward kinematics of a manipulator. | Lynxmotion 5DoF arm, Arduino interface | Arduino sketch programs for Lynxmotion | Lecture 13 Slides. | |
| Thurs. 2 Mar. | 14 | Robot arms IV | Inverse kinematics. From forward and inverse kinematics to internal simulation. Hesslow's simulation hypothesis. HAMMER. | Lynxmotion 5DoF arm, Arduino interface | Arduino 1.8.1 for Windows
Arduino inverse kinematics demo sketch CDM21224 COM port on USB.exe |
Lecture 14 Slides. Abu Qassem et al (2010). Gan et al (2005). | Assignment 3: Robot Manipulation |
Mini 2: Cognitive Robotics: Principles and Practice
| Date | Lecture | Topic | Material covered | Required hardware | Required software | Reading | Homework exercises |
|---|---|---|---|---|---|---|---|
| TBD | 1 | Cognitive architectures I | Role and requirements; cognitive architecture schemas; example cognitive architectures including Soar, ACT-R, Clarion, LIDA, and ISAC. The Standard Model. | Vernon (2014) Chapter 3. Chella et al. (2013). Scheutz et al. (2013). Vernon et al. (2016). | Group discussion on which cognitive architectures are suitable for cognitive robotics | ||
| TBD | 2 | Cognitive architectures II | CRAM: Cognitive Robot Abstract Machine. CRAM Plan Language (CPL). KnowRob knowledge processing and reasoning | CRAM | Beetz et al. (2010) | CRAM test programs | |
| TBD | 3 | Cognitive architectures III | Knowledge representation, processing, and reasoning. | KnowRob and OpenEASE | Beetz et al. (2015) | OpenEASE test programs | |
| TBD | 4 | Learning and development I | Supervised, unsupervised, and reinforcement learning. Hebbian learning. | MaxHebb library | Harmon and Harmon (1997) | Hebbian learning | |
| TBD | 5 | Learning and development II | Predictive sequence learning (PSL). | Anki Cozmo mobile robot | Anki Cozmo SDK, PSL library | Sun and Giles (2001). Billing et al. (2011, 2016). | PSL test programs |
| TBD | 6 | Learning and development III | Learning from demonstration | Anki Cozmo mobile robot | Anki Cozmo SDK, PSL library | Vernon (2014), Chapters 6 & 8. Billard et al. (2008). Argall (2009). | PSL test programs |
| TBD | 7 | Learning and development IV | Cognitive development in humans and robots. Value systems for developmental and cognitive robots. | Vernon (2014), Chapters 6 & 9. Lungarella et al. (2003). Asada et al. (2009). Cangelosi and Schlesinger (2015), Chapters 1 & 2. Merrick (2016). Vernon et al. (2016). | |||
| TBD | 8 | Memory and Prospection | Declarative vs. procedural memory. Semantic memory. Episodic memory | Anki Cozmo mobile robot | Anki Cozmo SDK, CINDY library, OpenCV | Vernon (2014), Chapter 7. | Implementation of episodic memory on Cozmo |
| TBD | 9 | Internal simulation I | Episodic future thinking. Forward and inverse models. Internal simulation hypothesis, Internal simulation with PSL | Anki Cozmo mobile robot | Anki Cozmo SDK, PSL library | Vernon (2014), Chapter 8. Billing et al. (2016). | PSL test programs |
| TBD | 10 | Internal simulation II | HAMMER cognitive architecture | Boost, Imperial College London HAMMER library | Demiris and Khadhouri (2006). Sarabia et al. (2011). | HAMMER tutorial using the ICL library | |
| TBD | 11 | Social interaction I | Joint action. Joint attention. Shared intention. Shared goals. Perspective taking. Theory of mind. | Kinect RGB-D sensor | Ubuntu 14.04, ROS, Imperial College London Perspective Taking library | Vernon (2014), Chapter 9. Fisher and Demiris 2016. | Perspective taking using the ICL library. |
| TBD | 12 | Social interaction II | Action and intention recognition. Embodied cognition. Humanoid robotics. | Kinect RGB-D sensor | Ubuntu 14.04, ROS, Imperial College London Perspective Taking library. | Vernon (2014), Chapter 9. | Perspective taking using the ICL library |
| TBD | 13 | ||||||
| 14 |
Back to Cognitive Robotics
