Cognitive Robotics Lectures and Labs

Week	Lecture	Topic	Material covered	Required hardware	Required software	Pre-class reading	Homework exercises
1	1	Cognitive robotics	Introduction to AI and cognition in robotics. Industrial requirements. Artificial cognitive systems. Cognitivist, emergent, and hybrid paradigms in cognitive science. Autonomy.	None	None	Vernon (2014), Chapters 1, 2, and 4.	Installation of software tools.
1	2	Robot  vision I	Optics, sensors, and image formation. Image acquisition. Image filtering. Edge detection.	USB camera	OpenCV	Kragic and Vincze (2010). Szeliski (2010), Sections 1.1, 1.2, 2.3, 3.2, 4.2. Vernon (1991), Sections 2.1, 2.1	Image acquisition and image processing using OpenCV
2	3	Robot  vision II	Segmentation. Hough transform: line, circle, and generalized transform; extension to codeword features. Colour-based segmentation.	USB camera	OpenCV	Szeliski (2010), Sections 3.1.2, 3.3.4, 4.3.2. Vernon (1991), Section 3.1, 3.2, 3.3, 4.2.1, 4.2.2, 5.3, 6.4.	Hough transforms and colour segmentation using OpenCV
2	4	Robot  vision III	Object recognition. Interest point operators. Gradient orientation histogram - SIFT descriptor. Colour histogram intersection. Haar features, boosting, face detection.	USB camera	OpenCV, Vienna University of Technology BLORT Library	Szeliski (2010), Sections 4.1.2, 4.1.3, 4.1.4, 4.1.5, 14.1.1.	Face detection and object recognition using OpenCV
3	5	Robot  vision IV	Homogeneous coordinates and transformations. Perspective transformation. Camera model and inverse perspective transformation. Stereo vision. Epipolar geometry. Structured light & RGB-D cameras.	USB camera	OpenCV	Szeliski (2010), Sections 2.1, 11.1, 11.2, 11.3. Vernon (1991), Section 8.6, 9.4.2.	Camera calibration
3	6	Robot  vision V	Visual attention. Plane pop-out. RANSAC. Differential geometry. Surface normals and Gaussian sphere. Point clouds. 3D descriptors.	Kinect RGB-D sensor	Vienna University of Technology RGB-D Segmentation Library and V4R Library	Szeliski (2010), Sections 12.4. Point Cloud Library tutorial.	Analysis of point cloud data from RGB-D camera
4	7	Mobile robots I	Differential drive locomotion. Forward and inverse kinematics. Holonomic and non-holonomic constraints. Cozmo mobile robot.	Anki Cozmo mobile robot	Anki Cozmo SDK, OpenCV	Python tutorial. Cozmo SDK API. OpenCV Python tutorial.	Cozmo locomotion (e.g. program Cozmo to drive along a pre-determined route and perform face detection)
4	8	Mobile robots II	Map representation. Probabilistic map-based localization. Landmark-based localization.	Anki Cozmo mobile robot	Anki Cozmo SDK, OpenCV	Python tutorial. Cozmo SDK API. OpenCV Python tutorial.	Cozmo landmark recognition
5	9	Mobile robots III	SLAM: simultaneous localization and mapping. Extended Kalman Filter (EKF) SLAM. Visual SLAM. Particle filter SLAM.	Anki Cozmo mobile robot	Anki Cozmo SDK, OpenCV	Python tutorial. Cozmo SDK API. OpenCV Python tutorial.	Cozmo object recognition
5	10	Mobile robots IV	Graph search path planning. Potential field path planning. Navigation. Obstacle avoidance. Object search.	Anki Cozmo mobile robot	Anki Cozmo SDK, OpenCV	Python tutorial. Cozmo SDK API. OpenCV Python tutorial.	Cozmo navigation
6	11	Robot arms I	Homogeneous transformations. Frame-based pose specification. Denavit-Hartenberg specifications. Robot kinematics.	Lynxmotion 5DoF arm, Arduino interface	Arduino sketch programs for Lynxmotion	Paul (1981), Chapters 1 & 2.	Move end-effector along various paths in joint space
6	12	Robot arms II	Analytic inverse kinematics. Iterative approaches. Kinematic structure learning. Kinematics structure correspondences.	Lynxmotion 5DoF arm, Arduino interface	Arduino sketch programs for Lynxmotion	Paul (1981), Chapter 3.	Move end-effector along various paths in Cartesian frame of reference
7	13	Robot arms III	Robot manipulation. Frame-based task specification. Vision-based pose estimation.	Lynxmotion 5DoF arm, Arduino interface	Arduino sketch programs for Lynxmotion	Vernon (1991), Sections 8.1-8.4.	Compute the pose of a light cube
7	14	Robot arms IV	Language-based programming. Programming by demonstration.	Lynxmotion 5DoF arm, Arduino interface	Arduino sketch programs for Lynxmotion	Vernon (1991), Sections 8.1-8.4	Implement a program to move light cube from one position/pose to another position/pose
8	15
8	16
9	17	Cognitive architectures I	Role and requirements; cognitive architecture schemas; example cognitive architectures including Soar, ACT-R, Clarion, LIDA, and ISAC. The Standard Model.			Vernon (2014) Chapter 3. Chella et al. (2013). Scheutz et al. (2013). Vernon et al. (2016).	Group discussion on which cognitive architectures are suitable for cognitive robotics
9	18	Cognitive architectures II	CRAM: Cognitive Robot Abstract Machine. CRAM Plan Language (CPL). KnowRob knowledge processing and reasoning		CRAM	Beetz et al. (2010)	CRAM test programs
10	19	Learning and development I	Supervised, unsupervised, and reinforcement learning. Hebbian learning.		MaxHebb library	Harmon and Harmon (1997)	Hebbian learning
10	20	Learning and development II	Predictive sequence learning (PSL).	Anki Cozmo mobile robot	Anki Cozmo SDK, PSL library	Sun and Giles (2001). Billing et al. (2011, 2016).	PSL test programs
11	21	Learning and development III	Cognitive development in humans and robots.	Anki Cozmo mobile robot	Anki Cozmo SDK	Lungarella et al. (2003). Asada et al. (2009). Cangelosi and Schlesinger (2015), Chapters 1 & 2.	PSL test programs
11	22	Learning and development IV	Value systems for developmental and cognitive robots.			Merrick (2016). Vernon et al. (2016).	Group discussion on cognitive development in robotics
12	23	Memory and Prospection	Declarative vs. procedural memory. Semantic memory. Episodic memory	Anki Cozmo mobile robot	Anki Cozmo SDK, CINDY library, OpenCV	Vernon (2014), Chapter 7.	Implementation of episodic memory on Cozmo
12	24	Internal simulation I	Forward and inverse models, internal simulation hypothesis, internal simulation with PSL	Anki Cozmo mobile robot	Anki Cozmo SDK, PSL library	Vernon (2014), Chapter 8. Billing et al. (2016).	Exercises on PSL test programs
13	25	Internal simulation II	HAMMER cognitive architecture		Boost, Imperial College London HAMMER library	Demiris and Khadhouri (2006). Sarabia et al. (2011).	Exercise on HAMMER tutorial using the ICL library
13	26	Visual attention	Visual attention. Spatial attention vs. selective attention. Saliency functions. Selective Tuning. Overt attention. Inhibition of return. Habituation. Top-down attention.	Anki Cozmo mobile robot	Anki Cozmo SDK, CINDY library	Borji and Itti (2013).	Implement visual attention on Cozmo
14	27	Social interaction I	Joint action. Joint attention. Shared intention. Shared goals. Perspective taking. Theory of mind.	Orabec Astra RGB-D sensor	Ubuntu 14.04, ROS, Imperial College London Perspective Taking library	Vernon (2014), Chapter 9.	Exercise on perspective taking using the ICL library
14	28	Social interaction II	Action and intention recognition. Learning from demonstration. Humanoid robotics.	Kinect RGB-D sensor	PSL library	Billard et al. (2008). Argall (2009).	Exercise on learning from demonstration using the PSL library

---

Back to Cognitive Robotics