Difference between revisions of "Applied Computer Vision"

From David Vernon's Wiki
Jump to: navigation, search
(→‎Content details)
 
(43 intermediate revisions by the same user not shown)
Line 13: Line 13:
 
<span style="color:#AB0000">Lecture/Lab/Rep hours/week: 4 hours lectures/week</span>
 
<span style="color:#AB0000">Lecture/Lab/Rep hours/week: 4 hours lectures/week</span>
  
−
<span style="color:#AB0000">Semester: Spring</span>
+
<span style="color:#AB0000">Semester: Fall</span>
  
 
<span style="color:#AB0000">Pre-requisites: programming skills </span>  
 
<span style="color:#AB0000">Pre-requisites: programming skills </span>  
  
−
Students are expected to be familiar with programming in at least one programming language, ideally C/C++.   
+
Students are expected to be proficient in programming in at least one programming language, ideally C/C++.   
  
 
==<span style="color:#AB0000">Course description</span> ==
 
==<span style="color:#AB0000">Course description</span> ==
  
−
Many organizations today are incorporating computer hardware and software into the products that they design and build. Most of these organizations' primary competencies are not computer science or software engineering, but rather they find that automation makes their products smarter, more capable, and more appealing in the market place. Because deep domain knowledge is needed to build these products, these organizations often hire engineers from traditional engineering disciplines to design and build the product platform, in many cases requiring them to write software to make the product actually work. These are capable engineers from many disciplines other than software engineering and unfortunately they usually learn software engineering on the job. This process typically involves considerable trial and error and often results in poorly designed and documented systems, defect laden software, bloated product development costs, unmaintainable software, and missed opportunities to leverage software development investments.
+
This course provides students with a solid foundation in the key elements of computer vision, emphasizing the practical application of the underlying theory. It focusses mainly on the techniques required to build robot vision applications but the algorithms can also be applied in other domains such as industrial inspection and video surveillance. A key focus of the course is on effective implementation of solutions to practical computer vision problems in a variety of environments using both bespoke software authored by the students and standard computer vision libraries.
−
 
+
−
In addition to developing mere functionality, some application domains are often highly constrained and unforgiving in their quality attribute needs such as performance, safety, and availability. These systems intimately depend upon software to provide these capabilities in addition to basic functionality. Designing software intensive systems with these properties in a cost-effective way requires first-class computer science and software engineering expertise. While many practicing engineers often have many years of industrial experience writing software applications, many lack a formal background in computer science principles. These engineers may have had a few courses or technical training in specific computer languages or technologies, but in general they often lack formal training in algorithms, computing theory, data structures, and design among other key topics. The result is that many of these engineers are not fully realizing their potential as software engineers. This course is designed to bridge these gaps in formal computer science training.
+
  
 
==<span style="color:#AB0000">Learning objectives</span> ==
 
==<span style="color:#AB0000">Learning objectives</span> ==
 +
The course covers optics, sensors, image formation, image acquisition & image representation before proceeding to the essentials of image processing and image filtering.  This provides the basis for a treatment of image segmentation, including edge detection, region growing, and boundary detection, the Hough transform, and colour-based segmentation.
 +
 +
Building on this, the course then proceeds to deal with object detection and recognition in 2D, addressing template matching, interest point operators, gradient orientation histograms, the SIFT descriptor, and colour histogram intersection and back-projection. 
  
−
The primary objective of the course is to provide engineers without formal training in computer science, a solid background in the key principles of computer science, in general, and of the algorithms and data-structures, in particular. The key purpose of this course is to complement the experience that engineers may already have in writing software with formal computer science underpinnings, making those engineers more capable in developing software intensive systems.  
+
The problem of recovery of 3D information is then addressed, introducing homogeneous coordinates and transformations, the perspective transformation, camera model, inverse perspective transformation, stereo vision, and epipolar geometry.  
  
−
The course begins by considering the main phases of the software development lifecycle, from requirements elicitation, to computational modelling, system specification, software design, implementation, and software quality assurance, including various forms of testing, verification, and validation. Then, building on the concept of abstract data types, the course provides an in in-depth treatment of the key elements of algorithms and data-structures, beginning with the fundamentals of searching, sorting, lists, stacks, and queues, but quickly progressing to more advanced topics, including trees, graphs, and algorithmic strategies. It also covers the analysis of the performance and tractability of algorithms, finishing with automata theory and computability theory. A key focus of the course is on effective implementation and good design principles.
+
The interpretation of visual information in unstructured environments poses many problems. To deal with these, the course then addresses visual attention, clustering, grouping, and segmentation, building on Gestalt principles, before proceeding to deal with object detection, object recognition, and object categorization in both 2D and 3D.
  
 
==<span style="color:#AB0000">Outcomes</span> ==
 
==<span style="color:#AB0000">Outcomes</span> ==
  
 
After completing this course, students should be able to:
 
After completing this course, students should be able to:
−
* Recognize and analyze critical computational problems, generate alternative solutions to problems, and assess their relative merits.
+
* Apply their knowledge of image acquisition, image processing, and image analysis to extract useful information from visual images.
−
* Understand, analyze, and characterize those factors that influence algorithmic computational performance and memory consumption.
+
* Design, implement, and document appropriate, effective, and efficient software solutions for a variety of real-world computer vision problems.
−
* Design, implement, and document appropriate, effective, and efficient data structures & algorithms for a variety of real-world problems.
+
* Exploit standard computer vision software libraries in the development of these solutions.
−
* Understand detailed software structures and their underlying strengths and weaknesses.
+
−
* Perform detailed, code-level design and document the design in an understandable way.
+
  
 
==<span style="color:#AB0000">Content details</span> ==
 
==<span style="color:#AB0000">Content details</span> ==
 
Refer to the  
 
Refer to the  
−
'''[[Data Structures and Algorithms for Engineers Lecture Schedule|Lecture Schedule]]'''  
+
'''[[Applied Computer Vision Lecture Schedule|Lecture Schedule]]'''  
−
<!-- '''[[Data Structures and Algorithms for Engineers Lecture Schedule 2|Lecture Schedule]]''' -->
+
 
for information on course delivery, including lectures, labs, assignments, and exercises.
 
for information on course delivery, including lectures, labs, assignments, and exercises.
  
−
The course will cover the following topics:
+
The course will cover the following topics (this list is provisional and will be finalized before the start of the course):
  
−
* Introduction: The Software Development Life Cycle
+
# Overview of human and computer vision.
−
* Formalisms for Representing Algorithms
+
# OpenCV and software development tools for course work
−
* Analysis of Complexity
+
# Optics, sensors, and image formation.
−
* Searching and Sorting Algorithms
+
# Image acquisition and image representation.
−
* Abstract Data Types (ADT)
+
# Image processing: point & neighbourhood operations, image filtering, convolution, Fourier transform.
−
* Containers, Dictionaries, and Lists
+
# Image processing: morphological operations.
−
* Stacks
+
# Image processing: geometric operations.
−
* Queues
+
# Segmentation: region-based approaches, binary thresholding, connected component analysis.
−
* Trees
+
# Segmentation: edge detection.
−
* Heaps
+
# Segmentation: colour-based approaches; k-means clustering.
−
* Graphs
+
# Image features: Harris and Difference of Gaussian interest point operators.
−
* Complex Networks
+
# Image features: SIFT feature descriptor.
−
* Hashing
+
# Object recognition: template matching;  normalized cross-correlation; chamfer matching.
−
* Algorithmic Strategies
+
# Object recognition: 2D shape features; statistical pattern recognition.
−
* Analysis of Correctness
+
# Object recognition: Hough transform for parametric curves: lines, circles, and ellipses.
−
<!-- * Databases -->
+
# Object recognition: generalized Hough transform; extension to codeword features. 
−
<!-- * Programming Paradigms -->
+
# Object recognition: colour histogram matching and back-projection.
−
* Automata Theory
+
# Object recognition: Haar features and boosted classifiers.
−
* Computability Theory
+
# Object recognition: Histogram of Oriented Gradients (HOG) feature descriptor.
−
 
+
# Video image processing: background subtraction and object tracking
−
''The detailed content for each of these topics follows.''
+
# 3D vision: Homogeneous coordinates and transformations. Perspective transformation.  Camera model and inverse perspective transformation.
−
 
+
# Stereo vision. Epipolar geometry. 
−
 
+
# Optical flow.
−
<span style="color:#AB0000">Introduction: The Software Development Life Cycle</span><BR>
+
# Visual attention. Saliency. Bottom-up and top-down attention.
−
* Motivation
+
# Clustering, grouping, and segmentation. Gestalt principles. Clustering algorithms.
−
* Goals of the course
+
# Object recognition in 3D. Object detection, object recognition, object categorisation.
−
* Syllabus and lecture schedule
+
# Affordances.
−
* Course operation
+
# Computer vision and machine learning.
−
* Software development tools for assignments
+
<!-- ''The detailed content for each of these topics follows.'' -->
−
* Preview of course material
+
<!-- <span style="color:#AB0000">Optics, sensors, image formation</span><BR> -->
−
* Levels of abstraction in information processing systems
+
<!-- * xyz -->
−
* The software development life cycle: Yourdon Structured Analysis - functional, data, and behavioural models (hierarchical decomposition trees, architecture diagrams, data flow diagrams DFD, data dictionaries, entity relationship ER diagrams, state transition diagrams)
+
−
* Software process models: waterfall, evolutionary, formal transformation, re-use, hybrid, spiral
+
−
 
+
−
<span style="color:#AB0000">Formalisms for Representing Algorithms</span><BR>
+
−
* Definition of an algorithm
+
−
* Modelling software
+
−
* Relational modelling
+
−
* State modelling
+
−
* Practical representations
+
−
* Pseudo code
+
−
* Flow charts
+
−
* Finite state machines
+
−
* UML
+
−
* Predicate logic
+
−
* Analysis
+
−
 
+
−
<span style="color:#AB0000">Analysis of Complexity</span><BR>
+
−
* Performance of algorithms, time and space tradeoff, worst case and average case performance
+
−
* Big O notation
+
−
* Recurrence relationships
+
−
* Analysis of complexity of iterative and recursive algorithms
+
−
* Recursive vs. iterative algorithms: runtime memory implications
+
−
* Complexity theory: tractable vs intractable algorithmic complexity
+
−
* Example intractable problems: travelling salesman problem, Hamiltonian circuit, 3-colour problem, SAT, cliques
+
−
* Determinism and non-determinism
+
−
* P, NP, and NP-Complete classes of algorithm
+
−
 
+
−
<span style="color:#AB0000">Searching and Sorting Algorithms</span><BR>
+
−
* Linear and binary search (iterative and recursive)
+
−
* In-place sorts: bubblesort (efficient and inefficient), selection sort, insertion sort.
+
−
* Not-in-place sorts: Quicksort, merge sort.
+
−
* Complexity analysis
+
−
* Characteristics of a good sort
+
−
* Speed, consistency, keys, memory usage, length & code complexity, stability
+
−
* Other sorts ordered by complexity
+
−
 
+
−
<span style="color:#AB0000">Abstract Data Types (ADT)</span><BR>
+
−
*  Vector example exercise
+
−
* History of abstraction
+
−
* Abstract Data Types (ADT)
+
−
* Information hiding
+
−
* Types and typing
+
−
* Encapsulation
+
−
* Efficiency
+
−
* Design practices
+
−
 
+
−
<span style="color:#AB0000">Containers, Dictionaries, and Lists</span><BR>
+
−
* Basic operations
+
−
* Implementation with arrays and linked lists
+
−
* Singly linked lists
+
−
* Doubly linked lists
+
−
* Performance considerations
+
−
 
+
−
<span style="color:#AB0000">Stacks</span><BR>
+
−
* Stack (LIFO): push, pop, peek, size, numItems operations
+
−
* Array implementation (directly and array of pointers to data)
+
−
* Stack applications, including evaluation of infix, prefix, and postfix expressions
+
−
 
+
−
<span style="color:#AB0000">Queues</span><BR>
+
−
* Queue (FIFO): enqueue, dequeue, peek, size, numItems operations
+
−
* Array implementation (directly and array of pointers to data)
+
−
* Linked list implementation
+
−
* Circular queues
+
−
* Performance considerations
+
−
<!-- * Deque -->
+
−
 
+
−
<span style="color:#AB0000">Trees</span><BR>
+
−
* Concepts and terminology: level, height, external and internal nodes, skinny, fat, complete, left-complete, perfect, multi-way, d-ary
+
−
* Types of tree: binary, binary search, B-tree, 2-3 tree, AVL, Red-Black
+
−
* Binary trees and binary search trees
+
−
* Tree traversals: inorder, preorder, postorder
+
−
* Fixed-length codes, variable length codes, optimal code trees, Huffman's algorithm and implementation
+
−
* Height-balanced trees: AVL Trees, RR, RL, LR, LL rotations
+
−
* Height-balanced trees: Red-Black Trees, single promotion, zig-zag promotion, recolouring and restructuring
+
−
<!-- * Non-search trees: parse trees, array implementation, linked list implementation -->
+
−
<!-- * Forests -->
+
−
 
+
−
<span style="color:#AB0000">Heaps</span><BR>
+
−
* Heap basics
+
−
* Types of heap: min heaps and max heap
+
−
* Heap characteristics
+
−
* Heap operations: delete max/min, down heap, up heap, merge, construct, heapify; complexity of operations
+
−
* Priority queues
+
−
* Operating systems heaps
+
−
* Implementation of heap
+
−
* Heap sort
+
−
<!-- * d-ary heaps -->
+
−
<!-- * Leftist  heaps -->
+
−
 
+
−
<span style="color:#AB0000">Graphs</span><BR>
+
−
* Types of graphs: directed, undirected, weighted, unweighted, cyclic, acyclic, directed acyclic, simple, non-simple, implicit, explicit, embedded, topological
+
−
* Adjacency matrix representation,
+
−
* Adjacency list representation
+
−
* Topological sort
+
−
* Euler's theorem
+
−
* Graph traversal: breadth-first and depth-first, uses of
+
−
* Depth-first search and maze traversal
+
−
* Spanning trees and minimum spanning trees, Kruskal's algorithm, Prim's algorithm
+
−
* Dijkstra's shortest path algorithm
+
−
* Floyd-Warshall's all-pairs algorithm
+
−
<!-- * Graphs problems: routes, Hamilton paths, network flows, covering problems, museum guard problem -->
+
−
<!-- * Fleury's Euler circuit algorithm -->
+
−
 
+
−
<span style="color:#AB0000">Complex Networks</span><BR>
+
−
<!-- * Complex systems and large networks
+
−
* Community detection
+
−
* Network evolution -->
+
−
* The importance of complex networks and network science
+
−
* Euler's theorem: the Bridges of Königsberg
+
−
* Networks vs. graphs
+
−
* Degree, average degree, and degree distribution
+
−
* Bipartite networks
+
−
* Path length, BFS, Connectivity, Components
+
−
* Clustering coefficient
+
−
* Random graph model
+
−
* Small world phenomena
+
−
* Scale free networks
+
−
* Communities
+
−
* Fundamental Hypothesis
+
−
* Connectedness and Density Hypothesis
+
−
* Strong and weak communities
+
−
* Graph partitioning
+
−
* Community detection
+
−
* Hierarchical clustering
+
−
* Girvan-Newman Algorithm
+
−
* Modularity
+
−
* Random Hypothesis
+
−
* Maximum Modularity Hypothesis
+
−
* Greedy algorithm for community detection by maximizing modularity
+
−
* Overlapping communities
+
−
* Clique percolation algorithm and CFinder
+
−
 
+
−
<span style="color:#AB0000">Hashing</span><BR>
+
−
*  Using keys to address data
+
−
* Mappings: injection, surjection, bijection
+
−
* Map ADT
+
−
* Hash functions
+
−
* Hash tables: current value tables, direct access tables
+
−
* Managing collisions: chaining, overflow areas, re-hashing, linear probing, quadratic probing
+
−
<!-- * Evaluating hash functions: prime division, mid-square, folding, load factor -->
+
−
* Example application: dictionaries
+
−
<!-- * Generating hash functions and using hash structures -->
+
−
 
+
−
<span style="color:#AB0000">Algorithmic Strategies</span><BR>
+
−
* Classes of algorithms
+
−
* Brute force
+
−
* Divide and conquer
+
−
* Greedy algorithms
+
−
* Dynamic programming
+
−
* Combinatorial search and backtracking
+
−
* Branch and bound
+
−
<!-- * Heuristics and heuristic algorithms -->
+
−
<!-- * Probabilistic algorithms -->
+
−
 
+
−
<span style="color:#AB0000">Analysis of Correctness</span><BR>
+
−
* Types of software defects
+
−
* Code module design
+
−
* Syntactic, semantic, logical defects
+
−
* (Semi-)formal verification: partial vs. total correctness
+
−
* Invariant assertion method
+
−
* Simple proof strategies: by contradiction, counterexample, induction
+
−
* Dynamic testing: unit tests, test harness, stubs, drivers, integration testing, regression testing.
+
−
* Static tests: reviews, walkthroughs, inspections, reviewing algorithms and software
+
−
* Pair programming
+
−
* Verification and validation strategies
+
−
 
+
−
<!--
+
−
<span style="color:#AB0000">Databases</span><BR>
+
−
* Databases: relational databases, hierarchical databases, NoSQL databases
+
−
* Relational databases: entity relationship modelling, relational algebra, SQL, normalization
+
−
* Compression strategies, dictionary algorithm, LZ algorithm
+
−
* File structure strategies.
+
−
-->
+
−
<!--  
+
−
<span style="color:#AB0000">Programming Paradigms</span><BR>
+
−
* Imperative programming
+
−
* Logic programming
+
−
* Functional programming
+
−
* OO programming; classes; type, operational and functional polymorphism; inheritance, attributes, methods, instantiations, abstract classes, object-oriented languages
+
−
* OO design methodology: UML class diagram, composite-structure diagram, architecture diagram, package diagram, object diagram, component diagram, deployment diagram, activity diagram, sequence diagram, communication diagram, interaction diagram, timing diagram, use case diagram, state machine diagram
+
−
* OO design principles: open/close principle, design by contract principle, dependency inversion principle, other design principles, documentation
+
−
-->
+
−
<span style="color:#AB0000">Automata Theory</span><BR>
+
−
* Regular Languages
+
−
* Finite Automata
+
−
* Nondeterminism
+
−
* Regular Expressions
+
−
* Nonregular Languages
+
−
*Context-free Languages
+
−
* Context-free Grammars
+
−
* Pushdown Automata
+
−
* Deterministic Context-Free Languages
+
−
 
+
−
<span style="color:#AB0000">Computability Theory</span><BR>
+
−
* The Church-Turing Thesis
+
−
* Turing Machines
+
−
* Variants of Turing Machines
+
−
* The Definition of Algorithm
+
−
* Decidability
+
−
* Decidable Languages
+
−
* Undecidability
+
−
* Reducibility
+
  
 
==<span style="color:#AB0000">Lecture Schedule</span> ==
 
==<span style="color:#AB0000">Lecture Schedule</span> ==
−
Refer to the [[Data Structures and Algorithms for Engineers Lecture Schedule|Lecture Schedule]] for information on course delivery, including lectures, labs, assignments, and exercises.
+
Refer to the [[Applied Computer Vision Lecture Schedule|Lecture Schedule]] for information on course delivery, including lectures, labs, assignments, and exercises.
  
 
==<span style="color:#AB0000">Faculty</span> ==
 
==<span style="color:#AB0000">Faculty</span> ==
Line 292: Line 88:
  
 
==<span style="color:#AB0000">Student assessment</span> ==  
 
==<span style="color:#AB0000">Student assessment</span> ==  
−
This course includes several hands-on programming and analysis assignments. Students will program mainly in C/C++. The programming assignments include individual assignments and a team capstone project in teams of 2-3 people. In addition to programming assignments, students will be assigned readings to support the lecture material.
+
This course includes several hands-on programming and analysis assignments. Students will program mainly in C/C++ using OpenCV and other computer vision libraries. In addition to programming assignments, students will be assigned readings to support the lecture material.
  
−
Marks will be awarded as follows.
+
Marks will be awarded as follows (yet to be finalized).
  
−
Seven individual assignments 70%.
+
Four individual assignments 60%.
−
Final examination  30%.
+
Final examination  40%.
−
<!-- Instructor Judgement 10% (We will use time tracking and observation to determine this part of the grade). -->
+
  
 
==<span style="color:#AB0000">Software tools</span> ==
 
==<span style="color:#AB0000">Software tools</span> ==
−
We will use Microsoft Visual C++ Express compiler, version 10.0 (also known as Visual C++ 2010) and CMake running on Windows 7 (64 bit). 
+
Please follow the instructions provided in the [[ACV Software Development Environment|Software Development Environment]] installation guide.
  
−
Please follow the instructions provided in the [[04-630 Software Development Environment|Software Development Environment]] installation guide.
+
==<span style="color:#AB0000">Course text</span> ==
  
−
==<span style="color:#AB0000">Course texts</span> ==
+
[http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf Szeliski, R. (2010). Computer Vision: Algorithms and Applications, Springer. ]
  
−
David Harel and Yishai Feldman, ''Algorithmics: The Spirit of Computing'', Third Edition.
+
==<span style="color:#AB0000">Recommended reading</span> ==
  
−
Alfred V. Aho,  Jeffrey D. Ullman, and John E. Hopcroft, ''Data Structures and Algorithms''.
+
[http://ilab.usc.edu/publications/doc/Borji_Itti13pami.pdf Borji, A. and Itti, L. (2013). "State-of-the-Art in Visual Attention Modeling", IEEE Transactions on Pattern Analysis and Machine intelligence, Vol. 35, No. 1, pp. 185-207. ]
  
−
A selection of examples will be taken from Steven Skiena, "The Algorithm Design Manual", Second Edition.
+
Dawson-Howe, K. (2014). ''A Practical Introduction to Computer Vision with OpenCV'', Wiley.
  
−
A selection of papers and readings will be provided to complement these required textbooks.
+
Hanbury, A. ''The Taming of the Hue, Saturation, and Brightness Colour Space'', Proc. Computer Vision Winter Workshop (CVWW), Austria, 2002.
 +
 
 +
Kragic, D. and Vincze, M. (2010). "Vision for Robotics", Foundation and Trends in Robotics, Vol 1, No 1, pp 1–78.
 +
 
 +
[http://homepages.inf.ed.ac.uk/rbf/BOOKS/VERNON/vernon.htm Vernon, D. (1991). ''Machine Vision: Automated Visual Inspection and Robot Vision'', Prentice-Hall.]
  
 
==<span style="color:#AB0000">Acknowledgments</span> ==
 
==<span style="color:#AB0000">Acknowledgments</span> ==
−
This syllabus is based mainly on Course 04-630 Computer Science Principles for Practicing Engineers given by Mel Rosso-Llopart and Anthony J. Lattanze at Carnegie Mellon University, USA, Course CS-CO-412 Algorithms and Data Structures given by David Vernon at Innopolis University, Russia, and Course IT706A Scientific Theory in Informatics given by David Vernon and others at the University of Skövde, Sweden.
+
The syllabus for this course drew inspiration from several sources. These include the following.
 +
 
 +
* Course VO 4.0 376.054 Machine Vision and Cognitive Robotics given by Markus Vincze, Michael Zillich, and Daniel Wolf at Technische Universität Wien.
 +
* Course 4BA10 Computer Vision given by David Vernon at Trinity College Dublin.
 +
* Course 4BA10 Computer Vision given by Kenneth-Dawson Howe at Trinity College Dublin.
 +
* Course on Computer Vision at VVV2017 by Francesca Odone, University of Genova.

Latest revision as of 03:37, 5 July 2017

|CARNEGIE MELLON UNIVERSITY AFRICA|

04-800
Applied Computer Vision

Elective

Units: 12

Lecture/Lab/Rep hours/week: 4 hours lectures/week

Semester: Fall

Pre-requisites: programming skills

Students are expected to be proficient in programming in at least one programming language, ideally C/C++.

Course description

This course provides students with a solid foundation in the key elements of computer vision, emphasizing the practical application of the underlying theory. It focusses mainly on the techniques required to build robot vision applications but the algorithms can also be applied in other domains such as industrial inspection and video surveillance. A key focus of the course is on effective implementation of solutions to practical computer vision problems in a variety of environments using both bespoke software authored by the students and standard computer vision libraries.

Learning objectives

The course covers optics, sensors, image formation, image acquisition & image representation before proceeding to the essentials of image processing and image filtering. This provides the basis for a treatment of image segmentation, including edge detection, region growing, and boundary detection, the Hough transform, and colour-based segmentation.

Building on this, the course then proceeds to deal with object detection and recognition in 2D, addressing template matching, interest point operators, gradient orientation histograms, the SIFT descriptor, and colour histogram intersection and back-projection.

The problem of recovery of 3D information is then addressed, introducing homogeneous coordinates and transformations, the perspective transformation, camera model, inverse perspective transformation, stereo vision, and epipolar geometry.

The interpretation of visual information in unstructured environments poses many problems. To deal with these, the course then addresses visual attention, clustering, grouping, and segmentation, building on Gestalt principles, before proceeding to deal with object detection, object recognition, and object categorization in both 2D and 3D.

Outcomes

After completing this course, students should be able to:

  • Apply their knowledge of image acquisition, image processing, and image analysis to extract useful information from visual images.
  • Design, implement, and document appropriate, effective, and efficient software solutions for a variety of real-world computer vision problems.
  • Exploit standard computer vision software libraries in the development of these solutions.

Content details

Refer to the Lecture Schedule for information on course delivery, including lectures, labs, assignments, and exercises.

The course will cover the following topics (this list is provisional and will be finalized before the start of the course):

  1. Overview of human and computer vision.
  2. OpenCV and software development tools for course work
  3. Optics, sensors, and image formation.
  4. Image acquisition and image representation.
  5. Image processing: point & neighbourhood operations, image filtering, convolution, Fourier transform.
  6. Image processing: morphological operations.
  7. Image processing: geometric operations.
  8. Segmentation: region-based approaches, binary thresholding, connected component analysis.
  9. Segmentation: edge detection.
  10. Segmentation: colour-based approaches; k-means clustering.
  11. Image features: Harris and Difference of Gaussian interest point operators.
  12. Image features: SIFT feature descriptor.
  13. Object recognition: template matching; normalized cross-correlation; chamfer matching.
  14. Object recognition: 2D shape features; statistical pattern recognition.
  15. Object recognition: Hough transform for parametric curves: lines, circles, and ellipses.
  16. Object recognition: generalized Hough transform; extension to codeword features.
  17. Object recognition: colour histogram matching and back-projection.
  18. Object recognition: Haar features and boosted classifiers.
  19. Object recognition: Histogram of Oriented Gradients (HOG) feature descriptor.
  20. Video image processing: background subtraction and object tracking
  21. 3D vision: Homogeneous coordinates and transformations. Perspective transformation. Camera model and inverse perspective transformation.
  22. Stereo vision. Epipolar geometry.
  23. Optical flow.
  24. Visual attention. Saliency. Bottom-up and top-down attention.
  25. Clustering, grouping, and segmentation. Gestalt principles. Clustering algorithms.
  26. Object recognition in 3D. Object detection, object recognition, object categorisation.
  27. Affordances.
  28. Computer vision and machine learning.

Lecture Schedule

Refer to the Lecture Schedule for information on course delivery, including lectures, labs, assignments, and exercises.

Faculty

David Vernon

Delivery

Face-to-face.

Student assessment

This course includes several hands-on programming and analysis assignments. Students will program mainly in C/C++ using OpenCV and other computer vision libraries. In addition to programming assignments, students will be assigned readings to support the lecture material.

Marks will be awarded as follows (yet to be finalized).

Four individual assignments 60%. Final examination 40%.

Software tools

Please follow the instructions provided in the Software Development Environment installation guide.

Course text

Szeliski, R. (2010). Computer Vision: Algorithms and Applications, Springer.

Recommended reading

Borji, A. and Itti, L. (2013). "State-of-the-Art in Visual Attention Modeling", IEEE Transactions on Pattern Analysis and Machine intelligence, Vol. 35, No. 1, pp. 185-207.

Dawson-Howe, K. (2014). A Practical Introduction to Computer Vision with OpenCV, Wiley.

Hanbury, A. The Taming of the Hue, Saturation, and Brightness Colour Space, Proc. Computer Vision Winter Workshop (CVWW), Austria, 2002.

Kragic, D. and Vincze, M. (2010). "Vision for Robotics", Foundation and Trends in Robotics, Vol 1, No 1, pp 1–78.

Vernon, D. (1991). Machine Vision: Automated Visual Inspection and Robot Vision, Prentice-Hall.

Acknowledgments

The syllabus for this course drew inspiration from several sources. These include the following.

  • Course VO 4.0 376.054 Machine Vision and Cognitive Robotics given by Markus Vincze, Michael Zillich, and Daniel Wolf at Technische Universität Wien.
  • Course 4BA10 Computer Vision given by David Vernon at Trinity College Dublin.
  • Course 4BA10 Computer Vision given by Kenneth-Dawson Howe at Trinity College Dublin.
  • Course on Computer Vision at VVV2017 by Francesca Odone, University of Genova.
Retrieved from "http://www.vernon.eu/w/index.php?title=Applied_Computer_Vision&oldid=1494"