The complete and accurate schedule for current and upcoming robotics courses is maintained at the University level and can be accessed by visiting the university courses page. This page serves as an overview of courses taught within the Robotics department.
All courses with a “16-” prefix are offered by the Robotics department. Other departments offering courses taught by Robotics faculty are Computer Science (CS), Electrical and Computer Engineering (ECE), Mechanical Engineering (MechE), Statistics (Stat), Psychology (Psych), the Tepper School of Business (GSIA), and the Institute for Complex Engineered Systems (ICES).
Finding Robotics 16-xxx Courses on the Registrar Site:
- Go to: Schedule of Classes Course Search.
- Select the “Semester/Year”
- Select “SCS: Robotics (16-xxx)” from the list of departments.
- Click “RETRIEVE SCHEDULE”
Computer vision is a discipline that attempts to extract information from images and videos. Nearly every smart device on the planet has a camera, and people are increasingly interested in how to develop apps that use computer vision to perform an ever expanding list of things including: 3D mapping, photo/image search, people/object tracking, augmented reality etc. This course is intended for graduate students who are familiar with computer vision, and are keen to learn more about the applying state of the art vision methods on smart devices and embedded systems. A strong programming background is a must (at a minimum good knowledge of C/C++), topics will include using conventional computer vision software tools (OpenCV, MATLAB toolboxes, VLFeat, CAFFE, Torch 7), and development on iOS devices using mobile vision libraries such as GPUImage, Metal and fast math libraries like Armadillo and Eigen. For consistency, all app development will be in iOS and it is expected that all students participating in the class have access to an Intel-based MAC running OS X Mavericks or later. Although the coursework will be focused on a single operating system, the knowledge gained from this class will easily generalize to other mobile platforms such as Android etc.
This course provides an overview of the current techniques that allow robots to locomote and interact with the world. The kinematics and dynamics of electromechanical systems will be covered with a particular focus on their application to robotic arms. Some basic principles of robot control will be discussed, ranging from independent- joint PID tracking to coupled computed torque approaches. The practice and theory of robotic mobility will be investigated through various mobile robot platforms, including wheeled and tracked vehicle and legged robots. Hands-on experience with some of the topics in the class will be provided through practical demonstrations and lab assignments. Please note that this course is for MRSD students only.
- MRSD students only
Practically everything around us is a system-from the cell phone in your pocket to the International Space System up in the sky. The higher the complexity of the system, the more its creators benefit from applying formal processes to its development-processes that are collectively known under the umbrella “systems engineering.” Systems Engineering is a formal discipline that guides a product from conception and design all the way to production, marketing, servicing, and disposal. In this course we will study the fundamental elements of systems engineering as they apply to the development of robotic systems. We will cover topics such as needs analysis, requirements elicitation and formalization, system architecture development, trade studies, verification and validation, etc. In addition, for this course we will cover core topics of Project Management that must be performed in tandem with Systems Engineering to achieve a successful project and product. For the Project Management we will cover work breakdown structures, scheduling, estimation, and risk management. We will study both classical and agile methods in project management. The students will apply most of the elements of this course in the MRSD Project Course I and II, thus giving them the opportunity to put the theory in practice in a real product design activity. Please note that this course is for MRSD students only. (Past project examples: http://mrsd.ri.cmu.edu/project-examples/)
Robot autonomy delves into the interplay between perception, manipulation, navigation, planning, and learning required to develop fully autonomous systems. We will focus on application domains like the home, retail, and healthcare and identify common themes and key bottlenecks. We will discuss the state of the art algorithms, their computational and hardware requirements, and their limitations. To enable you to create end-to-end systems, you will learn how to address clutter and uncertainty in manipulation tasks, develop robust object recognition algorithms in real-world scenes, plan robot trajectories in high-dimensional spaces, build behavior engines for high-level tasks, and learn to apply and connect those to create an autonomous robot system. The course emphasizes the implementation of the algorithms discussed in class in simulation through homework assignments as well as on real systems in a class project.
Kinematics, Dynamic Systems, and Control is a graduate level introduction to robotics. The course covers fundamental concepts and methods to analyze, model and control robotic mechanisms which move in the physical world and manipulate it. Main topics include the fundamentals of kinematics, dynamics and control applied to the kinematics, dynamics and control of rigid body chains. Additional topics include state estimation and dynamic parameter identification.
This course introduces the fundamental techniques used in computer vision, that is, the analysis of patterns in visual images to reconstruct and understand the objects and scenes that generated them. Topics covered include image formation and representation, camera geometry, and calibration, computational imaging, multi-view geometry, stereo, 3D reconstruction from images, motion analysis, physics-based vision, image segmentation and object recognition. The material is based on graduate-level texts augmented with research papers, as appropriate. Evaluation is based on homeworks and a final project. The homeworks involve considerable Matlab programming exercises.
- Richard Szeliski, "Computer Vision Algorithms and Applications", Texts in Computer Science, Springer ISBN: 978-1-84882-934-3, Book Requirement: Recommended
- David Forsyth and Jean Ponce, "Computer Vision: A Modern Approach", Prentice Hall ISBN: 0-13-085198-1, Book Requirement: Recommended
The principles and practices of quantitative perception (sensing) illustrated by the devices and algorithms (sensors) that implement them. Learn to critically examine the sensing requirements of robotics applications, to specify the required sensor characteristics, to analyze whether these specifications can be realized even in principle, to compare what can be realized in principle to what can actually be purchased or built, to understand the engineering factors that account for the discrepancies, and to design transducing, digitizing, and computing systems that come tolerably close to realizing the actual capabilities of available sensors. Grading will be based on homework assignments, class participation, and a final exam. Three or four of the homework assignments will be hands-on “take-home labs” done with an Arduino kit that students will purchase in lieu of purchasing a textbook. Top-level course modules will cover (1) sensors, signals, and measurement science, (2) origins, nature, and amelioration of noise, (3) end-to-end sensing systems, (4) cameras and other imaging sensors and systems, (5) range sensing and imaging, (6) navigation sensors and systems, (7) other topics of interest to the class (as time allows).
Students will gain theoretical and practical skills in 2D, 3D, and 4D biomedical image analysis, including skills relevant to general image analysis. The fundamentals of computational medical image analysis will be explored, leading to current research in applying geometry and statistics to segmentation, registration, visualization, and image understanding. Additional and related covered topics include de-noising/restoration, morphology, level sets, and shape/feature analysis. Students will develop practical experience through projects using the latest version of the National Library of Medicine Insight Toolkit ( ITK ) and SimpleITK, a popular open-source software library developed by a consortium of institutions including Carnegie Mellon University and the University of Pittsburgh. In addition to image analysis, the course will include interaction with radiologists and pathologist(s). *** Lectures are at CMU and students will visit clinicians at UPMC. Some or all of the class lectures may also be videoed for public distribution, but students may request to be excluded from distributed video. 16-725 is a graduate class, and 16-425 is a cross-listed undergraduate section. 16-425 has substantially reduced requirements for the final project and for the larger homework assignments, nor does it require shadowing the clinicians.
- Knowledge of vector calculus, basic probability, and either C++ or python, including basic command-line familiarity and how to pass arguments to your own command-line programs. Extensive expertise with C++ and templates is not necessary, but some students may find it helpful.
Kinematics, statics, and dynamics of robotic manipulator’s interaction with a task, focusing on intelligent use of kinematic constraint, gravity, and frictional forces. Automatic planning based on mechanics. Application examples drawn from manufacturing and other domains.
This course surveys the use of optimization (especially optimal control) to design behavior. We will explore ways to represent policies including hand-designed parametric functions, basis functions, tables, and trajectory libraries. We will also explore algorithms to create policies including parameter optimization and trajectory optimization (first and second order gradient methods, sequential quadratic programming, random search methods, evolutionary algorithms, etc.). We will discuss how to handle the discrepancy between models used to create policies and the actual system being controlled (evaluation and robustness issues). The course will combine lectures, student-presented material, and projects. The goal of this course will be to help participants find the most effective methods for their problems.
This class will introduce concepts from a broad spectrum of constrained optimization, optimal and feasible motion planning, as well as machine learning all in the context of controlling complex, contemporary robotic systems. The specific systems on which we will focus include nominal examples from the nonlinear control literature (e.g., the constrained simple pendulum, the acrobat, cart-pole, etc.) all the way to the hybrid systems that underly much of the leading work on humanoid robots (e.g., the compass gait walker, the SLIP model, etc.). The course material will primarily consist of lecture notes as well as seminal papers from the robotics literature. By the end of this class you will be able to readily understand, implement, and adapt methods that span a wide spectrum that includes optimal control theory and reinforcement learning. People and animals move through and interact with the world in a fundamentally dynamic way. In the vast majority of cases the same cannot be said for robots. In fact, many conventional approaches to motion planning and robot control attempt to explicitly cancel out the dynamics associated with different tasks. This class will consider underactuated robots, systems that do not have full control over their state and therefore cannot be planned for or controlled via conventional methods. Our goal will be to make novel locomoting robots act more naturally. This class will highlight the relationship between conventional ideas from deterministic motion planning and control design (e.g., dynamic programming and linear-quadratic regulators) and their contemporary counterparts, many of which help form the analytical basis for the probabilisitic reasoning that underlies contemporary AI systems (e.g., POMDPs). Note that this course is inspired by and, for the most part, will follow the format of Underactuated Robotics: Learning, Planning, and Control for Efficient and Agile Machines created by Prof. Russ Tedrake at MIT. We will take several tangents, but the course materials provided by Prof. Tedrake through MIT Open Courseware are an incredible resource for this course (and really just in general).
This course covers all aspects of mobile robot systems design and programming from both a theoretical and a practical perspective. The basic subsystems of control, localization, mapping, perception, and planning are presented. For each, the discussion will include relevant methods from applied mathematics. aspects of physics necessary in the construction of models of system and environmental behavior, and core algorithms which have proven to be valuable in a wide range of circumstances.
Mechatronics is the synergistic integration of mechanism, electronics, and computer control to achieve a functional system. This course is a semester-long multidisciplinary capstone hardware project design experience in which small (typically four-person) teams of electrical and computer engineering, mechanical engineering and robotics students deliver an end-of-course demonstration of a final integrated system capable of performing a mechatronic task. Throughout the semester, the students design, configure, implement, test and evaluate in the laboratory devices and subsystems culminating in the final integrated mechatronic system. Lectures will complement the laboratory experience with comparative surveys, operational principles, and integrated design issues associated with the spectrum of mechanism, microcontroller, electronic, sensor, and control components. CROSS-LISTED COURSES: 18-578, 24-778
Planning and Decision-making are critical components of autonomy in robotic systems. These components are responsible for making decisions that range from path planning and motion planning to coverage and task planning to taking actions that help robots understand the world around them better. This course studies underlying algorithmic techniques used for planning and decision-making in robotics and examines case studies in ground and aerial robots, humanoids, mobile manipulation platforms and multi-robot systems. The students will learn the algorithms and implement them in a series of programming-based projects.
This course covers the topics on building cognitive intelligence for robotic systems. Cognitive capabilities constitute high-level, humanlike intelligence that exhibits reasoning or problem solving skills. Such capabilities as semantic perception, language understanding, and task planning can be built on top of low-level robot autonomy that enables autonomous control of physical platforms. The topics generally bridge across multiple technical areas, for example, vision-language intersection and language-action/plan grounding. This course is composed of 50 lectures and 50 seminar classes.
- There are no explicit prerequisites for this class, but a general background knowledge in AI and machine learning is assumed.
This course explores the rapidly developing field of data science in the context of its pragmatic applications. Applied Data Science strives to achieve three main goals. The first is to optimize the efficacy of decision making by human managers. The second is to maximize the utilization of available data, so that no important clue is ever missed. The third is to improve understanding of data and the underlying processes that produce it. This course aims at building skills required to systematically achieve those goals in practice. The students will gain and solidify awareness of the most prevalent contemporary methods of Data Science, and develop intuition needed for assessing practical utility of the studied topics in application scenarios. They will be able to learn how to formulate analytic tasks in support of project objectives, how to define successful analytic projects, and how to evaluate utility of existing and potential applications of the discussed technologies in practice.
This course covers selected topics in applied mathematics useful in robotics, taken from the following list: 1. Solution of Linear Equations. 2. Polynomial Interpolation and Approximation. 3. Solution of Nonlinear Equations. 4. Roots of Polynomials, Resultants. 5. Approximation by Orthogonal Functions (includes Fourier series). 6. Integration of Ordinary Differential Equations. 7. Optimization. 8. Calculus of Variations (with applications to Mechanics). 9. Probability and Stochastic Processes (Markov chains). 10. Computational Geometry. 11. Differential Geometry.
The course focuses on the geometric aspects of computer vision: The geometry of image formation and its use for 3D reconstruction and calibration. The objective of the course is to introduce the formal tools and results that are necessary for developing multi-view reconstruction algorithms. The fundamental tools introduced study affine and projective geometry, which are essential to the development of image formation models. Additional algebraic tools, such as exterior algebras are also introduced at the beginning of the course. These tools are then used to develop formal models of geometric image formation for a single view (camera model), two views (fundamental matrix), and three views (trifocal tensor); 3D reconstruction from multiple images; and auto-calibration.
- Computer Vision (16-721 or equivalent)
- The Geometry of Multiple Images. Faugeras and Long, MIT Press., Book Requirement: Not Specified
- Multiple View Geometry in Computer Vision, Richard Hartley and Andrew Zisserman, Cambridge University Press, June 2000., Book Requirement: Not Specified
- Fundamentals of projective, affine, and Euclidean geometries
- Invariance and duality
- Algebraic tools
- Single view geometry: The pinhole model
- Calibration techniques
- 2-view geometry: The Fundamental matrix
- 2-view reconstruction
- 3-view geometry: The trifocal tensor
- Parameter estimation and uncertainty
- n-view reconstruction
Everyday we observe an extraordinary array of light and color phenomena around us, ranging from the dazzling effects of the atmosphere, the complex appearances of surfaces and materials and underwater scenarios. For a long time, artists, scientists and photographers have been fascinated by these effects, and have focused their attention on capturing and understanding these phenomena. In this course, we take a computational approach to modeling and analyzing these phenomena, which we collectively call as “visual appearance”. The first half of the course focuses on the physical fundamentals of visual appearance, while the second half of the course focuses on algorithms and applications in a variety of fields such as computer vision, graphics and remote sensing and technologies such as underwater and aerial imaging. This course unifies concepts usually learnt in physical sciences and their application in imaging sciences, and will include the latest research advances in this area. The course will also include a photography competition.
- An undergraduate or graduate class in Computer Vision or in Computer Graphics
A graduate seminar course in Computer Vision with emphasis on representation and reasoning for large amounts of data (images, videos and associated tags, text, gps-locations etc) toward the ultimate goal of Image Understanding. We will be reading an eclectic mix of classic and recent papers on topics including: Theories of Perception, Mid-level Vision (Grouping, Segmentation, Poselets), Object and Scene Recognition, 3D Scene Understanding, Action Recognition, Contextual Reasoning, Image Parsing, Joint Language and Vision Models, etc. We will be covering a wide range of supervised, semi-supervised and unsupervised approaches for each of the topics above.
- Graduate Computer Vision or Machine Learning course
Data-driven learning techniques are now an essential part of building robotic systems designed to operate in the real world. These systems must learn to adapt to changes in the environment, learn from experience, and learn from demonstration. In particular we will cover three important sub-fields of Machine Learning applied to robotic systems: (1) We will cover Online Learning, which can be used to give robotic systems the ability to adapt to changing environmental conditions. (2) We will cover Reinforcement Learning, which takes into account the tradeoffs