1. Introduction

Robotics is a fascinating, and broadly interdisciplinary field, combining aspects of engineering, computer science, mathematics, physics, and even psychology. Moreover, society now finds itself at the dawn of a new, golden age of robotics, as robots are increasingly moving out of highly regulated factory environments into commercial settings, hospitals, our homes, our streets, and even the skies. How do these systems work? What sensors do they have and how do they perceive our world through these senses? What are the mechanisms by which they can move and act in their environment, and how do they choose their actions at any given moment? How can they learn to adapt their behavior over time to improve their capabilities and enhance interactions with humans they encounter? These questions frame an incredibly broad and rich subject, and given the recent advances in artificial intelligence, the field of robotics is bound to see many breakthroughs over the next few years.

This introductory text is designed to guide the reader through the foundations of modern robotics, while at the same time showing how practical systems can be implemented. The theoretical concepts in each chapter are explained informally without sacrificing rigor, balanced with a unique hands-on approach. Theory is immediately put into practice, as every section in the book is interspersed with example code and can be executed as a python notebook, allowing the reader to experiment with implementations that are representative of many robotics systems. No experience is needed beyond a good basis in math, especially linear algebra and notions of statistics, and some knowledge of the python programming language. The code itself is based on popular open source libraries, most notably GTSAM and PyTorch, which provide a good starting point for those who want to explore robotics as a potential career choice.

The sense-think-act cycle is a recurring theme in this book. While the range of applications for modern robotic systems is quite diverse, at a high level, most all systems take the same approach. They use sensors and perception algorithms to assess their environment; they reason about their goals and how to achieve them; they perform actions to change the state of the world. This is known as the sense-think-act cycle. The robot senses its world, thinks about what to do, then acts. This idea is general enough to be applied at many levels of abstraction and time scales. We can imagine a warehouse robot tasked with retrieving an item from inventory. After using its sensors to determine its current location (sense), the robot constructs a path (think), then traverses the path (act). The time scale here might be on the order of seconds to minutes to complete one iteration of the sense-think-act cycle, and the level of abstraction is very high, ignoring details like controlling motors. On the other hand, we could consider a surgical robot tasked with precisely inserting a needle into a tumor. In this case, ultrasonic sensors could be used to localize the needle relative to the tumor (sense), small corrections to needle placement would be computed (think), and a low-level motion command would be sent to the device, possibly a voltage or current, to effect the motion (act). Here, the time scale is on the order of milliseconds, and the level of abstraction could be quite detailed, including tissue dynamics and flexibility of the needle.

The chapters of the book proceed through a sequence of increasingly complex robotic systems. We begin, in Chapter 2, by considering a simple trash-sorting robot. Pieces of trash arrive on a conveyor, and the robot’s task is to place these in an appropriate bin. For this first robotic system, we abstract away most details that would confront a real trash-sorting robot, focusing our attention on fundamental concepts from probability theory, both to solve a simple perception problem (categorizing pieces of trash) and to solve a planning problem (which bin to select). From there, in each chapter we incrementally add complexity to the system models and the mathematical methods for manipulating them, developing increasingly sophisticated algorithms to implement these methods. We consider a simple vacuum cleaning robot in Chapter 3, a mobile robot that navigates in a warehouse in Chapter 4, a two-wheeled, differential-drive robot (DDR) in Chapter 5, self-driving cars in Chapter 6, and quadrotor drones in Chapter 7.

The individual chapters share a common structure, examining how to represent the robot’s state, how to describe its actuators and sensors, the associated perception and planning methods and algorithms, and concluding with a section on learning from data. The expressiveness of the state space, which depends on how much we abstract away from reality, largely determines the complexity of the corresponding algorithms. By gradually increasing this complexity, we ensure that each chapter builds on the previous one, guiding the reader through a series of manageable steps. A key element of our approach is the use of graphical models to illustrate how state, actions, and sensing share common patterns across a wide range of autonomous robots. This structure also reflects the essence of the sense-think-act cycle. Each chapter includes sections on sensors and perception (sense), planning and learning (think), and state and action models (act).

In the state, action, and sensing sections of each chapter, we focus on the mathematical models. To model robots, we progress from discrete state spaces with abstract action and sensing to continuous state spaces with real-world sensing in later sections. We introduce the special Euclidean group of order two, \(SE(2)\), for differential-drive robots and extend it to \(SE(3)\) for drones. We develop the differential kinematics for wheeled mobile robots, both for omnidirectional robots (our logistics robot) and for robots with nonholonomic constraints. For drones, we derive a dynamics model that includes propeller thrust and aerodynamic effects such as drag. We take a similar approach with sensors. In Chapter 2, we introduce simple sensors that measure physical properties of a piece of trash. In later chapters, we expand to a variety of sensors, including proximity sensors, RFID beacons, GPS, cameras, LIDAR, and Inertial Measurement Units (IMUs).

In other places, we focus on probabilistic methods for representing and reasoning about uncertainty. We introduce discrete probability distributions for simple classification problems and develop maximum-likelihood and maximum-a-posteriori estimation methods using our trash-sorting robot. We then move to continuous distributions, specifically the uniform and Gaussian distributions, to model the motion of mobile robots and the effects of sensor noise. To reason about uncertainty, we introduce sampling-based methods such as particle filtering and Markov localization, along with Kalman filters, Bayes nets, and Markov decision processes. In each case, we start by formulating the appropriate mathematical models, then proceed to describe algorithmic solutions, often illustrated with code.

In the remainder of this chapter, we provide a slightly more detailed overview of the topics covered in this book.