Description: https://images.manning.com/360/480/resize/book/4/9d28fdb-9d68-448e-97a9-efbed9461940/Bihlmaier-MEAP-HI.png

An excerpt from Robotics for Software Engineers by Andreas Bihlmaier

This article shows what a typical robot’s software stack looks like.

Read this if you are a software developer who is interested in learning more about robotics.


Take 25% off Robotics for Software Engineers by entering fccbihlmaier into the discount code box at checkout at manning.com.


Real robot systems have a certain inherent complexity to them as they need to deal with the complexity of the real environment. In order to manage this complexity well, just like with any other complex software system, we need a suitable software architecture. This article gives an overview of a typical robot’s software stack. It is important to have a feeling for the different layers in the robot software stack, because each specific robot functionality exists in a wider context and not in isolation.

On the one hand, this wider context is the robot software stack. On the other hand, the context also consists of the different fields that have an impact on the robot as a whole. Each field has a different view of the robot system and also uses a different terminology. Ultimately, we are creating one robot system. The different fields and views must hence be integrated into one solution. We discuss several of these perspectives and connect them to our robots as software perspective.


Figure 1. The conceptual structure and essential six parts of all robot systems. Reproduced from chapter 1. The horizontal subdivision of robot systems.


The answer is that there are different views on a system’s (software) architecture. Each view focuses on different aspects of the same systems. Figure 1 focuses on the conceptual parts that make up the casual loop between robot and environment. We will refer to this as the horizontal subdivision. In contrast, figure 2 focuses on the layers of the robot software stack. We will use the term vertical subdivision for this one.


Figure 2. The robot software stack. The layers specific to robotics are highlighted. The vertical subdivision of robot systems.


Although 2 might contain some parts that are not yet familiar, my assumption is that the general structure of these layers is already familiar to you from working with other software systems. The bottom layer is composed of the hardware devices. These hardware resources are managed by the operating system (OS) layer. Another responsibility of the OS is to encompass drivers that provide a unified API to similar hardware devices. Furthermore, the OS provides the basic mechanisms for multitasking, memory management, data storage and networking. Above the OS layer are the generic services needed in most computer systems. These include managing installed software, performing software configuration and executing software. We also regard software runtimes, general-purpose software libraries and general-purpose services, such as web servers and databases, as part of this layer. Up until this layer, there is nothing robotics specific. Yet, as we will soon discuss, robot software relies heavily on these layers and in some cases uses them in unique ways.

The first robot specific layer is the robot base libraries & services layer. I refer to it as robot base layer, because it contains functionality that is independent from concrete robot applications. Not all robots contain all components listed here. For example, manipulators do not need navigation functionality and some robots might not need a symbolic task planner.

At the top of the robot software stack sits the robot application layer. This is the layer that defines what the robot is actually doing, i.e. what task(s) the robot is actually performing. In the simplest case, the application layers consists of a list of instructions that are sequentially executed. In other words, it consists of a (simple) imperative program. In addition, some configuration of the robot base layer is performed here. Such a program would directly call functions in the robot base layer, e.g. a sequence of instructions for manipulators or a sequence of instructions for mobile robots. The top instruction in this kind of program is an endless loop that performs the same sequence over and over. Usually, there is some kind of start condition, e.g. an external signal, which then starts the next loop iteration. You might be surprised to hear that the majority of all (industrial) robots deployed today work in this way. In more complex robot systems, the application layer contains a set of object, environment, task, skill and other models for the robot base services. Either there is still an imperative program as top-level control or a formalized description of the robot’s task is used, that can be interpreted by an automated reasoning component. We will see plenty examples of both ways to realize robot applications.

Before bringing the horizontal and the just discussed vertical view together, I want to direct your attention to the boundary between the robot base layer and the robot application layer. Using the example of a pancake making robot from the previous chapter, let’s think about two ways to realize it. In terms of the robot software, the easiest solution is to put all ingredients and kitchen utensils in a well-defined place. Given such an environment, we have nothing more to do than write down a sequence of commands (and gripper open/close commands) to get pancakes. The robot will only work in the exact environment with the exact utensils for which we have created the program. Therefore, it is clear that in this solution all pancake making capabilities are part of the robot application layer. On the other end of the spectrum, we could add sophisticated object recognition capabilities, reasoning skills and task planning functions for pancake making to our robot. The top level instruction would then just state the number of pancakes to make. Would all this functionality just be usable for making pancakes or could it not also be utilized for other recipes? The answer is likely yes. So, should we add “Chef” to the robot base layer in fig 2, if we can build generalized functionality around kitchen robots? My suggestion is to leave this “Chef” component in the robot application layer and rather think of having some layers nested within the application layer. The robot base layer should be reserved for really broadly applicable functionality that is independent of a particular domain. Still, as often with such classifications, there is some gray area, but it need not bother us.

Now, let’s bring the horizontal and vertical views from figure 1 and figure 2 together in figure 3:


Figure 3. Combined view of the horizontal subdivision (parts) and the vertical subdivision (layers).


Some horizontal layers are relevant for all of our vertical parts. Since we assume our robots are controlled by digital computer programs, we need computation capabilities in all parts. This does not only mean the computer hardware, but also a lot of functionality from the operating system and generic system layer. Other components such as specific program libraries are primarily associated with a specific part, e.g. computer vision libraries with sensing.

We see a similar pattern repeated in the robot base layer. Some functions, such as communication and logging, are relevant for all parts. Others have a much closer association with specific parts, such as sensor fusion with sensing, navigation with planning and control with acting. Components related to models tend to be used in multiple parts. For example, the robot model of a manipulator is updated in the sensing part based on encoder data and is also used in acting for the purpose of control. While this non-exhaustive list of components or robot software building blocks might seem daunting at first, they will soon become second nature to you.

We have talked quite a bit about robot systems already. Let’s briefly talk about the boundaries of these robot systems and how they relate to computer systems. Figure 4(a) shows the base case consisting of one computer connected to a set of sensors & actuators and running one robot software stack.


Figure 4. Robot systems and their system boundaries.


In figure 4(b) the robot system consists of multiple computers connected to each other. Each computer can have its own sensors & actuators and runs (parts of) the robot software stack. But only in combination do these devices make up a complete robot system that can perform intended tasks.

Finally, figure 4(c) shows a number of complete robot systems working together. Although each robot system is in itself complete and can perform specific tasks, additional benefits can be gained from coordinating them. Such a coordinated set of robot systems is called a robot fleet. The coordination instance is hence called a fleet manager. Consider a (large) number of mobile robots performing logistics tasks in a warehouse environment. Each robot can independently transport goods from location A to location B. The fleet manager has a global view of transport tasks and vehicle positions. In order to avoid traffic jams and to minimize the overall distance travelled, the fleet manager gives (high-level) orders to the individual mobile robots instead of each mobile robot deciding only based on its local information.

There is another dimension we should keep in mind: the robot system’s lifecycle. Everything we discussed in this chapter thus far was about an operational robot system. However, there are multiple phases in the life of a robot solution. These include

  • Development (incl. testing and simulation)
  • Commissioning
  • Operation
  • Maintenance

The development phase, for example, will include components such as a developer tools (IDEs, compilers, debuggers, build systems, CI/CD, CAD/CAE, etc.), development machines, simulators and other helpful engineering tools.

After we have seen the relationship between the essential robot parts (vertical subdivision) and the layers in the robot software stack (horizontal subdivision), I want to conclude this section with a few more words on having different complementary views in software design. The seminal paper “Architectural Blueprints – The 4+1 View Model of Software Architecture” by Philippe Kruchten explicates 4 complementary views on software design:

  • Logical view: structure of the functionality provided by the system.
  • Process view: the system’s runtime behavior.
  • Development/Implementation view: breakdown of the system into components.
  • Physical/Deployment view: physical realization of the system.

These 4 views are illustrated through a number of selected scenarios (the “+1” view). I find these views very helpful in talking about complex software systems. Explicating the view under discussion avoids confusion, since they help to clarify the semantics of the arrows and boxes we are usually drawing in architectural diagrams. Fig 1 focuses on the logical and process view. Fig 2 highlights the development/implementation view. Fig 4 is concerned with the physical/deployment view. Please note that while I do not strictly adhere the 4+1 separation of views, I will explicate the diagram’s semantics.

Another powerful thinking aid is the C4 model by Simon Brown, which emphasizes the hierarchical structure applicable to software systems. It supports mentally zooming in and out to the right level of abstraction when discussing software design. The four Cs in C4 stand for these zoom levels:

  • Level 1, System Context: relation of overall system to world around it.
  • Level 2, Container: high-level technical building blocks or subsystems.
  • Level 3, Component: components within a single container.
  • Level 4, Code: code-level diagrams (e.g. classes and objects) of a single component.

This concludes our introduction to the robot software stack.

If you want to learn more about the book, check it out on Manning’s liveBook platform here.