Robot Safety: Tools and Resources to Help You Build More Reliable Software
Building safe and reliable software is hard. Building safe and reliable robots is even harder.
Moreover, just developing bug-free software is not sufficient to ensure the safety of your robot. It’s fundamental with any robotic development endeavor to worry about the quality of the software developed, and that’s not something that can be achieved after the fact. Softwares are built in layers, and are only as reliable as their weakest link, so quality has to be infused in from the very beginning. Accumulating technical debt can be a startup killer – see Argo AI, for example.
The good news is there exist a lot of tools and resources to help developers build quality robotic software – and most of them free. Not only is it a good time investment to learn and set up these tools to manage technical debt long-term – but it’s an extra pair of (bionic) eyes that can be a real time-saver in the short term, by helping you avoid a lot of potentially lengthy bug hunts.
Languages
When choosing your programming language, there are three most common options: Python, Rust, and C++.
Python
- Rapid development
- Easy debugging
- Great resources for ML
- Bad performance
- Relies on interpreter
Rust
- Memory safety
- Thread safety
- Performance
- Immature ecosystem (limited libraries and legacy code)
- Verbose syntax
- Steep learning curve
C++
- Performance
- Mature ecosystem (libraries and legacy code)
- Control over resources
- Not (inherently) thread safe
- Not (inherently) memory safe
- Steep learning curve
Overall, we find C++ to be the best balance between safety, performance, and existing ecosystem. However, the inherent safety and similar performance of Rust would make it a more suitable option for robotics development when the ecosystem grows.
Package manager
Package managers help you control dependency trees, have repeatable build environments and ensure ABI compatibility between packages.
The two we use and recommend are:
- Conan
- vcpkg
Code IDE
Ditch your outdated notepad++ and jump onto vscode, it has many extensions to help you develop clean robotics code, regardless of your chosen language.
Notable extensions for c++ dev:
- C/C++
- Clang-format
- Clangd
- CMake Tools
- Error Lens
- Test Explorer UI
- Todo Tree
Notable extensions for ROS dev:
- Msg Language Suppor
- ROS
- URDF
- Yaml
Notable extensions for embedded dev:
- Platform IO IDE
- Arduino
Continuous Integration (CI)
All git repo providers provide some CI with it. Make heavy use of it, it’s (mostly) free extra code reviewers and QA engineers.
Code linting
Code linting prevents errors by catching common mistakes beforehand, maintains consistency in coding styles, enhances readability by suggesting improvements, facilitates collaboration through smoother code reviews, and ultimately improves efficiency by reducing manual review and debugging time.
Best options include:
- Clang-format
- Cpplint
- Flint++
- include-what-you-use
Unit testing framework
Unit testing ensures reliable code behavior, aids in early bug detection, guides design decisions, supports code documentation, and promotes modular and maintainable code.
- GTest
- Catch2
- Parasoft
- Boost Test Library
- boost-ext/ut
Why write tests yourself when someone has already done it for you : SolidSand
Static Analysis
Static analysis identifies complex issues early, enforces coding standards, improves code quality, detects security vulnerabilities, and enhances overall software reliability.
Paid:
- Coverity Scan
- PVS studio
- cppdepend
Free:
- Cppcheck
- Clang-tidy
- clang-sa
Meta:
- codechecker
Compiler flags:
- Clang: set(CMAKE_CXX_FLAGS “${CMAKE_CXX_FLAGS} -pedantic -Wall -Wshadow -Wextra -Werror -Wnon-virtual-dtor -Wno-long-long -Wcast-align -Wchar-subscripts -Wall -Wpointer-arith -Wformat-security -Woverloaded-virtual -fno-check-new -fno-common -Wconversion -Wextra-semi -Wshadow-all”)
- GCC: set(CMAKE_CXX_FLAGS “${CMAKE_CXX_FLAGS} -pedantic -Wall -Wshadow -Wextra -Werror -Wnon-virtual-dtor -Wno-long-long -Wcast-align -Wchar-subscripts -Wall -Wpointer-arith -Wformat-security -Woverloaded-virtual -fno-check-new -fno-common -Wconversion”)
Dynamic analysis and debugging
While static analysis can be great, it’s unlikely to catch every issue you’ll encounter. For this, you need some tools for dynamic analysis and debugging.
Debugging options:
- Valgrind
- GCC / Clang Sanitizers (https://github.com/RWTH-HPC/sanitizers-cmake)
- Ftime_trace
- american fuzzy lop
- LibFuzzer
Note: many of these debugging options (and more) interface well with vscode
Compiler flags:
- -D_LIBCPP_DEBUG
- -D_GLIBCXX_DEBUG
- -D_GLIBCXX_DEBUG_PEDANTIC
- -D_GLIBCXX_SANITIZE_VECTOR
Logging and visualization
Proper logging and visualization provide insights into runtime behavior, aid in debugging and troubleshooting, offer performance monitoring, support data-driven decision-making, and enhance overall system understanding and optimization.
Tools for logging:
- Spdlog
- Glog
- Easyloggingpp
- Easylogging++
- rosbag
Tools for visualization:
- Grafana
- Foxglove
- webviz
Code Coverage
Code coverage tools help assess testing completeness, identify untested code paths, guide test creation, enhance code quality, and provide valuable insights into the effectiveness of testing efforts.
- lcov/gcov (https://github.com/RWTH-HPC/CMake-codecov)
- Codecov
Benchmarking
Benchmarking measures performance, identifies bottlenecks, guides optimization efforts, supports informed hardware/software choices, and ensures the delivery of high-performing and efficient software solutions.
- perf | speedscope
- Heapstack
- Strace
- Dr Memory
Documentation
Documentation is critical. Don’t wait until you’ve forgotten why you wrote it the way you did. Two options we like are:
- Doxygen
- Read the doc (https://github.com/maltfield/rtd-github-pages)
Coding guidelines
As mentioned previously, C++ is not inherently safe. These coding guidelines exist to ensure the safety of code developed from them.
- AUTOSAR C++ 2014
- CERT C++ 2016
- MISRA-C++ 2008 (2023 coming out soon),
- C++ Core Guidelines
- High Integrity C++
- JSF AV C++
- JPL 10 rules (https://web.eecs.umich.edu/~imarkov/10rules.pdf)
Operating systems
Your software needs something to run on! Here are some of the most popular real-time (and non real-time) operating systems.
- Linux
- preempt-RT
- Xenomai
- Free-rtos
- QNX
- Vxworks
Robot safety: complex but essential
Merely developing bug-free software isn’t enough to ensure a robot’s safety. The software’s quality is paramount, and this quality should be integrated from the onset of development. We hope our resources list is helpful as you endeavor to ensure the very highest standards or robot safety.
Here at 3Laws, we offer an AI supervisor for autonomous and intelligent robotic systems, ensuring safety without compromising performance. We separate safety from the autonomy stack, and deploy real-time safety analytics, robust collision avoidance software, and technology that’s robust to uncertainties in modeling and perception.
Good luck in your mission to ensure that robotic systems function optimally in real-world scenarios!