Creating A Cloud-Integrated Test Bench

Alex Irwin, Go West RoboticsAlex Irwin

At Go West Robotics, we say it a lot but it can’t be said enough. “Test early and test often.” It’s one of our mantras. Our users should not be finding software bugs in their production environment. Given the nature of the systems we work on, a simple bug can be a major safety hazard - a self-driving vehicle or a 3-ton robot arm can cause serious damage and bodily harm.

Connect Your Test Bench to the Cloud
Connect Your Test Bench to the Cloud

The Importance of Integration Testing

Integration testing is meant to identify issues between new and existing components. Code may run well in isolation, but the same code might fail miserably when interacting with the other components of a system.

A good example of integration in the robotics world is the interaction between HMIs, PLCs, and robot controllers. A beautiful, intuitive HMI may not effectively start a job on a robot, or it may not accurately reflect the current status of the robot as it executes a task. To consistently create high-quality software, robot developers need to test the end-to-end process, from user input to robot execution and return active feedback from the robot to the user.

Effective integration testing can quickly become cost prohibitive, though, as robots and physical controllers are expensive. In addition, when you have more than one developer working on a project, hardware must be shared or multiple pieces of hardware need to be available to the team. This problem gets even worse when your team members aren’t in the same location.


Robot simulation is a great tool for providing rapid feedback to developers and for performing initial integration tests (although everything must ultimately be tested on actual hardware).

Simulation software can be expensive when multiple seat licenses are needed. Time is money, too, so any development time spent waiting on compilation is wasted money. Hardware is expensive, though, and the latest hardware, with high-end graphics processors to support sim, can be ridiculously expensive.

There is No Substitute for Hardware

For effective integration testing, you must be testing your software in an environment that is a true analog of your production environment. Therefore, you must test your software on, and against, the actual hardware you will be using.

To control costs, we use a network-connected test bench. The bench contains all items that our software will directly interface with and downstream items (where feasible). For example, if our team is developing a custom robot control system, our test bench will contain at a bare minimum both the PLC and the robot controller that our software will talk to. All hardware on our bench is connected to a network switch, so that it is “plug and play”.

The Cloud Saves the Day

How does a robotic software engineering manager put their team in the best position to succeed in spite of a finite budget? Easy: move everything you can to the cloud. Really. It’s that simple. Once everything is in the cloud, or connected to it, your team members can share everything from anywhere, and you can control costs by only paying for what you use.

Here’s how we typically do it:

  • Create a “virtual private cloud” (VPC) using your preferred cloud services provider
  • Configure virtual servers in your VPC and install your preferred simulators and software development environments (SDEs) on the servers
  • Create a physical test bench with the hardware the team will need to share
  • Connect the test bench to your VPC with a “site-to-site” VPN
  • Use a VPN client to allow your teams to access hardware and cloud resources from anywhere

Create a Virtual Private Cloud (VPC)

A VPC is a private network that you create in the cloud. Many cloud services configure a default VPC for you as part of your account creation process. Once your VPC is created, configure the network rules to appropriately restrict access. Next, connect a VPN endpoint that allows both on-demand software clients and permanent tunnels.

Run Simulation and Software Development Environments (SDEs) in Virtual Machines

Once your VPC is configured, launch virtual machines into the network. Most cloud service providers have standard server images that can be used for your virtual machines. For third-party software (like simulators and SDEs), we typically use Windows Server 2019. Our custom software usually runs Ubuntu 20.04, which is also a generally available image.

One important step here is to select an instance class that supports the most intensive processing needs you will require. Your initial server can use the least expensive version of the instance, but give yourself room to scale up. For example, in AWS, we like using the c5n processor class for running RoboGuide. We start with a c5n.large, but we upgrade to the top tier when we are performing computationally intensive tasks. That way, we only pay for the processing that we actually need and use.

If you choose a processor class that will not meet your long-term demands and and need to change processor types, you will often invalidate your licensing and at best, will be trapped in the lower performance instance class.

Connect your Test Bench to the Cloud with a VPN

Install a hardware switch on your network that supports a “site-to-site” VPN connection to your VPC. You can reference the documentation provided by your cloud service provider to determine which VPN protocols they support to make your hardware decision.

Once your switch’s site-to-site VPN tunnel is active, connect it to your test bench’s network switch. Now your test bench is connected to the servers in your VPC.

Use a VPN Client to Access Hardware and Cloud Resources from Anywhere

Configure your VPC to accept “client” VPN connections. We typically use an OpenVPN-based client, as it is easily accessible from all of our customers’ and team members’ operating systems.

Once your team has their VPN connections up and running, they will have access to the SDEs and simulators running in the cloud, allowing them to share servers and licenses. They will also have access to the network- connected devices on your hardware test bench.

The End Goal: Better Software

While we’ve glossed over the technical aspects of configuring your cloud environment and making your hardware network accessible, we have provided a general overview to illustrate a broader strategy. We believe that executing this strategy will help you reduce costs and increase the productivity of your software teams, while producing more stable software.

The configuration time to create this environment could be substantial if your environment is complex or changes often. Any configuration time or cost is easily recovered with savings from fewer bugs and easier releases.

If you want more information or have questions, please send us a note. We are always happy to help.

Is there a topic you need help with or would like us to cover? Drop us a quick note and let us know:

Want to learn more?

We'd love to talk to you. Contact us to see how we can help.

Trending Topics

Deciding If ROS Is The Best Option For You
Should You Build Your Robotics Platform On ROS?

ROS is, without a doubt, one of the most widely used frameworks for developing robotics applications but is a complex framework that can easily lead to more trouble than it is worth. Is ROS the best option for you?

Locking out Ransomware
Defending Your Automated Processes Against Cyberattacks

In the past, hackers did their best to quietly sneak into your systems to steal your data and leave without you ever knowing. Those days are gone, hackers want their presence to be felt, they will break into your systems and lock them down until you meet their demands. Are you prepared if this happens to your business?

Saving Money with Better Testing
How Robotic Integrators Can Save Money With Better Testing

Building a comprehensive test suite will allow you to spend less time on releasing hardware, testing cells, and on-site support, which frees up your team to focus on what matters.

Go West Robotics, Inc.
350 N 9th St, Ste B80
Boise, ID 83702

RIA Member