Understanding Robot Behavior with Log Parsing

Tuong Anh Ens

Log parsing helps find issues before customers do

Overview

Neato Robotics develops and sells consumer robotic vacuum cleaners. To keep pace with competition, they must update robot features on a regular schedule. Ongoing code changes have the potential to introduce new bugs. It’s imperative to find and resolve bugs quickly in order to satisfy customers and remain competitive in the marketplace.

Neato Robotics hired Go West to improve product quality, prevent regression and to speed up the pace of development. Go West assessed Neato’s approach to quality and identified log parsing as a powerful tool to help Neato achieve their goals.

What are Logs?

Raw log files contain a wealth of information about the actions of a robot. Log files can include sensor readings, memory and cpu usage, operating system information and hardware and electronic errors. Raw log files come in gigabytes of data. They are difficult to process and understand. When a bug is found on a robot, it can take a robotics engineer hours to sift through a log file and make sense of it.

The Right Tool

Go West built a log parser to analyze and catalogue complex robotic behavior. We used our in-depth industry experience to quantify problems that would not normally surface to a human observer. This included focusing on specific time frames, identifying errors and correlating bugs to those errors. The log parser organized this information in a way that made sense to the QA team and developers.

A More Efficient Process

Go West built the log parser specifically for Neato’s QA department, allowing the department to gather relevant data before escalating bugs to the appropriate engineering resource for debugging. Instead of placing the burden on developers to sift through log files, the QA department had a powerful tool to organize, research and prioritize bugs before using up valuable developer cycles.

Results

After the log parser was built and the QA team started using it, Neato could cover hundreds of test cases from version to version. This ensured that regression did not occur and it gave Neato a method to address many bugs quickly. The result was a time (and cost) savings for the engineering team, a significantly more robust product and ultimately happier customers!

Conclusion

A thorough understanding of the end-to-end design of complex robots allows Go West Robotics to help companies streamline their product testing and development. In the case of Neato robotics, we identified, designed and built a tool that improved product quality. We recommended an organizational process change that saved significant time and costs. In the end, Neato is able to keep up with regular feature releases, while maintaining product quality and customer satisfaction.