Machine Learning Bridge and External Compute
Spot provides capabilities to perform computation off-board and easily integrate it into your API programs. For example, you might want to run a deep-learning-based bounding box detector to identify a ball that you want to command the robot to pick up. The
Network Compute Bridge functionality in the system can help you by coordinating the networking between various components and capturing images for you as needed. The tablet application has a built-in capability to view the output images from the detectors online.
Components of the system
The diagram below shows the diagram of
Network Compute Bridge functionality.
Compute Server applications as
NetworkComputeBridgeWorker RPC services and register them with with Spot’s Directory. These workers can run on an onboard compute payload, such as the CORE I/O, or any server networked with the Spot system. The
NetworkComputeBridge service running on the robot periodically checks for new workers and queries them for available model names through the
ListAvailableModels. API clients, such as the tablet application or any application that uses the Spot SDK, communicate with the
NetworkComputeBridge service on the robot through the
NetworkCompute RPC to process any set of images with any available models in registered in the system.
You have trained a model to detect tennis balls. You want to build a script to walk to every tennis ball in an area. Because your tennis ball detector is computationally expensive, you decide to have a big server running on the same WiFi network as Spot to do the computation.
Here is how you might set that up:
Large server is running in a server room, on the same WiFi network as Spot, or a compute payload attached to Spot, such as the CORE I/O.
Your client laptop has a script that calls the
Network Compute Bridgeservice to:
capture an image
run the tennis ball detector on that image
return results to the script
With the results, your script commands the robot to walk to a tennis ball.
Example Clients and Servers
Examples are found in the network compute bridge examples folder.
NetworkComputeBridgeis the API interface between your API client and the robot.
NetworkComputeBridgeWorkeris the API interface between the robot and a compute server
External applications do not communicate directly with the worker services. They always communicate through the
NetworkComputeBridge service and API interface.
The workers return any processed data through the
other_data fields in
WorkerComputeResponse message. Those pieces of information are then forwarded through the
NetworkComputeResponse message to the main client.
Does your service require inputs or parameters? Do you want controls or widgets for those parameters to show up on the SpotApp android application or on Scout? Please refer to this document to learn how to add parameters to a network compute bridge worker.