Network Compute Bridge¶
The network compute bridge allows you to off-load computation to another computer on the network. For example, you might want to offload a deep neural-network call to a server in the cloud.
tensorflow_server.py: Processes requests to run a Faster R-CNN TensorFlow model on Spot image data.
Registers a Network Compute Bridge Worker with the Spot directory service
Handles Network Compute queries and executes the TensorFlow model on the image or image source input
Can be easily reconfigured to run other TensorFlow object detection models
identify_object.py: Requests identification on an image using a TensorFlow model.
Client requests an image source, such as
Robot takes image, rotates it to align with the horizon and sends it to a server running
Results are returned to the client. If depth data is available inside the bounding box, the robot adds depth to the result.
These examples require the bosdyn API and client to be installed, and must be run using python3.
tensorflow_server.py example requires TensorFlow to be installed. You can install its
For non-GPU installations: python3 -m pip install -r requirements_tensorflow_server_cpu.txt For CUDA / NVIDIA GPU installations: python3 -m pip install -r requirements_tensorflow_server_gpu.txt
Installation of NVIDIA drivers and CUDA is outside the scope of the document. There are many tutorials available such as this one.
The client example (
identify_object.py) does not require TensorFlow:
python3 -m pip install -r requirements_client_only.txt
To run this example, first launch the server and direct it to your Spot:
python3 tensorflow_server.py --username <USERNAME> --password <PASSWORD> -d <MODEL DIRECTORY> <ROBOT ADDRESS>
PASSWORDare your user credentials for your Spot.
MODEL DIRECTORYis the path to the directory containing the TensorFlow model to be hosted.
ROBOT ADDRESSis the IP address or hostname of your Spot.
An example model directory can be obtained here:
https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1_detection_zoo.md. As an example, use the
frozen_inference_graph.pb file from the
faster_rcnn_inception_v2_coco model to detect people in the camera images.
To associate numeric labels with names, the user may optionally provide a CSV file in the model directory with a matching file name, e.g. frozen_inference_graph.csv, with the following format:
2 are possible label outputs for detections from the TensorFlow model and
oranges are their respective semantic names. The tensorflow_server.py example output indicates if a CSV has been loaded to associate label values with names.
An example CSV file for the Faster R-CNN model described above is included in this directory as
After launching tensorflow_server.py, the user may post requests to it using the
identify_object.py client example. To run this example with the above example server and model, run:
python3 identify_object.py --username <USERNAME> --password <PASSWORD> --service <SERVICE_NAME> --model <MODEL_NAME> <ROBOT_IP> For example: python3 identify_object.py --username <USERNAME> --password <PASSWORD> --service tensorflow-server --confidence 0.5 --model frozen_inference_graph --image-source frontleft_fisheye_image <ROBOT_IP>
PASSWORD are your user credentials for your Spot.
Ensure that your firewall is allowing traffic on the specified port (default: 50051).