Image Service Helpers

class bosdyn.client.image_service_helpers.CameraInterface[source]

Bases: abc.ABC

Abstract class interface for capturing and decoding images from a camera.

This interface is used by the VisualImageSource to ensure that each image source has the expected capture and decoding methods with the right specification.

abstract blocking_capture()[source]

Communicates with the camera payload to collect the image data.

Returns

A tuple with image data (in any format), and the capture timestamp in seconds (float) in the service computer’s clock.

abstract image_decode(image_data, image_proto, image_format, quality_percent)[source]

Decode the image data into an Image proto based on the requested format and quality.

Parameters
  • image_data (Same format as the output of blocking_capture) – The image data output from a capture.

  • image_proto (image_pb2.Image) – The proto message to populate with decoded image data.

  • image_format (image_pb2.Image.Format) – Requested image format (e.g. raw, jpeg) for the decoded data. Note, if the image_format is None, the image service should return the appropriate format for the requested image.

  • quality_percent (int) – The desired quality percent for decoding the data.

Returns

None. Mutates the image_proto message with the decoded data. This function should set the image data, pixel format, image format, and potentially the transform snapshot fields within the image_proto protobuf message.

class bosdyn.client.image_service_helpers.VisualImageSource(image_name, camera_interface, rows=None, cols=None, gain=None, exposure=None, logger=None)[source]

Bases: object

Helper class to represent a single image source.

This helper class can be used to track state associated with an image source. It can capture images and decode image data for an image service. It can be configured to: - throw faults for capture or decode failures - request images continuously on a background thread to allow image services to respond more rapidly to GetImage requests.

Parameters
  • image_name (string) – The name of the image source.

  • camera_interface (subclass of CameraInterface) – A camera class which inherits from and implements the abstract methods of the CameraInterface class.

  • rows (int) – The number of rows of pixels in the image.

  • cols (int) – The number of cols of pixels in the image.

  • gain (float | function) – The sensor’s gain in dB. This can be a fixed value or a function which returns the gain as a float.

  • exposure (float | function) – The exposure time for an image in seconds. This can be a fixed value or a function which returns the exposure time as a float.

  • logger (logging.Logger) – Logger for debug and warning messages.

set_logger(logger)[source]

Override the existing logger for the VisualImageSource class.

create_capture_thread(capture_period=0)[source]

Initialize a background thread to continuously capture images.

Parameters

capture_period (int) – Amount of time (in seconds) between captures to wait before triggering the next capture. Defaults to “no wait” between captures.

initialize_faults(fault_client, image_service)[source]

Initailize a fault client and faults for the image source (linked to the image service).

The fault client can be used to throw faults for capture and decode errors. All faults associated with the image service name provided will be cleared.

Parameters
  • fault_client (FaultClient) – The fault client to communicate with the robot.

  • image_service (string) – The image service name to associate faults with.

trigger_fault(error_message, fault)[source]

Trigger a service fault for a failure to the fault service.

Parameters
  • error_message (string) – The error message to provide in the camera capture fault.

  • fault (service_fault_pb2.ServiceFault) – The complete service fault to be issued.

clear_fault(fault)[source]

Attempts to clear the camera capture fault from the fault service.

Parameters

fault (service_fault_pb2.ServiceFault) – The fault (which contains an ID) to be cleared.

get_image_and_timestamp()[source]

Retrieve the latest captured image and timestamp.

Returns

The latest captured image and the time (in seconds) associated with that image capture. Throws a camera capture fault and returns None if the image cannot be retrieved.

image_decode_with_error_checking(image_data, image_proto, decode_format, quality_percent)[source]

Decode the image data into an Image proto based on the requested format and quality.

Parameters
  • image_data (any format) – The image data returned by the camera interface’s blocking_capture function.

  • image_proto (image_pb2.Image) – The image proto to be muated with the decoded data.

  • decode_format (image_pb2.Image.Format) – The requested image format (e.g. jpeg, raw)

  • quality_percent (int) – The decode quality percent.

Returns

Boolean indicating if the decode succeeds. Throws a decode data fault if the image cannot be decoded to the desired format. Mutates the image_proto Image proto with the decoded data if successful.

stop_capturing(timeout_secs=10)[source]

Stop the image capture thread if one exists.

Parameters

timeout_secs (int) – The timeout [seconds] for attempting to join and destroy the capture thread.

static make_image_source(source_name, rows=None, cols=None, image_type=1)[source]

Create an instance of the image_pb2.ImageSource for a given source name.

Parameters
  • name (string) – The name for the camera source.

  • rows (int) – The number of rows of pixels in the image.

  • cols (int) – The number of cols of pixels in the image.

  • image_type (image_pb2.ImageType) – The type of image (e.g. visual, depth).

Returns

An ImageSource with the cols, rows, and image type populated.

static make_capture_parameters(gain=None, exposure=None)[source]

Creates an instance of the image_pb2.CaptureParameters protobuf message.

Parameters
  • gain (float | function) – The sensor’s gain in dB. This can be a fixed value or a function which returns the gain as a float.

  • exposure (float | function) – The exposure time for an image in seconds. This can be a fixed value or a function which returns the exposure time as a float.

Returns

An instance of the protobuf CaptureParameters message.

class bosdyn.client.image_service_helpers.ImageCaptureThread(image_source_name, capture_func, capture_period_secs=0)[source]

Bases: object

Continuously query and store the last successfully captured image and it’s associated timestamp for a single camera device.

Parameters
  • image_source_name (string) – The image source name.

  • capture_func (CameraInterface.blocking_capture) – The function capturing the image data and timestamp.

  • capture_period_secs (int) – Amount of time (in seconds) between captures to wait before triggering the next capture. Defaults to “no wait” between captures.

start_capturing()[source]

Start the background thread for the image captures.

set_last_captured_image(image_frame, capture_time)[source]

Update the last image capture and timestamp.

get_latest_captured_image()[source]

Returns the last found image and the timestamp it was acquired at.

stop_capturing(timeout_secs=10)[source]

Stop the image capture thread.

class bosdyn.client.image_service_helpers.CameraBaseImageServicer(bosdyn_sdk_robot, service_name, image_sources, logger=None, use_background_capture_thread=True)[source]

Bases: bosdyn.api.image_service_pb2_grpc.ImageServiceServicer

GRPC service to provide access to multiple different image sources.

The service can list the available image (device) sources and query each source for image data.

Parameters
  • bosdyn_sdk_robot (Robot) – The robot instance for the service to connect to.

  • service_name (string) – The name of the image service.

  • image_sources (List[VisualImageSource]) – The list of image sources (provided as a VisualImageSource).

  • logger (logging.Logger) – Logger for debug and warning messages.

  • use_background_capture_thread (bool) – If true, the image service will create a thread that continuously captures images so the image service can respond rapidly to the GetImage request. If false, the image service will call an image sources’ blocking_capture_function during the GetImage request.

ListImageSources(request, context)[source]

Obtain the list of ImageSources for this given service.

Parameters
  • request (image_pb2.ListImageSourcesRequest) – The list images request.

  • context (GRPC ClientContext) – tracks internal grpc statuses and information.

Returns

A list of all image sources known to this image service. Note, there could be multiple image services registered with the robot’s directory service that can have different image sources.

GetImage(request, context)[source]

Gets the latest image capture from all the image sources specified in the request.

Parameters
  • request (image_pb2.GetImageRequest) – The image request, which specifies the image sources to query, and other format parameters.

  • context (GRPC ClientContext) – tracks internal grpc statuses and information.

Returns

The ImageSource and Image data for the last captured image from each image source name specified in the request.