slmsuite.hardware.cameras.camera.Camera#

class Camera(resolution, bitdepth=8, pitch_um=None, name='camera', averaging=None, capture_attempts=5, hdr=None, rot='0', fliplr=False, flipud=False)[source]#

Bases: _Picklable

Abstract class for cameras. Comes with transformations, averaging and HDR, and helper functions like autoexpose().

name#

Camera identifier.

Type:

str

shape#

Stores (height, width) of the camera in pixels, the same convention as numpy.shape().

Type:

(int, int)

bitdepth#

Depth of a camera pixel well in bits.

Type:

int

bitresolution#

Stores 2**bitdepth.

Type:

int

dtype#

Stores the type returned by _get_image_hw(). This value is cached upon initialization.

Type:

type

pitch_um#

Pixel pitch in microns.

Type:

(float, float) OR None

exposure_s#

Caches the last result of get_exposure(). Can be used if the user wants to avoid the overhead of calling the method.

Type:

float

exposure_bounds_s#

Shortest and longest allowable integration in seconds.

Type:

(float, float) OR None

averaging#

Default setting for averaging. See get_image().

Type:

int OR None

hdr#

Default setting for multi-exposure High Dynamic Range imaging. See get_image().

Type:

(int, int) OR None

capture_attempts#

If the camera returns an error or exceeds a timeout, try again for a total of capture_attempts attempts. This is useful for resilience against errors that happen with low probability. Defaults to 5.

Type:

int

woi#

WOI (window of interest) in (x, width, y, height) form.

Warning

This feature is less fleshed out than most. There may be issues (e.g. get_image() with the averaging or hdr flags).

Type:

tuple

default_shape#

Default shape of the camera before any WOI or transform changes are made.

Type:

tuple

transform#

Flip and/or rotation operator specified by the user in __init__(). The user is expected to apply this transform to the matrix returned in get_image(). Note that WOI changes are applied on the camera hardware before this transformation.

Type:

function

last_image#

Last captured image. Note that this is a pointer to the same data that the user receives (to avoid copying overhead). Thus, if the user modifies the returned data, then this data will be modified also. This may be of dtype, or may be a float, depending on whether hdr is used and the type of averaging. Is None if no image has ever been taken.

Type:

numpy.ndarray OR None

Methods

autoexposure

Sets the exposure of the camera such that the maximum value is at set_fraction of the dynamic range.

autofocus

Uses a FFT contrast metric to find optimal focus when scanning over some variable z.

close

Abstract method to close the camera and delete related objects.

flush

Cycle the image buffer such that all new get_image() calls yield fresh frames.

get_exposure

Get the frame integration time in seconds.

get_image

Capture, process, and return images from a camera.

get_image_hdr

Often, the necessities of precision applications exceed the bitdepth of a camera.

get_image_hdr_analysis

Analyzes raw data for High Dynamic Range (HDR) imaging multiple exposures each with increasing exposure time.

get_images

Grab image_count images in succession.

info

Abstract method to load information about what cameras are available.

live

Creates and displays an IPython camera viewer.

pickle

Returns a dictionary containing selected attributes of this class.

plot

Plots the provided image.

save

Saves the dictionary returned from pickle() to a file like "path/name_id.h5".

set_exposure

Set the frame integration time in seconds.

set_woi

Abstract method to narrow the imaging region to a 'window of interest' for faster framerates.
__init__(resolution, bitdepth=8, pitch_um=None, name='camera', averaging=None, capture_attempts=5, hdr=None, rot='0', fliplr=False, flipud=False)[source]#

Initializes a camera.

In addition to the other class attributes, accepts the following parameters to set transform. See get_orientation_transformation().

Parameters:

  • resolution –

    The width and height of the camera in (width, height) form.

    Important

    This is the opposite of the numpy (height, width) convention stored in shape.

  • bitdepth – See bitdepth.

  • pitch_um ((float, float) OR None) – Fill in extra information about the pixel pitch in (dx_um, dy_um) form to use additional calibrations.

  • name (str) – Defaults to "camera".

  • averaging (int or None) – Number of frames to average. Used to increase the effective bit depth of a camera by using pre-quantization noise (e.g. dark current, read-noise, etc.) to “dither” the pixel output signal. If None, no averaging is performed.

  • hdr (int OR (int, int) OR None OR False) – Exposure information for Multi-exposure High Dynamic Range (HDR) imaging

  • capture_attempts (int) – If the camera returns an error or exceeds a timeout, try again for a total of capture_attempts attempts. This is useful for resilience against errors that happen with low probability. Defaults to 5.

  • rot (str or int) – Rotates returned image by the corresponding degrees in ["90", "180", "270"] or numpy.rot90() code in [1, 2, 3]. Defaults to no rotation. Used to determine shape and transform.

  • fliplr (bool) – Flips returned image left right. Used to determine transform.

  • flipud (bool) – Flips returned image up down. Used to determine transform.

close()[source]#

Abstract method to close the camera and delete related objects.

static info(verbose=True)[source]#

Abstract method to load information about what cameras are available.

Parameters:

verbose (bool) – Whether or not to print display information.

Returns:

An empty list.

Return type:

list

get_exposure()[source]#

Get the frame integration time in seconds. Used in autoexposure().

Returns:

Integration time in seconds.

Return type:

float

set_exposure(exposure_s)[source]#

Set the frame integration time in seconds. Used in autoexposure().

Parameters:

exposure_s (float) – The integration time in seconds.

Returns:

Set integration time in seconds.

Return type:

float

set_woi(woi=None)[source]#

Abstract method to narrow the imaging region to a ‘window of interest’ for faster framerates.

Parameters:

woi (list, None) – See woi. If None, defaults to largest possible.

Returns:

woi – woi.

Return type:

list

flush(timeout_s=1)[source]#

Cycle the image buffer such that all new get_image() calls yield fresh frames. Without this feature, optimizations could be working on outdated information.

Defaults to calling get_image() twice, though cameras can implement hardware-specific alternatives.

Parameters:

timeout_s (float) – The time in seconds to wait for each frame. The frame exposure time is added to this timeout such that there is always enough time to expose.

get_image(timeout_s=1, transform=True, hdr=None, averaging=None)[source]#

Capture, process, and return images from a camera.

Tip

This function includes two advanced capture options:

These methods can aid the user in capturing more precise data, beyond the default raw (and bitdepth-limited) output of the camera.

Parameters:

  • timeout_s (float) – The time in seconds to wait for the frame to be fetched. The frame exposure time is added to this timeout such that there is always enough time to expose.

  • transform (bool) – Whether or not to transform the output image according to transform. Defaults to True.

  • hdr (int OR (int, int) OR None OR False) –

    Exposure information for Multi-exposure High Dynamic Range (HDR) imaging If None, the value of hdr is used. If False, HDR is not used no matter the state of hdr.

    See also

    get_image_hdr() for more information.

  • averaging (int OR None OR False) –

    If int, the number of frames to average over. If None, the value of averaging is used. If False, averaging is not used no matter the state of averaging.

    Tip

    The datatype is promoted to float if necessary but otherwise tries to stick with the default datatype. For instance, a camera that returns a 12-bit image as a 16-bit type has four more bits to use for averaging, i.e. \(2^4 = 16\) possible averages without risk of overflow. Requesting more than 16 averages would cause the return type to be promoted to float.

    Note

    Averaging is a bit of a misnomer as the true functionality is to sum or integrate the images. This is done such that integer datatypes (useful for memory compactness) can still be returned; a general mean would need to be floating point.

Returns:

Array of shape shape.

Return type:

numpy.ndarray of int OR float

get_images(image_count, timeout_s=1, out=None, transform=True, flush=False)[source]#

Grab image_count images in succession.

Important

This method does not support averaging or HDR features. Rather, it just returns a series of raw images.

Parameters:

  • image_count (int) – Number of images to grab.

  • timeout_s (float) – The time in seconds to wait for each frame to be fetched. The frame exposure time is added to this timeout such that there is always enough time to expose.

  • out (None OR numpy.ndarray) – If not None, output data in this memory. Useful to avoid excessive allocation.

  • transform (bool) – Whether or not to transform the output image according to transform. Defaults to True.

  • flush (bool) – Whether to flush before grabbing.

Returns:

Array of shape (image_count, height, width).

Return type:

numpy.ndarray

get_image_hdr(exposures=None, return_raw=False, **kwargs)[source]#

Often, the necessities of precision applications exceed the bitdepth of a camera. One way to recover High Dynamic Range (HDR) imaging is to use multiple exposures each with increasing exposure time. Then, these images can be stitched together as floating-point data, omitting data which is under- or over- exposed.

Tip

This feature can be accessed in get_image() using hdr or the hdr= flag. This function is exposed here also to reveal the raw data using return_raw= and to draw attention to this useful feature.

Caution

Camera exposure is sometimes poorly defined. This might cause incorrect assumptions of the exposure. In general, a larger base exposure will produce more accurate results as a greater number of sample clock periods are rounded to for smaller relative variation. Future modifications to get_image_hdr_analysis() might improve image stitching.

Parameters:

  • exposures (int OR (int, int)) – The number of exposures to take. Each exposure increases in time multiplicatively from the base value (original get_exposure()) by a factor \(p\). The \(i\text{th}\) image has exposure time \(\tau \times p^i\), zero-indexed. The default base of \(p = 2\) leads to exposures being equivalent to spots. This base can be changed to another number by instead passing a tuple, where the second int defines the desired base.

  • return_raw (bool) – If True, returns the raw data (stack of images with count exposures) instead of the processed data. The data can be processed using get_image_hdr_analysis()

  • **kwargs – Passed to get_image().

Returns:

Array of shape shape.

Important

The scale of the returned image is the same as the original exposure.

Return type:

numpy.ndarray of float

static get_image_hdr_analysis(imgs, overexposure_threshold=None, exposure_power=2)[source]#

Analyzes raw data for High Dynamic Range (HDR) imaging multiple exposures each with increasing exposure time.

Parameters:

  • imgs (array_like) – Stack of images with increasing exposure.

  • overexposure_threshold (float OR None) – For each image (except the first), data is thrown out if values are above this threshold. If None, the threshold defaults to half the maximum.

  • exposure_power (int) –

    Each exposure increases in time multiplicatively from the base value (original get_exposure()) by this factor \(p\). The \(i\text{th}\) image has exposure time \(\tau \times p^i\), zero-indexed. The default value of 2 leads to exposures being equivalent to spots.

Returns:

Array of shape shape.

Important

The scale of the returned image is the same as the original exposure.

Return type:

numpy.ndarray of float

plot(image=None, limits=None, title='Image', ax=None, cbar=True)[source]#

Plots the provided image.

Parameters:

  • image (ndarray OR None) – Image to be plotted. If None, grabs an image from the camera.

  • limits (None OR float OR [[float, float], [float, float]]) – Scales the limits by a given factor or uses the passed limits directly.

  • title (str) – Title the axis.

  • ax (matplotlib.pyplot.axis OR None) – Axis to plot upon.

  • cbar (bool) – Also plot a colorbar.

Returns:

Axis of the plotted image.

Return type:

matplotlib.pyplot.axis

live(activate=None, widgets=True, backend='ipython', **kwargs)[source]#

Creates and displays an IPython camera viewer. This viewer displays the result of get_image() or the last image of get_images() whenever these methods are called. Averaging and HDR are displayed with the same color scaling as without.

If True is passed to the widgets argument, this viewer is accompanied by a series of IPython widgets in the form of slides and buttons for controlling the color scale, colormap, viewer scale, and live viewing. By toggling the Live widget button, this viewer can be used as a realtime camera monitor within the jupyter notebook. Note that any user-execution will block the monitoring loop. Regardless, any image polling during the blocked period will still update the viewer, which provides useful active feedback for what is happening during the execution.

This limitation is imposed by the python Global Interpreter Lock (GIL) which restricts operation to a single thread, especially operation connecting to a diverse set of camera and SLM hardware. We use asyncio to allow the realtime monitoring loop to be interrupted by user-execution (e.g. running a cell in jupyter), blocking until the execution is finished.

Running multiple viewers at once might not play nicely right now.

Parameters:

  • activate (bool OR None) – If True, creates a live viewer in the current cell, destroying any other attached viewer. If False, destroys any other attached viewer. If None, toggles the live viewer, destroying any attached viewer or creating one in the current cell if none is attached. Defaults to None.

  • widgets (bool) – If True, also displays sliders and controls used to hone the display properties.

  • backend (str) – Placeholder option for different types of viewers. The default is "ipython".

  • **kwargs – Options passed to the _CameraViewer to customize the default settings. These features will be made less hidden in the future. Most things are customizable via these keywords. For instance, the user can pass a custom list of colormaps to appear in the widget dropdown as cmap_options=.

autoexposure(set_fraction=0.5, tol=0.05, exposure_bounds_s=None, window=None, average_count=5, timeout_s=5, verbose=True)[source]#

Sets the exposure of the camera such that the maximum value is at set_fraction of the dynamic range. Useful for mitigating deleterious over- or under- exposure.

Parameters:

  • set_fraction (float) – Fraction of camera dynamic range to target image maximum.

  • tol (float) – Fractional tolerance for exposure adjustment.

  • exposure_bounds_s ((float, float) OR None) – Shortest and longest allowable integration in seconds. If None, defaults to exposure_bounds_s. If this attribute was not set (or not availible on a particular camera), then None instead defaults to unbounded.

  • window (array_like or None) – See window. If None, the full camera frame will be used.

  • average_count (int) – Number of frames to average intensity over for noise reduction.

  • timeout_s (float) – Stop attempting adjusting exposure after timeout_s seconds.

  • verbose (bool) – Whether to print exposure updates.

Returns:

Resulting exposure in seconds.

Return type:

float

autofocus(get_z, set_z, z_list=None, plot=False)[source]#

Uses a FFT contrast metric to find optimal focus when scanning over some variable z. This z often takes the form of a vertical stage to position a sample precisely at the plane of imaging of a lens or objective. The contrast metric works particularly well when combined with a projected spot array hologram.

Parameters:

  • get_z (function) – Gets the current position of the focusing stage. Should return a float.

  • set_z (function) – Sets the position of the focusing stage to a given float.

  • z_list (array_like or None) – z values to sweep over during search. Defaults (when None) to numpy.linspace(-4,4,16).

  • plot (bool) – Whether to provide illustrative plots.

pickle(attributes=True, metadata=True)[source]#

Returns a dictionary containing selected attributes of this class.

Parameters:

  • attributes (bool OR list of str) – If False, pickles only baseline attributes, usually single floats. If True, also pickles ‘heavy’ attributes such as large images and calibrations. If list of str, pickles the keys in the given list. By default, the chosen attributes should be things that can be written to .h5 files: scalars and lists of scalars.

  • metadata (bool) – If True, package the dictionary as the "__meta__" value of a superdictionary which also contains: "__version__", the current slmsuite version, "__time__", the time formatted as a date string, and "__timestamp__", the time formatting as a floating point timestamp. This information is used as standard metadata for calibrations and saving.

save(path='.', name=None, **kwargs)[source]#

Saves the dictionary returned from pickle() to a file like "path/name_id.h5".

Parameters:

  • path (str) – Path to directory to save in. Default is current directory.

  • name (str OR None) – Name of the save file. If None, will use name + '-pickle'.

  • **kwargs – Passed to pickle() to customize how and what data is saved.

Returns:

The file path that the pickled data was saved to.

Return type:

str