slmsuite.hardware.cameras.simulated.SimulatedCamera#

class SimulatedCamera(slm, resolution=None, M=None, b=None, noise=None, pitch_um=None, gain=1, **kwargs)[source]#

Bases: Camera

Simulated camera.

Outputs simulated images (i.e., the far-field of an SLM interpolated to camera pixels based on the camera’s location and orientation. Serves as a future testbed for simulation of other imaging artifacts, including non-affine aberrations (e.g. pincushion distortion) and imaging readout noise.

Note

For fastest simulation, initialize SimulatedCamera with a SimulatedSLM only. Simulated camera images will directly sample the (quickly) computed SLM far-field ("knm") via a one-to-one mapping instead of interpolating the SLM’s far-field intensity at each camera pixel location (i.e. "knm"->``”ij”`` basis change), which may also require additional padding (computed automatically upon initialization) for sufficient resolution.

grid#

Pixel column/row number (x_grid, y_grid) in the "ij" basis used for far-field interpolation.

Type:: (numpy.ndarray, numpy.ndarray)

shape_padded#

Size of the FFT computational space required to faithfully reproduce the far-field at full camera resolution.

Type:: (int, int)

noise#

Dictionary of single-argument functions (returning the normalized noise amplitude for any normalized input pixel amplitude) to simulate various noise sources. Currently, 'dark' and 'read', representing exposure-dependent dark current/background noise and exposure-independent readout noise, respectively, are the only accepted keys.

Example

The following code adds a Gaussian background with 50% mean and 5% standard deviation (relative to the dynamic range at the default self.exposure_s = 1) and a Poisson readout noise (independent of self.exposure_s) with an average value of 20% of the camera’s dynamic range.

self.noise = {
    'dark': lambda img: np.random.normal(0.5*img, 0.05*img),
    'read': lambda img: np.random.poisson(0.2*img)
}

Type:: dict

Methods

`autoexposure`	Sets the exposure of the camera such that the maximum value is at `set_fraction` of the dynamic range.
`autofocus`	Finds optimal focus when scanning over some variable `z`.
`build_affine`	Builds an affine transform defining the SLM to camera transformation as detailed in `kxyslm_to_ijcam()`.
`close`	Abstract method to close the camera and delete related objects.
`flush`	See `Camera.flush()`.
`get_exposure`	Get the frame integration time in seconds.
`get_image`	Capture, process, and return images from a camera.
`get_image_hdr`	Often, the necessities of precision applications exceed the bitdepth of a camera.
`get_image_hdr_analysis`	Analyzes raw data for High Dynamic Range (HDR) imaging multiple exposures each with increasing exposure time.
`get_images`	Grab `image_count` images in succession.
`info`	Abstract method to load information about what cameras are available.
`live`	Creates and displays an IPython camera viewer.
`pickle`	Returns a dictionary containing selected attributes of this class.
`plot`	Plots the provided image.
`save`	Saves the dictionary returned from `pickle()` to a file like `"path/name_id.h5"`.
`set_affine`	Set the camera's placement in the SLM's k-space.
`set_exposure`	Set the frame integration time in seconds.
`set_woi`	Abstract method to narrow the imaging region to a 'window of interest' for faster framerates.
`test`	Tests the core hardware methods of `Camera`.

Attributes

bitresolution

__init__(slm, resolution=None, M=None, b=None, noise=None, pitch_um=None, gain=1, **kwargs)[source]#

Initialize simulated camera.

Parameters:

slm (SimulatedSLM) – Simulated SLM creating the image.
resolution ((int, int)) – See resolution. If None, defaults to the resolution of slm.
M (array_like) – Passed to set_affine(). Can be set later, but the camera cannot be used until then.
b (array_like) – Passed to set_affine(). Can be set later, but the camera cannot be used until then.
noise (dict) – See noise.
pitch_um ((float, float) OR None) – Pixel pitch in microns. If None, certain calibrations and conversions are not available (e.g. build_affine() for certain units).
gain (float) – Gain to emulate physical cameras while keeping the same values for exposure time.
**kwargs – See Camera.__init__() for permissible options.

close()[source]#: Abstract method to close the camera and delete related objects.

set_affine(M=None, b=None, **kwargs)[source]#

Set the camera’s placement in the SLM’s k-space. M and/or b, if provided, are used to transform the SimulatedCamera’s "ij" grid to a "knm" grid for interpolation against the SimulatedSLM’s "knm" grid. Keyword arguments, if provided, are passed to build_affine() to build M and b.

Parameters:

M (array_like) – 2 x 2 affine transform matrix to convert between SLM’s \(k\)-space and the simulated camera’s pixel basis ("ij"). If None, defaults to the identity matrix.
b (array_like) – Lateral displacement (in pixels) of the camera center from the SLM’s optical axis. If None, defaults to (0,0) offset.
**kwargs (dict, optional) – Various orientation parameters passed to build_affine() to build M and b, if not provided. See options documented in this method. f_eff is a required keyword.

build_affine(f_eff, units='norm', theta=0, shear_angle=0, offset=None)[source]#

Builds an affine transform defining the SLM to camera transformation as detailed in kxyslm_to_ijcam().

Parameters:

f_eff (float OR (float, float)) – Effective focal length of the optical train separating the Fourier-domain SLM from the camera. If a float is provided, f_eff is isotropic; otherwise, f_eff is defined along the SLM’s \(x\) and \(y\) axes.
units (str {"norm", "ij", "m", "cm", "mm", "um", "nm"}) –
Units for the focal length f_eff.
- "norm"
  Normalized focal length in wavelengths according to the SLM’s wav_um. This is the default unit.
- "ij"
  Focal length in units of camera pixels.
- "m", "cm", "mm", "um", "nm"
  Focal length in metric units.
theta (float) – Rotation angle (in radians, ccw) of the camera relative to the SLM orientation. Defaults to zero (i.e., aligned with the SLM).
shear_angle (float OR (float, float)) – Shearing angles (in radians) along the SLM’s \(x\) and \(y\) axes. If a float is provided, shear is applied isotropically. Defaults to zero (i.e., no shear).
offset ((float, float) OR None) – Lateral displacement (in pixels units) of the SLM’s optical axis from the camera’s origin. If None, defaults to be centered on the center of the camera.

Returns:

numpy.ndarray – Affine matrix \(M\). Shape (2, 2).
numpy.ndarray – Affine vector \(b\). Shape (1, 2).

flush(timeout_s=1)[source]#: See Camera.flush().

autoexposure(set_fraction=0.5, tol=0.05, exposure_bounds_s=None, window=None, timeout_s=5, verbose=True)[source]#

Sets the exposure of the camera such that the maximum value is at set_fraction of the dynamic range. Useful for mitigating over- or under- exposure.

Parameters:

set_fraction (float) – Fraction of camera dynamic range to use as a target image maximum.
tol (float) – Fractional tolerance for exposure adjustment.
exposure_bounds_s ((float, float) OR None) – Shortest and longest allowable integration in seconds. If None, defaults to exposure_bounds_s. If this attribute was not set (or not available on a particular camera), then None instead defaults to unbounded.
window (array_like or None) – See woi. If None, the full camera frame will be used.
timeout_s (float) – Stop attempting adjusting exposure after timeout_s seconds.
verbose (bool) – Whether to print exposure updates.

Returns:

Resulting exposure in seconds.

Return type:

float

autofocus(set_z, get_z=0, range_z=2, metric=None, plot=False, verbose=False)[source]#

Finds optimal focus when scanning over some variable z. This z often takes the form of a vertical stage to position a sample precisely at the plane of imaging of a lens or objective. The default metric is based on the Fourier contrast of the image, and works particularly well when combined with a projected spot array hologram.

Parameters:

set_z (function OR SLM) – Sets the position of the focusing stage to a given float. If an SLM is passed, adds a lens phase to the SLM to focus the existing pattern. In this case, the units of z are in Zernike defocus terms (wavefronts). The optimal defocus is added to the wavefront calibration (source["phase"]) of the SLM.
get_z (function OR float) – Gets the current position of the focusing stage. Should return a float. Can also pass a float representing the center of the search range.
range_z (array_like OR float OR None) – z values to sweep over during search relative to the base position get_z. If a single float is passed, sweeps from -range_z to +range_z with 11 steps.
metric (function OR None) – Function which evaluates the focus quality of an image. Should take in an image and return a scalar figure-of-merit (FoM). Defaults to Camera._autofocus_metric(), which approximates the sharpness of the images (implemented as the Fourier contrast of the image, the sum of the normalized Fourier amplitudes). The sharper the image, the higher the FoM.
plot (bool) – Whether to provide illustrative plots.

Returns:

Optimal z value found.

Return type:

float

get_exposure()[source]#

Get the frame integration time in seconds. Used in autoexposure().

Returns:: Integration time in seconds.
Return type:: float

get_image(timeout_s=1, transform=True, hdr=None, averaging=None)[source]#

Capture, process, and return images from a camera.

Tip

This function includes two advanced capture options:

Multi-exposure High Dynamic Range (HDR) imaging and
Software frame averaging (integrating).

These methods can aid the user in capturing more precise data, beyond the default raw (and bitdepth-limited) output of the camera.

Parameters:

timeout_s (float) – The time in seconds to wait for the frame to be fetched. The frame exposure time is added to this timeout such that there is always enough time to expose.
transform (bool) – Whether or not to transform the output image according to transform. Defaults to True.
hdr (int OR (int, int) OR None OR False) –
Exposure information for Multi-exposure High Dynamic Range (HDR) imaging If None, the value of hdr is used. If False, HDR is not used no matter the state of hdr.

See also

get_image_hdr() for more information.
averaging (int OR None OR False) –
If int, the number of frames to average over. If None, the value of averaging is used. If False, averaging is not used no matter the state of averaging.

Tip

The datatype is promoted to float if necessary but otherwise tries to stick with the default datatype. For instance, a camera that returns a 12-bit image as a 16-bit type has four more bits to use for averaging, i.e. \(2^4 = 16\) possible averages without risk of overflow. Requesting more than 16 averages would cause the return type to be promoted to float.

Important

This feature sums many measurements together (does not mean), thereby averaging without floating point operations. This is done such that integer datatypes (useful for memory compactness) can still be returned, whereas a general mean would need to be floating point.

Returns:

Array of shape shape.

Return type:

numpy.ndarray of int OR float

get_image_hdr(exposures=None, return_raw=False, **kwargs)[source]#

Often, the necessities of precision applications exceed the bitdepth of a camera. One way to recover High Dynamic Range (HDR) imaging is to use multiple exposures each with increasing exposure time. Then, these images can be stitched together as floating-point data, omitting data which is under- or over- exposed.

Tip

This feature can be accessed in get_image() using hdr or the hdr= flag. This function is exposed here also to reveal the raw data using return_raw= and to draw attention to this useful feature.

Caution

Camera exposure is sometimes poorly defined. This might cause incorrect assumptions of the exposure. In general, a larger base exposure will produce more accurate results as a greater number of sample clock periods are rounded to for smaller relative variation. Future modifications to get_image_hdr_analysis() might improve image stitching.

Parameters:

exposures (int OR (int, int)) – The number of exposures to take. Each exposure increases in time multiplicatively from the base value (original get_exposure()) by a factor \(p\). The \(i\text{th}\) image has exposure time \(\tau \times p^i\), zero-indexed. The default base of \(p = 2\) leads to exposures being equivalent to spots. This base can be changed to another number by instead passing a tuple, where the second int defines the desired base.
return_raw (bool) – If True, returns the raw data (stack of images with count exposures) instead of the processed data. The data can be processed using get_image_hdr_analysis()
**kwargs – Passed to get_image().

Returns:

Array of shape shape.

Important

The scale of the returned image is the same as the original exposure.

Return type:

numpy.ndarray of float

static get_image_hdr_analysis(imgs, overexposure_threshold=None, exposure_power=2)[source]#

Analyzes raw data for High Dynamic Range (HDR) imaging multiple exposures each with increasing exposure time.

Parameters:

imgs (array_like) – Stack of images with increasing exposure.
overexposure_threshold (float OR None) – For each image (except the first), data is thrown out if values are above this threshold. If None, the threshold defaults to half the maximum.
exposure_power (int or list of float) –
Each exposure increases in time multiplicatively from the base value (original get_exposure()) by this factor \(p\). The \(i\text{th}\) image has exposure time \(\tau \times p^i\), zero-indexed. The default value of 2 leads to exposures being equivalent to spots.

Returns:

Array of shape shape.

Important

The scale of the returned image is the same as the original exposure.

Return type:

numpy.ndarray of float

get_images(image_count, timeout_s=1, out=None, transform=True, flush=False)[source]#

Grab image_count images in succession.

Important

This method does not support averaging or HDR features. Rather, it just returns a series of raw images.

Parameters:

image_count (int) – Number of images to grab.
timeout_s (float) – The time in seconds to wait for each frame to be fetched. The frame exposure time is added to this timeout such that there is always enough time to expose.
out (None OR numpy.ndarray) – If not None, output data in this memory. Useful to avoid excessive allocation.
transform (bool) – Whether or not to transform the output image according to transform. Defaults to True.
flush (bool) – Whether to flush before grabbing.

Returns:

Array of shape (image_count, height, width).

Return type:

numpy.ndarray

static info(verbose=True)[source]#

Abstract method to load information about what cameras are available.

Parameters:: verbose (bool) – Whether or not to print display information.
Returns:: An empty list.
Return type:: list

live(activate=None, widgets=True, backend='ipython', **kwargs)[source]#

Creates and displays an IPython camera viewer. This viewer displays the result of get_image() or the last image of get_images() whenever these methods are called. Averaging and HDR are displayed with the same color scaling as without.

If True is passed to the widgets argument, this viewer is accompanied by a series of IPython widgets in the form of slides and buttons for controlling the color scale, colormap, viewer scale, and live viewing. By toggling the Live widget button, this viewer can be used as a realtime camera monitor within the jupyter notebook. Note that any user-execution will block the monitoring loop. Regardless, any image polling during the blocked period will still update the viewer, which provides useful active feedback for what is happening during the execution.

This limitation is imposed by the python Global Interpreter Lock (GIL) which restricts operation to a single thread, especially operation connecting to a diverse set of camera and SLM hardware. We use asyncio to allow the realtime monitoring loop to be interrupted by user-execution (e.g. running a cell in jupyter), blocking until the execution is finished.

Running multiple viewers at once might not play nicely right now.

Parameters:

activate (bool OR None) – If True, creates a live viewer in the current cell, destroying any other attached viewer. If False, destroys any other attached viewer. If None, toggles the live viewer, destroying any attached viewer or creating one in the current cell if none is attached. Defaults to None.
widgets (bool) – If True, also displays sliders and controls used to hone the display properties.
backend (str) – Placeholder option for different types of viewers. The default is "ipython".
**kwargs – Options passed to the _CameraViewer to customize the default settings. These features will be made less hidden in the future. Most things are customizable via these keywords. For instance, the user can pass a custom list of colormaps to appear in the widget dropdown as cmap_options=.

pickle(attributes=True, metadata=True)[source]#

Returns a dictionary containing selected attributes of this class.

Parameters:

attributes (bool OR list of str) – If False, pickles only baseline attributes, usually single floats. If True, also pickles ‘heavy’ attributes such as large images and calibrations. If list of str, pickles the keys in the given list. By default, the chosen attributes should be things that can be written to .h5 files: scalars and lists of scalars.
metadata (bool) – If True, package the dictionary as the "__meta__" value of a superdictionary which also contains: "__version__", the current slmsuite version, "__time__", the time formatted as a date string, and "__timestamp__", the time formatting as a floating point timestamp. This information is used as standard metadata for calibrations and saving.

plot(image=None, limits=None, title='Image', ax=None, cbar=True)[source]#

Plots the provided image.

Parameters:

image (ndarray OR None OR bool) – Image to be plotted. If None, grabs an image from the camera. If False, uses the last_image attribute.
limits (None OR float OR [[float, float], [float, float]]) – Scales the limits by a given factor or uses the passed limits directly.
title (str) – Title the axis.
ax (matplotlib.pyplot.axis OR None) – Axis to plot upon.
cbar (bool) – Also plot a colorbar.

Returns:

Axis of the plotted image.

Return type:

matplotlib.pyplot.axis

save(path='.', name=None, **kwargs)[source]#

Saves the dictionary returned from pickle() to a file like "path/name_id.h5".

Parameters:

path (str) – Path to directory to save in. Default is current directory.
name (str OR None) – Name of the save file. If None, will use name + '-pickle'.
**kwargs – Passed to pickle() to customize how and what data is saved.

Returns:

The file path that the pickled data was saved to.

Return type:

str

set_exposure(exposure_s)[source]#

Set the frame integration time in seconds. Used in autoexposure().

Parameters:: exposure_s (float) – The integration time in seconds.
Returns:: The resulting integration time in seconds (pulled from get_exposure()).
Return type:: float

set_woi(woi=None)[source]#

Abstract method to narrow the imaging region to a ‘window of interest’ for faster framerates.

Parameters:: woi (list, None) – See woi. If None, defaults to largest possible.
Returns:: woi – woi.
Return type:: list

test()[source]#: Tests the core hardware methods of Camera. Validates that the camera is connected correctly and all hardware features are supported.