slmsuite.holography.algorithms.FeedbackHologram

class FeedbackHologram(shape, target_ij=None, cameraslm=None, **kwargs)

Bases: Hologram

Experimental holography aided by camera feedback. Contains mechanisms for hologram positioning and camera feedback aided by a FourierSLM.

cameraslm

A hologram with experimental feedback needs access to an SLM and camera. If None, no feedback is applied (mostly defaults to Hologram).

Type

slmsuite.hardware.cameraslms.FourierSLM OR None

cam_shape

Shape of the camera in the meaning of numpy.shape().

Type

(int, int)

cam_points

Array containing points corresponding to the corners of the camera in the SLM’s k-space. First point is repeated at the end for easy plotting.

Type

numpy.ndarray

target_ij

Amplitude target in the "ij" (camera) basis. Of same shape as the camera in cameraslm. Counterpart to target which is in the "knm" (computational k-space) basis.

Type

array_like OR None

img_ij, img_knm

Cached amplitude feedback image in the "ij" (raw camera) basis or "knm" (transformed to computational k-space) basis. Measured with measure().

Methods

GS	GPU-accelerated Gerchberg-Saxton (GS) iterative phase retrieval.
calculate_padded_shape	Helper function to calculate the shape of the computational space.
export_stats	Uses write_h5() to export the statistics hierarchy to a given h5 file.
extract_farfield	Collects the current complex farfield from the GPU with cupy.ndarray.get().
extract_phase	Collects the current nearfield phase from the GPU with cupy.ndarray.get().
get_mempool_limit	Helper function to get the cupy memory pool size.
ijcam_to_knmslm	Convert an image in the camera domain to computational SLM k-space using, in part, the affine transformation stored in a cameraslm's Fourier calibration.
import_stats	Uses write_h5() to import the statistics hierarchy from a given h5 file.
measure	Method to request a measurement to occur.
optimize	Optimizers to solve the "phase problem": approximating the near-field phase that transforms a known near-field source amplitude to a desired far-field target amplitude.
plot_farfield	Plots an overview (left) and zoom (right) view of source.
plot_nearfield	Plots the amplitude (left) and phase (right) of the nearfield (plane of the SLM).
plot_stats	Plots the statistics contained in the given dictionary.
refine_offset	(NotImplemented) Hones the position of the produced image to the desired target image to compensate for Fourier calibration imperfections.
reset	Resets the hologram to an initial state.
reset_phase	Resets the hologram to a random state or to a provided phase.
reset_weights	Resets the hologram weights to the target defaults.
set_mempool_limit	Helper function to set the cupy memory pool size.
update_stats	Calculate statistics corresponding to the desired stat_groups.
update_target	Change the target to something new.

__init__(shape, target_ij=None, cameraslm=None, **kwargs)

Initializes a hologram with camera feedback.

Parameters

• shape ((int, int)) – Computational shape of the SLM in numpy (h, w) form. See Hologram.__init__().

• target_ij (array_like OR None) – See target_ij. Should only be None if the target will be generated by other means (see SpotHologram), so the user should generally provide an array.

• cameraslm (slmsuite.hardware.cameraslms.FourierSLM OR slmsuite.hardware.slms.SLM OR None) – Provides access to experimental feedback. If an slmsuite.hardware.slms.SLM is passed, this is set to None, but the information contained in the SLM is passed to the superclass Hologram. See cameraslm.

• kwargs – See Hologram.__init__().

ijcam_to_knmslm(img, out=None, blur_ij=None, order=3)

Convert an image in the camera domain to computational SLM k-space using, in part, the affine transformation stored in a cameraslm’s Fourier calibration.

Note

This includes two transformations:

• The affine transformation "ij" -> "kxy" (camera pixels to normalized k-space).

• The scaling "kxy" -> "knm" (normalized k-space to computational k-space pixels).

Parameters

• img (numpy.ndarray OR cupy.ndarray) – Image to transform. This should be the same shape as images returned by the camera.

• out (numpy.ndarray OR cupy.ndarray OR None) – If out is not None, this array will be used to write the memory in-place.

• blur_ij (int OR None) – Applies a blur_ij pixel-width Gaussian blur to img. If None, defaults to the "blur_ij" flag if present; otherwise zero.

• order (int) – Order of interpolation used for transformation. Defaults to 3 (cubic).

Returns

Image transformed into "knm" space.

Return type

numpy.ndarray OR cupy.ndarray

update_target(new_target, reset_weights=False, plot=False)

Change the target to something new. This method handles cleaning and normalization.

Parameters

• new_target (array_like OR None) – New target to optimize towards. Should be of shape shape. If None, target is zeroed (used internally, but probably should not be used by a user).

• reset_weights (bool) – Whether to update the weights to this new target.

• plot (bool) – Calls plot_farfield() on target.

measure(basis='ij')

Method to request a measurement to occur. If img_ij is None, then a new image will be grabbed from the camera (this is done automatically in algorithms).

Parameters

basis (str) –

The cached image to be sure to fill with new data. Can be "ij" or "knm".

• If "knm", then img_ij and img_knm are filled.

• If "ij", then img_ij is filled, and img_knm is ignored.

This is useful to avoid (expensive) transformation from the "ij" to the "knm" basis if img_knm is not needed.

refine_offset(img, basis='kxy')

(NotImplemented) Hones the position of the produced image to the desired target image to compensate for Fourier calibration imperfections. Works either by moving the desired camera target to align where the image ended up (basis="ij") or by moving the \(k\)-space image to target the desired camera target (basis="knm"/basis="kxy"). This should be run at the user’s request inbetween optimize() iterations.

Parameters

• img (numpy.ndarray) – Image measured by the camera.

• basis (str) – The correction can be in any of the following bases: - "ij" changes the pixel that the spot is expected at, - "kxy" or "knm" changes the k-vector which the SLM targets. Defaults to "kxy" if None.

Returns

Euclidean pixel error in the "ij" basis for each spot.

Return type

numpy.ndarray

update_stats(stat_groups=[])

Calculate statistics corresponding to the desired stat_groups.

Parameters

stat_groups (list of str) – Which groups or types of statistics to analyze.

GS(iterations, callback)

GPU-accelerated Gerchberg-Saxton (GS) iterative phase retrieval.

Solves the “phase problem”: approximates the near-field phase that transforms a known near-field source amplitude to a desired far-field target amplitude.

Caution

This function should be called through optimize() and not called directly. It is left as a public function exposed in documentation to clarify how the internals of optimize() work.

Note

FFTs are not in-place in this algorithm. In both non-cupy and cupy implementations, numpy.fft does not support in-place operations. However, scipy.fft does in both. In the future, we may move to the scipy implementation. However, neither numpy or scipy fftshift support in-place movement (for obvious reasons). For even faster computation, algorithms should consider not shifting the FFT result, and instead shifting measurement data / etc to this unshifted basis. We might also implement get_fft_plan for even faster FFTing.

Parameters

• iterations (iterable) – Number of loop iterations to run. Is an iterable to pass a tqdm iterable.

• callback (callable OR None) – See optimize().

static calculate_padded_shape(slm_shape, padding_order=1, square_padding=True, precision=inf, precision_basis='kxy')

Helper function to calculate the shape of the computational space. For a given base slm_shape, pads to the user’s requirements. If the user chooses multiple requirements, the largest dimensions for the shape are selected. By default, pads to the smallest square power of two that encapsulates the original slm_shape.

Tip

See also the first tip in the constructor of Hologram for more information about the importance of padding.

Note

Under development: a parameter to pad based on available memory (see _calculate_memory_constrained_shape()).

Parameters

• slm_shape ((int, int) OR slmsuite.hardware.FourierSLM) – The original shape of the SLM in numpy (h, w) form. The user can pass a slmsuite.hardware.FourierSLM instead, and should pass this when using the precision parameter.

• padding_order (int) – Scales to the padding_order th larger power of 2. A padding_order of zero does nothing. For instance, an SLM with shape (720, 1280) would yield (720, 1280) for padding_order=0, (1024, 2048) for padding_order=1, and (2048, 4096) for padding_order=2.

• square_padding (bool) – If True, sets the smaller shape dimension to that of the larger, yielding a square.

• precision (float) – Returns the shape that produces a computational k-space with resolution smaller than precision. The default, infinity, requests a padded shape larger than zero, so padding_order will dominate.

• precision_basis (str) – Basis for the precision. Can be "ij" (camera) or "kxy" (normalized blaze).

Returns

Shape of the computational space which satisfies the above requirements.

Return type

(int, int)

export_stats(file_path, include_state=True)

Uses write_h5() to export the statistics hierarchy to a given h5 file.

Parameters

• file_path (str) – Full path to the file to read the data from.

• include_state (bool) – If True, also includes all other attributes of Hologram except for dtype (cannot pickle) and amp_ff (can regenerate). These attributes are converted to numpy if necessary. Note that the intent is not to produce a runnable Hologram by default (as this would require pickling hardware interfaces), but rather to provide extra information for debugging.

extract_farfield()

Collects the current complex farfield from the GPU with cupy.ndarray.get().

Returns

Current farfield computed by GS.

Return type

numpy.ndarray

extract_phase()

Collects the current nearfield phase from the GPU with cupy.ndarray.get(). Also shifts the \([-\pi, \pi]\) range of numpy.arctan2() to \([0, 2\pi]\) for faster writing to the SLM (see write()).

Returns

Current nearfield phase computed by GS.

Return type

numpy.ndarray

static get_mempool_limit(device=0)

Helper function to get the cupy memory pool size.

Parameters

device (int) – Which GPU to set the limit on. Passed to cupy.cuda.Device().

Returns

Current memory pool limit in bytes

Return type

int

import_stats(file_path, include_state=True)

Uses write_h5() to import the statistics hierarchy from a given h5 file.

Tip

Enabling the "raw_stats" flag will export feedback data from each iteration instead of only derived statistics. Consider enabling this to save more detailed information upon export.

Parameters

• file_path (str) – Full path to the file to read the data from.

• include_state (bool) – If True, also overwrite all other attributes of Hologram except for dtype and amp_ff.

optimize(method='GS', maxiter=20, verbose=True, callback=None, feedback=None, stat_groups=[], **kwargs)

Optimizers to solve the “phase problem”: approximating the near-field phase that transforms a known near-field source amplitude to a desired far-field target amplitude. Supported optimization methods include:

• Gerchberg-Saxton (GS) phase retrieval.

  'GS'

  An iterative algorithm for phase retrieval, accomplished by moving back and forth between the imaging and Fourier domains, with amplitude corrections applied to each. This is implemented using fast Fourier transforms, potentially GPU-accelerated.

• Weighted Gerchberg-Saxton (WGS) phase retrieval algorithms of various flavors. Improves the uniformity of GS-computed focus arrays using weighting methods and techniques from literature. The method keywords are:

  'WGS-Leonardo'

  The original WGS algorithm. Weights the target amplitudes by the ratio of mean amplitude to computed amplitude, which amplifies weak spots while attenuating strong spots. Uses the following weighting function:

  \[\mathcal{W} = \mathcal{W}\left(\frac{\mathcal{T}}{\mathcal{F}}\right)^p\]

  where \(\mathcal{W}\), \(\mathcal{T}\), and \(\mathcal{F}\) are the weight amplitudes, target (goal) amplitudes, and feedback (measured) amplitudes, and \(p\) is the power passed as "feedback_exponent" in flags (see kwargs). The power \(p\) defaults to .9 if not passed. In general, smaller \(p\) will lead to slower yet more stable optimization.

  'WGS-Kim'

  Improves the convergence of WGS-Leonardo by fixing the far-field phase strictly after a desired number of net iterations specified by "fix_phase_iteration" or after exceeding a desired efficiency (fraction of far-field energy at the desired points) specified by "fix_phase_efficiency"

  'WGS-Nogrette'

  Weights target intensities by a tunable gain factor.

  \[\mathcal{W} = \mathcal{W}/\left(1 - f\left(1 - \mathcal{F}/\mathcal{T}\right)\right)\]

  where \(f\) is the gain factor passed as "feedback_factor" in flags (see kwargs). The factor \(f\) defaults to .1 if not passed.

  Note that while Nogrette et al compares powers, this implementation compares amplitudes for speed. These are identical to first order.

• The option for Mixed Region Amplitude Freedom (MRAF) feedback. In standard iterative algorithms, the entire Fourier-domain unpatterned field is replaced with zeros. This is disadvantageous because a desired farfield pattern might not be especially compatible with a given nearfield amplitude, or otherwise. MRAF enables “noise regions” where some fraction of the given farfield is not replaced with zeros and instead is allowed to vary. In practice, MRAF is enabled by setting parts of the target to nan; these regions act as the noise regions. The "mraf_factor" flag in flags allows the user to partially attenuate the noise regions. A factor of 0 fully attenuates the noise region (normal WGS behavior). A factor of 1 does not attenuate the noise region at all (the default). Middle ground is recommended, but is application-dependent as a tradeoff between improving pattern fidelity and maintaining pattern efficiency.

  As examples, consider two cases where MRAF can be useful:

  • Sloping a top hat. Suppose we want very flat amplitude on a beam. Requesting a sharp edge to this beam can lead to fringing effects at the boundary which mitigate flatness both inside and outside the beam. If instead a noise region is defined in a band surrounding the beam, the noise region will be filled with whatever slope best enables the desired flat beam.

  • Mitigating diffractive orders. Without MRAF, spot patterns with high crystallinity often have “ghost” diffractive orders which continue the pattern past the edges of requested spots. Even though these orders are attenuated during each phase retrieval iteration, they remain part of the best solution for the recovered phase. With MRAF, a noise region can help solve for retrieved phase which does not generate these undesired orders.

Caution

Requesting stat_groups will slow the speed of optimization due to the overhead of processing and saving statistics, especially in the case of GPU-accelerated optimization where significant time cost is incurred by moving these statistics to the CPU. This is especially apparent in the case of fully-computational holography, where this effect can slow what is otherwise a fully-GPU-contained loop by an order magnitude.

Tip

This function uses a parameter naming convention borrowed from scipy.optimize.minimize() and other functions in scipy.optimize. The parameters method, maxiter, and callback have the same functionality as the equivalently-named parameters in scipy.optimize.minimize().

Parameters

• method (str) – Optimization method to use. See the list of optimization methods above.

• maxiter (int) – Number of iterations to optimize before terminating.

• verbose (bool OR int) – Whether to display tqdm progress bars. These bars are also not displayed for maxiter <= 1. If verbose is greater than 1, then flags are printed as a preamble.

• callback (callable OR None) – Same functionality as the equivalently-named parameter in scipy.optimize.minimize(). callback must accept a Hologram or Hologram subclass as the single argument. If callback returns True, then the optimization exits. Ignored if None.

• feedback (str OR None) –

  Type of feedback to use during optimization, for instance when weighting in "WGS". For direct instances of Hologram, this can only be "computational" feedback. Subclasses support more types of feedback. Supported feedback options include the following:

  • "computational" Uses the the projected farfield pattern (transform of the complex nearfield) as feedback.

  • "experimental" Uses a camera contained in a passed cameraslm as feedback. Specific to subclasses of FeedbackHologram.

  • "computational_spot" Takes the computational result (the projected farfield pattern) and integrates regions around the expected positions of spots in an optical focus array. More stable than "computational" for spots. Specific to subclasses of SpotHologram.

  • "experimental_spot" Takes the experimental result (the image from a camera) and integrates regions around the expected positions of spots in an optical focus array. More stable than "experimental" for spots. Specific to subclasses of SpotHologram.

  • "external_spot" Uses some external user-provided metric for spot feedback. See external_spot_amp. Specific to subclasses of SpotHologram.

• stat_groups (list of str OR None) – Strings representing types of feedback (data gathering) upon which statistics should be derived. These strings correspond to valid types of feedback (see above). For instance, if "experimental" is passed as a stat group, statistics on the pixels in the experimental feedback image will automatically be computed and stored for each iteration of optimization. However, this comes with overhead (see above warning).

• **kwargs (dict, optional) – Various weight keywords and values to pass depending on the weight method. These are passed into flags. See options documented in the constructor.

plot_farfield(source=None, title='', limits=None, units='knm', limit_padding=0.1, figsize=(8, 4), cbar=False)

Plots an overview (left) and zoom (right) view of source.

Parameters

• source (array_like OR None) – Should have shape equal to shape. If None, defaults to amp_ff.

• title (str) – Title of the plots. If "phase" is a substring of title, then the source is treated as a phase.

• limits (((float, float), (float, float)) OR None) – \(x\) and \(y\) limits for the zoom plot in "knm" space. If None, limits are autocomputed as the smallest bounds that show all non-zero values (plus limit_padding). Note that autocomputing on target will perform well, as zero values are set to actually be zero. However, doing so on computational or experimental outputs (e.g. amp_ff) will likely perform poorly, as values in the field deviate slightly from zero and artificially expand the limits.

• units (str) – Far-field units for plots (see convert_blaze_vector() for options). If units requiring a SLM are desired, the attribute cameraslm must be filled.

• limit_padding (float) – Fraction of the width and height to expand the limits of the zoom plot by, only if the passed limits is None (autocompute).

• figsize (tuple) – Size of the plot.

• cbar (bool) – Whether to add colorbars to the plots. Defaults to False.

Returns

Used limits, which may be autocomputed. Autocomputed limits are returned as integers.

Return type

((float, float), (float, float))

plot_nearfield(title='', padded=False, figsize=(8, 4), cbar=False)

Plots the amplitude (left) and phase (right) of the nearfield (plane of the SLM). The amplitude is assumed (whether uniform, or experimentally computed) while the phase is the result of optimization.

Parameters

• title (str) – Title of the plots.

• padded (bool) – If True, shows the full computational nearfield of shape shape. Otherwise, shows the region at the center of the computational space of size slm_shape corresponding to the unpadded SLM.

• figsize (tuple) – Size of the plot.

• cbar (bool) – Whether to add colorbars to the plots. Defaults to False.

plot_stats(stats_dict=None, stat_groups=[], ylim=None)

Plots the statistics contained in the given dictionary.

Parameters

• stats_dict (dict OR None) – Stats to plot in dictionary form. If None, defaults to stats.

• stat_groups (list of str OR None) – Which

• ylim ((int, int) OR None) – Allows the user to pass in desired y limits. If None, the default y limits are used.

reset(reset_phase=True, reset_flags=False)

Resets the hologram to an initial state. Does not restore the preconditioned phase that may have been passed to the constructor (as this information is lost upon optimization). Also uses the current target rather than the target that may have been passed to the constructor (e.g. includes current refine_offset() changes, etc).

Parameters

• reset_phase (bool) – Whether to additionally call reset_phase().

• reset_flags (bool:) – Whether to erase the information (including passed kwargs) stored in flags.

reset_phase(phase=None)

Resets the hologram to a random state or to a provided phase.

Parameters

phase (array_like OR None) – The near-field initial phase. See phase. phase should only be passed if the user wants to precondition the optimization. Of shape slm_shape.

reset_weights()

Resets the hologram weights to the target defaults.

static set_mempool_limit(device=0, size=None, fraction=None)

Helper function to set the cupy memory pool size.

Parameters

• device (int) – Which GPU to set the limit on. Passed to cupy.cuda.Device().

• size (int) – Desired number of bytes in the pool. Passed to cupy.cuda.MemoryPool.set_limit().

• fraction (float) – Fraction of available memory to use. Passed to cupy.cuda.MemoryPool.set_limit().