slmsuite.holography.algorithms.MultiplaneHologram#

class MultiplaneHologram(holograms, weights=None)[source]#

Bases: MultiplaneHologram

Holography combining multiple objectives, potentially across planes of focus or color. Other Hologram subclasses are restricted to either optimizing a hologram within a fixed basis of spots or within the grid of a discrete Fourier transform at a fixed plane of focus. This MultiplaneHologram acts as a metaclass to optimize many individual holograms simultaneously—over many planes or pointsets—producing a composite phase pattern.

Note

Though the infrastructure to make this trivial is not yet in place, the idea of a ‘plane’ extends to planes of color. That is, this class MultiplaneHologram could also be used to optimize a multicolor hologram and account for how the farfield of each color scales with wavelength.

Tip

Calls to optimize() which update flags also update the flags of any child hologram.

holograms#

List of sub-holograms to optimize simultaneously.

Type:: list of Hologram

weights#

Weight for each hologram. This allows the user to redistribute power between holograms. Keep in mind that each hologram will normalize itself, so differences in intensity between target patterns cannot be relied upon.

Type:: list of float

Methods

`get_amp`	Collects the current nearfield amplitude regardless of np/cp configuration.
`get_farfield`	Collects the current complex DFT farfield, potentially with transformations.
`get_mempool_limit`	Helper function to get the cupy memory pool size.
`get_multiplane_defocus_blur`	From a stack of target (power) images at `target_depths`, generate a stack of images at `return_depths`, accounting for defocus blur.
`get_padded_shape`	Helper function to calculate the shape of the computational space.
`get_phase`	Collects the current nearfield phase from the GPU with `cupy.ndarray.get()`.
`get_weights`	Returns the current weights.
`load_stats`	Uses `save_h5()` to import the statistics hierarchy from a given h5 file.
`optimize`	Optimizers to solve the "phase problem": approximating the nearfield phase that transforms a known nearfield source amplitude to a desired farfield target amplitude.
`optimize_cg`	Conjugate Gradient (CG) iterative phase retrieval.
`optimize_gs`	GPU-accelerated Gerchberg-Saxton (GS) iterative phase retrieval.
`plot_farfield`	Plots an overview (left) and zoom (right) view of `source`.
`plot_nearfield`	Plots the amplitude (left) and phase (right) of the nearfield (plane of the SLM).
`plot_stats`	Plots the statistics contained in the given dictionary.
`remove_vortices`
`reset`	Resets the hologram to an initial state.
`reset_phase`	Resets the hologram to a provided phase, to a random state, or to a quadratic phase which overlaps with the target pattern.
`reset_weights`	Resets the hologram weights to the `target` defaults.
`save_stats`	Uses `save_h5()` to export the statistics hierarchy to a given h5 file.
`set_mempool_limit`
`set_target`	Change the target to something new.
`set_weights`	Sets the weights to a new value.

__init__(holograms, weights=None)[source]#

Initializes a ‘meta’ hologram consisting of several sub-holograms optimizing at the same time.

Parameters:

holograms (list of Hologram) – List of N sub-holograms to optimize simultaneously.
weights (array_like of float OR None) – List of N floats. If None, defaults to even power.

get_amp()[source]#

Collects the current nearfield amplitude regardless of np/cp configuration.

Returns:: Nearfield amplitude.
Return type:: numpy.ndarray

get_farfield(shape=None, propagation_kernel=None, affine=None, get=True)[source]#

Collects the current complex DFT farfield, potentially with transformations. This includes collecting the data from the GPU with cupy.ndarray.get().

Parameters:

shape ((int, int)) – Shape of the DFT. Useful to change the resolution of the farfield. If None, defaults to shape, and falls back to slm_shape.
propagation_kernel (array_like) – Used to check the result of the hologram at different depths. See propagation_kernel. If None, defaults to propagation_kernel if one is present. Otherwise, no kernel is applied. Zeroing can force no kernel to be applied and yield the raw DFT.
affine (dict) – Affine transformation to apply to farfield data (in the form of a dictionary with keys "M" and "b"). If None, no transformation is applied.
get (bool) – Whether or not to convert the cupy array to a numpy array if cupy is used. This is ignored if numpy is used.

Returns:

Current farfield expected from the current phase.

Return type:

numpy.ndarray

static get_mempool_limit(device=0)[source]#

Helper function to get the cupy memory pool size.

Parameters:: device (int) – Which GPU to set the limit on. Passed to cupy.cuda.Device().
Returns:: Current memory pool limit in bytes
Return type:: int

static get_multiplane_defocus_blur(cameraslm, targets, target_depths, return_depths=None, sharp_focus=True)[source]#

From a stack of target (power) images at target_depths, generate a stack of images at return_depths, accounting for defocus blur. Power is summed as if all depths were transparent; i.e. objects do no block objects further behind. This is a partial farfield implementation of realistic defocus blur.

Warning

This feature seems to lead to less stable holography without, perhaps, some additional optimizations.

Parameters:

cameraslm (FourierSLM) –
Hardware to implement blur for. Calibrations are necessary to determine how much to blur.

Tip

Right now, the blurring is Gaussian and analytic, but in the future, the measurement of the point spread function should be used.
targets (array_like) – Stack of images of shape (image_count, h, w).
target_depths (list of float) – Depths in "kxy" focal power units corresponding to the targets.
return_depths (list of float) – Depths to return images at. If None, use target_depths.
sharp_focus (bool) – If False, depths at focal planes are blurred by the point spread radius of a focussed spot. If True, all the blurring is reduced by the focused point spread radius, keeping images that are in focus sharp.

static get_padded_shape(slm_shape, padding_order=1, square_padding=True, precision=inf, precision_basis='kxy')[source]#

Helper function to calculate the shape of the computational space. For a given base slm_shape, pads to the user’s requirements. If the user chooses multiple requirements, the largest dimensions for the shape are selected. By default, pads to the smallest square power of two that encapsulates the original slm_shape.

See also

Hologram

Note

Under development: a parameter to pad based on available memory (see _calculate_memory_constrained_shape()).

Parameters:

slm_shape ((int, int) OR slmsuite.hardware.FourierSLM) – The original shape of the SLM in numpy (h, w) form. The user can pass a FourierSLM or SLM instead, and should pass this when using the precision parameter.
padding_order (int) – Scales to the padding_order th larger power of 2. A padding_order of zero does nothing. For instance, an SLM with shape (720, 1280) would yield (720, 1280) for padding_order=0, (1024, 2048) for padding_order=1, and (2048, 4096) for padding_order=2.
square_padding (bool) – If True, sets the smaller shape dimension to that of the larger, yielding a square.
precision (float) – Returns the shape that produces a computational k-space with resolution smaller than precision. The default, infinity, requests a padded shape larger than zero, so padding_order will dominate.
precision_basis (str) – Basis for the precision. Can be "ij" (camera) or "kxy" (normalized blaze).

Returns:

Shape of the computational space which satisfies the above requirements.

Return type:

(int, int)

get_phase(include_propagation=False)[source]#

Collects the current nearfield phase from the GPU with cupy.ndarray.get(). Also shifts the \([-\pi, \pi]\) range of numpy.arctan2() to \([0, 2\pi]\) for faster writing to the SLM (see set_phase()).

Parameters:: include_propagation (bool) – Whether to include the propagation_kernel, if available.
Returns:: Current nearfield phase of the optimization.
Return type:: numpy.ndarray

get_weights()[source]#: Returns the current weights. Collects the current weights from the GPU if applicable.

load_stats(file_path, include_state=True)[source]#

Uses save_h5() to import the statistics hierarchy from a given h5 file.

Tip

Enabling the "raw_stats" flag will export feedback data from each iteration instead of only derived statistics. Consider enabling this to save more detailed information upon export.

Parameters:

file_path (str) – Full path to the file to read the data from.
include_state (bool) – If True, also overwrite all other attributes of Hologram except for dtype and amp_ff.

optimize(method='GS', maxiter=20, verbose=True, callback=None, feedback=None, stat_groups=[], **kwargs)[source]#

Optimizers to solve the “phase problem”: approximating the nearfield phase that transforms a known nearfield source amplitude to a desired farfield target amplitude. Supported optimization methods include:

Gerchberg-Saxton (GS) phase retrieval.
- 'GS'
  
  An iterative algorithm for phase retrieval, accomplished by moving back and forth between the imaging and Fourier domains, with amplitude corrections applied to each. This is usually implemented using fast discrete Fourier transforms, potentially GPU-accelerated.
Weighted Gerchberg-Saxton (WGS) phase retrieval algorithms of various flavors. Improves the uniformity of GS-computed focus arrays using weighting methods and techniques from literature. The method keywords are:
- 'WGS-Leonardo'
  
  The original WGS algorithm. Weights the target amplitudes by the ratio of mean amplitude to computed amplitude, which amplifies weak spots while attenuating strong spots. Uses the following weighting function:
  
  \[\mathcal{W} = \mathcal{W}\left(\frac{\mathcal{T}}{\mathcal{F}}\right)^p\]
  
  where \(\mathcal{W}\), \(\mathcal{T}\), and \(\mathcal{F}\) are the weight amplitudes, target (goal) amplitudes, and feedback (measured) amplitudes, and \(p\) is the power passed as "feedback_exponent" in flags (see kwargs). The power \(p\) defaults to .8 if not passed. In general, smaller \(p\) will lead to slower yet more stable optimization.
- 'WGS-Kim'
  
  Improves the convergence of WGS-Leonardo by fixing the farfield phase strictly after a desired number of net iterations specified by "fix_phase_iteration" or after exceeding a desired efficiency (fraction of farfield energy at the desired points) specified by "fix_phase_efficiency"
- 'WGS-Nogrette'
  
  Weights target intensities by a tunable gain factor.
  
  \[\mathcal{W} = \mathcal{W}/\left(1 - f\left(1 - \mathcal{F}/\mathcal{T}\right)\right)\]
  
  where \(f\) is the gain factor passed as "feedback_factor" in flags (see kwargs). The factor \(f\) defaults to .1 if not passed.
  
  Note that while Nogrette et al compares powers, this implementation compares amplitudes for speed. These are identical to first order.
- 'WGS-Wu'
  
  Weights using an exponential function, which is less sensitive to near-zero values of \(\mathcal{F}\) or \(\mathcal{T}\).
  
  \[\mathcal{W} = \mathcal{W}\exp\left( p (\mathcal{T} - \mathcal{F}) \right)\]
  
  The speed of correction is controlled by \(p\), the power passed as "feedback_exponent".
- 'WGS-tanh'
  
  Weights by hyperbolic tangent, commonly used as an activation function in machine learning.
  
  \[\mathcal{W} = \mathcal{W}\left[1 + f\text{tanh}\left( p (\mathcal{T} - \mathcal{F}) \right) \right]\]
  
  This weighting limits each update to a relative change of \(\pm f\), passed as "feedback_factor", which is useful to prevent large changes. The speed of correction is controlled by \(p\), the power passed as "feedback_exponent".
Conjugate Gradient (CG) phase retrieval.
- 'CG'
  
  (This feature is experimental.)
  
  Some holography—especially that with more complicated holographic objectives—can be better treated with gradient-based methods. In these cases, the phase is guided to an optimized state by following the back-propagated gradients (with respect to phase) of given objective loss which is passed as one of the flags to optimize(). Weighting different components of the objective leads to tradeoffs between those components: for instance a tradeoff between power guided into a given pattern and the uniformity of the realized pattern. slmsuite uses pytorch as a backend for gradient computation. Notably, memory is still owned and initialized by cupy, but gradients can be calculated by using pytorch-cupy interoperability.
  
  The objective loss is expected to be a torch.nn.Module and defaults to a complex variant of torch.nn.MSELoss(). loss is called in the style of pytorch, using (as arguments) the computed farfield (with gradient tree intact) and the target values for the farfield. Internally, this looks like:
```
result = loss(      # The user provides this nn.Module to .optimize()
    farfield,       # The farfield (with gradients), calculated from `phase` by slmsuite
    target          # The target, initialized by the user and processed by slmsuite
)
result.backward()   # Gradients are back-propagated to the input `phase`.
```
  For FeedbackHologram and subclasses, the gradients are computed computationally, but the computational values are then replaced with the experimental results. This allows optimization of the experimental results using the computational gradients (correct to first order) as a guide. Currently, feedback is not supported for spot arrays with "experimental_spot" or "computational_spot" feedback (WGS probably works better for such spot array objectives anyway).
  
  Creating a custom objective is as simple as making a custom torch.nn.Module.forward() method. These methods can be as simple as a single expression or as complicated as a full neural network operating on the input parameters. However, remember to use pytorch methods because the arguments are of type torch.Tensor. Here’s an example of a custom torch.nn.Module.forward() which implements the Huber loss:
```
# Define the loss as a class.
class HuberLoss(nn.Module):
    def __init__(self, delta=1.0):
        super(HuberLoss, self).__init__()
        self.delta = delta

    def forward(self, farfield, target):
        residual = torch.abs(farfield - target)
        quadratic = torch.clamp(residual, max=self.delta)
        linear = residual - quadratic
        loss = 0.5 * quadratic ** 2 + self.delta * linear

        return torch.mean(loss)

# Initialize the class. Remember that we can pass arguments (delta) here.
loss = HuberLoss(delta=2.0)

# Pass the loss to the hologram by one of two methods:
hologram.optimize(..., loss=loss)       # 1. Pass as **kwarg.
hologram.flags["loss"] = loss           # 2. Set directly.
```
  MRAF (next section), if desired, needs to be handled by the loss function. MRAF information is encoded in the target, with the noise region being nan.
The option for Mixed Region Amplitude Freedom (MRAF) feedback. In standard iterative algorithms, the entire Fourier-domain unpatterned field is replaced with zeros. This is disadvantageous because a desired farfield pattern might not be especially compatible with a given nearfield amplitude, or otherwise. MRAF enables “noise regions” where some fraction of the given farfield is not replaced with zeros and instead is allowed to vary. In practice, MRAF is enabled by setting parts of the target to nan; these regions act as the noise regions. The "mraf_factor" flag in flags allows the user to partially attenuate the noise regions. A factor of 0 fully attenuates the noise region (normal WGS behavior). A factor of 1 does not attenuate the noise region at all (the default). Middle ground is recommended, but is application-dependent as a tradeoff between improving pattern fidelity and maintaining pattern efficiency.

As examples, consider two cases where MRAF can be useful:
- Sloping a top hat. Suppose we want very flat amplitude on a beam. Requesting a sharp edge to this beam can lead to fringing effects at the boundary which mitigate flatness both inside and outside the beam. If instead a noise region is defined in a band surrounding the beam, the noise region will be filled with whatever slope best enables the desired flat beam.
- Mitigating diffractive orders. Without MRAF, spot patterns with high crystallinity often have “ghost” diffractive orders which continue the pattern past the edges of requested spots. Even though these orders are attenuated during each phase retrieval iteration, they remain part of the best solution for the recovered phase. With MRAF, a noise region can help solve for retrieved phase which does not generate these undesired orders.

Caution

Requesting stat_groups will slow the speed of optimization due to the overhead of processing and saving statistics, especially in the case of GPU-accelerated optimization where significant time cost is incurred by moving these statistics to the CPU. This is especially apparent in the case of fully-computational holography, where this effect can slow what is otherwise a fully-GPU-contained loop by an order magnitude.

Tip

This function uses a parameter naming convention borrowed from scipy.optimize.minimize() and other functions in scipy.optimize. The parameters method, maxiter, and callback have the same functionality as the equivalently-named parameters in scipy.optimize.minimize().

Parameters:

method (str) – Optimization method to use. See the list of optimization methods above.
maxiter (int) – Number of iterations to optimize before terminating.
verbose (bool OR int) – Whether to display tqdm progress bars. These bars are also not displayed for maxiter <= 1. If verbose is greater than 1, then flags are printed as a preamble.
callback (callable OR None) – Same functionality as the equivalently-named parameter in scipy.optimize.minimize(). callback must accept a Hologram or Hologram subclass as the single argument. If callback returns True, then the optimization exits. Ignored if None.
feedback (str OR None) –
Type of feedback to use during optimization, for instance when weighting in "WGS". For direct instances of Hologram, this can only be "computational" feedback. Subclasses support more types of feedback. Supported feedback options include the following:
- "computational" Uses the the projected farfield pattern (transform of the complex nearfield) as feedback.
- "experimental" Uses a camera contained in a passed cameraslm as feedback. Specific to subclasses of FeedbackHologram.
- "computational_spot" Takes the computational result (the projected farfield pattern) and integrates regions around the expected positions of spots in an optical focus array. More stable than "computational" for spots. Specific to subclasses of SpotHologram.
- "experimental_spot" Takes the experimental result (the image from a camera) and integrates regions around the expected positions of spots in an optical focus array. More stable than "experimental" for spots. Specific to subclasses of SpotHologram.
- "external_spot" Uses some external user-provided metric for spot feedback. See external_spot_amp. Specific to subclasses of SpotHologram.
stat_groups (list of str OR None) – Strings representing types of feedback (data gathering) upon which statistics should be derived. These strings correspond to valid types of feedback (see above). For instance, if "experimental" is passed as a stat group, statistics on the pixels in the experimental feedback image will automatically be computed and stored for each iteration of optimization. However, this comes with overhead (see above warning).
**kwargs (dict, optional) – Various weight keywords and values to pass depending on the weight method. These are passed into flags. See options documented in the constructor.

optimize_cg(iterations, callback)[source]#

Conjugate Gradient (CG) iterative phase retrieval.

(This feature is experimental.)

Solves the “phase problem”: approximates the nearfield phase that transforms a known nearfield source amplitude to a desired farfield target amplitude.

Caution

This function should be called through optimize() and not called directly. It is left as a public function exposed in documentation to clarify how the internals of optimize() work.

Parameters:

iterations (iterable) – Number of loop iterations to run. Is an iterable to pass a tqdm iterable.
callback (callable OR None) – See optimize().

optimize_gs(iterations, callback)[source]#

GPU-accelerated Gerchberg-Saxton (GS) iterative phase retrieval.

Solves the “phase problem”: approximates the nearfield phase that transforms a known nearfield source amplitude to a desired farfield target amplitude.

Caution

This function should be called through optimize() and not called directly. It is left as a public function exposed in documentation to clarify how the internals of optimize() work.

Note

Default FFTs are not in-place in this algorithm. In both non-cupy and cupy implementations, numpy.fft does not support in-place operations. However, scipy.fft does in both. In the future, we may move to the scipy implementation. However, neither numpy or scipy fftshift support in-place movement (for obvious reasons). For even faster computation, algorithms should consider not shifting the FFT result, and instead shifting measurement data / etc to this unshifted basis. We might also implement get_fft_plan for even faster FFTing. However, in practice, speed is limited by other peripherals (especially feedback and stats) rather than FFT speed or memory.

Parameters:

iterations (iterable) – Number of loop iterations to run. Is an iterable to pass a tqdm iterable.
callback (callable OR None) – See optimize().

plot_farfield(*args, **kwargs)[source]#

Plots an overview (left) and zoom (right) view of source.

Parameters:

source (array_like OR None) – Should have shape equal to shape. If None, defaults to amp_ff.
title (str) – Title of the plots. If "phase" is a substring of title, then the source is treated as a phase.
limits (((float, float), (float, float)) OR None) – \(x\) and \(y\) limits for the zoom plot in "knm" space. If None, limits are autocomputed as the smallest bounds that show all non-zero values (plus limit_padding). Note that autocomputing on target will perform well, as zero values are set to actually be zero. However, doing so on computational or experimental outputs (e.g. amp_ff) will likely perform poorly, as values in the field deviate slightly from zero and artificially expand the limits.
units (str) – Far-field units for plots (see convert_vector() for options). If units requiring a SLM are desired, the attribute cameraslm must be filled.
limit_padding (float) – Fraction of the width and height to expand the limits of the zoom plot by, only if the passed limits is None (autocompute).
figsize (tuple) – Size of the plot.
cbar (bool) – Whether to add colorbars to the plots. Defaults to False.
axs (matplotlib.axes.Axes OR None) – If provided, uses these axes instead of creating new ones.

Returns:

Used limits, which may be autocomputed. Autocomputed limits are returned as integers.

Return type:

((float, float), (float, float))

plot_nearfield(source=None, title='', padded=False, figsize=(8, 4), cbar=False)[source]#

Plots the amplitude (left) and phase (right) of the nearfield (plane of the SLM). The amplitude is assumed (whether uniform, assumed, or measured) while the phase is the result of optimization.

Parameters:

title (str) – Title of the plots.
padded (bool) – If True, shows the full computational nearfield of shape shape. Otherwise, shows the region at the center of the computational space of size slm_shape corresponding to the unpadded SLM.
figsize (tuple) – Size of the plot.
cbar (bool) – Whether to add colorbars to the plots. Defaults to False.

plot_stats(*args, **kwargs)[source]#

Plots the statistics contained in the given dictionary.

Parameters:

stats_dict (dict OR None) – Stats to plot in dictionary form. If None, defaults to stats.
stat_groups (list of str OR None) – Which statistics groups to plot. If empty or None is provided, defaults to all groups present in stats.
ylim ((int, int) OR None) – Allows the user to pass in desired y limits. If None, the default y limits are used.
show (bool) – Whether or not to immediately show the plot. Defaults to false.

reset(reset_phase=True, reset_flags=False)[source]#

Resets the hologram to an initial state. Does not restore the preconditioned phase that may have been passed to the constructor (as this information is lost upon optimization). Instead, phase is randomized if reset_phase=True. Also uses the current target rather than the target that may have been passed to the constructor (e.g. includes current refine_offset() changes, etc).

Parameters:

reset_phase (bool) – Whether to additionally call reset_phase().
reset_flags (bool:) – Whether to erase the information (including passed kwargs) stored in flags.

reset_phase(custom_phase=None, random_phase=None, quadratic_phase=None)[source]#

Resets the hologram to a provided phase, to a random state, or to a quadratic phase which overlaps with the target pattern.

Parameters:

custom_phase (array_like OR None) – Custom nearfield initial phase. If not None, then all other parameters are ignored. See phase. phase should only be passed if the user wants to precondition the optimization. Of shape slm_shape.
random_phase (float OR None) – Sets the phase to uniformly random phase, scaled to \(2\pi\). Setting random_phase to a fraction of 1 likewise scales the randomness. If None, looks for "random_phase" in flags. This adds with the quadratic_phase parameter.
quadratic_phase (bool OR float OR None) – We can also precondition the phase analytically (with a lens and blaze) to roughly the size of the target hologram, according to the first and second order image_moments(). This quadratic preconditioning is thought to help reduce the formation of optical vortices or speckle compared to random initialization, as the analytic distribution is smooth in phase. If None, looks for "quadratic_phase" in flags. If a float is provided, the size of the beam in the farfield is scaled accordingly. This feature is ignored if phase is not None.

reset_weights()[source]#: Resets the hologram weights to the target defaults.

save_stats(file_path, include_state=True)[source]#

Uses save_h5() to export the statistics hierarchy to a given h5 file.

Parameters:

file_path (str) – Full path to the file to read the data from.
include_state (bool) – If True, also includes all other attributes of Hologram except for dtype (cannot pickle) and amp_ff (can regenerate). These attributes are converted to numpy if necessary. Note that the intent is not to produce a runnable Hologram by default (as this would require pickling hardware interfaces), but rather to provide extra information for debugging.

static set_mempool_limit(device=0, size=None, fraction=None)[source]#

Helper function to set the cupy memory pool size.

Parameters:

device (int) – Which GPU to set the limit on. Passed to cupy.cuda.Device().
size (int) – Desired number of bytes in the pool. Passed to cupy.cuda.MemoryPool.set_limit().
fraction (float) – Fraction of available memory to use. Passed to cupy.cuda.MemoryPool.set_limit().

set_target(*args, **kwargs)[source]#

Change the target to something new. This method handles cleaning and normalization.

Parameters:

new_target (array_like OR None) – New target to optimize towards. Should be of shape shape. If None, target is zeroed (used internally, but probably should not be used by a user).
reset_weights (bool) – Whether to update the weights to this new target.

set_weights(new_weights)[source]#: Sets the weights to a new value. Handles moving data onto the GPU if applicable.