ptyrad.params.recon_params

ptyrad.params.recon_params#

Defines available options and validation rules for the recon_params dictionary.

pydantic model ptyrad.params.recon_params.ReconParams[source]#

Bases: BaseModel

The PtyRAD results are organized into folder structures with 2 (reconstruction) or 3 (hypertune) levels not including the ‘output/’. The main (1st) output directory is specified by ‘output_dir’, this is usually separated by material systems or projects, and presumably you’ll have multiple reconstructions for this material system / project. Each PtyRAD reconstruction would be saved into a “reconstruction folder” that will be automatically generated by PtyRAD if ‘SAVE_ITERS’ is not null. For reconstruction mode, the folder structure might look like ‘output/<MATERIALS>/<RECONSTRUCTION>’ Note that ‘recon_dir_affixes’, ‘prefix_time’, ‘prefix’, and ‘postfix’ all operates on the reconstruction folder, and have no effect if ‘SAVE_ITERS’ is null because reconstuction folder would not be generated from the first place The date, pre- and postfix are automatically connected by ‘_’, and the / behind ‘output_dir’ will also be automatically generated. For hypertune mode, a hypertune folder is automatically inserted as the 2nd level between ‘output_dir’ and the (optional) resonstruction folders. For example, ‘output_dir/<MATERIALS>/<HYPERTUNE>/<RECONSTRUCTION>’. This way the hypertune folders are organized under the <MATERIALS> folder just like other reconstruction folders. Note that in hypertune mode, ‘prefix_time’, ‘prefix’, and ‘postfix’ would be applied (i.e., hijacked) on this hypertune folder and have no effect to the reconstruction folder name. In other words, ‘prefix_time’, ‘prefix’, and ‘postfix’ would always be applied to the folders under ‘output_dir’ for both reconstruction and hypertune modes.

Fields:
param NITER: int = 200#

Total number of reconstruction iterations. 1 iteration means a full pass of all selected diffraction patterns. Usually 20-50 iterations can get 90% of the work done with a proper learning rate between 1e-3 to 1e-4. For faster trials in hypertune mode, set ‘NITER’ to a smaller number than your typical reconstruction to save time. Usually 10-20 iterations are enough for the hypertune parameters to show their relative performance.

param INDICES_MODE: IndicesMode [Optional]#

Indices mode determines multiple ways to use diffraction patterns at each probe positions for reconstructions. Each probe position (or each diffraction pattern) has a unique index, by selecting a subset of the indices, we can conveniently change the effective reconstruction area (‘center’), or effective scan step size or real space overlap (‘sub’). You may choose ‘full’ for reconstructing with all probe positions, ‘sub’ for subsampling by selecting only every few probe positions with the full FOV (i.e., this will increase the effective scan step size and is a good way to test whether we can reduce the real space overlap), or ‘center’ for only using the center rectangular region in real space with a effectively reduced FOV but full sized object canvas. ‘subscan_slow’ and ‘subscan_fast’ determine the number of scan positions chosen for ‘sub’ and ‘center’, and have no effect to ‘full’. If ‘subscan_slow’ and ‘subscan_fast’ are not provided (or null), they’ll be set to half of the ‘N_scan_slow’ and half of the ‘N_scan_fast’ by default. Typically we can start from ‘INDICES_MODE’: {‘mode’: ‘sub’, ‘subscan_slow: null, ‘subscan_fast’: null} to get an quick idea of the entire object by reconstructing the entire object FOV but using only every other diffraction pattern along fast and slow scan directions (so only 1/4 diffraction patterns are used, hence 4x speedup in the iteration time). Similarly we can use ‘center’ to keep the effective scan step size, but reconstruct a smaller FOV. Once the ‘sub’ or ‘center’ show reasonable results, we can then switch to ‘full’ to further refine it without starting from scratch because there’s no object dimension mismatch between the ‘INDICES_MODEs’.

param BATCH_SIZE: BatchSize [Optional]#

Batch size is the number of diffraction patterns processed simultaneously to get the gradient update. ‘size’ is the number of diffraction pattern in a sub-batch, and ‘grad_accumulation’ is how many sub-batches’ gradients are accumulated before applying the update. Effective batch size (for 1 update) is batch_size * grad_accumulation. Gradient accumulation is a ML technique that allows people to use large effective batch size by trading the iteration time with memory requirement, so if you can fit the entire batch inside your memory, you should always set ‘grad_accumulation’: 1 for performance. “Batch size” is commonly used in machine learning community, while it’s called “grouping” in PtychoShelves. Batch size has an effect on both convergence speed and final quality, usually smaller batch size leads to better final quality for iterative gradient descent, but smaller batch size would also lead to longer computation time per iteration because the GPU isn’t as utilized as large batch sizes (due to less GPU parallelism). On the other hand, large batch size is known to be more robust (noise-resilient) but converges slower. Generally batch size of 32 to 128 is used, although certain algorithms (like DM) would prefer a large batch size that is equal to the dataset size for robustness. For extremely large object (or with a lot of object modes), you’ll need to reduce batch size to save GPU memory, or use grad_accumulation to split a batch into multiple sub-batches for 1 update.

param GROUP_MODE: Literal['random', 'sparse', 'compact', 'hilbert', 'fps'] = 'random'#

Group mode determines the spatial distribution of the selected probe positions within a batch (group), this is similar to the ‘MLs’ for ‘sparse’ and ‘MLc’ for ‘compact’ in PtychoShelves. Available options are ‘random’, ‘sparse’, ‘compact’, ‘hilbert’, and ‘fps’. Usually ‘random’ is good enough with small batch sizes and is the suggested option for most cases. ‘compact’ is believed to provide best final quality, although it’s converging much slower. ‘sparse’ gives the most uniform coverage on the object so converges the fastest, and is also preferred for reconstructions with few scan positions to prevent any locally biased update. Since generating sparse grouping can be computationally expensive (a few minutes with 256x256 scan), PtyRAD provides 2 methods for sparse grouping, (1) ‘fps’ (Farthest Point Sampling) gives highest quality groups but is slower, or (2) ‘hilbert’ (Hilbert Curve Sorting) produces good quality groups within 0.5 sec and is fastest with O(N). These methods can be explictly chosen, or use ‘sparse’ to automatically select them based on the number of probe positions. ‘fps’ is chosen for datasets with less than 256x256 scan pattern. Note that PtychoShelves automatically switches from ‘sparse’ to ‘random’ for Nscans > 1e3. The grouping in PtyRAD is fixed during optimization, but the order between each group is shuffled for every iteration.

param SAVE_ITERS: int | None = 10#

Number of completed iterations before saving the current reconstruction results (model, probe, object) and summary figures. If ‘SAVE_ITERS’ is 50, it’ll create an output reconstruction folder and save the results and figures into it every 50 iterations. If null, the output reconstruction folder would not be created and no reconstruction results or summary figures would be saved. If ‘SAVE_ITERS’ > ‘NITER’, it’ll create the output reconstruction folder but no results / figs would be saved. Typically we set ‘SAVE_ITERS’ to 50 for reconstruction mode with ‘NITER’ around 200 to 500. For hypertune mode, it’s suggested to set ‘SAVE_ITERS’ to null and set ‘collate_results’ to true to save the disk space, while also provide an convenient way to check the hypertune performance by the collated results.

param output_dir: str = 'output/'#

Path and name of the main output directory. Ideally the ‘output_dir’ keeps a series of reconstruction of the same materials system or project. The PtyRAD results and figs will be saved into a reconstruction-specific folder under ‘output_dir’. The ‘output_dir’ folder will be automatically created if it doesn’t exist.

param recon_dir_affixes: List[Literal['minimal', 'default', 'all', 'indices', 'meas', 'batch', 'pmode', 'omode', 'nlayer', 'lr', 'optimizer', 'scheduler', 'start_iter', 'model', 'constraint', 'loss', 'conv_angle', 'aberrations', 'Ls', 'z_shift', 'dx', 'affine', 'tilt']] = ['default']#

This list specifies the optional affixes to the reconstruction folder name for file management. The order of strings has NO effect to the output folder name. PtyRAD provides high-level presets including ‘minimal’, ‘default’, and ‘all’, while each of them corresponds to a subset of all available options. There are currently 20 available options, including ‘indices’, ‘meas’, ‘batch’, ‘pmode’, ‘omode’, ‘nlayer’, ‘lr’, ‘optimizer’, ‘scheduler’, ‘start_iter’, ‘model’, ‘constraint’, ‘loss’, ‘conv_angle’, ‘aberrations’, ‘Ls’, ‘z_shift’, ‘dx’, ‘affine’, and ‘tilt’. Each option corresponds to specific fields in the params file. These individual tags can be combined with the presets, e.g. [‘minimal’, ‘tilt’]. A typical output folder name of ‘default’ looks like: ‘ptyrad/demo/output/tBL_WSe2/20250607_full_N16384_dp128_flipT100_random32_p6_1obj_6slice_dz2_plr1e-4_oalr5e-4_oplr5e-4_slr1e-4_orblur0.5_ozblur1_mamp0.03_4_oathr0.98_oposc_sng1.0_spr0.1’. Note that certain trivial values might not be shown even it’s specified, e.g. tilt of [0,0] mrad, slice thickness for single-slice ptychography, or start_iter = 1, etc. It’s recommended to use ‘default’ for ‘reconstruction’ mode and adjust if needed. For ‘hypertune’ mode, you can set to [] (empty list) or [‘minimal’] if you’re considering saving intermediate results, because a unique identifier (trial number) would be appended and detail information are fully stored in the sqlite, and the hypertuned params are appended to the collated result anyway if you have ‘collate_results’: true.

param prefix_time: bool | str = 'date'#

Set to true to prepend a date str like ‘20240903_XXX’ in front of the reconstruction folder name, so that reconstruction with the same ‘recon_dir_affixes’ setting won’t get incorrectly saved to the same output folder. Available options are None, True, False, ‘date’, ‘time’, ‘datetime’, and time format string like ‘%Y%m%d_%H%M%S’. Suggested value is ‘date’ for both ‘reconstruction’ and ‘hypertune’ modes. In hypertune mode, the date string would be applied on the hypertune folder instead of the reconstruction folder. Also note that if you’re using hypertune mode on multiple GPUs, you should set prefix_time to ‘date’ or False, and handle any additional identifier using ‘prefix’, otherwise different workers launched at different times would each generate their own output folder with different time strings despite using the same sqlite database file.

param prefix: str = ''#

Prefix this string to the reconstruction folder name. Note that “_” will be automatically generated, and the attached str would be after the time str if ‘prefix_time’ is true. In hypertune mode, the prefix string would be applied on the hypertune folder instead of the reconsstruction folder.

param postfix: str = ''#

Postfix this string to the reconstruction folder name. Note that “_” will be automatically generated. In hypertune mode, the postfix string would be applied on the hypertune folder instead of the reconsstruction folder.

param save_result: List[Literal['model', 'obja', 'objp', 'probe', 'probe_prop', 'optim_state', 'scheduler_state']] = ['model', 'objp']#

This list specifies the available results to save every SAVE_ITERS, so it keeps the intermediate progress. Available options are ‘model’, ‘obja’, ‘objp’, ‘probe’, ‘probe_prop’, ‘optim_state’, and ‘scheduler_state’. ‘model’ is a nested dict that later got stored as an hdf5 file. ‘model’ contains optimizable tensors and metadata so that you can always refine from it and load whatever optimizable tensors (object, probe, positions, tilts) if you want to continue the reconstruction. It’s similar to the NiterXXX.mat from PtychoShelves. ‘object’ and ‘probe’ output the reconstructed object and probe as ‘.tif’. If you don’t want to save anything, set ‘SAVE_ITERS’ to null. Suggested setting is to save everything (i.e., [‘model’, ‘obja’, ‘objp’, ‘probe’]). For hypertune mode, you can set ‘collate_results’ to true and set ‘SAVE_ITERS’ to null to disable result saving.

param result_modes: ResultModes [Optional]#

This dict specifies which object output is saved by their final dimension (‘obj_dim’), whether to save the full or cropped FOV (‘FOV’) of object, and whether to save the raw or normalized bit depth version of object and probe. A comprehensive (but probably redundant) saving option looks like {‘obj_dim’: [2,3,4], ‘FOV’: [‘full’, ‘crop’], ‘bit’: [‘raw’, ‘32’, ‘16’, ‘8’]}. ‘obj_dim’ takes a list of int, the int ranges between 2 to 4, corresponding to 2D to 4D object output. Set ‘obj_dim’: [2] if you only want the zsum from multislice ptychography. Suggested value is [2,3,4] to save all possible output. ‘FOV’ takes a list of strings, the available strings are either ‘full’ or ‘crop’. Suggested value is ‘crop’ so the lateral padded region of object is not saved. ‘bit’ takes a list of strings, the available strings are ‘raw’, ‘32’, ‘16’, and ‘8’. ‘raw’ is the original value range, while ‘32’ normalizes the value from 0 to 1. ‘16’ and ‘8’ will normalize the value from 0 to 65535 and 255 correspondingly. Defualt is ‘8’ to save only the normalized 8bit result for quick visualization. You can set it to [‘raw’, ‘8’] if you want to keep the original float32 bit results with normalized 8bit results. These postprocessing would postfix corresponding labels to the result files.

param selected_figs: List[Literal['loss', 'learning_rates', 'forward', 'probe_r_amp', 'probe_k_amp', 'probe_k_phase', 'pos', 'tilt', 'tilt_avg', 'slice_thickness', 'convergence', 'convergence_full', 'convergence_dynamic', 'all']] = ['convergence', 'forward', 'probe_r_amp', 'pos']#

This list specifies the selected figures that will be plotted/saved at each save interval. Available keys: - ‘convergence’ / ‘convergence_full’ : Unified time-series dashboard showing the full iteration

history from iter 0 (no zoom). Saved as summary_convergence.png (fixed name, overwritten each cycle). Works even when convergence_monitor is null — tensor panels are simply absent. Recommended default.

  • ‘convergence_dynamic’Same dashboard but each panel is zoomed to its most informative

    x-range via the Kneedle algorithm. Useful for inspecting the long tail of an qualitatively converged run. Saved as summary_convergence_dynamic.png.

  • ‘loss’ : Standalone loss curve with zoom inset (iter-stamped). Covered by ‘convergence’.

  • ‘learning_rates’ : Standalone LR schedule (iter-stamped). Covered by ‘convergence’.

  • ‘slice_thickness’ : Standalone slice thickness curve with zoom inset (iter-stamped). Covered by ‘convergence’.

  • ‘tilt_avg’ : Standalone avg tilt curves with zoom inset (iter-stamped). Covered by ‘convergence’.

  • ‘forward’ : Forward pass snapshot (probe, object patches, model DP vs measured DP).

  • ‘probe_r_amp’ : Probe modes amplitude in real space.

  • ‘probe_k_amp’ : Probe modes amplitude in Fourier space.

  • ‘probe_k_phase’ : Probe modes phase in Fourier space.

  • ‘pos’ : Scan positions overlay (init vs optimized).

  • ‘tilt’ : Object tilt quiver plot (snapshot).

  • ‘all’ : All of the above.

Suggested value: [‘convergence’, ‘forward’, ‘probe_r_amp’, ‘pos’].

param copy_params: bool = True#

Set to true if you want to copy the .yml params file to the hypertune folder (hypertune mode) or individual reconstruction folders (reconsturction mode). Suggested value is true for better record keeping, although most information is saved in model.pt and can be loaded by ckpt = torch.load(‘model.pt’), params = ckpt[‘params’].

param compiler_configs: CompilerConfigs | None [Optional]#

This dict specifies the PyTorch JIT compiler configurations. Set to {‘enable’: true} to enable PyTorch JIT compilation for a 1.3-1.9x speedup on supported hardware. See https://docs.pytorch.org/docs/stable/generated/torch.compile.html for more details.

Generally, for torch.compile with Triton, you’ll need CUDA GPU with Compute Capability >= 7.0. Linux and macOS should support the PyTorch JIT compiler out-of-the-box. For Windows users, please follow the instruction and download triton-windows from woct0rdho/triton-windows.

param convergence_monitor: ConvergenceMonitorParams | None [Optional]#

Convergence monitoring configuration. Enabled by default — periodically snapshots optimizable tensors and records how they change over the reconstruction into model.hdf5. Set to null to disable. Add ‘convergence’ to selected_figs to generate convergence plots at each save interval. Example: {tensors: [obja, objp, probe, probe_pos_shifts], every_n_iters: null, percentile_range: [15, 85]}.

pydantic model ptyrad.params.recon_params.IndicesMode[source]#

Bases: BaseModel

Fields:
param mode: Literal['full', 'sub', 'center'] = 'full'#
param subscan_slow: int | None = None#
param subscan_fast: int | None = None#
pydantic model ptyrad.params.recon_params.BatchSize[source]#

Bases: BaseModel

Fields:
param size: int = 32#
param grad_accumulation: int = 1#
pydantic model ptyrad.params.recon_params.ResultModes[source]#

Bases: BaseModel

Fields:
param obj_dim: List[int] = [2, 3, 4]#
param FOV: List[str] = ['crop']#
param bit: List[str] = ['8']#
pydantic model ptyrad.params.recon_params.CompilerConfigs[source]#

Bases: BaseModel

Fields:
param enable: bool = False#
param fullgraph: bool = False#
param dynamic: bool | None = None#
param backend: Literal['inductor', 'cudagraphs', 'ipex', 'onnxrt'] = 'inductor'#
param mode: Literal['default', 'reduce-overhead', 'max-autotune', 'max-autotune-no-cudagraphs'] = 'default'#
param options: dict[str, str | int | bool] | None = None#
pydantic model ptyrad.params.recon_params.ConvergenceMonitorParams[source]#

Bases: BaseModel

Fields:
param tensors: List[Literal['obja', 'objp', 'probe', 'probe_pos_shifts']] = ['obja', 'objp', 'probe', 'probe_pos_shifts']#

Which optimizable tensors to track convergence for. - ‘obja’: mean absolute change of (1 - obja) in the scanned ROI, split into background

(pixels below p_low) and signal (pixels above p_high) regions. Stored as obja_bg / obja_fg. The 1-obja transform maps vacuum → 0 and material → >0 for a consistent sign convention.

  • ‘objp’: same percentile-masked mean absolute change applied to objp directly, stored as objp_bg / objp_fg.

  • ‘probe’: fractional intensity change (sum|ΔI|/sum(I)) of mode-summed probe intensity (robust to mode-order switching).

  • ‘probe_pos_shifts’: RMS displacement change in Å between consecutive snapshots.

‘slice_thickness’ and ‘obj_tilts’ are excluded — they are already recorded every iteration via model.dz_iters and model.avg_tilt_iters and plotted directly by the dashboard.

param every_n_iters: int | None = None#

Snapshot interval in iterations. When null (default), aligns with SAVE_ITERS so convergence snapshots happen at the same cadence as result saving. When set explicitly, snapshots are taken at every_n_iters regardless of SAVE_ITERS. Has no effect if both every_n_iters and SAVE_ITERS are null (monitor silently disabled). Note: each metric measures the accumulated change since the previous snapshot, so its magnitude scales with this interval. Tracking every iteration is noisy so not recommended; 10-50 iters (matching SAVE_ITERS) is the intended use.

param percentile_range: List[float] = [15.0, 85.0]#

Percentile range [p_low, p_high] used to separate background from signal pixels when computing obja and objp convergence metrics. The background metric tracks mean |Δ| for pixels below p_low (vacuum/near-vacuum region), and the signal metric tracks mean |Δ| for pixels above p_high (material region). The middle range [p_low, p_high] is excluded to avoid the transition boundary. Has no effect on probe or probe_pos_shifts metrics.

Contents