Get reconstruction provenance#
Created with PtyRAD 0.1.0b13
Requires PtyRAD >= 0.1.0b13
Documentation: https://ptyrad.readthedocs.io/en/latest/
PtyRAD paper: https://doi.org/10.1093/mam/ozaf070
PtyRAD arXiv: https://arxiv.org/abs/2505.07814
Zenodo record: https://doi.org/10.5281/zenodo.15273176
Box folder: https://cornell.box.com/s/n5balzf88jixescp9l15ojx7di4xn1uo
Youtube channel: https://www.youtube.com/@ptyrad_official
Before running this notebook, you must first follow the instruction in README.md to:
Create the Python environment with all dependant Python packages like PyTorch
Activate that python environment
Install
ptyradpackage into your activated Python environement (only need to install once)
Note: This notebook is designed for retrieving and exporting the full reconstruction history (i.e., provenance) for sequential reconstructions
Author: Chia-Hao Lee, cl2696@cornell.edu
Why we want the full provenance record?#
It’s very common to run sequential reconstructions to refine your ptychography results.
For example, we might do the following:
Run a reduced FOV using 1 nm slice
Load the previous probe from run #1, run a new reconstruction for full FOV
Take the previous probe, object, and positions from run #2, and run a new reconstruction with 0.5 nm slice
Take the previous probe, object, and positions from run #3, and further reduce the depth regularization strength
For full reproducibility, one needs to keep track of all the output HDF5s and param files. This process is quite demanding and error prone, often leads to incorrect or incomplete record if not done carefully. Therefore, a light-weight automatic provenance tracking feature is highly desirable.
How PtyRAD resolve it?#
PtyRAD collects the metadata and params for each run and saves it as the provenance record in the HDF5 output. For sequantial runs, it will collect the provenance record during initialization (specifically, while loading previous runs), and combine with the current record while saving the HDF5 output. This way, the full history of metadata and participating params are inherited, making it much easier to reproduce results from sequential reconstructions.
00. Setup working directory and imports#
import os
# Set this to your desired working directory so you can easily access the data, model, param files
work_dir = "H:/workspace/ptyrad/"
os.chdir(work_dir)
print("Current working dir: ", os.getcwd())
# Note that the output/ directory will be automatically generated under your working directory
Current working dir: H:\workspace\ptyrad
from ptyrad.io.load import load_ptyrad
from ptyrad.io.provenance import load_provenance_from_h5, export_hdf5_provenance_to_json
01. Load PtyRAD HDF5 output file#
hdf5_path = "output/tBL_WSe2/20260209_full_N16384_dp128_flipT100_random32_p6_1obj_6slice_dz2_orblur0.4_ozblur1.0_oposc_sng1.0_spr0.1/model_iter0003.hdf5"
ckpt = load_ptyrad(hdf5_path)
# The reconstructed arrays (probe, object, positions) and metadata are stored as HDF5 groups/datasets and parsed as Python dict.
print("ckpt: ")
for k in ckpt.keys():
print(' ', k)
ckpt:
avg_iter_t
avg_losses
batch_losses
dz_iters
indices
iter_times
loss_iters
model_attributes
niter
optim_state_dict
optimizable_tensors
output_path
params
ptyrad_version
random_seed
02. Load the provenance record as Python dict from HDF5 attr#
provenance = load_provenance_from_h5(hdf5_path)
provenance
{'probe': [{'uid': '00c5aa2f',
'timestamp': '2026-02-09T23:51:32.115382',
'run_name': '<Simulation>',
'action': 'Simulated',
'metadata': {'sim_params': None}},
{'uid': '2d5dd63e',
'timestamp': '2026-02-09T23:53:17.382249',
'ptyrad_version': '0.1.0b13',
'run_name': 'h:\\workspace\\ptyrad-test\\output\\tBL_WSe2\\20260209_full_N16384_dp128_flipT100_random32_p6_1obj_6slice_dz2_orblur0.4_ozblur1.0_oposc_sng1.0_spr0.1\\model_iter0003.hdf5',
'note': 'PtyRAD Run',
'params': {'init_params': {'random_seed': None,
'probe_illum_type': 'electron',
'probe_kv': 80.0,
'probe_conv_angle': 24.9,
'probe_aberrations': {},
'beam_kev': None,
'probe_dRn': None,
'probe_Rn': None,
'probe_D_H': None,
'probe_D_FZP': None,
'probe_Ls': None,
'meas_Npix': 128,
'pos_N_scans': 16384,
'pos_N_scan_slow': 128,
'pos_N_scan_fast': 128,
'pos_scan_step_size': 0.429,
'meas_calibration': {'mode': 'fitRBF', 'value': None},
'probe_pmode_max': 6,
'probe_pmode_init_pows': [0.02],
'obj_omode_max': 1,
'obj_omode_init_occu': {'occu_type': 'uniform', 'init_occu': None},
'obj_Nlayer': 6,
'obj_slice_thickness': 2.0,
'meas_permute': None,
'meas_reshape': None,
'meas_flipT': [1, 0, 0],
'meas_crop': None,
'meas_pad': None,
'meas_resample': None,
'meas_add_source_size': None,
'meas_add_detector_blur': None,
'meas_remove_neg_values': {'mode': 'clip_neg',
'value': None,
'force': False},
'meas_normalization': {'mode': 'max_at_one', 'value': None},
'meas_add_poisson_noise': None,
'meas_export': None,
'probe_permute': None,
'probe_z_shift': None,
'probe_normalization': {'mode': 'mean_total_ints', 'value': None},
'pos_scan_flipT': None,
'pos_scan_affine': None,
'pos_scan_rand_std': 0.15,
'obj_z_crop': None,
'obj_z_pad': None,
'obj_z_resample': None,
'meas_source': 'file',
'meas_params': {'path': 'data\\tBL_WSe2\\Panel_g-h_Themis\\scan_x128_y128.raw',
'key': None,
'shape': None,
'offset': None,
'gap': None},
'probe_source': 'simu',
'probe_params': None,
'pos_source': 'simu',
'pos_params': None,
'obj_source': 'simu',
'obj_params': None,
'tilt_source': 'simu',
'tilt_params': {'tilt_type': 'all', 'init_tilts': [[0, 0]]}},
'hypertune_params': {'if_hypertune': False,
'collate_results': True,
'append_params': True,
'n_trials': 5,
'timeout': None,
'sampler_params': {'name': 'TPESampler',
'configs': {'multivariate': True, 'group': True, 'constant_liar': True}},
'pruner_params': {'name': 'HyperbandPruner',
'configs': {'min_resource': 5, 'reduction_factor': 2}},
'storage_path': 'sqlite:///hypertune.sqlite3',
'study_name': 'study',
'error_metric': 'loss',
'tune_params': {'optimizer': {'state': False,
'suggest': 'cat',
'kwargs': {'choices': ['Adam', 'AdamW', 'RMSprop', 'SGD'],
'optim_configs': {}}},
'batch_size': {'state': False,
'suggest': 'int',
'kwargs': {'low': 16, 'high': 512, 'log': True}},
'plr': {'state': False,
'suggest': 'cat',
'kwargs': {'choices': [0.01, 0.0001, 0.0001]}},
'oalr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'oplr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'slr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'tlr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'dzlr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'dx': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.14, 'high': 0.16, 'step': 0.001}},
'pmode_max': {'state': False,
'suggest': 'int',
'kwargs': {'low': 1, 'high': 8, 'step': 1}},
'conv_angle': {'state': False,
'suggest': 'float',
'kwargs': {'low': 24, 'high': 26, 'step': 1}},
'C10': {'state': False,
'suggest': 'float',
'kwargs': {'low': -100, 'high': 100, 'step': 1}},
'C12': {'state': False,
'suggest': 'float',
'kwargs': {'low': -100, 'high': 100, 'step': 1}},
'C21': {'state': False,
'suggest': 'float',
'kwargs': {'low': -1000, 'high': 1000, 'step': 1}},
'C23': {'state': False,
'suggest': 'float',
'kwargs': {'low': -1000, 'high': 1000, 'step': 1}},
'C30': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5000, 'high': 5000, 'step': 1}},
'C32': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5000, 'high': 5000, 'step': 1}},
'C34': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5000, 'high': 5000, 'step': 1}},
'C41': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50000.0, 'high': 50000.0, 'step': 1}},
'C43': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50000.0, 'high': 50000.0, 'step': 1}},
'C45': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50000.0, 'high': 50000.0, 'step': 1}},
'C50': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'C52': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'C54': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'C56': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'z_shift': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50, 'high': 50, 'step': 10}},
'Nlayer': {'state': False,
'suggest': 'int',
'kwargs': {'low': 1, 'high': 8, 'step': 1}},
'dz': {'state': False,
'suggest': 'float',
'kwargs': {'low': 4, 'high': 8, 'step': 0.5}},
'scale': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.8, 'high': 1.2, 'step': 0.02}},
'asymmetry': {'state': False,
'suggest': 'float',
'kwargs': {'low': -0.2, 'high': 0.2, 'step': 0.05}},
'rotation': {'state': False,
'suggest': 'float',
'kwargs': {'low': -4, 'high': 4, 'step': 0.5}},
'shear': {'state': False,
'suggest': 'float',
'kwargs': {'low': -4, 'high': 4, 'step': 0.5}},
'tilt_y': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5, 'high': 5, 'step': 0.5}},
'tilt_x': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5, 'high': 5, 'step': 0.5}}}},
'model_params': {'obj_preblur_std': None,
'detector_blur_std': None,
'preload_data': True,
'optimizer_params': {'name': 'Adam', 'configs': {}, 'load_state': None},
'update_params': {'obja': {'start_iter': 1, 'lr': 0.0005},
'objp': {'start_iter': 1, 'lr': 0.0005},
'obj_tilts': {'start_iter': None, 'lr': 0.0},
'slice_thickness': {'start_iter': None, 'lr': 0.0},
'probe': {'start_iter': 1, 'lr': 0.0001},
'probe_pos_shifts': {'start_iter': 1, 'lr': 0.0005}}},
'loss_params': {'loss_single': {'state': True,
'weight': 1.0,
'dp_pow': 0.5},
'loss_poissn': {'state': False,
'weight': 1.0,
'dp_pow': 1.0,
'eps': 1e-06},
'loss_pacbed': {'state': False, 'weight': 0.5, 'dp_pow': 0.2},
'loss_sparse': {'state': True, 'weight': 0.1, 'ln_order': 1},
'loss_simlar': {'state': False,
'weight': 0.1,
'obj_type': 'both',
'scale_factor': [1.0, 1.0, 1.0],
'blur_std': 1.0}},
'constraint_params': {'ortho_pmode': {'start_iter': 1,
'step': 1,
'end_iter': None},
'probe_mask_k': {'start_iter': None,
'step': 1,
'end_iter': None,
'radius': 0.22,
'width': 0.05,
'power_thresh': 0.95},
'fix_probe_int': {'start_iter': 1, 'step': 1, 'end_iter': None},
'obj_rblur': {'start_iter': 1,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'kernel_size': 5,
'std': 0.4},
'obj_zblur': {'start_iter': 1,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'kernel_size': 5,
'std': 1.0},
'kr_filter': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'radius': 0.15,
'width': 0.05},
'kz_filter': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'beta': 1.0,
'alpha': 1.0},
'kr_thresh': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'thresh': 0.05},
'complex_ratio': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'alpha1': 1.0,
'alpha2': 0.0},
'mirrored_amp': {'start_iter': None,
'step': 1,
'end_iter': None,
'relax': 0.1,
'scale': 0.03,
'power': 4.0},
'obj_z_recenter': {'start_iter': None,
'step': 1,
'end_iter': None,
'thresh': 95,
'scale': 1,
'max_shift': 10},
'obja_thresh': {'start_iter': None,
'step': 1,
'end_iter': None,
'relax': 0.0,
'thresh': [0.96, 1.04]},
'objp_postiv': {'start_iter': 1,
'step': 1,
'end_iter': None,
'relax': 0.0},
'pos_recenter': {'start_iter': 1,
'step': 1,
'end_iter': None,
'relax': 0.0},
'tilt_smooth': {'start_iter': None,
'step': 1,
'end_iter': None,
'std': 2.0}},
'recon_params': {'NITER': 200,
'INDICES_MODE': {'mode': 'full',
'subscan_slow': None,
'subscan_fast': None},
'BATCH_SIZE': {'size': 32, 'grad_accumulation': 1},
'GROUP_MODE': 'random',
'SAVE_ITERS': 1,
'output_dir': 'output/tBL_WSe2/',
'recon_dir_affixes': ['minimal', 'constraint', 'loss'],
'prefix_time': 'date',
'prefix': '',
'postfix': '',
'save_result': ['model', 'objp'],
'result_modes': {'obj_dim': [2, 3, 4], 'FOV': ['crop'], 'bit': ['8']},
'selected_figs': ['loss', 'forward', 'probe_r_amp', 'pos'],
'copy_params': True,
'if_quiet': False,
'compiler_configs': {'fullgraph': False,
'dynamic': None,
'backend': 'inductor',
'mode': 'default',
'options': None,
'disable': True}},
'params_path': 'params/examples/tBL_WSe2.yaml'}}],
'pos': [{'uid': 'ffaacc18',
'timestamp': '2026-02-09T23:51:32.115382',
'run_name': '<Simulation>',
'action': 'Simulated',
'metadata': {'sim_params': None}},
{'uid': '2d5dd63e',
'timestamp': '2026-02-09T23:53:17.382249',
'ptyrad_version': '0.1.0b13',
'run_name': 'h:\\workspace\\ptyrad-test\\output\\tBL_WSe2\\20260209_full_N16384_dp128_flipT100_random32_p6_1obj_6slice_dz2_orblur0.4_ozblur1.0_oposc_sng1.0_spr0.1\\model_iter0003.hdf5',
'note': 'PtyRAD Run',
'params': {'init_params': {'random_seed': None,
'probe_illum_type': 'electron',
'probe_kv': 80.0,
'probe_conv_angle': 24.9,
'probe_aberrations': {},
'beam_kev': None,
'probe_dRn': None,
'probe_Rn': None,
'probe_D_H': None,
'probe_D_FZP': None,
'probe_Ls': None,
'meas_Npix': 128,
'pos_N_scans': 16384,
'pos_N_scan_slow': 128,
'pos_N_scan_fast': 128,
'pos_scan_step_size': 0.429,
'meas_calibration': {'mode': 'fitRBF', 'value': None},
'probe_pmode_max': 6,
'probe_pmode_init_pows': [0.02],
'obj_omode_max': 1,
'obj_omode_init_occu': {'occu_type': 'uniform', 'init_occu': None},
'obj_Nlayer': 6,
'obj_slice_thickness': 2.0,
'meas_permute': None,
'meas_reshape': None,
'meas_flipT': [1, 0, 0],
'meas_crop': None,
'meas_pad': None,
'meas_resample': None,
'meas_add_source_size': None,
'meas_add_detector_blur': None,
'meas_remove_neg_values': {'mode': 'clip_neg',
'value': None,
'force': False},
'meas_normalization': {'mode': 'max_at_one', 'value': None},
'meas_add_poisson_noise': None,
'meas_export': None,
'probe_permute': None,
'probe_z_shift': None,
'probe_normalization': {'mode': 'mean_total_ints', 'value': None},
'pos_scan_flipT': None,
'pos_scan_affine': None,
'pos_scan_rand_std': 0.15,
'obj_z_crop': None,
'obj_z_pad': None,
'obj_z_resample': None,
'meas_source': 'file',
'meas_params': {'path': 'data\\tBL_WSe2\\Panel_g-h_Themis\\scan_x128_y128.raw',
'key': None,
'shape': None,
'offset': None,
'gap': None},
'probe_source': 'simu',
'probe_params': None,
'pos_source': 'simu',
'pos_params': None,
'obj_source': 'simu',
'obj_params': None,
'tilt_source': 'simu',
'tilt_params': {'tilt_type': 'all', 'init_tilts': [[0, 0]]}},
'hypertune_params': {'if_hypertune': False,
'collate_results': True,
'append_params': True,
'n_trials': 5,
'timeout': None,
'sampler_params': {'name': 'TPESampler',
'configs': {'multivariate': True, 'group': True, 'constant_liar': True}},
'pruner_params': {'name': 'HyperbandPruner',
'configs': {'min_resource': 5, 'reduction_factor': 2}},
'storage_path': 'sqlite:///hypertune.sqlite3',
'study_name': 'study',
'error_metric': 'loss',
'tune_params': {'optimizer': {'state': False,
'suggest': 'cat',
'kwargs': {'choices': ['Adam', 'AdamW', 'RMSprop', 'SGD'],
'optim_configs': {}}},
'batch_size': {'state': False,
'suggest': 'int',
'kwargs': {'low': 16, 'high': 512, 'log': True}},
'plr': {'state': False,
'suggest': 'cat',
'kwargs': {'choices': [0.01, 0.0001, 0.0001]}},
'oalr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'oplr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'slr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'tlr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'dzlr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'dx': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.14, 'high': 0.16, 'step': 0.001}},
'pmode_max': {'state': False,
'suggest': 'int',
'kwargs': {'low': 1, 'high': 8, 'step': 1}},
'conv_angle': {'state': False,
'suggest': 'float',
'kwargs': {'low': 24, 'high': 26, 'step': 1}},
'C10': {'state': False,
'suggest': 'float',
'kwargs': {'low': -100, 'high': 100, 'step': 1}},
'C12': {'state': False,
'suggest': 'float',
'kwargs': {'low': -100, 'high': 100, 'step': 1}},
'C21': {'state': False,
'suggest': 'float',
'kwargs': {'low': -1000, 'high': 1000, 'step': 1}},
'C23': {'state': False,
'suggest': 'float',
'kwargs': {'low': -1000, 'high': 1000, 'step': 1}},
'C30': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5000, 'high': 5000, 'step': 1}},
'C32': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5000, 'high': 5000, 'step': 1}},
'C34': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5000, 'high': 5000, 'step': 1}},
'C41': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50000.0, 'high': 50000.0, 'step': 1}},
'C43': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50000.0, 'high': 50000.0, 'step': 1}},
'C45': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50000.0, 'high': 50000.0, 'step': 1}},
'C50': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'C52': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'C54': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'C56': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'z_shift': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50, 'high': 50, 'step': 10}},
'Nlayer': {'state': False,
'suggest': 'int',
'kwargs': {'low': 1, 'high': 8, 'step': 1}},
'dz': {'state': False,
'suggest': 'float',
'kwargs': {'low': 4, 'high': 8, 'step': 0.5}},
'scale': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.8, 'high': 1.2, 'step': 0.02}},
'asymmetry': {'state': False,
'suggest': 'float',
'kwargs': {'low': -0.2, 'high': 0.2, 'step': 0.05}},
'rotation': {'state': False,
'suggest': 'float',
'kwargs': {'low': -4, 'high': 4, 'step': 0.5}},
'shear': {'state': False,
'suggest': 'float',
'kwargs': {'low': -4, 'high': 4, 'step': 0.5}},
'tilt_y': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5, 'high': 5, 'step': 0.5}},
'tilt_x': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5, 'high': 5, 'step': 0.5}}}},
'model_params': {'obj_preblur_std': None,
'detector_blur_std': None,
'preload_data': True,
'optimizer_params': {'name': 'Adam', 'configs': {}, 'load_state': None},
'update_params': {'obja': {'start_iter': 1, 'lr': 0.0005},
'objp': {'start_iter': 1, 'lr': 0.0005},
'obj_tilts': {'start_iter': None, 'lr': 0.0},
'slice_thickness': {'start_iter': None, 'lr': 0.0},
'probe': {'start_iter': 1, 'lr': 0.0001},
'probe_pos_shifts': {'start_iter': 1, 'lr': 0.0005}}},
'loss_params': {'loss_single': {'state': True,
'weight': 1.0,
'dp_pow': 0.5},
'loss_poissn': {'state': False,
'weight': 1.0,
'dp_pow': 1.0,
'eps': 1e-06},
'loss_pacbed': {'state': False, 'weight': 0.5, 'dp_pow': 0.2},
'loss_sparse': {'state': True, 'weight': 0.1, 'ln_order': 1},
'loss_simlar': {'state': False,
'weight': 0.1,
'obj_type': 'both',
'scale_factor': [1.0, 1.0, 1.0],
'blur_std': 1.0}},
'constraint_params': {'ortho_pmode': {'start_iter': 1,
'step': 1,
'end_iter': None},
'probe_mask_k': {'start_iter': None,
'step': 1,
'end_iter': None,
'radius': 0.22,
'width': 0.05,
'power_thresh': 0.95},
'fix_probe_int': {'start_iter': 1, 'step': 1, 'end_iter': None},
'obj_rblur': {'start_iter': 1,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'kernel_size': 5,
'std': 0.4},
'obj_zblur': {'start_iter': 1,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'kernel_size': 5,
'std': 1.0},
'kr_filter': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'radius': 0.15,
'width': 0.05},
'kz_filter': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'beta': 1.0,
'alpha': 1.0},
'kr_thresh': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'thresh': 0.05},
'complex_ratio': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'alpha1': 1.0,
'alpha2': 0.0},
'mirrored_amp': {'start_iter': None,
'step': 1,
'end_iter': None,
'relax': 0.1,
'scale': 0.03,
'power': 4.0},
'obj_z_recenter': {'start_iter': None,
'step': 1,
'end_iter': None,
'thresh': 95,
'scale': 1,
'max_shift': 10},
'obja_thresh': {'start_iter': None,
'step': 1,
'end_iter': None,
'relax': 0.0,
'thresh': [0.96, 1.04]},
'objp_postiv': {'start_iter': 1,
'step': 1,
'end_iter': None,
'relax': 0.0},
'pos_recenter': {'start_iter': 1,
'step': 1,
'end_iter': None,
'relax': 0.0},
'tilt_smooth': {'start_iter': None,
'step': 1,
'end_iter': None,
'std': 2.0}},
'recon_params': {'NITER': 200,
'INDICES_MODE': {'mode': 'full',
'subscan_slow': None,
'subscan_fast': None},
'BATCH_SIZE': {'size': 32, 'grad_accumulation': 1},
'GROUP_MODE': 'random',
'SAVE_ITERS': 1,
'output_dir': 'output/tBL_WSe2/',
'recon_dir_affixes': ['minimal', 'constraint', 'loss'],
'prefix_time': 'date',
'prefix': '',
'postfix': '',
'save_result': ['model', 'objp'],
'result_modes': {'obj_dim': [2, 3, 4], 'FOV': ['crop'], 'bit': ['8']},
'selected_figs': ['loss', 'forward', 'probe_r_amp', 'pos'],
'copy_params': True,
'if_quiet': False,
'compiler_configs': {'fullgraph': False,
'dynamic': None,
'backend': 'inductor',
'mode': 'default',
'options': None,
'disable': True}},
'params_path': 'params/examples/tBL_WSe2.yaml'}}],
'obj': [{'uid': 'b3783236',
'timestamp': '2026-02-09T23:51:32.115382',
'run_name': '<Simulation>',
'action': 'Simulated',
'metadata': {'sim_params': None}},
{'uid': '2d5dd63e',
'timestamp': '2026-02-09T23:53:17.382249',
'ptyrad_version': '0.1.0b13',
'run_name': 'h:\\workspace\\ptyrad-test\\output\\tBL_WSe2\\20260209_full_N16384_dp128_flipT100_random32_p6_1obj_6slice_dz2_orblur0.4_ozblur1.0_oposc_sng1.0_spr0.1\\model_iter0003.hdf5',
'note': 'PtyRAD Run',
'params': {'init_params': {'random_seed': None,
'probe_illum_type': 'electron',
'probe_kv': 80.0,
'probe_conv_angle': 24.9,
'probe_aberrations': {},
'beam_kev': None,
'probe_dRn': None,
'probe_Rn': None,
'probe_D_H': None,
'probe_D_FZP': None,
'probe_Ls': None,
'meas_Npix': 128,
'pos_N_scans': 16384,
'pos_N_scan_slow': 128,
'pos_N_scan_fast': 128,
'pos_scan_step_size': 0.429,
'meas_calibration': {'mode': 'fitRBF', 'value': None},
'probe_pmode_max': 6,
'probe_pmode_init_pows': [0.02],
'obj_omode_max': 1,
'obj_omode_init_occu': {'occu_type': 'uniform', 'init_occu': None},
'obj_Nlayer': 6,
'obj_slice_thickness': 2.0,
'meas_permute': None,
'meas_reshape': None,
'meas_flipT': [1, 0, 0],
'meas_crop': None,
'meas_pad': None,
'meas_resample': None,
'meas_add_source_size': None,
'meas_add_detector_blur': None,
'meas_remove_neg_values': {'mode': 'clip_neg',
'value': None,
'force': False},
'meas_normalization': {'mode': 'max_at_one', 'value': None},
'meas_add_poisson_noise': None,
'meas_export': None,
'probe_permute': None,
'probe_z_shift': None,
'probe_normalization': {'mode': 'mean_total_ints', 'value': None},
'pos_scan_flipT': None,
'pos_scan_affine': None,
'pos_scan_rand_std': 0.15,
'obj_z_crop': None,
'obj_z_pad': None,
'obj_z_resample': None,
'meas_source': 'file',
'meas_params': {'path': 'data\\tBL_WSe2\\Panel_g-h_Themis\\scan_x128_y128.raw',
'key': None,
'shape': None,
'offset': None,
'gap': None},
'probe_source': 'simu',
'probe_params': None,
'pos_source': 'simu',
'pos_params': None,
'obj_source': 'simu',
'obj_params': None,
'tilt_source': 'simu',
'tilt_params': {'tilt_type': 'all', 'init_tilts': [[0, 0]]}},
'hypertune_params': {'if_hypertune': False,
'collate_results': True,
'append_params': True,
'n_trials': 5,
'timeout': None,
'sampler_params': {'name': 'TPESampler',
'configs': {'multivariate': True, 'group': True, 'constant_liar': True}},
'pruner_params': {'name': 'HyperbandPruner',
'configs': {'min_resource': 5, 'reduction_factor': 2}},
'storage_path': 'sqlite:///hypertune.sqlite3',
'study_name': 'study',
'error_metric': 'loss',
'tune_params': {'optimizer': {'state': False,
'suggest': 'cat',
'kwargs': {'choices': ['Adam', 'AdamW', 'RMSprop', 'SGD'],
'optim_configs': {}}},
'batch_size': {'state': False,
'suggest': 'int',
'kwargs': {'low': 16, 'high': 512, 'log': True}},
'plr': {'state': False,
'suggest': 'cat',
'kwargs': {'choices': [0.01, 0.0001, 0.0001]}},
'oalr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'oplr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'slr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'tlr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'dzlr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'dx': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.14, 'high': 0.16, 'step': 0.001}},
'pmode_max': {'state': False,
'suggest': 'int',
'kwargs': {'low': 1, 'high': 8, 'step': 1}},
'conv_angle': {'state': False,
'suggest': 'float',
'kwargs': {'low': 24, 'high': 26, 'step': 1}},
'C10': {'state': False,
'suggest': 'float',
'kwargs': {'low': -100, 'high': 100, 'step': 1}},
'C12': {'state': False,
'suggest': 'float',
'kwargs': {'low': -100, 'high': 100, 'step': 1}},
'C21': {'state': False,
'suggest': 'float',
'kwargs': {'low': -1000, 'high': 1000, 'step': 1}},
'C23': {'state': False,
'suggest': 'float',
'kwargs': {'low': -1000, 'high': 1000, 'step': 1}},
'C30': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5000, 'high': 5000, 'step': 1}},
'C32': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5000, 'high': 5000, 'step': 1}},
'C34': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5000, 'high': 5000, 'step': 1}},
'C41': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50000.0, 'high': 50000.0, 'step': 1}},
'C43': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50000.0, 'high': 50000.0, 'step': 1}},
'C45': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50000.0, 'high': 50000.0, 'step': 1}},
'C50': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'C52': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'C54': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'C56': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'z_shift': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50, 'high': 50, 'step': 10}},
'Nlayer': {'state': False,
'suggest': 'int',
'kwargs': {'low': 1, 'high': 8, 'step': 1}},
'dz': {'state': False,
'suggest': 'float',
'kwargs': {'low': 4, 'high': 8, 'step': 0.5}},
'scale': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.8, 'high': 1.2, 'step': 0.02}},
'asymmetry': {'state': False,
'suggest': 'float',
'kwargs': {'low': -0.2, 'high': 0.2, 'step': 0.05}},
'rotation': {'state': False,
'suggest': 'float',
'kwargs': {'low': -4, 'high': 4, 'step': 0.5}},
'shear': {'state': False,
'suggest': 'float',
'kwargs': {'low': -4, 'high': 4, 'step': 0.5}},
'tilt_y': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5, 'high': 5, 'step': 0.5}},
'tilt_x': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5, 'high': 5, 'step': 0.5}}}},
'model_params': {'obj_preblur_std': None,
'detector_blur_std': None,
'preload_data': True,
'optimizer_params': {'name': 'Adam', 'configs': {}, 'load_state': None},
'update_params': {'obja': {'start_iter': 1, 'lr': 0.0005},
'objp': {'start_iter': 1, 'lr': 0.0005},
'obj_tilts': {'start_iter': None, 'lr': 0.0},
'slice_thickness': {'start_iter': None, 'lr': 0.0},
'probe': {'start_iter': 1, 'lr': 0.0001},
'probe_pos_shifts': {'start_iter': 1, 'lr': 0.0005}}},
'loss_params': {'loss_single': {'state': True,
'weight': 1.0,
'dp_pow': 0.5},
'loss_poissn': {'state': False,
'weight': 1.0,
'dp_pow': 1.0,
'eps': 1e-06},
'loss_pacbed': {'state': False, 'weight': 0.5, 'dp_pow': 0.2},
'loss_sparse': {'state': True, 'weight': 0.1, 'ln_order': 1},
'loss_simlar': {'state': False,
'weight': 0.1,
'obj_type': 'both',
'scale_factor': [1.0, 1.0, 1.0],
'blur_std': 1.0}},
'constraint_params': {'ortho_pmode': {'start_iter': 1,
'step': 1,
'end_iter': None},
'probe_mask_k': {'start_iter': None,
'step': 1,
'end_iter': None,
'radius': 0.22,
'width': 0.05,
'power_thresh': 0.95},
'fix_probe_int': {'start_iter': 1, 'step': 1, 'end_iter': None},
'obj_rblur': {'start_iter': 1,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'kernel_size': 5,
'std': 0.4},
'obj_zblur': {'start_iter': 1,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'kernel_size': 5,
'std': 1.0},
'kr_filter': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'radius': 0.15,
'width': 0.05},
'kz_filter': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'beta': 1.0,
'alpha': 1.0},
'kr_thresh': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'thresh': 0.05},
'complex_ratio': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'alpha1': 1.0,
'alpha2': 0.0},
'mirrored_amp': {'start_iter': None,
'step': 1,
'end_iter': None,
'relax': 0.1,
'scale': 0.03,
'power': 4.0},
'obj_z_recenter': {'start_iter': None,
'step': 1,
'end_iter': None,
'thresh': 95,
'scale': 1,
'max_shift': 10},
'obja_thresh': {'start_iter': None,
'step': 1,
'end_iter': None,
'relax': 0.0,
'thresh': [0.96, 1.04]},
'objp_postiv': {'start_iter': 1,
'step': 1,
'end_iter': None,
'relax': 0.0},
'pos_recenter': {'start_iter': 1,
'step': 1,
'end_iter': None,
'relax': 0.0},
'tilt_smooth': {'start_iter': None,
'step': 1,
'end_iter': None,
'std': 2.0}},
'recon_params': {'NITER': 200,
'INDICES_MODE': {'mode': 'full',
'subscan_slow': None,
'subscan_fast': None},
'BATCH_SIZE': {'size': 32, 'grad_accumulation': 1},
'GROUP_MODE': 'random',
'SAVE_ITERS': 1,
'output_dir': 'output/tBL_WSe2/',
'recon_dir_affixes': ['minimal', 'constraint', 'loss'],
'prefix_time': 'date',
'prefix': '',
'postfix': '',
'save_result': ['model', 'objp'],
'result_modes': {'obj_dim': [2, 3, 4], 'FOV': ['crop'], 'bit': ['8']},
'selected_figs': ['loss', 'forward', 'probe_r_amp', 'pos'],
'copy_params': True,
'if_quiet': False,
'compiler_configs': {'fullgraph': False,
'dynamic': None,
'backend': 'inductor',
'mode': 'default',
'options': None,
'disable': True}},
'params_path': 'params/examples/tBL_WSe2.yaml'}}],
'tilt': [{'uid': '96b428a0',
'timestamp': '2026-02-09T23:51:32.115382',
'run_name': '<Simulation>',
'action': 'Simulated',
'metadata': {'sim_params': {'tilt_type': 'all', 'init_tilts': [[0, 0]]}}},
{'uid': '2d5dd63e',
'timestamp': '2026-02-09T23:53:17.382249',
'ptyrad_version': '0.1.0b13',
'run_name': 'h:\\workspace\\ptyrad-test\\output\\tBL_WSe2\\20260209_full_N16384_dp128_flipT100_random32_p6_1obj_6slice_dz2_orblur0.4_ozblur1.0_oposc_sng1.0_spr0.1\\model_iter0003.hdf5',
'note': 'PtyRAD Run',
'params': {'init_params': {'random_seed': None,
'probe_illum_type': 'electron',
'probe_kv': 80.0,
'probe_conv_angle': 24.9,
'probe_aberrations': {},
'beam_kev': None,
'probe_dRn': None,
'probe_Rn': None,
'probe_D_H': None,
'probe_D_FZP': None,
'probe_Ls': None,
'meas_Npix': 128,
'pos_N_scans': 16384,
'pos_N_scan_slow': 128,
'pos_N_scan_fast': 128,
'pos_scan_step_size': 0.429,
'meas_calibration': {'mode': 'fitRBF', 'value': None},
'probe_pmode_max': 6,
'probe_pmode_init_pows': [0.02],
'obj_omode_max': 1,
'obj_omode_init_occu': {'occu_type': 'uniform', 'init_occu': None},
'obj_Nlayer': 6,
'obj_slice_thickness': 2.0,
'meas_permute': None,
'meas_reshape': None,
'meas_flipT': [1, 0, 0],
'meas_crop': None,
'meas_pad': None,
'meas_resample': None,
'meas_add_source_size': None,
'meas_add_detector_blur': None,
'meas_remove_neg_values': {'mode': 'clip_neg',
'value': None,
'force': False},
'meas_normalization': {'mode': 'max_at_one', 'value': None},
'meas_add_poisson_noise': None,
'meas_export': None,
'probe_permute': None,
'probe_z_shift': None,
'probe_normalization': {'mode': 'mean_total_ints', 'value': None},
'pos_scan_flipT': None,
'pos_scan_affine': None,
'pos_scan_rand_std': 0.15,
'obj_z_crop': None,
'obj_z_pad': None,
'obj_z_resample': None,
'meas_source': 'file',
'meas_params': {'path': 'data\\tBL_WSe2\\Panel_g-h_Themis\\scan_x128_y128.raw',
'key': None,
'shape': None,
'offset': None,
'gap': None},
'probe_source': 'simu',
'probe_params': None,
'pos_source': 'simu',
'pos_params': None,
'obj_source': 'simu',
'obj_params': None,
'tilt_source': 'simu',
'tilt_params': {'tilt_type': 'all', 'init_tilts': [[0, 0]]}},
'hypertune_params': {'if_hypertune': False,
'collate_results': True,
'append_params': True,
'n_trials': 5,
'timeout': None,
'sampler_params': {'name': 'TPESampler',
'configs': {'multivariate': True, 'group': True, 'constant_liar': True}},
'pruner_params': {'name': 'HyperbandPruner',
'configs': {'min_resource': 5, 'reduction_factor': 2}},
'storage_path': 'sqlite:///hypertune.sqlite3',
'study_name': 'study',
'error_metric': 'loss',
'tune_params': {'optimizer': {'state': False,
'suggest': 'cat',
'kwargs': {'choices': ['Adam', 'AdamW', 'RMSprop', 'SGD'],
'optim_configs': {}}},
'batch_size': {'state': False,
'suggest': 'int',
'kwargs': {'low': 16, 'high': 512, 'log': True}},
'plr': {'state': False,
'suggest': 'cat',
'kwargs': {'choices': [0.01, 0.0001, 0.0001]}},
'oalr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'oplr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'slr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'tlr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'dzlr': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.0001, 'high': 0.01, 'log': True}},
'dx': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.14, 'high': 0.16, 'step': 0.001}},
'pmode_max': {'state': False,
'suggest': 'int',
'kwargs': {'low': 1, 'high': 8, 'step': 1}},
'conv_angle': {'state': False,
'suggest': 'float',
'kwargs': {'low': 24, 'high': 26, 'step': 1}},
'C10': {'state': False,
'suggest': 'float',
'kwargs': {'low': -100, 'high': 100, 'step': 1}},
'C12': {'state': False,
'suggest': 'float',
'kwargs': {'low': -100, 'high': 100, 'step': 1}},
'C21': {'state': False,
'suggest': 'float',
'kwargs': {'low': -1000, 'high': 1000, 'step': 1}},
'C23': {'state': False,
'suggest': 'float',
'kwargs': {'low': -1000, 'high': 1000, 'step': 1}},
'C30': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5000, 'high': 5000, 'step': 1}},
'C32': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5000, 'high': 5000, 'step': 1}},
'C34': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5000, 'high': 5000, 'step': 1}},
'C41': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50000.0, 'high': 50000.0, 'step': 1}},
'C43': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50000.0, 'high': 50000.0, 'step': 1}},
'C45': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50000.0, 'high': 50000.0, 'step': 1}},
'C50': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'C52': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'C54': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'C56': {'state': False,
'suggest': 'float',
'kwargs': {'low': -500000.0, 'high': 500000.0, 'step': 1}},
'z_shift': {'state': False,
'suggest': 'float',
'kwargs': {'low': -50, 'high': 50, 'step': 10}},
'Nlayer': {'state': False,
'suggest': 'int',
'kwargs': {'low': 1, 'high': 8, 'step': 1}},
'dz': {'state': False,
'suggest': 'float',
'kwargs': {'low': 4, 'high': 8, 'step': 0.5}},
'scale': {'state': False,
'suggest': 'float',
'kwargs': {'low': 0.8, 'high': 1.2, 'step': 0.02}},
'asymmetry': {'state': False,
'suggest': 'float',
'kwargs': {'low': -0.2, 'high': 0.2, 'step': 0.05}},
'rotation': {'state': False,
'suggest': 'float',
'kwargs': {'low': -4, 'high': 4, 'step': 0.5}},
'shear': {'state': False,
'suggest': 'float',
'kwargs': {'low': -4, 'high': 4, 'step': 0.5}},
'tilt_y': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5, 'high': 5, 'step': 0.5}},
'tilt_x': {'state': False,
'suggest': 'float',
'kwargs': {'low': -5, 'high': 5, 'step': 0.5}}}},
'model_params': {'obj_preblur_std': None,
'detector_blur_std': None,
'preload_data': True,
'optimizer_params': {'name': 'Adam', 'configs': {}, 'load_state': None},
'update_params': {'obja': {'start_iter': 1, 'lr': 0.0005},
'objp': {'start_iter': 1, 'lr': 0.0005},
'obj_tilts': {'start_iter': None, 'lr': 0.0},
'slice_thickness': {'start_iter': None, 'lr': 0.0},
'probe': {'start_iter': 1, 'lr': 0.0001},
'probe_pos_shifts': {'start_iter': 1, 'lr': 0.0005}}},
'loss_params': {'loss_single': {'state': True,
'weight': 1.0,
'dp_pow': 0.5},
'loss_poissn': {'state': False,
'weight': 1.0,
'dp_pow': 1.0,
'eps': 1e-06},
'loss_pacbed': {'state': False, 'weight': 0.5, 'dp_pow': 0.2},
'loss_sparse': {'state': True, 'weight': 0.1, 'ln_order': 1},
'loss_simlar': {'state': False,
'weight': 0.1,
'obj_type': 'both',
'scale_factor': [1.0, 1.0, 1.0],
'blur_std': 1.0}},
'constraint_params': {'ortho_pmode': {'start_iter': 1,
'step': 1,
'end_iter': None},
'probe_mask_k': {'start_iter': None,
'step': 1,
'end_iter': None,
'radius': 0.22,
'width': 0.05,
'power_thresh': 0.95},
'fix_probe_int': {'start_iter': 1, 'step': 1, 'end_iter': None},
'obj_rblur': {'start_iter': 1,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'kernel_size': 5,
'std': 0.4},
'obj_zblur': {'start_iter': 1,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'kernel_size': 5,
'std': 1.0},
'kr_filter': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'radius': 0.15,
'width': 0.05},
'kz_filter': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'beta': 1.0,
'alpha': 1.0},
'kr_thresh': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'thresh': 0.05},
'complex_ratio': {'start_iter': None,
'step': 1,
'end_iter': None,
'obj_type': 'both',
'alpha1': 1.0,
'alpha2': 0.0},
'mirrored_amp': {'start_iter': None,
'step': 1,
'end_iter': None,
'relax': 0.1,
'scale': 0.03,
'power': 4.0},
'obj_z_recenter': {'start_iter': None,
'step': 1,
'end_iter': None,
'thresh': 95,
'scale': 1,
'max_shift': 10},
'obja_thresh': {'start_iter': None,
'step': 1,
'end_iter': None,
'relax': 0.0,
'thresh': [0.96, 1.04]},
'objp_postiv': {'start_iter': 1,
'step': 1,
'end_iter': None,
'relax': 0.0},
'pos_recenter': {'start_iter': 1,
'step': 1,
'end_iter': None,
'relax': 0.0},
'tilt_smooth': {'start_iter': None,
'step': 1,
'end_iter': None,
'std': 2.0}},
'recon_params': {'NITER': 200,
'INDICES_MODE': {'mode': 'full',
'subscan_slow': None,
'subscan_fast': None},
'BATCH_SIZE': {'size': 32, 'grad_accumulation': 1},
'GROUP_MODE': 'random',
'SAVE_ITERS': 1,
'output_dir': 'output/tBL_WSe2/',
'recon_dir_affixes': ['minimal', 'constraint', 'loss'],
'prefix_time': 'date',
'prefix': '',
'postfix': '',
'save_result': ['model', 'objp'],
'result_modes': {'obj_dim': [2, 3, 4], 'FOV': ['crop'], 'bit': ['8']},
'selected_figs': ['loss', 'forward', 'probe_r_amp', 'pos'],
'copy_params': True,
'if_quiet': False,
'compiler_configs': {'fullgraph': False,
'dynamic': None,
'backend': 'inductor',
'mode': 'default',
'options': None,
'disable': True}},
'params_path': 'params/examples/tBL_WSe2.yaml'}}]}
03. Export the provenance record as JSON from HDF5 attr#
export_hdf5_provenance_to_json(hdf5_path, output_json_path=None); # If output_json_path=None, the JSON would be output to the same folder with hdf5_path)
Reading provenance from: output/tBL_WSe2/20260209_full_N16384_dp128_flipT100_random32_p6_1obj_6slice_dz2_orblur0.4_ozblur1.0_oposc_sng1.0_spr0.1/model_iter0003.hdf5
Successfully exported to: output/tBL_WSe2/20260209_full_N16384_dp128_flipT100_random32_p6_1obj_6slice_dz2_orblur0.4_ozblur1.0_oposc_sng1.0_spr0.1/model_iter0003_provenance.json
04. Understanding Your Provenance Record#
Now that you have exported the provenance history, you might wonder how to read it. The structure is designed to answer one question: “Where did this component come from, and what happened to it along the way?”
To understand it, think of your reconstruction as a Family Tree.
a. The Bloodlines (Components)#
Your reconstruction is the “child” of multiple parents. The Probe, Object, Positions, and Tilts each have their own separate family history.
The Probe might be an “heir” to a long line of previous refinements (Run A → Run B → Current Run).
The Object might be a “newcomer,” freshly imported from a simulation or an external file.
In the JSON file, these are stored as separate lists:
{
"probe": [ ...history... ],
"object": [ ...history... ],
"pos": [ ...history... ]
}
b. The Timeline (The List)#
Each list represents a chronological timeline for that specific component.
The Genesis (Start): The first entry in the list tells you how the component was born. Was it simulated? Was it loaded from a legacy file? Was it a raw NumPy array?
The Inheritance (Middle): As you load previous results (e.g., using
probe_source='PtyRAD'), the new run inherits the entire history list from the parent file. The history flows from the old file into the new one automatically.The Current Run (End): The final entry in the list is always THIS run. It stamps the component with the parameters used in the current reconstruction. -
Therefore, the involved files, parameters, and the full history are all encapsulated in “1 single HDF5 file”.
c. Reading a Trace#
Here is how to read a typical trace for a Probe that has gone through 3 sequential runs:
Entry 1 (Grandparent):
Simulated
Meaning: The probe started life as a simulation with 300kV and 20 mrad convergence angle.
Entry 2 (Parent):
Run_Step1.h5
Meaning: The simulated probe was used in “Step 1” to refine a coarse image. The specific parameters (learning rate, iterations) are stored here.
Entry 3 (Current Run):
Run_Step2.h5
Meaning: We loaded the result from Step 1 and refined it further in “Step 2”. This file contains the result of this step.
05. Example JSON output#
{
"probe": [
{
"uid": "62c2e472",
"timestamp": "2026-01-28T01:36:35.908309",
"run_name": "<Simulation>",
"action": "Simulated",
"metadata": {
"sim_params": null
}
},
{
"uid": "9f15b82d",
"timestamp": "2026-01-28T01:36:40.968223",
"run_name": "h:\\workspace\\ptyrad\\demo\\output\\tBL_WSe2\\20260128_full_N1024_dp128_flipT100_random32_p6_1obj_6slice_dz2_plr1e-4_oalr5e-4_oplr5e-4_slr5e-4_orblur0.4_ozblur1.0_mamp0.03_4.0_oathr0.96_oposc_sng1.0_spr0.1_new\\model_iter0001.hdf5",
"note": "PtyRAD Run",
"params": {"NEGLECTED FOR CLEANER NOTEBOOK DISPLAY"},
},
{
"uid": "3cbfa081",
"timestamp": "2026-01-28T01:45:09.575841",
"run_name": "h:\\workspace\\ptyrad\\demo\\output\\tBL_WSe2\\20260128_full_N1024_dp128_flipT100_random32_p6_1obj_6slice_dz2_plr1e-4_oalr5e-4_oplr5e-4_slr5e-4_orblur0.4_ozblur1.0_mamp0.03_4.0_oathr0.96_oposc_sng1.0_spr0.1_new_r2\\model_iter0005.hdf5",
"note": "PtyRAD Run",
"params": {"NEGLECTED FOR CLEANER NOTEBOOK DISPLAY"},
},
{
"uid": "bcaa4222",
"timestamp": "2026-01-28T11:25:15.814207",
"run_name": "h:\\workspace\\ptyrad\\demo\\output\\tBL_WSe2\\20260128_full_N16384_dp128_flipT100_random32_p6_1obj_6slice_dz2_plr1e-4_oalr5e-4_oplr5e-4_slr5e-4_orblur0.4_ozblur1.0_mamp0.03_4.0_oathr0.96_oposc_sng1.0_spr0.1_test1\\model_iter0003.hdf5",
"note": "PtyRAD Run",
"params": {"NEGLECTED FOR CLEANER NOTEBOOK DISPLAY"},
}