Tutorial

The below tutorial outlines how to perform SFX data processing from the command line on the psana cluster, and specifically the psffb nodes. For processing using the psanaq nodes, the following arguments should be added to each command: -f SLAC -q psanaq. Tasks (-t) that benefit from parallelization can be run using ${ncores} CPUs.

Step 0: setup btx and config file

Add this line in your ~/.bashrc file:

export PATH=/cds/sw/package/autosfx/btx/scripts:$PATH

Parameters for all of the data processing steps are listed in a config (.yaml) file, organized by task. More information about the config file can be found here.

Step 1: metrology set up

The most recent mask and geometry files can be retrieved from the metrology archive mrxv as follows:

$ elog_submit.sh -c ../tutorial/mfxlv4920.yaml -t fetch_mask -n 1
$ elog_submit.sh -c ../tutorial/mfxlv4920.yaml -t fetch_geom -n 1

The above commands generate the following files: ${config.root_dir}/mask/r000.npy and ${config.root_dir}/geom/r000.geom.

Step 2: run analysis

If running from the command line, the run entry under the .config file's setup field should be updated to the correct value. Trajectories of the image statistics and powder patterns can be generated by:

$ elog_submit.sh -c ../tutorial/mfxlv4920.yaml -t run_analysis -n ${ncores}

The associated figures will be saved to ${config.root_dir}/powder/fig/*_r{run:04}.png.

If the run is of silver behenate, this:

$ elog_submit.sh -c ../tutorial/mfxlv4920.yaml -t opt_distance -n 1

will estimate the distance to the detector and generate a new CrystFEL-style geometry file with the correct coffset parameter. A figure of the fit can be found at: ${config.root_dir}/geom/figs/r${run:04}.png.

These plots may give a sense of good parameters to use for generating a new, experiment-specific mask. The following:

$ elog_submit.sh -c ../tutorial/mfxlv4920.yaml -t build_mask -n 1

pulls a collection of random images from the run and masks pixels whose intensities fall outside the user-defined threshold values. This mask can be combined with a previous mask, and a set width of border pixels can be masked as well depending on the config task arguments.

Step 3: crystal analysis

If the run is a crystal sample, peak finding and indexing should be sequentially run:

$ elog_submit.sh -c ../tutorial/mfxlv4920.yaml -t find_peaks -n ${ncores}
$ elog_submit.sh -c ../tutorial/mfxlv4920.yaml -t index -n ${ncores}

Once stream files are generated, the following:

$ elog_submit.sh -c ../tutorial/mfxlv4920.yaml -t stream_analysis -n ${ncores}

can be used to produce the peakogram and cell distribution plots, which will be saved in: ${config.root_dir}/index/figs.