.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_experiments/experimental_abinitio_pipeline_10028_jsb.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_experiments_experimental_abinitio_pipeline_10028_jsb.py: Abinitio Pipeline - Experimental Data EMPIAR 10028 ================================================== This notebook introduces a selection of components corresponding to loading real Relion picked particle cryo-EM data and running key ASPIRE-Python ab initio model components as a pipeline. This demonstrates reproducing results similar to those found in: .. admonition:: Publication | Common lines modeling for reference free Ab-initio reconstruction in cryo-EM | Journal of Structural Biology 2017 | https://doi.org/10.1016/j.jsb.2017.09.007 Specifically this pipeline uses the EMPIAR 10028 picked particles data, available here: https://www.ebi.ac.uk/empiar/EMPIAR-10028 .. GENERATED FROM PYTHON SOURCE LINES 25-28 Imports ------- Import packages that will be used throughout this experiment. .. GENERATED FROM PYTHON SOURCE LINES 28-39 .. code-block:: Python import logging from pathlib import Path from aspire.denoising import LegacyClassAvgSource from aspire.reconstruction import MeanEstimator from aspire.source import OrientedSource, RelionSource logger = logging.getLogger(__name__) .. GENERATED FROM PYTHON SOURCE LINES 40-47 Parameters --------------- Use of GPU is expected for a large configuration. If running on a less capable machine, or simply experimenting, it is strongly recommended to reduce ``img_size``, ``n_imgs``, and ``n_nbor``. .. GENERATED FROM PYTHON SOURCE LINES 47-67 .. code-block:: Python # Inputs # Note the published ``shiny_2sets.star`` requires removal of a stray '9' character on line 5476. starfile_in = "10028/data/shiny_2sets_fixed9.star" data_folder = "." # This depends on the specific starfile entries. pixel_size = 1.34 # Defined with the dataset from EMPIAR # Config n_imgs = None # Set to None for all images in starfile, can set smaller for tests. img_size = 179 # Downsample the images/reconstruction to a desired resolution n_classes = 3000 # How many class averages to compute. n_nbor = 50 # How many neighbors to stack # Outputs preprocessed_fn = f"10028_preprocessed_{img_size}px.star" class_avg_fn = f"10028_var_sorted_cls_avgs_m{n_nbor}_{img_size}px.star" oriented_fn = f"10028_oriented_class_averages_{img_size}px.star" volume_output_filename = f"10028_abinitio_c{n_classes}_m{n_nbor}_{img_size}px.mrc" .. GENERATED FROM PYTHON SOURCE LINES 68-80 Source data and Preprocessing ----------------------------- ``RelionSource`` is used to access the experimental data via a `STAR` file. Begin by preprocessing to correct for CTF, then downsample to ``img_size`` and apply noise correction. ASPIRE-Python has the ability to automatically adjust CTF filters for downsampling, and this can be employed simply by changing the order of preprocessing steps, saving time by phase flipping lower resolution images. However, this script intentionally follows the order described in the original publication. .. GENERATED FROM PYTHON SOURCE LINES 80-110 .. code-block:: Python # Create a source object for the experimental images src = RelionSource( starfile_in, pixel_size=pixel_size, max_rows=n_imgs, data_folder=data_folder ) # Use phase_flip to attempt correcting for CTF. # Caching is used throughout for speeding up large datasets on high memory machines. logger.info("Perform phase flip to input images.") src = src.phase_flip().cache() # Downsample the images. logger.info(f"Set the resolution to {img_size} X {img_size}") src = src.downsample(img_size).cache() # Normalize the background of the images. src = src.normalize_background().cache() # Estimate the noise and whiten based on the estimated noise. src = src.legacy_whiten().cache() # Optionally invert image contrast. logger.info("Invert the global density contrast") src = src.invert_contrast().cache() # Save the preprocessed images. # These can be reused to experiment with later stages of the pipeline # without repeating the preprocessing computations. src.save(preprocessed_fn, save_mode="single", overwrite=True) .. GENERATED FROM PYTHON SOURCE LINES 111-115 Class Averaging ---------------------- Now perform classification and averaging for each class. .. GENERATED FROM PYTHON SOURCE LINES 115-130 .. code-block:: Python logger.info("Begin Class Averaging") avgs = LegacyClassAvgSource(src, n_nbor=n_nbor).cache() # Save the entire set of class averages to disk so they can be re-used. avgs.save(class_avg_fn, save_mode="single", overwrite=True) # We'll continue our pipeline with the first ``n_classes`` from # ``avgs``. The classes will be selected by the ``class_selector`` of a # ``ClassAvgSource``, which in this case will be the class averages # having the largest variance. Note global sorting requires computing # all class averages, which is computationally intensive. avgs = avgs[:n_classes].cache() .. GENERATED FROM PYTHON SOURCE LINES 131-138 Common Line Estimation ---------------------- Estimate orientation of projections and assign to source by applying ``OrientedSource`` to the class averages from the prior step. By default this applies the Common Line with Synchronization Voting ``CLSync3N`` method. .. GENERATED FROM PYTHON SOURCE LINES 138-147 .. code-block:: Python logger.info("Begin Orientation Estimation") oriented_src = OrientedSource(avgs) # Save off the selected set of class average images, along with the # estimated orientations and shifts. These can be reused to # experiment with alternative volume reconstructions. oriented_src.save(oriented_fn, save_mode="single", overwrite=True) .. GENERATED FROM PYTHON SOURCE LINES 148-152 Volume Reconstruction ---------------------- Using the oriented source, attempt to reconstruct a volume. .. GENERATED FROM PYTHON SOURCE LINES 152-162 .. code-block:: Python logger.info("Begin Volume reconstruction") # Set up an estimator to perform the backprojection. estimator = MeanEstimator(oriented_src) # Perform the estimation and save the volume. estimated_volume = estimator.estimate() estimated_volume.save(volume_output_filename, overwrite=True) logger.info(f"Saved Volume to {str(Path(volume_output_filename).resolve())}") .. _sphx_glr_download_auto_experiments_experimental_abinitio_pipeline_10028_jsb.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: experimental_abinitio_pipeline_10028_jsb.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: experimental_abinitio_pipeline_10028_jsb.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: experimental_abinitio_pipeline_10028_jsb.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_