.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_tutorials/tutorials/micrograph_source.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_tutorials_tutorials_micrograph_source.py: ===================== Micrograph Sources ===================== This tutorial will demonstrate how to set up and use ASPIRE's ``MicrographSource`` classes. .. GENERATED FROM PYTHON SOURCE LINES 9-17 .. code-block:: Python import os import tempfile import numpy as np from aspire.source import ArrayMicrographSource .. GENERATED FROM PYTHON SOURCE LINES 18-28 Overview -------- ``MicrographSource`` is an abstract class which provides access to three distinct subclasses. The first two are ``ArrayMicrographSource`` and ``DiskMicrographSource`` which provide access to array and disk backed micrograph data respectively. ``MicrographSimulation`` takes a volume and generates projection images which are aggregated into synthetic microgaphs. The following illustrates an overview of the interfaces, and the tutorial will go on to demonstrate common operations for each class. .. GENERATED FROM PYTHON SOURCE LINES 30-92 .. mermaid:: classDiagram class MicrographSource{ micrograph_count: int micrograph_size: int dtype: np.dtype +asnumpy() +dtype +len() +repr() +images[] +micrograph_count +micrograph_size +save() +show() } class ArrayMicrographSource{ micrographs: np.ndarray } class DiskMicrographSource{ micrographs_path: str, Path, or list } class MicrographSimulation{ volume: Volume micrograph_size: Optional, int micrograph_count: Optional, int particles_per_micrograph: Optional, int particle_amplitudes: Optional, np.ndarray projection_angles: Optional, np.ndarray seed: Optional, int ctf_filters: Optional, list noise_adder: Optional, NoiseAdder boundary: Optional, int interparticle_distance: Optional, int +boundary +centers +ctf_filters +clean_images[] +filter_indices +get_micrograph_index() +get_particle_index() +interparticle_distance +noise_adder +simulation +particle_amplitudes +particle_box_size +particle_per_micrograph +projection_angles +total_particle_count +volume } MicrographSource <|-- ArrayMicrographSource MicrographSource <|-- DiskMicrographSource MicrographSource <|-- MicrographSimulation MicrographSimulation o-- Volume MicrographSimulation *-- CTFFilter MicrographSimulation *-- NoiseAdder .. GENERATED FROM PYTHON SOURCE LINES 95-101 Creating an ArrayMicrographSource --------------------------------- An ``ArrayMicrographSource`` is populated with an array. For this demonstration, random data will initialize the object, then this data will be saved off for use in the next example (which loads data from files). .. GENERATED FROM PYTHON SOURCE LINES 101-119 .. code-block:: Python # Create an (2,512,512) array of data. # This represents two (512,512) micrographs. mgs_np = np.random.rand(2, 512, 512) # Construct the source src = ArrayMicrographSource(mgs_np) # Create a tmp dir for saving the data to. # This just for ensuring the tutorial script is portable, tmp_dir = tempfile.TemporaryDirectory() # Save the data as multiple MRC files # This method returns a file_list, # which might be useful for loading or other operations. file_list = src.save(tmp_dir.name) .. GENERATED FROM PYTHON SOURCE LINES 120-124 Creating a DiskMicrographSource ------------------------------- A ``DiskMicrographSource`` is populated with str or list representing the location of MRC files. .. GENERATED FROM PYTHON SOURCE LINES 124-133 .. code-block:: Python from aspire.source import DiskMicrographSource # Load files in directory src = DiskMicrographSource(tmp_dir.name) # Load files from a list src = DiskMicrographSource(file_list) .. GENERATED FROM PYTHON SOURCE LINES 134-138 Creating a Micrograph Simulation -------------------------------- A ``MicrographSimulation`` is populated with particle projections from a ``Volume``, so we'll begin by generating a ``Volume``. .. GENERATED FROM PYTHON SOURCE LINES 138-150 .. code-block:: Python from aspire.source import MicrographSimulation from aspire.volume import AsymmetricVolume # Generate one (100,100,100) ``Volume``. vol = AsymmetricVolume( L=100, C=1, seed=1234, dtype=np.float32, ).generate() .. GENERATED FROM PYTHON SOURCE LINES 151-155 We'll pass our ``Volume`` as an argument and configure our ``MicrographSimulation``. In this example, the ``MicrographSimulation`` has 4 micrographs of size 1024, each with 10 particles. .. GENERATED FROM PYTHON SOURCE LINES 155-171 .. code-block:: Python n_particles_per_micrograph = 10 n_micrographs = 3 src = MicrographSimulation( vol, particles_per_micrograph=n_particles_per_micrograph, particle_amplitudes=1, micrograph_size=1024, micrograph_count=n_micrographs, seed=1234, ) # Plot the micrographs src.images[:].show() .. image-sg:: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_001.png :alt: micrograph source :srcset: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 172-178 CTF Filters ----------- By default, no CTF corruption is configured. To apply CTF filters, we have to pass them as arguments to the ``MicrographSimulation``. It is possible to apply a single CTF, different CTF per-micrograph or different CTF per-particle by configuring a list of matching size. .. GENERATED FROM PYTHON SOURCE LINES 178-199 .. code-block:: Python from aspire.operators import RadialCTFFilter # Create our CTF Filter and add it to a list. # This configuration will apply the same CTF to all particles. ctfs = [ RadialCTFFilter(pixel_size=4, voltage=200, defocus=15000, Cs=2.26, alpha=0.07, B=0), ] src = MicrographSimulation( vol, particles_per_micrograph=n_particles_per_micrograph, micrograph_size=1024, micrograph_count=n_micrographs, ctf_filters=ctfs, seed=1234, ) # Plot the micrographs src.images[:].show() .. image-sg:: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_002.png :alt: micrograph source :srcset: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 200-204 Noise ----- By default, no noise corruption is configured. To apply noise, pass a ``NoiseAdder`` to ``MicrographSimulation``. .. GENERATED FROM PYTHON SOURCE LINES 204-224 .. code-block:: Python from aspire.noise import WhiteNoiseAdder # Create our noise using WhiteNoiseAdder noise = WhiteNoiseAdder(4e-3, seed=1234) # Add noise to our MicrographSimulation using the noise_adder argument src = MicrographSimulation( vol, noise_adder=noise, particles_per_micrograph=n_particles_per_micrograph, micrograph_size=1024, micrograph_count=n_micrographs, ctf_filters=ctfs, seed=1234, ) # Plot the micrographs src.images[:].show() .. image-sg:: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_003.png :alt: micrograph source :srcset: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_003.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 225-226 Plot the clean micrographs using the ``clean_images`` accessor. .. GENERATED FROM PYTHON SOURCE LINES 226-228 .. code-block:: Python src.clean_images[:].show() .. image-sg:: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_004.png :alt: micrograph source :srcset: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_004.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 229-235 Interparticle Distance ---------------------- By default, particle distance is set to avoid collisions. We can use the ``interparticle_distance`` argument to control the minimum distance between particle centers. However, setting this argument too large may generate insufficient centers. .. GENERATED FROM PYTHON SOURCE LINES 235-254 .. code-block:: Python # Let's increase the number of particles to show overlap. n_particles_per_micrograph = 50 # Set the interparticle distance to 1, which adds at least one pixel # of separation between center and allows particles to collide. src = MicrographSimulation( vol, interparticle_distance=1, noise_adder=noise, particles_per_micrograph=n_particles_per_micrograph, micrograph_size=1024, micrograph_count=n_micrographs, ctf_filters=ctfs, ) # Plot the micrographs src.images[:].show() .. image-sg:: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_005.png :alt: micrograph source :srcset: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_005.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 255-262 Boundary -------- By default, the boundary is set to half of the particle width, which will completely contain every particle inside the micrograph. Setting ``boundary=0`` will allow particles to be placed along the edges. Positive values (measured in pixels) move the boundaries inward, while negative values move the boundaries outward. .. GENERATED FROM PYTHON SOURCE LINES 262-279 .. code-block:: Python # Create a micrograph with a negative boundary, allowing particles to # generate outward. out_src = MicrographSimulation( vol, boundary=-20, interparticle_distance=1, noise_adder=noise, particles_per_micrograph=n_particles_per_micrograph, micrograph_size=1024, micrograph_count=n_micrographs, ctf_filters=ctfs, ) # Plot the micrographs out_src.images[:].show() .. image-sg:: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_006.png :alt: micrograph source :srcset: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_006.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 280-288 Particle Indices ---------------- Each particle comes from a ``Simulation`` internal to ``MicrographSimulation``. This simulation can be accessed directly by the attribute ``MicrographSimulation.simulation``. A map is provided between each particle's indexing relative to that ``Simulation`` and micrograph based indexing. This relationship is demonstrated below. .. GENERATED FROM PYTHON SOURCE LINES 288-296 .. code-block:: Python # Let's choose four random numbers as our global (``Simulation``) # particle indices from ``test_micrograph=1``. test_micrograph = 1 n_particles = 3 local_particle_indices = np.random.choice(n_particles_per_micrograph, n_particles) print(f"Local particle indices: {local_particle_indices}") .. rst-class:: sphx-glr-script-out .. code-block:: none Local particle indices: [24 20 36] .. GENERATED FROM PYTHON SOURCE LINES 297-300 We can obtain the individual particle images from our ``MicrographSimulation`` by retrieving their centers and plotting the boundary boxes. .. GENERATED FROM PYTHON SOURCE LINES 300-328 .. code-block:: Python centers = np.zeros((n_particles, 2), dtype=int) for i in range(n_particles): centers[i] = src.centers[test_micrograph][local_particle_indices[i]] # Let's use the particles' centers and sizes to perform "perfect # particle picking" on this test micrograph. p_size = src.particle_box_size micrograph_picked_particles = np.zeros( ( n_particles, src.particle_box_size, src.particle_box_size, ) ) for i, center in enumerate(centers): x, y = center[0], center[1] # Calculate the square of the particle particle = src.clean_images[test_micrograph].asnumpy()[0][ x - p_size // 2 : x + p_size // 2, y - p_size // 2 : y + p_size // 2 ] micrograph_picked_particles[i] = particle # Let's plot and look at the particles! from aspire.image import Image Image(micrograph_picked_particles)[:].show() .. image-sg:: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_007.png :alt: micrograph source :srcset: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_007.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 329-332 .. note:: There may be overlap with nearby particles in the above images. To reduce overlap, increase ``interparticle_distance``. .. GENERATED FROM PYTHON SOURCE LINES 335-337 Let's find the images from the ``Simulation`` using the ``get_particle_indices`` method to retrieve their global indices. .. GENERATED FROM PYTHON SOURCE LINES 337-346 .. code-block:: Python global_particle_indices = np.zeros((n_particles), dtype=int) for i in range(n_particles): global_particle_indices[i] = src.get_particle_indices( test_micrograph, local_particle_indices[i] ) # Plot the simulation's images src.simulation.images[global_particle_indices].show() .. image-sg:: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_008.png :alt: micrograph source :srcset: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_008.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 347-349 We can check if these global indices match our local particle indices with the ``get_micrograph_index`` method. .. GENERATED FROM PYTHON SOURCE LINES 349-359 .. code-block:: Python check_local_indices = np.zeros((n_particles), dtype=int) for i in range(n_particles): # Get each particle's corresponding micrograph index and local particle index micrograph_index, check_local_indices[i] = src.get_micrograph_index( global_particle_indices[i] ) assert micrograph_index == 1 np.testing.assert_array_equal(local_particle_indices, check_local_indices) print(f"Local particle indices: {check_local_indices}") .. rst-class:: sphx-glr-script-out .. code-block:: none Local particle indices: [24 20 36] .. GENERATED FROM PYTHON SOURCE LINES 360-370 Saving a MicrographSimulation ----------------------------- In addition to saving the raw MRC files, ``MicrographSimulation`` populates STAR files with the particle centers, particle box size (``rlnImageSize``), and projection rotations. Additionally, CTF parameters are saved when CTF is used in the simulation. Each micrograph will have a corresponidng STAR file. The collection of these files are returned from ``MicrographSimulation.save`` as a list of tuples which is designed to work directly with ``CentersCoordinateSource``. .. GENERATED FROM PYTHON SOURCE LINES 370-376 .. code-block:: Python from aspire.source import CentersCoordinateSource # Save the simulation results = src.save(os.path.join(tmp_dir.name, "mg_sim")) .. GENERATED FROM PYTHON SOURCE LINES 377-381 .. code-block:: Python # Review the resulting files print(results) .. rst-class:: sphx-glr-script-out .. code-block:: none [('/tmp/tmprlauckns/mg_sim/micrograph_0.mrc', '/tmp/tmprlauckns/mg_sim/micrograph_0.star'), ('/tmp/tmprlauckns/mg_sim/micrograph_1.mrc', '/tmp/tmprlauckns/mg_sim/micrograph_1.star'), ('/tmp/tmprlauckns/mg_sim/micrograph_2.mrc', '/tmp/tmprlauckns/mg_sim/micrograph_2.star')] .. GENERATED FROM PYTHON SOURCE LINES 382-387 .. code-block:: Python # Review the example STAR file contents with open(results[0][1], "r") as f: print(f.read()) .. rst-class:: sphx-glr-script-out .. code-block:: none data_ loop_ _rlnImageName _rlnImageSize _rlnCoordinateX _rlnCoordinateY _rlnAngleRot _rlnAngleTilt _rlnAnglePsi _rlnVoltage _rlnDefocusU _rlnDefocusV _rlnDefocusAngle _rlnSphericalAberration _rlnAmplitudeContrast 000001@micrograph_0 100 357 514 225.17323 70.42997 85.89395 200.0 15000.0 15000.0 0.0 2.26 0.07 000002@micrograph_0 100 118 467 67.95413 99.76551 37.444767 200.0 15000.0 15000.0 0.0 2.26 0.07 000003@micrograph_0 100 590 277 7.2576475 140.38687 300.22638 200.0 15000.0 15000.0 0.0 2.26 0.07 000004@micrograph_0 100 522 248 63.480827 84.10157 7.4896197 200.0 15000.0 15000.0 0.0 2.26 0.07 000005@micrograph_0 100 223 745 229.15794 21.15329 197.15344 200.0 15000.0 15000.0 0.0 2.26 0.07 000006@micrograph_0 100 174 445 260.969 149.42964 175.6943 200.0 15000.0 15000.0 0.0 2.26 0.07 000007@micrograph_0 100 299 59 47.95209 136.74622 193.82863 200.0 15000.0 15000.0 0.0 2.26 0.07 000008@micrograph_0 100 659 434 91.8869 100.16265 202.57196 200.0 15000.0 15000.0 0.0 2.26 0.07 000009@micrograph_0 100 117 647 329.64502 75.08341 13.788601 200.0 15000.0 15000.0 0.0 2.26 0.07 000010@micrograph_0 100 735 339 46.58517 129.15901 187.30682 200.0 15000.0 15000.0 0.0 2.26 0.07 000011@micrograph_0 100 491 789 110.339386 99.2707 85.04628 200.0 15000.0 15000.0 0.0 2.26 0.07 000012@micrograph_0 100 809 818 336.5936 52.53211 220.0516 200.0 15000.0 15000.0 0.0 2.26 0.07 000013@micrograph_0 100 436 431 134.82375 125.791466 127.273315 200.0 15000.0 15000.0 0.0 2.26 0.07 000014@micrograph_0 100 212 212 88.836 131.61661 170.09749 200.0 15000.0 15000.0 0.0 2.26 0.07 000015@micrograph_0 100 374 937 71.1484 50.87036 117.835976 200.0 15000.0 15000.0 0.0 2.26 0.07 000016@micrograph_0 100 159 101 63.209724 61.319878 215.21576 200.0 15000.0 15000.0 0.0 2.26 0.07 000017@micrograph_0 100 727 717 171.41907 103.669914 162.73087 200.0 15000.0 15000.0 0.0 2.26 0.07 000018@micrograph_0 100 434 566 148.2073 23.667522 192.28221 200.0 15000.0 15000.0 0.0 2.26 0.07 000019@micrograph_0 100 514 940 124.225586 101.704155 352.64386 200.0 15000.0 15000.0 0.0 2.26 0.07 000020@micrograph_0 100 644 172 242.63022 54.44209 61.728706 200.0 15000.0 15000.0 0.0 2.26 0.07 000021@micrograph_0 100 758 943 197.08356 43.91972 100.298325 200.0 15000.0 15000.0 0.0 2.26 0.07 000022@micrograph_0 100 394 628 359.41455 75.2847 220.48091 200.0 15000.0 15000.0 0.0 2.26 0.07 000023@micrograph_0 100 87 144 171.61751 15.32889 111.023254 200.0 15000.0 15000.0 0.0 2.26 0.07 000024@micrograph_0 100 204 235 326.69586 57.931633 271.72266 200.0 15000.0 15000.0 0.0 2.26 0.07 000025@micrograph_0 100 775 436 24.332895 108.68852 9.214272 200.0 15000.0 15000.0 0.0 2.26 0.07 000026@micrograph_0 100 702 85 166.2759 55.460697 243.91948 200.0 15000.0 15000.0 0.0 2.26 0.07 000027@micrograph_0 100 816 257 342.33093 164.46213 154.5533 200.0 15000.0 15000.0 0.0 2.26 0.07 000028@micrograph_0 100 245 494 205.81186 160.12997 93.87625 200.0 15000.0 15000.0 0.0 2.26 0.07 000029@micrograph_0 100 222 116 39.215176 119.0958 321.86478 200.0 15000.0 15000.0 0.0 2.26 0.07 000030@micrograph_0 100 499 733 108.81932 61.365723 181.91663 200.0 15000.0 15000.0 0.0 2.26 0.07 000031@micrograph_0 100 947 736 109.85279 42.58259 210.66025 200.0 15000.0 15000.0 0.0 2.26 0.07 000032@micrograph_0 100 134 621 36.03223 105.68081 324.1332 200.0 15000.0 15000.0 0.0 2.26 0.07 000033@micrograph_0 100 106 707 312.10165 33.21698 55.839603 200.0 15000.0 15000.0 0.0 2.26 0.07 000034@micrograph_0 100 233 487 336.63525 81.33271 271.0917 200.0 15000.0 15000.0 0.0 2.26 0.07 000035@micrograph_0 100 576 110 79.40814 74.74897 337.01068 200.0 15000.0 15000.0 0.0 2.26 0.07 000036@micrograph_0 100 439 755 267.8921 65.83309 255.31674 200.0 15000.0 15000.0 0.0 2.26 0.07 000037@micrograph_0 100 395 504 204.60135 136.76486 181.031 200.0 15000.0 15000.0 0.0 2.26 0.07 000038@micrograph_0 100 738 721 47.220955 58.597233 144.94357 200.0 15000.0 15000.0 0.0 2.26 0.07 000039@micrograph_0 100 952 243 78.088165 78.69344 2.2200234 200.0 15000.0 15000.0 0.0 2.26 0.07 000040@micrograph_0 100 631 543 220.26422 161.4182 308.45044 200.0 15000.0 15000.0 0.0 2.26 0.07 000041@micrograph_0 100 903 635 317.36963 141.55984 150.86116 200.0 15000.0 15000.0 0.0 2.26 0.07 000042@micrograph_0 100 599 527 276.6933 48.701714 132.33043 200.0 15000.0 15000.0 0.0 2.26 0.07 000043@micrograph_0 100 215 962 188.79645 59.49243 256.79306 200.0 15000.0 15000.0 0.0 2.26 0.07 000044@micrograph_0 100 120 455 274.22302 16.362684 58.064697 200.0 15000.0 15000.0 0.0 2.26 0.07 000045@micrograph_0 100 135 738 183.78815 115.328545 255.8863 200.0 15000.0 15000.0 0.0 2.26 0.07 000046@micrograph_0 100 937 717 356.82742 117.60184 176.06554 200.0 15000.0 15000.0 0.0 2.26 0.07 000047@micrograph_0 100 71 684 257.85233 63.155346 49.60995 200.0 15000.0 15000.0 0.0 2.26 0.07 000048@micrograph_0 100 644 568 130.81349 123.28017 78.281044 200.0 15000.0 15000.0 0.0 2.26 0.07 000049@micrograph_0 100 672 915 146.05128 137.0732 329.7856 200.0 15000.0 15000.0 0.0 2.26 0.07 000050@micrograph_0 100 130 103 86.410545 123.06443 121.30503 200.0 15000.0 15000.0 0.0 2.26 0.07 .. GENERATED FROM PYTHON SOURCE LINES 388-395 .. code-block:: Python img_src = CentersCoordinateSource(results, src.particle_box_size) # Show the first five images from the image source. img_src.images[:3].show() # Cleanup the tmp_dir tmp_dir.cleanup() .. image-sg:: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_009.png :alt: micrograph source :srcset: /auto_tutorials/tutorials/images/sphx_glr_micrograph_source_009.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 13.427 seconds) .. _sphx_glr_download_auto_tutorials_tutorials_micrograph_source.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: micrograph_source.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: micrograph_source.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: micrograph_source.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_