Note
Go to the end to download the full example code.
ASPIRE Configuration¶
This tutorial reviews the default ASPIRE configuration and common patterns for overriding.
Default Configuration¶
ASPIRE uses the confuse library for managing configuration. While this document should cover common uses, advanced users and developers should consider reviewing their documentation.
The ASPIRE package ships with config_default.yaml
.
This represents a base configuration.
The shipped configuration for this version of ASPIRE is:
version: 0.13.0
common:
# numeric module to use - one of numpy/cupy
numeric: numpy
# fft backend to use - one of pyfftw/scipy/cupy/mkl
fft: scipy
# Set cache directory for ASPIRE example data.
# By default the cache location will be set by pooch.os_cache(),
# which sets cache based on operating system as follows:
# Mac: ~/Library/Caches/<AppName>
# Unix: ~/.cache/<AppName> or the value of the XDG_CACHE_HOME environment variable, if defined.
# Windows: C:\Users\<user>\AppData\Local\<AppAuthor>\<AppName>\Cache
cache_dir: ""
logging:
# Set log_dir to a relative or absolute directory
# Default is a subfolder `logs` in your current working directory.
log_dir: logs
log_exceptions: 1
# tqdm_disable=True disables progress bars.
tqdm_disable: False
# set Python logging level: DEBUG, INFO, WARNING, ERROR, CRITICAL
# logging level for console output:
console_level: INFO
# logging level for log file output:
log_file_level: DEBUG
# custom log file name prefix can be set using 'log_prefix'
# will appear as '[log_dir]/[log_prefix]-{timestamp}.log'
log_prefix: aspire
nufft:
# NUFFT backends should be one of finufft, cufinufft, pynfft.
# They will be attempted in order from left to right.
backends: [cufinufft, finufft, pynfft]
ray:
# Ray will default to a OS specific tmp dir.
# By default on linux this is `/tmp/ray`.
# If you find your machine has a very small /tmp,
# try setting `temp_dir` to `/dev/shm`
# or some other fast scratch dir.
temp_dir: /tmp/ray
System Overrides¶
From here we can override with a custom config file in your home dir,
specifically $HOME/.config/ASPIRE/config.yaml
on most Linux platforms.
Items in this file will take precedence over the default configuration.
For other platforms, refer to the confuse documentation.
As an example, suppose you want to change the ray
temp_dir
variable
when working on a specific machine.
By creating $HOME/.config/ASPIRE/config.yaml
with the following contents
on that machine, ASPIRE’s configuration utility will overload
the temp_dir
directory from a /tmp/ray
folder to /scratch/tmp/ray
.
ray: temp_dir: /scratch/tmp/ray
Override Configuration Directory¶
Users may specify a directory containing the configuration file.
This is done by using the enviornment variable ASPIREDIR
If you wanted a file in your working directory to take
precedence over system-overrides, we can create a local config.yaml
.
Suppose you want to store ASPIRE logs at /tmp/my_proj/aspire_logs
when working on a specific project. Create the following config.yaml
.
logging: log_dir: /tmp/my_proj/aspire_logs
This directory must then be set before invoking any code.
export ASPIREDIR=$PWD
Similarly, you could specify any directory you like that might contain a configuration. This allows you to store configurations for reuse.
In-Code Overrides¶
You can also specify your own file from an arbitrary location from within Python code. For precise behavior refer to the confuse documentation.
aspire.config.set_file('/path/to/some_experimental_config.yaml')
Or simply set specific variables as needed.
Here we will disable progress bars displayed by
aspire.utils.trange
and aspire.utils.tqdm
.
import time
from aspire import config
from aspire.utils import trange
# Progress bars are displayed by default.
print("Disabling progress bars")
config["logging"]["tqdm_disable"] = True
for _ in trange(3):
time.sleep(1)
print("Done Loop 1\n")
print("Re-enabling progress bars")
config["logging"]["tqdm_disable"] = False
for _ in trange(3):
time.sleep(1)
print("Done Loop 2\n")
Disabling progress bars
Done Loop 1
Re-enabling progress bars
0%| | 0/3 [00:00<?, ?it/s]
33%|███▎ | 1/3 [00:01<00:02, 1.00s/it]
67%|██████▋ | 2/3 [00:02<00:01, 1.00s/it]
100%|██████████| 3/3 [00:03<00:00, 1.00s/it]
100%|██████████| 3/3 [00:03<00:00, 1.00s/it]
Done Loop 2
Enabling GPU Acceleration¶
Enabling GPU acceleration requires installing supporting software
packages and small config changes. Installing the supporting
software is most easily accomplished by installing ASPIRE with one
of the published GPU extensions, for example pip install
"aspire[dev,gpu_12x]"
. Once the packages are installed users
should find that the NUFFT calls are automatically running on the
GPU. Additional acceleration is achieved by enabling cupy for
numeric and fft components.
common: # numeric module to use - one of numpy/cupy numeric: cupy # fft backend to use - one of pyfftw/scipy/cupy/mkl fft: cupy
Alternatively, like other config options, this can be changed dynamically with code.
from aspire import config config["common"]["numeric"] = "cupy" config["common"]["fft"] = "cupy"
Resolution¶
ASPIRE logs the config_dir()
for your system at startup,
along with the configuration sources and resolved configuration at import time.
This should give an accurate snapshot of the configuration before any in-code overrides.
To view these as saved in your log, you will need to locate your log_dir.
If you are not sure where it is, we can ask the config:
import aspire
print(aspire.config["logging"]["log_dir"].as_filename())
/home/runner/work/ASPIRE-Python/ASPIRE-Python/gallery/tutorials/logs
You can also resolve the config in code
print(aspire.config.dump())
logging:
tqdm_disable: no
log_dir: /home/runner/work/ASPIRE-Python/ASPIRE-Python/gallery/tutorials/logs
log_exceptions: 1
console_level: INFO
log_file_level: DEBUG
log_prefix: aspire
common:
cache_dir: /home/runner/.cache/ASPIRE-data
numeric: numpy
fft: scipy
version: 0.13.0
nufft:
backends: [cufinufft, finufft, pynfft]
ray:
temp_dir: /tmp/ray
Logging preferences¶
The “Logging” section contains options for controlling the verbosity and destination of log messages generated by ASPIRE.
By default, the log_dir
is a directory called logs
within the current working directory from which ASPIRE was invoked, but
it can be changed to an absolute path (for instance, to go to the same folder as a custom configuration file stored elsewhere, such
as in the example above):
logging: log_dir=/tmp/my_proj/aspire_logs
ASPIRE directs output streams to both the console and to a log file during each session. The logging verbosity for both of these
can be specified individually via console_level
and log_file_level
. These levels are passed through to Python’s logging
system, so the labels are the same: "DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"
. In general, it may make sense to have the log file
store more verbose output than the console. By default, only INFO
and above level messages are printed to the terminal, while
detailed DEBUG
messages are saved to disk:
logging: console_level: INFO log_file_level: DEBUG
The following modification would save all DEBUG
messages to the log file, but only print ERROR
and CRITICAL
messages
in the terminal:
logging: console_level: ERROR log_file_level: DEBUG
By default, the filename of the log file is aspire-{%Y-%m-%dT%H-%M-%S.%f}.log
. The prefix before the timestamp can be customized
via the log_prefix
option.
More details on logging behavior in Python can be found in the Python logging HOWTO.
Advanced logging options¶
Users with sophisticated knowledge of Python’s logging
library are able to directly work with the logging.conf
file stored in
src/aspire
in the source.
[loggers]
keys = root
[handlers]
keys = consoleHandler, fileHandler
[formatters]
keys = simpleFormatter
[logger_root]
level = DEBUG
handlers = consoleHandler, fileHandler
[handler_consoleHandler]
class = StreamHandler
level = %(console_level)s
formatter = simpleFormatter
args = (sys.stdout,)
[handler_fileHandler]
class = FileHandler
level = %(log_file_level)s
formatter = simpleFormatter
# You may set a custom filename here:
args =(os.path.join('%(log_dir)s', '%(log_prefix)s-%(dt_stamp)s.log'),)
[formatter_simpleFormatter]
format = %(asctime)s %(levelname)s [%(name)s] %(message)s
This file exposes options that ASPIRE’s config.yaml
does not, for example the precise formatting options for log messages. However,
values from ASPIRE’s configuration are passed through this file, so caution is needed when modifying it.
Total running time of the script: (0 minutes 7.173 seconds)