This is a convenience class with the MMAccess already implemented. This class initializes the MMAccess and provides logic to load data based on the data_origin. It also provides a simple repr.

In addition to any abstractmethod from the MMA, this class contains three abstractmethods you must override when subclassing.

_load_object_from_file: defines data load/handling from a local file
_load_object_from_db: defines data load/handling from a local database
_load_object_from_api: defines data load/handling from a remote API

Parameters

data_input (str) – The file or name of target data to load
filename (str) – The absolute filepath to data to load
objectid (str) – The object identifier of the data to load
mode (str) – The operating mode: auto, local, or remote
release (str) – The data release of the object, e.g. “DR16”
data (object) – Optional data to instantiate the object with
download (bool) – If True, downloads the object locally with sdss_access
ignore_db (bool) – If True, ignores any database connection for local access
use_db (DatabaseConnection) – a database connection to override the default with

Variables

_db (DatabaseConnection) – A relevant sdssdb database connection for the object
mapped_version (str) – The name of survey/category in the mapped_versions dictionary

mapped_version = None¶

Mixins¶

class sdss_brain.mixins.mma.MMAMixIn(data_input=None, filename=None, objectid=None, mode=None, release=None, download=None, ignore_db=False, use_db=None)[source]¶

Bases: abc.ABC

Mixin for implementing multi-modal data access

This is a mixin class that adds multi-modal data access to any class that subclasses from this one. The MMA allows toggling between local and remote data access modes, or leaving it on automatic. Local mode access tries to load data via a database, if one exists, otherwise it loads data via a local filepath. Remote mode will try to load data over an API. When the mode is set to “auto”, it automatically tries to first load things locally, and then remotely. Depending on the mode and logic, the MMA will set data_origin to either file, db, or api.

Note that this class does not provide the logic for loading data from a db, over an API, or from a file. The user must provide that logic in a subclass.

This mixin contains three abstractmethods you must override when subclassing.

_parse_inputs: provides logic to parse data_input into either filename or objectid
download: a method for downloading a data file to a local disk
get_full_path: a method for generating the absolute file path on disk to a file

Parameters

data_input (str) – The file or name of target data to load
filename (str) – The absolute filepath to data to load
objectid (str) – The object identifier of the data to load
mode (str) – The operating mode: auto, local, or remote
release (str) – The data release of the object, e.g. “DR16”
download (bool) – If True, downloads the object locally with sdss_access
ignore_db (bool) – If True, ignores any database connection for local access
use_db (DatabaseConnection) – a database connection to override the default with

Variables

release (str) – The current data release loaded

abstract download()[source]¶: Abstract method to download a file

abstract get_full_path()[source]¶: Abstract method to return a full local file path

property is_access_mixedin¶: Checks if the AccessMixIn is included

property release¶: Returns the release.

class sdss_brain.mixins.mma.MMAccess(*args, **kwargs)[source]¶

Bases: sdss_brain.mixins.access.AccessMixIn, sdss_brain.mixins.mma.MMAMixIn

Class that mixes in the sdss_access functionality with the MMA

Note that this class does not provide the logic for loading data from a db, over an API, or from a file. The user must provide that logic in a subclass.

This mixin contains two abstractmethods you must override when subclassing.

_set_access_path_params: sets the arguments needed by sdss_access
_parse_inputs: provides logic to parse data_input into either filename or objectid

Parameters

data_input (str) – The file or name of target data to load
filename (str) – The absolute filepath to data to load
objectid (str) – The object identifier of the data to load
mode (str) – The operating mode: auto, local, or remote
release (str) – The data release of the object, e.g. “DR16”
download (bool) – If True, downloads the object locally with sdss_access
ignore_db (bool) – If True, ignores any database connection for local access
use_db (DatabaseConnection) – a database connection to override the default with

Variables

release (str) – The current data release loaded
access (Access) – An instance of sdss_access using for all path creation and file downloads

class sdss_brain.mixins.access.AccessMixIn(*args, **kwargs)[source]¶

Bases: abc.ABC

Mixin for implementing multi-modal data access

This is a class that adds support for dynamic path operations using sdss_access. Given a template path name and a defined set of template keyword argument, provides convenience methods for constructing the full local or url-based pathname, downloading the file with sdss_access. Also provides the complete Access object as a property for full range of functionality. The access property automatically reconfigures itself according to the specified data release on each call.

This mixin contains one abstractmethod you must override when subclassing.

_set_access_path_params: sets the arguments needed by sdss_access

Parameters

release (str) – The data release of the object, e.g. “DR16”

Variables

path_name (str) – The sdss_access template path name
path_params (dict) – The set of sdss_access template path keyword arguments
access (Access) – An instance of sdss_access using for all path creation and file downloads

download()[source]¶: Download the file using sdss_access

get_full_path(url=None, force_file=None)[source]¶

Returns the full path of the file in the tree.

Parameters

url (bool) – If True, specifies the url location rather than the local file location
force_file (bool) – If True, explicitly returns any set filename attribute instead of constructing a path from keyword arguments.

Returns

fullpath (str) – The full path as built by sdss_access

property access¶: Returns an instance of Access

path_name: str = None¶

Helpers¶

sdss_brain.helpers.io.get_mapped_version(name: str, release: str = None, key: str = None) → Union[dict, str][source]¶

Get a version id mapped to a release number

For a given named category, looks up the “mapped_versions” attribute from the configuration yaml file and returns a version number that has been mapped to a specific release. For example, for manga, DR16 maps to drpver=’v2_4_3’ and dapver=’2.2.1’. This can be useful when needing to specify certain versions when defining paths to files.

Parameters

name (str) – The name of the set of versions to access
release (str) – The SDSS release. Default is config.release.
key (str) – Optional name of dictionary key to access specific value

Returns

version (dict|str) – A version number corresponding to a given release

Example

>>> # access the MaNGA versions for release DR16
>>> get_mapped_version('manga', release='DR16')
    {'drpver': 'v2_4_3', 'dapver': '2.2.1'}

>>> # access specific key
>>> get_mapped_version('manga', release='DR16', key='drpver')
    'v2_4_3'

sdss_brain.helpers.io.load_fits_file(filename: str) → astropy.io.fits.hdu.hdulist.HDUList[source]¶

Load a FITS file

Opens and loads a FITS file with astropy.io.fits.

Parameters: filename (str) – A FITS filen to open
Returns: hdulist (HDUList) – an Astropy HDUList

sdss_brain.helpers.parsing.create_object_pattern(regex: str = None, keys: list = None, keymap: dict = None, delimiter: str = '-', exclude: list = None, include: list = None, order: list = None) → str [source]¶

Create a regex pattern to parse data input by

Parameters

regex (str) – A custom regex pattern
keys (list) – A list of (access) names to build a pattern out of
keymap (dict) – A dict of key name and pattern to build a pattern out of
delimiter (str) – The delimiter to use when joining the keys. Default is “-“.
exclude (list) – A list of names to exclude from the keys
include (list) – A list of names to only include from the keys
order (list) – A list of names specifying the order in which to create the keyed pattern

Returns

pattern (str) – A regex pattern to use for parsing an objectid

sdss_brain.helpers.parsing.parse_data_input(value: str, regex: str = None, keys: list = None, keymap: dict = None, delimiter: str = '-', exclude: list = None, include: list = None, order: list = None, inputs: bool = False) → dict [source]¶

Parse data input for a filename or an object id

Parameters

value (str) – The input string to perform a pattern match on
regex (str) – A custom regex pattern
keys (list) – A list of (access) names to build a pattern out of
keymap (dict) – A dict of key name and pattern to build a pattern out of
delimiter (str) – The delimiter to use when joining the keys. Default is “-“.
exclude (list) – A list of names to exclude from the keys
include (list) – A list of names to only include from the keys
order (list) – A list of names specifying the order in which to create the keyed pattern
inputs (bool) – If True, returns the parser inputs. Default is False.

Returns

matches (dict) – A dict with keys “filename”, “objectid”, and any other matches

Example

>>> # parse a filename
>>> parse_data_input('/path/to/a/file.txt')
    {'filename': '/path/to/a/file.txt', 'objectid': None, 'parsed_groups': None}

>>> # parse an objectid as is
>>> parse_data_input('8485-1901')
    {'filename': None, 'objectid': '8485-1901', 'parsed_groups': ['8485-1901', '485-1901']}

>>> # parse an objectid using a custom pattern
>>> parse_data_input('8485-1901', regex=r'(?P<plate>\d{4,5})-(?P<ifu>\d{3,5})')
    {'filename': None, 'objectid': '8485-1901', 'plate': '8485', 'ifu': '1901', 'parsed_groups': ['8485-1901', '8485', '1901']}

>>> # parse an objectid using access keywords
>>> keys=['drpver', 'plate', 'ifu', 'wave']
>>> parse_data_input('v1-8485-1901-LOG', keys=keys)
    {'filename': None, 'objectid': 'v1-8485-1901-LOG', 'drpver': 'v1', 'plate': '8485',
     'ifu': '1901', 'wave': 'LOG', 'parsed_groups': ['v1-8485-1901-LOG', 'v1', '8485', '1901', 'LOG']}

>>> # parse an objectid specifying the input order of the keys
>>> parse_data_input('8485-1901', keys=keys, order=['plate', 'ifu'])
    {'filename': None, 'objectid': '8485-1901', 'plate': '8485', 'ifu': '1901', 'parsed_groups': ['8485-1901', '8485', '1901']}

sdss_brain.helpers.parsing.raw_parse(value: str, regex: str = None) → Union[dict, tuple][source]¶

Match a string via a regex pattern with no frills

Parameters

value (str) – The input string to match on
regex (str) – The regex pattern to use for matching

Returns

A matched group

sdss_brain.helpers.decorators.access_loader(kls=None, *, name=None, defaults={}, mapped_version=None)[source]¶

Class decorator to reduce boilerplate around setting of sdss_access parameters

Decorator to generate a default _set_access_path_params method given a template path name. The default method creates an empty path_params dictionary using the template keywords given a path name. Default values for template kwargs can be specified using the “defaults” argument.

mapped_version is a mapping key, “[mapping]:[property,]”, where [mapping] is the name of the key in the “mapped_version” attribute in the brain configuration yaml file, and [property,] is a list of version ids to become properties. See get_mapped_version for more details.

For example a key of “manga:drpver” creates a new read-only property called “drpver” and uses get_mapped_version to extract the correct version number for a given release from the “mapped_version[‘manga’]” key in ~sdss_brain.yaml.

Parameters

name (str) – The sdss_access template path name
defaults (dict) – Default values for the sdss_access template keyword arguments
mapped_version (str) – A mapping key to map a specific version onto a release, e.g. “manga:drpver”

Returns

kls (class) – The decorated class

sdss_brain.helpers.decorators.create_mapped_properties(kls, mapped_version)[source]¶

Create new read-only properties on a given class

Creates new read-only properties that extracts a specific version id to an input release. This allows the version id to be updated when the global “release” is changed. See get_mapped_version for more details. mapped_version is a mapping key, “[mapping]:[property,]”, where [mapping] is the name of the key in the “mapped_version” attribute in the brain configuration yaml file, and [property,] is a list of version ids to become properties.

Parameters

kls (Type) – The class object
mapped_version (str) – The mapping key to map a specific version onto a release

sdss_brain.helpers.decorators.get_parse_input(regex=None, keys=None, keymap=None, include=None, exclude=None, order=None, delimiter=None)[source]¶: Generate a default parse_input method to be attached to a class

sdss_brain.helpers.decorators.parser_loader(kls=None, *, pattern=None, keys=None, keymap=None, include=None, exclude=None, order=None, delimiter=None)[source]¶

Class decorator to reduce boilerplate around definition of parse_input method

Decorator to generate a default _parse_input method to be attached to a class. The created default method uses parse_data_input to match the input string value against the provided regex pattern and returns a dictionary containing the parsed “filename” or “objectid”. If the pattern is a named-group regex pattern the method extracts the names and adds them as attributes on the class. If no pattern is specified, the method checks for available sdss_access template keys and, if found, constructs a regex matching pattern from them. Otherwise, if no keys are found, it performs a greedy match and sets the result as the objectid. The return dictonary adds a parsed_groups attribute to the instance containing the match group output from the regex match. This allows the user to access the extracted matches when the input pattern is simple, i.e. containing no named groups or grouping regex structure.

Parameters

pattern (str) – The regex pattern to match with
keys (list) – Optional list of names to construct a named pattern. Default is to use any sdss_access keys.
keymap (dict) – Optional dict mapping names to specific patterns to use. Default is None.
include (list) – A list of access keywords to include in the objectid pattern creation
exclude (list) – A list of access keywords to exclude in the objectid pattern creation
order (list) – A list of access keywords to order the objectid pattern by
delimiter (str) – The delimiter to use when joining keys for the objectid pattern creation

Returns

kls (class) – The decorated class

sdss_brain.helpers.decorators.register(func)[source]¶: Decorator to add class decorators to the global registry

sdss_brain.helpers.decorators.sdss_loader(kls=None, *args, **kwargs)[source]¶

Class decorator that combines and applies other class decorators

This decorator applies the decorators specified in use_decorators to the input class instance. It effectively combines individual decorators allowing a global entry point for the keyword arguments. This is equivalvent to stacking decorators, e.g.:

@access_loader()
@parser_loader()
class B(object):
    pass

Parameters

args (list) – Any appropriate decorator arguments
kwargs (list) – Any appropriate decoratoy keyword arguments

Returns

The class that has been decorated

sdss_brain.helpers.decorators.use_decorators(*args)[source]¶: Decorator that adds the specified decorators to the function as an attribute