Setting Work Versions¶

sdss_brain can work with data files not yet explicitly released in a public or internal data release by using what are called “work” versions, attached to a “work” release. “Work” versions are survey specific version numbers needed to identify file paths typically listed under the SAS “sdsswork” directory or module, or the equivalent survey-specific work directories, e.g. “ebosswork”, “apogeework”, “mangawork”. For example, work versions might be run2d = v5_10_0 for EBOSS data, apred = r12 for APOGEE data, or drpver = v2_4_3, dapver = 2.2.1 for MaNGA data.

Defining Work Versions¶

Specifying release="WORK" and these versions tells the Brain which files to look for when operating under a work directory. Work versions take a dictionary format with key:value set to the version_name:version_number, where version_name is a valid version name normally as specified by the sdss_access path template keyword argument. version_number is a valid version number. There are several ways to set valid work versions.

By the config class:

>>> # set the new work versions
>>> from sdss_brain.config import config
>>> config.set_work_versions({'drpver': 'v2_4_3', 'run2d': 'v5_10_0', 'apred': 'r12'})

or by using the configuration file to set them permanently. See The Custom Config File for an example of how to set the work versions. You can also do so individually on a tool.

>>> from sdss_brain.tools import Eboss
>>> e=Eboss('3606-55182-0537', version={'run2d': 'v5_10_0'})

Version Order Precedence¶

The order of precendence the Brain uses when deciding which work versions to use in a given tool is tool > setting the config > using the custom config file. Specifying versions explicitly on a tool takes precedence over using the config set_work_versions method which takes precedence over any values found in the custom sdss_brain.yml configuration file.

Setting the Work Release¶

Additionally to “work” versions, the global, or tool, release attribute must be set to “WORK” rather than a data release ID. You can set the work release globally.

>>> from sdss_brain.config
>>> config.set_release('work')

Or individually on each tools.

>>> from sdss_brain.tools import Eboss
>>> e=Eboss('3606-55182-0537', release='WORK')

Example Access of “Work” Data¶

Here is an example highlighting how to access different work versions for an EBOSS spec-lite file. First we set our default work versions using the config class, and set our release to “work”.

>>> # set the new work versions
>>> from sdss_brain.config import config
>>> config.set_work_versions({'drpver': 'v2_4_3', 'run2d': 'v5_10_0', 'apred': 'r12'})
>>> config.work_versions
{'drpver': 'v2_4_3', 'dapver': '2.2.1', 'apred': 'r12', 'run2d': 'v5_10_0'}

>>> # set the global release
config.set_release("work")

Now we import the Eboss spectrum tool, and load some data. We can see the full file path is set to use the ebosswork directory, with run2d version set to v5_10_0.

>>> from sdss_brain.tools import Eboss
>>> e = Eboss('3606-55182-0537')
>>> e
<Eboss objectid='3606-55182-0537', mode='local', data_origin='file', lite=True>

>>> # check the filename path
>>> e.get_full_path()
'/Users/Brian/Work/sdss/sas/ebosswork/eboss/spectro/redux/v5_10_0/spectra/lite/3606/spec-3606-55182-0537.fits'

>>> # check the version number
>>> e.run2d
'v5_10_0'

We can explicitly change the work version accessed by specifying the version keyword as input. This lets us load a different work file without changing the default work version set. Let’s load the same file but for run2d = v5_13_0.

>>> from sdss_brain.tools import Eboss
>>> e = Eboss('3606-55182-0537', version={'run2d': 'v5_13_0')
>>> e
<Eboss objectid='3606-55182-0537', mode='remote', data_origin='api', lite=True>

>>> # check the filename path
>>> e.get_full_path()
'/Users/Brian/Work/sdss/sas/ebosswork/eboss/spectro/redux/v5_13_0/spectra/lite/3606/spec-3606-55182-0537.fits'

>>> # check the version number
>>> e.run2d
'v5_13_0'

Note

Even though the object is instantiated in “remote” mode with data_origin set to API, you can still construct a valid file path using get_full_path which uses sdss_access under the hood which specifies what would the file location be. Checking e.filename in the above will verify that the file does not actually exist locally. You can always download the file with e.download().

Work Authentication¶

Accessing unreleased “work” data requires proper SDSS authentication. For remote access with sdss_access, requests, httpx or any HTTP request library, this uses the ~/.netrc file. Any attempt to access data without a properly set .netrc file will result in an error.

>>> from sdss_brain.tools import Eboss
>>> e = Eboss('3606-55182-0537', version={'run2d': 'v5_13_0')
HTTPStatusError: 401 Client Error: Unauthorized for url: https://data.sdss.org/sas/ebosswork/eboss/spectro/redux/v5_13_0/spectra/lite/3606/spec-3606-55182-0537.fits
For more information check: https://httpstatuses.com/401

See Netrc Authentication for how to set up a .netrc file for authentication to access proprietary SDSS data content.