.. _version:

Mapping Versions to Data Releases
---------------------------------

Paths to data products often contain a version number either in the directory path or filename itself.  These
version numbers may be tagged pipeline reduction versions, or software post-processing versions.  Eventually these
files are frozen and released in a public Data Release (DR) or in a survey-specific internal product launch, e.g.
IPLs or MaNGA's MPLs.  It may be necessary to map these specific version numbers to release numbers so that tools
like ``sdss_access`` can understand where to look for files given a specific overall data release.  ``sdss_brain``
provides a method for doing so.

Version Metadata
^^^^^^^^^^^^^^^^

The metadata describing the various software version names and tags associated for each SDSS data release is
handled by the SDSS datamodel product.  ``sdss_brain`` contains convenience methods for accessing version
metadata either from a local copy of the ``datamodel`` product, or remotely via the SDSS valis API.  To
retrieve all the version metadata, use the `~.sdss_brain.datamodel.versions.get_versions` function.

::

    >>> from sdss_brain.datamodel import get_versions
    >>> vers = get_versions()
    {'IPL1': {'apred_vers': '1.0', 'v_astra': '0.2.6', 'run2d': 'v6_0_9', 'run1d': 'v6_0_9'},
     'DR18': {'run2d': 'v6_0_4', 'run1d': 'v6_0_4', 'v_speccomp': 'v1.4.3', 'v_targ': '1.0.1'},
     'DR17': {'run2d': 'v5_13_2', 'apred_vers': 'dr17', 'apstar_vers': 'stars', 'aspcap_vers': 'synspec_rev1',
              'results_vers': 'synspec_rev1', 'run1d': 'v5_13_2', 'drpver': 'v3_1_1', 'dapver': '3.1.0'},
    ...}


Accessing a Mapping
^^^^^^^^^^^^^^^^^^^
To access a mapping you can use the `~.sdss_brain.datamodel.versions.get_mapped_version` helper function.
It accepts as input the version reference name, and a release to lookup.  If no version name is specified,
it returns the entire mapping for the given release.
::

    >>> # access the entire version mapping for DR16
    >>> from sdss_brain.datamodel import get_mapped_version
    >>> get_mapped_version(release='DR16')
    {'run2d': 'v5_13_0', 'apred_vers': 'r12', 'apstar_vers': 'stars', 'aspcap_vers': 'l33',
    'results_vers': 'l33', 'run1d': 'v5_13_0', 'drpver': 'v2_4_3', 'dapver': '2.2.1'}

Specifiy a version reference name to extract only the individual item from the dictionary.
::

    >>> # access only the manga DRP version mapping for DR16
    >>> from sdss_brain.datamodel import get_mapped_version
    >>> get_mapped_version('drpver', release='DR16')
    'v2_4_3'

Using a Mapping
^^^^^^^^^^^^^^^

As mentioned, the utility of this is to extract version numbers needed by ``sdss_access`` given a single release.
Consider a defined class for MaNGA datacubes:
::

    from sdss_brain.core import Brain
    from sdssdb.sqlalchemy.mangadb import database
    from sdss_brain.datamodel import get_mapped_version


    class Cube(Brain):
        _db = database
        path_name = 'mangacube'

        def _parse_input(self, value):
            plateifu_pattern = re.compile(r'([0-9]{4,5})-([0-9]{4,9})')
            plateifu_match = re.match(plateifu_pattern, value)

            data = {'filename': None, 'objectid': None}
            # match on plate-ifu or else assume a filename
            if plateifu_match is not None:
                data['objectid'] = value

                # extract and set additional parameters
                self.plateifu = plateifu_match.group(0)
                self.plate, self.ifu = plateifu_match.groups(0)
            else:
                data['filename']
            return data

        def _set_access_path_params(self):
            drpver = get_mapped_version("drpver", release=self.release)
            self.path_params = {'plate': self.plate, 'ifu': self.ifu, 'drpver': drpver}

Inside the ``_set_access_path_params`` method we use ``get_mapped_version`` to access the
DRP version for the given release of the Cube.  Now as we load different cubes of different
releases, the correct versions and paths are updated.
::

    >>> # load a cube for DR16
    >>> cube = Cube('8485-1901', release='DR16')
    >>> cube.get_full_path()
    '/Users/Brian/Work/sdss/sas/dr16/manga/spectro/redux/v2_4_3/8485/stack/manga-8485-1901-LOGCUBE.fits.gz'

    >>> # load a cube for DR13
    >>> cube = Cube('8485-1901', release='DR13')
    >>> cube.get_full_path()
    '/Users/Brian/Work/sdss/sas/dr13/manga/spectro/redux/v1_5_4/8485/stack/manga-8485-1901-LOGCUBE.fits.gz'


Version Name Differences
^^^^^^^^^^^^^^^^^^^^^^^^

Sometimes the version name specified in an ``sdss_access`` path template and in a datamodel can be different.
This can happen due to the longetivity of SDSS, a lack of standards around naming conventions, and multiple
people contributing to the same project/code.  For example, a version name could be referenced as
``drpver`` in the datamodel, but as ``drp_ver`` or ``ver_drp`` or ``drpvers`` in various ``sdss_access``
path templates describing data products.  A real example is the version of the APOGEE pipeline is
often referenced as ``apred`` in older path templates, but ``apred_vers`` in the datamodel and in
newer path templates.

To accommodate these differences, aliases can be defined using the ``version_aliases`` parameter in the
`sdss_brain.yml` configuration file, which is a dictionary of parameters read in by ``sdss_brain``.  Each
key in ``version_aliases`` is a mapping between a version alias and the true datamodel version name.  Using
the above example, the entry in the config file would like that
::

    version_aliases:
      drp_ver: drpver
      ver_drp: drpver
      drpvers: drpver
      apred: apred_vers