.. doctest-skip-all .. _whatsnew-3.0: ************************** What's New in Astropy 3.0? ************************** Overview ======== Astropy 3.0 is a major release that adds significant new functionality since the 2.0.x series of releases. It is the first version that supports only Python 3. In particular, this release includes: * :ref:`whatsnew-3.0-velocities-skycoord-vel` * :ref:`whatsnew-3.0-velocities-skycoord-space-motion` * :ref:`whatsnew-3.0-constants-contextmanager` * :ref:`whatsnew-3.0-quantities-scipy-ufuncs` * :ref:`whatsnew-3.0-showtable` * :ref:`whatsnew-3.0-hdf5-mixins` * :ref:`whatsnew-3.0-fits-time-support` * :ref:`whatsnew-3.0-fits-performance` * :ref:`whatsnew-3.0-lomb-scargle-false-alarm` * :ref:`whatsnew-3.0-ascii-chunks` * :ref:`whatsnew-3.0-python3` * :ref:`whatsnew-3.0-pytest-plugins` In addition to these major changes, Astropy 3.0 includes a large number of smaller improvements and bug fixes, which are described in the :ref:`changelog`. By the numbers: * 757 issues have been closed since v2.0 * 399 pull requests have been merged since v2.0 * 253 distinct people have contributed code .. _whatsnew-3.0-python3: Astropy now only supports Python 3 ================================== Astropy v3.0 is the first release to support only Python 3. Python 2.7 users will be able to continue downloading and installing older releases of Astropy, and we will continue to release new v2.0.x versions (which fix bugs but don't add new features compared to v2.0) until the end of 2019 (see `the APE10 document `_ for more details). .. _whatsnew-3.0-velocities-skycoord-vel: Full support for velocities in ``astropy.coordinates`` ====================================================== The :class:`~astropy.coordinates.SkyCoord` object now supports representing and transforming velocity components along with positional coordinates. For example, to transform equatorial (ICRS) proper motions to the Galactic coordinate system:: >>> import astropy.units as u >>> import astropy.coordinates as coord >>> c = coord.SkyCoord(ra=86.7 * u.deg, dec=53.09 * u.deg, ... pm_ra_cosdec=112*u.mas/u.yr, ... pm_dec=35*u.mas/u.yr) >>> c.galactic # doctest: +FLOAT_CMP .. _whatsnew-3.0-velocities-skycoord-space-motion: Support for position updates based on source motion for ``SkyCoord`` objects ============================================================================ :class:`~astropy.coordinates.SkyCoord` now also has a method for computing the position of a source at a new epoch, given position and some velocity data at a specified epoch. For example, the reference epoch for the Tycho-Gaia Astrometric Solution (TGAS) is J2015.0, but we might want to compute the position of a source at a prior epoch to cross-match to an old catalog, or compute the position at a future epoch to observe the source. We can do this with :meth:`~astropy.coordinates.SkyCoord.apply_space_motion`:: >>> from astropy.time import Time >>> c = coord.SkyCoord(ra=86.7 * u.deg, dec=53.09 * u.deg, ... distance=100*u.pc, ... pm_ra_cosdec=112*u.mas/u.yr, ... pm_dec=35*u.mas/u.yr, ... obstime=Time('J2015.0')) >>> c.apply_space_motion(Time('J2000')) # doctest: +FLOAT_CMP >>> c.apply_space_motion(Time(58228.62, format='mjd')) # doctest: +FLOAT_CMP For a more in-depth example and more explanation of this feature, see :ref:`astropy-coordinates-apply-space-motion`. .. _whatsnew-3.0-constants-contextmanager: Context managers for versioned constants ======================================== The context manager :func:`~astropy.constants.set_enabled_constants` is now available for temporarily switching to an older version of constants:: >>> from astropy import constants as const >>> with const.set_enabled_constants('astropyconst13'): ... print(const.h) Name = Planck constant Value = 6.62606957e-34 Uncertainty = 2.9e-41 Unit = J s Reference = CODATA 2010 >>> print(const.h) Name = Planck constant Value = 6.62607004e-34 Uncertainty = 8.1e-42 Unit = J s Reference = CODATA 2014 .. _whatsnew-3.0-quantities-scipy-ufuncs: Support for quantities in scipy special functions ================================================= :class:`~astropy.units.Quantity`'s support for all relevant numpy "universal functions" or ufuncs has been extended to many of the `scipy.special` functions (with more to be added in future releases; PRs welcome!), so that these can now be used with any :class:`~astropy.units.Quantity` that has appropriate units. For instance, using the error function to calculate confidence intervals: .. doctest-requires:: scipy >>> import astropy.units as u >>> import scipy.special as sps >>> sigma = [1., 3.] * u.minute / (60. * u.s) >>> sps.erf(sigma / 2**0.5).to(u.percent) # doctest: +FLOAT_CMP .. _whatsnew-3.0-showtable: ``showtable`` command line script ================================= The new command-line tool ``showtable`` can be used to print the content of tables for the formats supported by the unified I/O interface (ASCII, FITS, HDF5 or VOTable):: $ showtable astropy/io/fits/tests/data/table.fits target V_mag ------- ----- NGC1001 11.1 NGC1002 12.3 NGC1003 15.2 To get full documentation on the usage and available options do ``showtable --help``. .. _whatsnew-3.0-hdf5-mixins: Lossless writing and reading mixin columns to HDF5 and FITS =========================================================== It is now possible to write Table :ref:`mixin_columns` such as :class:`~astropy.coordinates.SkyCoord`, :class:`~astropy.time.Time`, and :class:`~astropy.units.Quantity` (and subclasses like :class:`~astropy.coordinates.EarthLocation`) to both HDF5 and FITS with no loss of information. All object attributes and metadata are stored to file and the object is exactly regenerated (to the bit level) when reading back into astropy. For details see the :ref:`fits_astropy_native` section. .. _whatsnew-3.0-fits-time-support: Support for FITS time standard ============================== The `FITS Time standard paper `_ defines the formats and keywords used to represent timing information in FITS files. The Astropy FITS package now provides support for reading and writing native :class:`~astropy.time.Time` table columns using this format. This is described within the :ref:`FITS unified I/O interface ` and examples of usage can be found in the :ref:`fits_astropy_native` section. Implementation of this feature was made possible through the support of the Google Summer of Code 2017 program. To learn more about this project please see the final report, `A mixin protocol for seamless interoperability `_. .. _whatsnew-3.0-fits-performance: FITS table performance enhancements =================================== The performance for reading and writing FITS tables has been significantly improved, in particular for cases where the tables contain one or more string columns and when done through the :class:`~astropy.table.Table` interface. The :meth:`Table.read() ` method now supports a ``memmap=`` keyword argument to control whether or not to use memory mapping when reading the table, and :func:`fits.open() ` supports a new keyword argument ``character_as_bytes`` to return character columns as Numpy byte arrays (Numpy type 'S'), which uses less memory. .. _whatsnew-3.0-lomb-scargle-false-alarm: False alarm probabilities in ``astropy.stats.LombScargle`` ========================================================== :class:`~astropy.stats.LombScargle` now supports estimation of false alarm probabilities, using the :meth:`~astropy.stats.LombScargle.false_alarm_probability` and :meth:`~astropy.stats.LombScargle.false_alarm_level` methods. Supported approaches to false alarm estimation include a common heuristic based on an estimated number of independent frequencies (``method='naive'``), a more careful approach based on extreme value statistics (``method='baluev'``; first explored by Baluev (2007)), and a more accurate (but computationally intensive) approach based on bootstrap resampling (``method='bootstrap'``). These methods are implemented for all of the available periodogram normalizations. Further explanation and examples are available in :ref:`stats-lombscargle`. .. _whatsnew-3.0-ascii-chunks: Reading large ASCII files in chunks =================================== Astropy now supports reading very large ASCII tables in smaller chunks, allowing tables to be loaded that approach or exceed the maximum memory available. There are two possible ways to do this: - Read the table in chunks and aggregate the final table along the way. This uses only somewhat more memory than the final table requires. In contrast, the default reading process may temporarily require up to 10 times more memory than the size of the file. - Use a Python generator function to return a :class:`~astropy.table.Table` object for each chunk of the input table. This allows for scanning through arbitrarily large tables since it never returns the final aggregate table. For details see :ref:`chunk_reading`. Full change log =============== To see a detailed list of all changes in version v3.0, including changes in API, please see the :ref:`changelog`. Renamed/removed functionality ============================= .. _whatsnew-3.0-pytest-plugins: pytest plugins moved to external packages ***************************************** The following ``pytest`` plugins were previously provided as part of the Astropy core package but have now been moved to separate packages: * `pytest-remotedata `__ * `pytest-doctestplus `__ * `pytest-openfiles `__ Affiliated packages that make use of one or more of these plugins may install them individually using ``pip``, or they may install the Astropy testing package ``pytest-astropy``, which includes all of these plugins as dependencies. Packages that use Astropy's test runner will also need to install ``pytest-astropy``. See :ref:`testing-dependencies` for more details. The ``config`` and ``display`` plugins are still provided as part of the Astropy core package, but they have moved from ``astropy.tests.pytest_plugins`` to ``astropy.tests.plugins.config`` and ``astropy.tests.plugins.display``, respectively. Affiliated packages that currently access plugin decorators and other variables from ``astropy.tests.pytest_plugins`` will need to make updates, although backwards compatibility will be maintained in the meantime. Below is an outline of the required changes: * Affiliated packages that use the ``remote_data`` decorator to mark tests that require remote data access should now mark them with ``@pytest.mark.remote_data``. This requires installing either ``pytest-remotedata`` or ``pytest-astropy``. Importing ``remote_data`` from ``astropy.tests.helper`` is maintained for backwards compatibility but will eventually be removed. * Packages should no longer import ``disable_internet`` from ``astropy.tests``. It is available using ``from pytest_remotedata import disable_internet`` if required, but developers should consider whether they can avoid importing it directly by instead using ``@pytest.mark.remote_data`` and ``@pytest.mark.internet_off`` provided by ``pytest-remotedata``. Use of ``disable_internet.INTERNET_OFF`` is not recommended and should be replaced by use of ``remote_data`` and ``internet_off``. * Packages that access the display configuration variables ``PYTEST_HEADER_MODULES`` and ``TESTED_VERSIONS`` should no longer import them from ``astropy.tests.pytest_plugins``. They should instead be imported from ``astropy.tests.plugins.display``. We hope that eventually these will be configurable using ``setup.cfg`` instead. * The function ``enable_deprecations_as_exceptions`` should not be imported from ``astropy.tests.pytest_plugins``. It should be imported from ``astropy.tests.helper`` instead.