.. _building-c-or-cython-extensions:

**********************
C or Cython Extensions
**********************

Astropy supports using C extensions for wrapping C libraries and Cython for
speeding up computationally-intensive calculations. Both Cython and C extension
building can be customized using the ``get_extensions`` function of the
``setup_package.py`` file. If defined, this function must return a list of
`distutils.core.Extension` objects. The creation process is left to the
subpackage designer, and can be customized however is relevant for the
extensions in the subpackage.

While C extensions must always be defined through the ``get_extensions``
mechanism, Cython files (ending in ``.pyx``) are automatically located and
loaded in separate extensions if they are not in ``get_extensions``. For
Cython extensions located in this way, headers for numpy C functions are
included in the build, but no other external headers are included. ``.pyx``
files present in the extensions returned by ``get_extensions`` are not
included in the list of extensions automatically generated extensions. Note
that this allows disabling a Cython file by providing an extension that
includes the Cython file, but giving it the special ``name`` 'cython_skip'. Any
extension with this package name will not be built by ``setup.py``.

.. note::

    If an :class:`~distutils.core.Extension` object is provided for Cython
    source files using the ``get_extensions`` mechanism, it is very
    important that the ``.pyx`` files be given as the ``source``, rather than the
    ``.c`` files generated by Cython.

Using Numpy C headers
=====================

If your C or Cython extensions uses `numpy` at the C level, you probably
need access to the numpy C headers.  A common idiom you can find in the numpy
docs or other examples involves getting the include directory by calling
``numpy.get_include()``.  However, using this in ``setup_package.py`` will *not*
work, because ``setup_package.py`` needs to be able to import even when none of
the dependencies are present.  To work around this need, simply include the
string ``'numpy'`` in the list that is passed to the ``include_dirs`` argument
of `distutils.core.Extension`.  The astropy setup helpers will then use
``numpy.get_include()`` downstream once it is certain that the dependencies
have actually been processed.  For example::

    from distutils.extension import Extension

    def get_extensions():
        return Extension(name='myextension', sources=['myext.pyx'],
                         include_dirs=['numpy'])



Installing C header files
=========================

If your C extension needs to be linked from other third-party C code,
you probably want to install its header files along side the Python module.

    1) Create an ``include`` directory inside of your package for
       all of the header files.

    2) Use the ``get_package_data`` hook in ``setup_package.py`` to
       install those header files.  For example, the `astropy.wcs`
       package has this::

           def get_package_data():
               return {'astropy.wcs': ['include/*.h']}

Preventing importing at build time
==================================

In rare cases, some packages may need to be imported at build time.
Unfortunately, anything that requires a C or Cython extension will fail to
import until the build phase has completed. In this cases, the
``_ASTROPY_SETUP_`` variable can be used to determine if the package is being
imported as part of the build and choose to not import problematic modules.
``_ASTROPY_SETUP_`` is inserted into the builtins, and is `True` when inside
of astropy's ``setup.py`` script, and `False` otherwise.

For example, suppose there is a subpackage ``foo`` that needs to
import a module called ``version.py`` at build time in order to set
some version information, and also has a C extension, ``process``,
that will not be available in the source tree.  In this case,
``astropy/foo/__init__.py`` would probably want to check the value of
``_ASTROPY_SETUP_`` before importing the C extension::

    try:
        from . import process
    except ImportError:
        if not _ASTROPY_SETUP_:
            raise

    from . import version

Speed up your builds with ccache
================================

`ccache <https://en.wikipedia.org/wiki/Ccache>`_ is a tool that caches
compiled sources so that they don't have to be recompiled (so long as they are
unchanged) even if the outputs have been deleted.  This means that if you
switch branches or clean your source checkout you can save a lot of time by
avoiding the majority of re-compiles from scratch.

Because installation and configuration of ccache varies from platform to
platform, please consult the ccache documentation and/or Google to set up
ccache on your system--this is strongly encouraged for anyone doing significant
development of Astropy or scientific programming in general.