Utility functions for handling bit masks and mask arrays.¶
It is common to use bit fields - e.g., integer variables whose individual bits represent some attributes - to characterize the state of data. For example, HST uses arrays of bit fields to characterize data quality (DQ) of HST images, see, e.g., DQ field values for WFPC2 image data and WFC3 image data. As one can see, the meaning assigned to various bit flags in for the two instruments is generally different.
Bit fields can be thought of as tightly packed collections of bit flags. Using masking we can “inspect” the status of individual bits.
One common operation performed on bit field arrays is their conversion to
boolean masks, for example by simply assigning boolean True
(in the boolean
mask) to those elements that correspond to non-zero-valued bit fields
(bit fields with at least one bit set to 1
) or, oftentimes, by assigning
True
to elements whose corresponding bit fields have only specific fields
set (to 1
). This more sophisticated analysis of bit fields can be
accomplished using bit masks and the aforementioned masking operation.
The bitmask
module provides two functions that facilitate
conversion of bit field arrays (i.e., DQ arrays) to boolean masks:
bitfield_to_boolean_mask
to convert an input bit
fields array to a boolean mask using an input bit mask (or list of individual
bit flags) and interpret_bit_flags
to create bit mask
from input list of individual bit flags.
Creating boolean masks¶
Overview¶
bitfield_to_boolean_mask
by default assumes that
all input bit fields that have at least one bit turned “ON” correspond to
“bad” data (i.e., pixels) and converts them to boolean True
in the output
boolean mask (otherwise output boolean mask values are set to False
).
Often, for specific algorithms and situations, some bit flags are OK and
can be ignored. bitfield_to_boolean_mask
accepts
lists of bit flags that by default must be ignored in the input bit fields
when creating boolean masks.
Fundamentally, by default, bitfield_to_boolean_mask
performs the following operation:
(1) boolean_mask = (bitfield & ~bit_mask) != 0
(here &
is bitwise and
and ~
is the bitwise not
operations).
In the previous formula, bit_mask
is a bit mask created from individual
bit flags that need to be ignored in the bit field.
Bit Field | Bit Mask | ~(Bit Mask) | Bit Field & ~(Bit Mask) | Boolean Mask |
---|---|---|---|---|
11011001 (217) | 01010000 (80) | 10101111 (175) | 10001001 (137) | True |
11011001 (217) | 10101111 (175) | 01010000 (80) | 01010000 (80) | True |
00001001 (9) | 01001001 (73) | 10110110 (182) | 00000000 (0) | False |
00001001 (9) | 00000000 (0) | 11111111 (255) | 00001001 (9) | True |
00001001 (9) | 11111111 (255) | 00000000 (0) | 00000000 (0) | False |
Specifying bit flags¶
bitfield_to_boolean_mask
accepts either an integer
bit mask or lists of bit flags. Lists of bit flags will be combined into a
bit mask and can be provided either as a Python list of
integer bit flag values or as a comma-separated (or +
-separated)
list of integer bit flag values. Consider the bit mask from the first example
in Table 1. In this case ignore_flags
can be set either to:
- an integer value bit mask 80, or
- a Python list indicating individual non-zero bit flag values:
[16, 64]
, or- a string of comma-separated bit flag values:
'16,64'
, or- a string of
+
-separated bit flag values:'16+64'
For example,
>>> from astropy.nddata import bitmask
>>> import numpy as np
>>> bitmask.bitfield_to_boolean_mask(217, ignore_flags=80)
array(True...)
>>> bitmask.bitfield_to_boolean_mask(217, ignore_flags='16,64')
array(True...)
>>> bitmask.bitfield_to_boolean_mask(217, ignore_flags=[16, 64])
array(True...)
>>> bitmask.bitfield_to_boolean_mask(9, ignore_flags=[1, 8, 64])
array(False...)
>>> bitmask.bitfield_to_boolean_mask([9, 10, 73, 217], ignore_flags='1,8,64')
array([False, True, False, True]...)
It is also possible to specify the type of the output mask:
>>> bitmask.bitfield_to_boolean_mask([9, 10, 73, 217], ignore_flags='1,8,64', dtype=np.uint8)
array([0, 1, 0, 1], dtype=uint8)
Modifying the Formula for Creating Boolean Masks¶
bitfield_to_boolean_mask
provides several parameters
that can be used to modify the formula used to create boolean masks.
Inverting Bit Mask¶
Sometimes it is more convenient to be able to specify those bit
flags that must be considered when creating the boolean mask and all other
flags should be ignored. In bitfield_to_boolean_mask
this can be accomplished by setting parameter flip_bits
to True
.
This effectively modifies equation (1) to:
(2) boolean_mask = (bitfield & bit_mask) != 0
So, instead of
>>> bitmask.bitfield_to_boolean_mask([9, 10, 73, 217], ignore_flags=[1, 8, 64])
array([False, True, False, True]...)
one can obtain the same result as
>>> bitmask.bitfield_to_boolean_mask(
... [9, 10, 73, 217], ignore_flags=[2, 4, 16, 32, 128], flip_bits=True
... )
array([False, True, False, True]...)
Note however, when ignore_flags
is a comma-separated list of bit flag
values, flip_bits
cannot be set to neither True
or False
. Instead,
to flip bits of the bit mask formed from a string list of comma-separated
bit flag values, one can prepend a single ~
to the list:
>>> bitmask.bitfield_to_boolean_mask([9, 10, 73, 217], ignore_flags='~2+4+16+32+128')
array([False, True, False, True]...)
Inverting Boolean Mask¶
Other times, it may be more convenient to obtain an inverted mask in which
flagged data are converted to False
instead of True
:
(3) boolean_mask = (bitfield & ~bit_mask) == 0
This can be accomplished by changing good_mask_value
parameter from
its default value (False
) to True
. For example,
>>> bitmask.bitfield_to_boolean_mask([9, 10, 73, 217], ignore_flags=[1, 8, 64],
... good_mask_value=True)
array([ True, False, True, False]...)