.. _modern_cipher_modes: Modern modes of operation for symmetric block ciphers ===================================================== Classic modes of operation such as CBC only provide guarantees over the *confidentiality* of the message but not over its *integrity*. In other words, they don't allow the receiver to establish if the ciphertext was modified in transit or if it really originates from a certain source. For that reason, classic modes of operation have been often paired with a MAC primitive (such as :mod:`Crypto.Hash.HMAC`), but the combination is not always straightforward, efficient or secure. Recently, new modes of operations (AEAD, for `Authenticated Encryption with Associated Data `_) have been designed to combine *encryption* and *authentication* into a single, efficient primitive. Optionally, some part of the message can also be left in the clear (non-confidential *associated data*, such as headers), while the whole message remains fully authenticated. In addition to the **ciphertext** and a **nonce** (or **IV** - Initialization Vector), AEAD modes require the additional delivery of a **MAC tag**. This is the state machine for a cipher object: .. figure:: aead.png :align: center :figwidth: 80% Generic state diagram for a AEAD cipher mode Beside the usual :meth:`encrypt()` and :meth:`decrypt()` already available for classic modes of operation, several other methods are present: .. method:: update(data) Authenticate those parts of the message that get delivered as is, without any encryption (like headers). It is similar to the ``update()`` method of a MAC object. Note that all data passed to ``encrypt()`` and ``decrypt()`` get automatically authenticated already. :param bytes data: the extra data to authenticate .. method:: digest() Create the final authentication tag (MAC tag) for a message. :return bytes: the MAC tag .. method:: hexdigest() Equivalent to ``digest()``, with the output encoded in hexadecimal. :return str: the MAC tag as a hexadecimal string .. method:: verify(mac_tag) Check if the provided authentication tag (MAC tag) is valid, that is, if the message has been decrypted using the right key and if no modification has taken place in transit. :param bytes mac_tag: the MAC tag :raise ValueError: if the MAC tag is not valid, that is, if the entire message should not be trusted. .. method:: hexverify(mac_tag_hex) Same as ``verify()`` but accepts the MAC tag encoded as an hexadecimal string. :param str mac_tag_hex: the MAC tag as a hexadecimal string :raise ValueError: if the MAC tag is not valid, that is, if the entire message should not be trusted. .. method:: encrypt_and_digest(plaintext, output=None) Perform ``encrypt()`` and ``digest()`` in one go. :param bytes plaintext: the last piece of plaintext to encrypt :keyword bytes/bytearray/memoryview output: the pre-allocated buffer where the ciphertext must be stored (as opposed to being returned). :return: a tuple with two items - the ciphertext, as ``bytes`` - the MAC tag, as ``bytes`` The first item becomes ``None`` when the ``output`` parameter specified a location for the result. .. method:: decrypt_and_verify(ciphertext, mac_tag, output=None) Perform ``decrypt()`` and ``verify()`` in one go. :param bytes ciphertext: the last piece of ciphertext to decrypt :keyword bytes/bytearray/memoryview output: the pre-allocated buffer where the plaintext must be stored (as opposed to being returned). :raise ValueError: if the MAC tag is not valid, that is, if the entire message should not be trusted. .. _ccm_mode: CCM mode -------- `Counter with CBC-MAC `_, defined in `RFC3610 `_ or `NIST SP 800-38C `_. It only works with ciphers having block size 128 bits (like AES). The :func:`new` function at the module level under ``Crypto.Cipher`` instantiates a new CCM cipher object for the relevant base algorithm. In the following definition, ```` can only be ``AES`` today: .. function:: Crypto.Cipher..new(key, mode, *, nonce=None, mac_len=None, msg_len=None, assoc_len=None) Create a new CCM object, using as the base block cipher. :param bytes key: the cryptographic key :param mode: the constant ``Crypto.Cipher..MODE_CCM`` :param bytes nonce: the value of the fixed nonce. It must be unique for the combination message/key. For AES, its length varies from 7 to 13 bytes. The longer the nonce, the smaller the allowed message size (with a nonce of 13 bytes, the message cannot exceed 64KB). If not present, the library creates a 11 bytes random nonce (the maximum message size is 8GB). :param integer mac_len: the desired length of the MAC tag (default if not present: 16 bytes). :param integer msg_len: pre-declaration of the length of the message to encipher. If not specified, :func:`encrypt` and :func:`decrypt` can only be called once. :param integer assoc_len: pre-declaration of the length of the associated data. If not specified, some extra buffering will take place internally. :return: a CTR cipher object The cipher object has a read-only attribute :attr:`nonce`. Example (encryption):: >>> import json >>> from base64 import b64encode >>> from Crypto.Cipher import AES >>> from Crypto.Random import get_random_bytes >>> >>> header = b"header" >>> data = b"secret" >>> key = get_random_bytes(16) >>> cipher = AES.new(key, AES.MODE_CCM) >>> cipher.update(header) >>> ciphertext, tag = cipher.encrypt_and_digest(data) >>> >>> json_k = [ 'nonce', 'header', 'ciphertext', 'tag' ] >>> json_v = [ b64encode(x).decode('utf-8') for x in (cipher.nonce, header, ciphertext, tag) ] >>> result = json.dumps(dict(zip(json_k, json_v))) >>> print(result) {"nonce": "p6ffzcKw+6xopVQ=", "header": "aGVhZGVy", "ciphertext": "860kZo/G", "tag": "Ck5YpVCM6fdWnFkFxw8K6A=="} Example (decryption):: >>> import json >>> from base64 import b64decode >>> from Crypto.Cipher import AES >>> >>> # We assume that the key was securely shared beforehand >>> try: >>> b64 = json.loads(json_input) >>> json_k = [ 'nonce', 'header', 'ciphertext', 'tag' ] >>> jv = {k:b64decode(b64[k]) for k in json_k} >>> >>> cipher = AES.new(key, AES.MODE_CCM, nonce=jv['nonce']) >>> cipher.update(jv['header']) >>> plaintext = cipher.decrypt_and_verify(jv['ciphertext'], jv['tag']) >>> print("The message was: " + plaintext.decode('utf-8')) >>> except (ValueError, KeyError): >>> print("Incorrect decryption") .. _eax_mode: EAX mode -------- An AEAD mode designed for NIST by `Bellare, Rogaway, and Wagner in 2003 `_. The :func:`new` function at the module level under ``Crypto.Cipher`` instantiates a new EAX cipher object for the relevant base algorithm. .. function:: Crypto.Cipher..new(key, mode, *, nonce=None, mac_len=None) Create a new EAX object, using as the base block cipher. :param bytes key: the cryptographic key :param mode: the constant ``Crypto.Cipher..MODE_EAX`` :param bytes nonce: the value of the fixed nonce. It must be unique for the combination message/key. If not present, the library creates a random nonce (16 bytes long for AES). :param integer mac_len: the length of the MAC tag, in bytes. At least 2, and not larger than the cipher's block size (default), which is 16 bytes for AES. :return: an EAX cipher object The cipher object has a read-only attribute :attr:`nonce`. Example (encryption):: >>> import json >>> from base64 import b64encode >>> from Crypto.Cipher import AES >>> from Crypto.Random import get_random_bytes >>> >>> header = b"header" >>> data = b"secret" >>> key = get_random_bytes(16) >>> cipher = AES.new(key, AES.MODE_EAX) >>> cipher.update(header) >>> ciphertext, tag = cipher.encrypt_and_digest(data) >>> >>> json_k = [ 'nonce', 'header', 'ciphertext', 'tag' ] >>> json_v = [ b64encode(x).decode('utf-8') for x in (cipher.nonce, header, ciphertext, tag) ] >>> result = json.dumps(dict(zip(json_k, json_v))) >>> print(result) {"nonce": "CSIJ+e8KP7HJo+hC4RXIyQ==", "header": "aGVhZGVy", "ciphertext": "9YYjuAn6", "tag": "kXHrs9ZwYmjDkmfEJx7Clg=="} Example (decryption):: >>> import json >>> from base64 import b64decode >>> from Crypto.Cipher import AES >>> >>> # We assume that the key was securely shared beforehand >>> try: >>> b64 = json.loads(json_input) >>> json_k = [ 'nonce', 'header', 'ciphertext', 'tag' ] >>> jv = {k:b64decode(b64[k]) for k in json_k} >>> >>> cipher = AES.new(key, AES.MODE_EAX, nonce=jv['nonce']) >>> cipher.update(jv['header']) >>> plaintext = cipher.decrypt_and_verify(jv['ciphertext'], jv['tag']) >>> print("The message was: " + plaintext.decode('utf-8')) >>> except (ValueError, KeyError): >>> print("Incorrect decryption") .. _gcm_mode: GCM mode -------- `Galois/Counter Mode `_, defined in `NIST SP 800-38D `_. It only works in combination with a 128 bits cipher like AES. The :func:`new` function at the module level under ``Crypto.Cipher`` instantiates a new GCM cipher object for the relevant base algorithm. .. function:: Crypto.Cipher..new(key, mode, *, nonce=None, mac_len=None) Create a new GCM object, using as the base block cipher. :param bytes key: the cryptographic key :param mode: the constant ``Crypto.Cipher..MODE_GCM`` :param bytes nonce: the value of the fixed nonce. It must be unique for the combination message/key. If not present, the library creates a random nonce (16 bytes long for AES). :param integer mac_len: the desired length of the MAC tag, from 4 to 16 bytes (default: 16). :return: a GCM cipher object The cipher object has a read-only attribute :attr:`nonce`. Example (encryption):: >>> import json >>> from base64 import b64encode >>> from Crypto.Cipher import AES >>> from Crypto.Random import get_random_bytes >>> >>> header = b"header" >>> data = b"secret" >>> key = get_random_bytes(16) >>> cipher = AES.new(key, AES.MODE_GCM) >>> cipher.update(header) >>> ciphertext, tag = cipher.encrypt_and_digest(data) >>> >>> json_k = [ 'nonce', 'header', 'ciphertext', 'tag' ] >>> json_v = [ b64encode(x).decode('utf-8') for x in (cipher.nonce, header, ciphertext, tag) ] >>> result = json.dumps(dict(zip(json_k, json_v))) >>> print(result) {"nonce": "DpOK8NIOuSOQlTq+BphKWw==", "header": "aGVhZGVy", "ciphertext": "CZVqyacc", "tag": "B2tBgICbyw+Wji9KpLVa8w=="} Example (decryption):: >>> import json >>> from base64 import b64decode >>> from Crypto.Cipher import AES >>> >>> # We assume that the key was securely shared beforehand >>> try: >>> b64 = json.loads(json_input) >>> json_k = [ 'nonce', 'header', 'ciphertext', 'tag' ] >>> jv = {k:b64decode(b64[k]) for k in json_k} >>> >>> cipher = AES.new(key, AES.MODE_GCM, nonce=jv['nonce']) >>> cipher.update(jv['header']) >>> plaintext = cipher.decrypt_and_verify(jv['ciphertext'], jv['tag']) >>> print("The message was: " + plaintext.decode('utf-8')) >>> except (ValueError, KeyError): >>> print("Incorrect decryption") .. note:: GCM is most commonly used with 96-bit (12-byte) nonces, which is also the length recommended by NIST SP 800-38D. If interoperability is important, one should take into account that the library default of a 128-bit random nonce may not be (easily) supported by other implementations. A 96-bit nonce can be explicitly generated for a new encryption cipher:: >>> key = get_random_bytes(16) >>> nonce = get_random_bytes(12) >>> cipher = AES.new(key, AES.MODE_GCM, nonce=nonce) .. _siv_mode: SIV mode -------- Synthetic Initialization Vector (SIV), defined in `RFC5297 `_. It only works with ciphers with a block size of 128 bits (like AES). Although less efficient than other modes, SIV is *nonce misuse-resistant*: accidental reuse of the nonce does not jeopardize the security as it happens with CCM or GCM. As a matter of fact, operating **without** a nonce is not an error per se: the cipher simply becomes **deterministic**. In other words, a message gets always encrypted into the same ciphertext. The :func:`new` function at the module level under ``Crypto.Cipher`` instantiates a new SIV cipher object for the relevant base algorithm. .. function:: Crypto.Cipher..new(key, mode, *, nonce=None) Create a new SIV object, using as the base block cipher. :param bytes key: the cryptographic key; it must be twice the size of the key required by the underlying cipher (e.g. 32 bytes for AES-128). :param mode: the constant ``Crypto.Cipher..MODE_SIV`` :param bytes nonce: the value of the fixed nonce. It must be unique for the combination message/key. If not present, the encryption will be deterministic. :return: a SIV cipher object If the `nonce` parameter was provided to :func:`new`, the resulting cipher object has a read-only attribute :attr:`nonce`. Example (encryption):: >>> import json >>> from base64 import b64encode >>> from Crypto.Cipher import AES >>> from Crypto.Random import get_random_bytes >>> >>> header = b"header" >>> data = b"secret" >>> key = get_random_bytes(16 * 2) >>> nonce = get_random_bytes(16) >>> cipher = AES.new(key, AES.MODE_SIV, nonce=nonce) # Without nonce, the encryption >>> # becomes deterministic >>> cipher.update(header) >>> ciphertext, tag = cipher.encrypt_and_digest(data) >>> >>> json_k = [ 'nonce', 'header', 'ciphertext', 'tag' ] >>> json_v = [ b64encode(x).decode('utf-8') for x in (nonce, header, ciphertext, tag) ] >>> result = json.dumps(dict(zip(json_k, json_v))) >>> print(result) {"nonce": "zMiifAVvDpMS8hnGK/z+iw==", "header": "aGVhZGVy", "ciphertext": "Q7lReEAF", "tag": "KgdnBVbCee6B/wGmMf/wQA=="} Example (decryption):: >>> import json >>> from base64 import b64decode >>> from Crypto.Cipher import AES >>> >>> # We assume that the key was securely shared beforehand >>> try: >>> b64 = json.loads(json_input) >>> json_k = [ 'nonce', 'header', 'ciphertext', 'tag' ] >>> jv = {k:b64decode(b64[k]) for k in json_k} >>> >>> cipher = AES.new(key, AES.MODE_SIV, nonce=jv['nonce']) >>> cipher.update(jv['header']) >>> plaintext = cipher.decrypt_and_verify(jv['ciphertext'], jv['tag']) >>> print("The message was: " + plaintext.decode('utf-8')) >>> except (ValueError, KeyError): >>> print("Incorrect decryption") One side-effect is that encryption (or decryption) must take place in one go with the method ``encrypt_and_digest()`` (or ``decrypt_and_verify()``). You cannot use ``encrypt()`` or ``decrypt()``. The state diagram is therefore: .. figure:: siv.png :align: center :figwidth: 60% State diagram for the SIV cipher mode The length of the key passed to ``new()`` must be twice as required by the underlying block cipher (e.g. 32 bytes for AES-128). Each call to the method ``update()`` consumes an full piece of associated data. That is, the sequence:: >>> siv_cipher.update(b"builtin") >>> siv_cipher.update(b"securely") is **not** equivalent to:: >>> siv_cipher.update(b"built") >>> siv_cipher.update(b"insecurely") .. _ocb_mode: OCB mode -------- `Offset CodeBook mode `_, a cipher designed by Rogaway and specified in `RFC7253 `_ (more specifically, this module implements the last variant, OCB3). It only works in combination with a 128 bits cipher like AES. OCB was patented in USA but the author eventually `abandoned the patents `_. The :func:`new` function at the module level under ``Crypto.Cipher`` instantiates a new OCB cipher object for the relevant base algorithm. .. note:: The OCB state machine is slightly different compared to other modes: if you encrypt (or decrypt) multiple chunks, at the end you MUST call the method ``encrypt`` (or ``decrypt``) with no parameters. This last call will return any piece of internally cached ciphertext (or plaintext). .. figure:: ocb_mode.png :align: center :figwidth: 80% State diagram for the OCB mode .. function:: Crypto.Cipher..new(key, mode, *, nonce=None, mac_len=None) Create a new OCB object, using as the base block cipher. :param bytes key: the cryptographic key :param mode: the constant ``Crypto.Cipher..MODE_OCB`` :param bytes nonce: the value of the fixed nonce, with length between 1 and 15 bytes. It must be unique for the combination message/key. If not present, the library creates a 15 bytes random nonce. :param integer mac_len: the desired length of the MAC tag (default if not present: 16 bytes). :return: an OCB cipher object The cipher object has two read-only attributes: :attr:`nonce` and :attr:`block_size`. Example (encryption as a once-off operation):: >>> import json >>> from base64 import b64encode >>> from Crypto.Cipher import AES >>> from Crypto.Random import get_random_bytes >>> >>> header = b"header" >>> data = b"secret" >>> key = get_random_bytes(16) >>> cipher = AES.new(key, AES.MODE_OCB) >>> cipher.update(header) >>> ciphertext, tag = cipher.encrypt_and_digest(data) >>> >>> json_k = [ 'nonce', 'header', 'ciphertext', 'tag' ] >>> json_v = [ b64encode(x).decode('utf-8') for x in (cipher.nonce, header, ciphertext, tag) ] >>> result = json.dumps(dict(zip(json_k, json_v))) >>> print(result) {"nonce": "I7E6PKxHNYo2i9sz8W98", "header": "aGVhZGVy", "ciphertext": "nYJnJ8jC", "tag": "0UbFcmO9lqGknCIDWRLALA=="} Example (decryption as a once-off operation):: >>> import json >>> from base64 import b64decode >>> from Crypto.Cipher import AES >>> >>> # We assume that the key was securely shared beforehand >>> try: >>> b64 = json.loads(json_input) >>> json_k = [ 'nonce', 'header', 'ciphertext', 'tag' ] >>> jv = {k:b64decode(b64[k]) for k in json_k} >>> >>> cipher = AES.new(key, AES.MODE_OCB, nonce=jv['nonce']) >>> cipher.update(jv['header']) >>> plaintext = cipher.decrypt_and_verify(jv['ciphertext'], jv['tag']) >>> print("The message was: " + plaintext.decode('utf-8')) >>> except (ValueError, KeyError): >>> print("Incorrect decryption") Example (encryption with multiple chunks):: >>> import json >>> from base64 import b64encode >>> from Crypto.Cipher import AES >>> from Crypto.Random import get_random_bytes >>> >>> header = b'header' >>> data = [b'chunk1', b'chunk2', b'chunk3'] >>> key = get_random_bytes(16) >>> cipher = AES.new(key, AES.MODE_OCB) >>> cipher.update(header) >>> ciphertext = b'' >>> for chunk in data: >>> ciphertext += cipher.encrypt(chunk) >>> ciphertext += cipher.encrypt() >>> tag = cipher.digest() >>> >>> json_k = [ 'nonce', 'header', 'ciphertext', 'tag' ] >>> json_v = [ b64encode(x).decode('utf-8') for x in (cipher.nonce, header, ciphertext, tag) ] >>> result = json.dumps(dict(zip(json_k, json_v))) >>> print(result) {"nonce": "IABQ/ww8vGsu7F4sbHXK", "header": "aGVhZGVy", "ciphertext": "7Amm2DoiMHVkYC8dY7NEX86M", "tag": "qOPnjAxF63MOAx6xjwRvJQ=="} Example (decryption with multiple chunks):: >>> import json >>> from base64 import b64decode >>> from Crypto.Cipher import AES >>> >>> # We assume that the key was securely shared beforehand >>> try: >>> b64 = json.loads(json_input) >>> json_k = [ 'nonce', 'header', 'ciphertext', 'tag' ] >>> jv = {k:b64decode(b64[k]) for k in json_k} >>> >>> cipher = AES.new(key, AES.MODE_OCB, nonce=jv['nonce']) >>> cipher.update(jv['header']) >>> ciphertext = jv['ciphertext'] >>> >>> # Split into small chunks, just for demo purposes >>> chunks = [ ciphertext[i:i+2] for i in range(0, len(ciphertext), 2) ] >>> >>> plaintext = b'' >>> for chunk in chunks: >>> plaintext += cipher.decrypt(chunk) >>> plaintext += cipher.decrypt() >>> cipher.verify(jv['tag']) >>> >>> print("The message was: " + plaintext.decode('utf-8')) >>> except (ValueError, KeyError): >>> print("Incorrect decryption")