[python-delta-tar] / docs / recovery.rst

The “disaster recovery” mode serves as a blunt instrument to retrieve data
from damaged backup sets.

In itself, the mechanism is not very sophisticated. It operates by ignoring
errors in the various validations performed on the individual layers: e. g. the
GCM tag, the gzip CRC32, tar header checksums. Recovery requires a valid index
file to access objects in the tarballs directly by their offsets instead of
iterating contents in a linear fashion. Should extraction of an object fail,
Deltatar will skip ahead to the next one. The assumption is always that the
offsets of objects in the corrupted archive did not move.

On the command line, recovery mode is entered by passing the parameter
``--recover``. Deltatar will then require that an index be specified as well.
Apart from that, the invocation is the same as with ``--restore``.

::
    $ python3 backup.py --recover --mode '#gz' --targetpath     \
            /tmp/corrupt/out --indexes                          \
            /tmp/corrupt/in/bfull-2017-08-14-1441.index.gz

After extraction is complete, Deltatar will emit a list of the files whose
corruption was detected. Note that some corruptions may go undetected without
GCM encryption. Plaintext, uncompressed tarballs are especially susceptible
since only the object headers are checksummed.

Depending on which part of an object the corruption affected the results can
be quite damaging, especially if compression is involved. Garbled output may
look like this:

    6876REF,06875) VPNCONN_XAUTH_SER2ER_EN72:"72.1"ad
    6877REF,06875) VPNCONN_SECUR")
    C42"ESP"
    6878REF,06875) VPNCONN_RETRIES72.1"3d
    6879REF,06875) VPNCONN_REMOTE_TYPE72.1"CUSTOMd

Needless to say, care must be taken to inspect the files reported damaged.

Security considerations
-------------------------------------------------------------------------------

With encrypted backup sets, recovery mode omits the GCM authentication tag on
ciphertext. This introduces a severe flaw in that bad initalization vectors
will no longer cause decryption to fail. An attacker controlling the backup
sets (e. g. after subverting the file server) can exploit this to recover
plaintext from encrypted objects. Thus recovery mode should only ever be used
as the last resort when dealing with known corrupted files. The decrypted
result must be conscientiously inspected for manipulation attempts.
Commit	Line	Data
7118811d PG	1	The “disaster recovery” mode serves as a blunt instrument to retrieve data
	2	from damaged backup sets.
	3
	4	In itself, the mechanism is not very sophisticated. It operates by ignoring
	5	errors in the various validations performed on the individual layers: e. g. the
	6	GCM tag, the gzip CRC32, tar header checksums. Recovery requires a valid index
	7	file to access objects in the tarballs directly by their offsets instead of
	8	iterating contents in a linear fashion. Should extraction of an object fail,
	9	Deltatar will skip ahead to the next one. The assumption is always that the
	10	offsets of objects in the corrupted archive did not move.
	11
	12	On the command line, recovery mode is entered by passing the parameter
	13	``--recover``. Deltatar will then require that an index be specified as well.
	14	Apart from that, the invocation is the same as with ``--restore``.
	15
	16	::
	17	$ python3 backup.py --recover --mode '#gz' --targetpath \
	18	/tmp/corrupt/out --indexes \
	19	/tmp/corrupt/in/bfull-2017-08-14-1441.index.gz
	20
	21	After extraction is complete, Deltatar will emit a list of the files whose
	22	corruption was detected. Note that some corruptions may go undetected without
	23	GCM encryption. Plaintext, uncompressed tarballs are especially susceptible
	24	since only the object headers are checksummed.
	25
	26	Depending on which part of an object the corruption affected the results can
	27	be quite damaging, especially if compression is involved. Garbled output may
	28	look like this:
	29
	30	6876REF,06875) VPNCONN_XAUTH_SER2ER_EN72:"72.1"ad
	31	6877REF,06875) VPNCONN_SECUR")
	32	C42"ESP"
	33	6878REF,06875) VPNCONN_RETRIES72.1"3d
	34	6879REF,06875) VPNCONN_REMOTE_TYPE72.1"CUSTOMd
	35
	36	Needless to say, care must be taken to inspect the files reported damaged.
	37
8b8b0d32 PG	38	Security considerations
	39	-------------------------------------------------------------------------------
	40
	41	With encrypted backup sets, recovery mode omits the GCM authentication tag on
	42	ciphertext. This introduces a severe flaw in that bad initalization vectors
	43	will no longer cause decryption to fail. An attacker controlling the backup
	44	sets (e. g. after subverting the file server) can exploit this to recover
	45	plaintext from encrypted objects. Thus recovery mode should only ever be used
	46	as the last resort when dealing with known corrupted files. The decrypted
	47	result must be conscientiously inspected for manipulation attempts.
	48