Make and restore incremental backups with GNU tar
Use GNU tar’s --listed-incremental option to create a full backup followed by archives of changes.
This walkthrough gives you a local backup chain and a restore drill that checks file contents,
including what happens when you add, modify, or delete files. Each backup gets its own snapshot copy,
so a failed run cannot advance the last successful backup’s state.
Set up a small backup source
Use Linux, Bash, GNU tar, gzip, and the standard diff and file utilities. The examples were tested
with GNU tar 1.35 and Bash 5; check that tar --version identifies GNU tar. GNU incremental archives
use extensions that other tar implementations may not support. See the
GNU tar incremental backup manual.
tar --version
bash --version
gzip --version
Run the blocks below in Bash, from the same starting directory. Parentheses keep directory changes
and shell options local to each block. Setup refuses to reuse an existing tar-demo directory.
(
set -euo pipefail
mkdir tar-demo
cd -- tar-demo
mkdir -p project/cache backups
printf 'version one\n' > project/notes.txt
printf 'keep this file\n' > project/unchanged.txt
printf 'remove this file later\n' > project/obsolete.txt
printf 'rebuildable cache\n' > project/cache/item.txt
)
Keep backups outside the source tree, as shown here, so archives cannot include their own output.
Use a source that is not being written to during a backup. This exercise covers ordinary files in a
local directory; it does not make a consistent backup of a running database or preserve every kind
of system metadata.
Save each backup with its snapshot
The snapshot file records what tar saw during the previous backup. An absent snapshot starts a full, or level-zero, backup. Reusing one records changes since that snapshot, and tar updates it during creation. Copy the previous snapshot before making the next archive; keep the previous copy intact.
Save this script as tar-demo/backup.sh. Its first argument names a new backup directory. The optional
second argument names the previous successful backup. Names may contain ASCII letters, digits,
underscores, and hyphens.
#!/usr/bin/env bash
set -euo pipefail
unset TAR_OPTIONS
cd -- "$(dirname -- "$0")"
if (( $# < 1 || $# > 2 )); then
printf 'Usage: bash backup.sh NAME [PREVIOUS]\n' >&2
exit 2
fi
for name in "$@"; do
case "$name" in
''|*[!a-zA-Z0-9_-]*)
printf 'Invalid backup name: %s\n' "$name" >&2
exit 2
;;
esac
done
output="backups/$1"
previous=${2:-}
if [[ -n "$previous" ]]; then
for file in complete archive.tar.gz state.snar; do
if [[ ! -f "backups/$previous/$file" ]]; then
printf 'Previous backup is incomplete: %s\n' "$previous" >&2
exit 1
fi
done
fi
# Reserve a new directory before installing cleanup that can remove it.
mkdir -- "$output"
trap 'rm -rf -- "$output"' EXIT
trap 'exit 1' HUP INT TERM
if [[ -n "$previous" ]]; then
cp -- "backups/$previous/state.snar" "$output/state.snar"
fi
tar --create --file=- \
--listed-incremental="$output/state.snar" \
--exclude='project/cache' \
-- project | gzip > "$output/archive.tar.gz"
gzip --test -- "$output/archive.tar.gz"
touch -- "$output/complete"
trap - EXIT
printf 'Created %s\n' "$output"
Run it once without a predecessor:
bash tar-demo/backup.sh 00-full
This creates backups/00-full/archive.tar.gz, state.snar, and an empty complete marker inside
tar-demo. The marker records that archive creation and the gzip check completed. It is not a
substitute for a restore test. An existing output directory makes the script fail before changing
its contents, even when input is noninteractive. Run only one backup at a time.
Keep exclusions consistent
The quoted --exclude='project/cache' omits that directory and its contents. Keep the source path and
exclusions the same throughout a chain. If you change what belongs in the backup, start a new full
backup under a new name. Selecting recent files with find -mtime does not provide this workflow’s
record of deleted directory entries; let tar traverse the source directory.
Gzip compresses each archive independently. It does not decide which files have changed. For a separate look at compression choices, see tar with Zstd and LZ4.
Capture changes in order
Modify one file, add another, and delete the obsolete file. The short pause separates the tiny example’s timestamps on filesystems with coarse time resolution. GNU tar’s incremental selection depends on timestamps, so do not move the clock backward during a chain.
(
set -euo pipefail
cd -- tar-demo
sleep 1
printf 'version two\n' > project/notes.txt
printf 'new file\n' > project/added.txt
rm -- project/obsolete.txt
bash backup.sh 01-change 00-full
)
Inspect the changed-file archive:
tar --list --gzip --file=tar-demo/backups/01-change/archive.tar.gz
It contains project/, project/notes.txt, and project/added.txt; the order may vary. The directory
entry also carries incremental metadata. unchanged.txt still depends on the full backup, while the
deletion of obsolete.txt will be applied during incremental extraction.
Make one more change, using 01-change as the predecessor:
(
set -euo pipefail
cd -- tar-demo
sleep 1
printf 'version three\n' > project/notes.txt
bash backup.sh 02-change 01-change
)
The resulting chain is 00-full → 01-change → 02-change. To reach the final state, keep all three
archives. Choosing 00-full as the predecessor again would create another backup relative to that
full backup, rather than extending this sequence.
Restore the full chain into an empty directory
GNU tar’s incremental extraction can delete files that were absent from an archived directory.
Restore only your own trusted archives into a new destination, never over your live source. Use
--listed-incremental=/dev/null on every extraction, including the full backup. Extraction reads
the directory history from the archives; it does not need the saved .snar files.
(
set -euo pipefail
cd -- tar-demo
mkdir restore
for generation in 00-full 01-change 02-change; do
test -f "backups/$generation/complete"
gzip --test -- "backups/$generation/archive.tar.gz"
tar --extract --gzip --listed-incremental=/dev/null \
--file="backups/$generation/archive.tar.gz" --directory=restore
done
diff --recursive --exclude=cache project restore/project
test ! -e restore/project/obsolete.txt
printf 'Restored files match the source, excluding cache.\n'
)
On success, diff prints nothing and the final line confirms the comparison. The restored
notes.txt contains version three, added.txt contains new file, unchanged.txt is retained, and
obsolete.txt is gone. No cache directory is restored. This compares file names and contents, not
ownership, permissions, or extended attributes.
A rerun refuses an existing restore directory before extracting anything. After a failed restore,
the directory may contain partial results; inspect the error and choose a new empty destination for
the next drill. Do not treat a partial restore as recovered data.
Reject incomplete backups
A successful compressor does not prove its input producer succeeded. Bash’s
pipefail option makes a failure
in tar | gzip fail the pipeline even if gzip successfully finishes compressing a partial archive.
The script then removes only the new directory it reserved. The previous archive and snapshot stay
unchanged, so retry with the same predecessor after fixing the problem.
Do not add --ignore-failed-read to this script. GNU tar documents that it makes missing or
unreadable input, and files changing while being read, cease to affect the exit status. That is the
opposite of detecting an incomplete backup. See
--ignore-failed-read.
Keep diagnostics visible and reject any nonzero exit, including status 1 for files changed during
creation. Stop the writer or use an appropriate filesystem snapshot before retrying a changing source.
tar --list checks whether tar can read the archive structure; tar’s header checksums do not cover
file contents. gzip --test
checks the compressed stream’s integrity, but cannot tell whether an input file was omitted.
That is why this workflow checks command failures and restores the result for comparison. The
GNU tar corruption notes
explain this distinction.
Keep complete backup chains together
Keep each successful directory’s archive and snapshot together. The archive is needed for recovery;
the snapshot is needed to continue the chain. A power loss or SIGKILL can bypass the cleanup trap
and leave a directory without complete. Once no backup process is running, inspect that incomplete
directory and remove it before retrying; do not use it as a predecessor.
These local copies do not protect against losing the disk. Copy completed backup directories to separate storage and repeat the restore drill from those copies. This example does not implement an SSH transfer or archive splitting. If a storage system splits an archive, all its pieces must be reassembled before the gzip check and restore.
Retain the full backup and every incremental archive needed by a retained recovery point. Deleting individual archives by age can break that chain. Start a new full backup periodically, test its recovery path, and retire older chains as units under your retention policy.