(This message will disappear, once this node revised.)
While an archive may contain many files, the archive itself is a single
ordinary file. Like any other file, an archive file can be written to a
storage device such as a tape or disk, sent through a pipe or over a network,
saved on the active file system, or even stored in another archive. An archive
file is not easy to read or manipulate without using the tar
utility or Tar mode in GNU Emacs.
Physically, an archive consists of a series of file entries terminated by
an end-of-archive entry, which consists of 512 zero bytes. A file entry
usually describes one of the files in the archive (an archive member),
and consists of a file header and the contents of the file. File headers
contain file names and statistics, checksum information which tar
uses to detect file corruption, and information about file types.
Archives are permitted to have more than one member with the same member name. One way this situation can occur is if more than one version of a file has been stored in the archive. For information about adding new versions of a file to an archive, see 4.2.4 Updating an Archive, and to learn more about having more than one archive member with the same name, see -backup node, when it's written .
In addition to entries describing archive members, an archive may contain
entries which tar itself uses to store information. See section 9.7
Including a Label in the Archive, for an example of such an archive entry.
A tar archive file contains a series of blocks. Each block
contains BLOCKSIZE bytes. Although this format may be thought of
as being on magnetic tape, other media are often used.
Each file archived is represented by a header block which describes the file, followed by zero or more blocks which give the contents of the file. At the end of the archive file there may be a block filled with binary zeros as an end-of-file marker. A reasonable system should write a block of zeros at the end, but must not assume that such a block exists when reading an archive.
The blocks may be blocked for physical I/O operations. Each record
of n blocks (where n is set by the --blocking-factor=512-size
(-b 512-size) option to tar) is written
with a single `write ()' operation. On magnetic tapes, the result
of such a write is a single record. When writing an archive, the last record
of blocks should be written at the full size, with blocks after the zero block
containing all zeros. When reading an archive, a reasonable system should
properly handle an archive whose last record is shorter than the rest, or
which contains garbage records after a zero block.
The header block is defined in C as follows. In the GNU tar
distribution, this is part of file `src/tar.h':
/* GNU tar Archive Format description. */
/* If OLDGNU_COMPATIBILITY is not zero, tar produces archives which, by
default, are readable by older versions of GNU tar. This can be
overriden by using --posix; in this case, POSIXLY_CORRECT in environment
may be set for enforcing stricter conformance. If OLDGNU_COMPATIBILITY
is zero or undefined, tar will eventually produces archives which, by
default, POSIX compatible; then either using --posix or defining
POSIXLY_CORRECT enforces stricter conformance.
This #define will disappear in a few years. FP, June 1995. */
#define OLDGNU_COMPATIBILITY 1
/*---------------------------------------------.
| `tar' Header Block, from POSIX 1003.1-1990. |
`---------------------------------------------*/
/* POSIX header. */
struct posix_header
{ /* byte offset */
char name[100]; /* 0 = 0x000 */
char mode[8]; /* 100 = 0x064 */
char uid[8]; /* 108 = 0x06C */
char gid[8]; /* 116 = 0x074 */
char size[12]; /* 124 = 0x07C */
char mtime[12]; /* 136 = 0x088 */
char chksum[8]; /* 148 = 0x094 */
char typeflag; /* 156 = 0x09C */
char linkname[100]; /* 157 = 0x09D */
char magic[6]; /* 257 = 0x101 */
char version[2]; /* 263 = 0x107 */
char uname[32]; /* 265 = 0x109 */
char gname[32]; /* 297 = 0x129 */
char devmajor[8]; /* 329 = 0x149 */
char devminor[8]; /* 337 = 0x151 */
char prefix[155]; /* 345 = 0x159 */
/* 500 = 0x1F4 */
};
#define TMAGIC "ustar" /* ustar and a null */
#define TMAGLEN 6
#define TVERSION "00" /* 00 and no null */
#define TVERSLEN 2
/* Values used in typeflag field. */
#define REGTYPE '0' /* regular file */
#define AREGTYPE '\0' /* regular file */
#define LNKTYPE '1' /* link */
#define SYMTYPE '2' /* reserved */
#define CHRTYPE '3' /* character special */
#define BLKTYPE '4' /* block special */
#define DIRTYPE '5' /* directory */
#define FIFOTYPE '6' /* FIFO special */
#define CONTTYPE '7' /* reserved */
/* Bits used in the mode field, values in octal. */
#define TSUID 04000 /* set UID on execution */
#define TSGID 02000 /* set GID on execution */
#define TSVTX 01000 /* reserved */
/* file permissions */
#define TUREAD 00400 /* read by owner */
#define TUWRITE 00200 /* write by owner */
#define TUEXEC 00100 /* execute/search by owner */
#define TGREAD 00040 /* read by group */
#define TGWRITE 00020 /* write by group */
#define TGEXEC 00010 /* execute/search by group */
#define TOREAD 00004 /* read by other */
#define TOWRITE 00002 /* write by other */
#define TOEXEC 00001 /* execute/search by other */
/*-------------------------------------.
| `tar' Header Block, GNU extensions. |
`-------------------------------------*/
/* In GNU tar, SYMTYPE is for to symbolic links, and CONTTYPE is for
contiguous files, so maybe disobeying the `reserved' comment in POSIX
header description. I suspect these were meant to be used this way, and
should not have really been `reserved' in the published standards. */
/* *BEWARE* *BEWARE* *BEWARE* that the following information is still
boiling, and may change. Even if the OLDGNU format description should be
accurate, the so-called GNU format is not yet fully decided. It is
surely meant to use only extensions allowed by POSIX, but the sketch
below repeats some ugliness from the OLDGNU format, which should rather
go away. Sparse files should be saved in such a way that they do *not*
require two passes at archive creation time. Huge files get some POSIX
fields to overflow, alternate solutions have to be sought for this. */
/* Descriptor for a single file hole. */
struct sparse
{ /* byte offset */
char offset[12]; /* 0 */
char numbytes[12]; /* 12 */
/* 24 */
};
/* Sparse files are not supported in POSIX ustar format. For sparse files
with a POSIX header, a GNU extra header is provided which holds overall
sparse information and a few sparse descriptors. When an old GNU header
replaces both the POSIX header and the GNU extra header, it holds some
sparse descriptors too. Whether POSIX or not, if more sparse descriptors
are still needed, they are put into as many successive sparse headers as
necessary. The following constants tell how many sparse descriptors fit
in each kind of header able to hold them. */
#define SPARSES_IN_EXTRA_HEADER 16
#define SPARSES_IN_OLDGNU_HEADER 4
#define SPARSES_IN_SPARSE_HEADER 21
/* The GNU extra header contains some information GNU tar needs, but not
foreseen in POSIX header format. It is only used after a POSIX header
(and never with old GNU headers), and immediately follows this POSIX
header, when typeflag is a letter rather than a digit, so signaling a GNU
extension. */
struct extra_header
{ /* byte offset */
char atime[12]; /* 0 */
char ctime[12]; /* 12 */
char offset[12]; /* 24 */
char realsize[12]; /* 36 */
char longnames[4]; /* 48 */
char unused_pad1[68]; /* 52 */
struct sparse sp[SPARSES_IN_EXTRA_HEADER];
/* 120 */
char isextended; /* 504 */
/* 505 */
};
/* Extension header for sparse files, used immediately after the GNU extra
header, and used only if all sparse information cannot fit into that
extra header. There might even be many such extension headers, one after
the other, until all sparse information has been recorded. */
struct sparse_header
{ /* byte offset */
struct sparse sp[SPARSES_IN_SPARSE_HEADER];
/* 0 */
char isextended; /* 504 */
/* 505 */
};
/* The old GNU format header conflicts with POSIX format in such a way that
POSIX archives may fool old GNU tar's, and POSIX tar's might well be
fooled by old GNU tar archives. An old GNU format header uses the space
used by the prefix field in a POSIX header, and cumulates information
normally found in a GNU extra header. With an old GNU tar header, we
never see any POSIX header nor GNU extra header. Supplementary sparse
headers are allowed, however. */
struct oldgnu_header
{ /* byte offset */
char unused_pad1[345]; /* 0 */
char atime[12]; /* 345 */
char ctime[12]; /* 357 */
char offset[12]; /* 369 */
char longnames[4]; /* 381 */
char unused_pad2; /* 385 */
struct sparse sp[SPARSES_IN_OLDGNU_HEADER];
/* 386 */
char isextended; /* 482 */
char realsize[12]; /* 483 */
/* 495 */
};
/* OLDGNU_MAGIC uses both magic and version fields, which are contiguous.
Found in an archive, it indicates an old GNU header format, which will be
hopefully become obsolescent. With OLDGNU_MAGIC, uname and gname are
valid, though the header is not truly POSIX conforming.
*/
#define OLDGNU_MAGIC "ustar " /* 7 chars and a null */
/* The standards committee allows only capital A through capital Z for
user-defined expansion. */
/* This is a dir entry that contains the names of files that were in the
dir at the time the dump was made. */
#define GNUTYPE_DUMPDIR 'D'
/* Identifies the *next* file on the tape as having a long linkname. */
#define GNUTYPE_LONGLINK 'K'
/* Identifies the *next* file on the tape as having a long name. */
#define GNUTYPE_LONGNAME 'L'
/* This is the continuation of a file that began on another volume. */
#define GNUTYPE_MULTIVOL 'M'
/* For storing filenames that do not fit into the main header. */
#define GNUTYPE_NAMES 'N'
/* This is for sparse files. */
#define GNUTYPE_SPARSE 'S'
/* This file is a tape/volume header. Ignore it on extraction. */
#define GNUTYPE_VOLHDR 'V'
/*--------------------------------------.
| tar Header Block, overall structure. |
`--------------------------------------*/
/* tar files are made in basic blocks of this size. */
#define BLOCKSIZE 512
enum archive_format
{
DEFAULT_FORMAT, /* format to be decided later */
V7_FORMAT, /* old V7 tar format */
OLDGNU_FORMAT, /* GNU format as per before tar 1.12 */
POSIX_FORMAT, /* restricted, pure POSIX format */
GNU_FORMAT /* POSIX format with GNU extensions */
};
union block
{
char buffer[BLOCKSIZE];
struct posix_header header;
struct extra_header extra_header;
struct oldgnu_header oldgnu_header;
struct sparse_header sparse_header;
};
/* End of Format description. */
All characters in header blocks are represented by using 8-bit characters in the local variant of ASCII. Each field within the structure is contiguous; that is, there is no padding used within the structure. Each character on the archive medium is stored contiguously.
Bytes representing the contents of files (after the header block of each
file) are not translated in any way and are not constrained to represent
characters in any character set. The tar format does not
distinguish text files from binary files, and no translation of file contents
is performed.
The name, linkname, magic,
uname, and gname are null-terminated
character strings. All other fileds are zero-filled octal numbers in ASCII.
Each numeric field of width w contains w minus 2 digits,
a space, and a null, except size, and mtime,
which do not contain the trailing null.
The name field is the file name of the file, with
directory names (if any) preceding the file name, separated by slashes.
The mode field provides nine bits specifying file
permissions and three bits to specify the Set UID, Set GID, and Save Text (sticky)
modes. Values for these bits are defined above. When special permissions are
required to create a file with a given mode, and the user restoring files from
the archive does not hold such permissions, the mode bit(s) specifying those
special permissions are ignored. Modes which are not supported by the
operating system restoring files from the archive will be ignored. Unsupported
modes should be faked up when creating or updating an archive; e.g. the group
permission could be copied from the other permission.
The uid and gid fields are the
numeric user and group ID of the file owners, respectively. If the operating
system does not support numeric user or group IDs, these fields should be
ignored.
The size field is the size of the file in bytes; linked
files are archived with this field specified as zero. Modifiers </>
, in particular the --incremental (-G) option.
The mtime field is the modification time of the file at
the time it was archived. It is the ASCII representation of the octal value of
the last time the file was modified, represented as an integer number of
seconds since January 1, 1970, 00:00 Coordinated Universal Time.
The chksum field is the ASCII representation of the
octal value of the simple sum of all bytes in the header block. Each 8-bit
byte in the header is added to an unsigned integer, initialized to zero, the
precision of which shall be no less than seventeen bits. When calculating the
checksum, the chksum field is treated as if it were all blanks.
The typeflag field specifies the type of file archived.
If a particular implementation does not recognize or permit the specified type,
the file will be extracted as if it were a regular file. As this action occurs,
tar issues a warning to the standard error.
The atime and ctime fields are used
in making incremental backups; they store, respectively, the particular file's
access time and last inode-change time.
The offset is used by the --multi-volume (-M)
option, when making a multi-volume archive. The offset is number of bytes into
the file that we need to restart at to continue the file on the next tape, i.e.,
where we store the location that a continued file is continued at.
The following fields were added to deal with sparse files. A file is sparse
if it takes in unallocated blocks which end up being represented as zeros, i.e.,
no useful data. A test to see if a file is sparse is to look at the number
blocks allocated for it versus the number of characters in the file; if there
are fewer blocks allocated for the file than would normally be allocated for a
file of that size, then the file is sparse. This is the method tar
uses to detect a sparse file, and once such a file is detected, it is treated
differently from non-sparse files.
Sparse files are often dbm files, or other database-type files
which have data at some points and emptiness in the greater part of the file.
Such files can appear to be very large when an `ls -l' is done on
them, when in truth, there may be a very small amount of important data
contained in the file. It is thus undesirable to have tar think
that it must back up this entire file, as great quantities of room are wasted
on empty blocks, which can lead to running out of room on a tape far earlier
than is necessary. Thus, sparse files are dealt with so that these empty
blocks are not written to the tape. Instead, what is written to the tape is a
description, of sorts, of the sparse file: where the holes are, how big the
holes are, and how much data is found at the end of the hole. This way, the
file takes up potentially far less room on the tape, and when the file is
extracted later on, it will look exactly the way it looked beforehand. The
following is a description of the fields used to handle a sparse file:
The sp is an array of struct sparse. Each struct
sparse contains two 12-character strings which represent an offset into
the file and a number of bytes to be written at that offset. The offset is
absolute, and not relative to the offset in preceding array element.
The header can hold four of these struct sparse at the moment;
if more are needed, they are not stored in the header.
The isextended flag is set when an extended_header
is needed to deal with a file. Note that this means that this flag can only be
set when dealing with a sparse file, and it is only set in the event that the
description of the file will not fit in the alloted room for sparse structures
in the header. In other words, an extended_header is needed.
The extended_header structure is used for sparse files
which need more sparse structures than can fit in the header. The header can
fit 4 such structures; if more are needed, the flag isextended
gets set and the next block is an extended_header.
Each extended_header structure contains an array of 21 sparse
structures, along with a similar isextended flag that the header
had. There can be an indeterminate number of such extended_headers
to describe a sparse file.
REGTYPE
AREGTYPE
tar, a typeflag value of AREGTYPE
should be silently recognized as a regular file. New archives should be
created using REGTYPE. Also, for backward compatibility, tar
treats a regular file whose name ends with a slash as a directory.
LNKTYPE
linkname
field with a trailing null.
SYMTYPE
linkname field with a trailing null.
CHRTYPE
BLKTYPE
devmajor and devminor
fields will contain the major and minor device numbers respectively.
Operating systems may map the device specifications to their own local
specification, or may ignore the entry.
DIRTYPE
name field should end with a slash. On systems where disk
allocation is performed on a directory basis, the size field
will contain the maximum number of bytes (which may be rounded to the
nearest disk block allocation unit) which the directory may hold. A size
field of zero indicates no such limiting. Systems which do not support
limiting in this manner should ignore the size field.
FIFOTYPE
CONTTYPE
A ... Z
Other values are reserved for specification in future revisions of the
P1003 standard, and should not be used by any tar program.
The magic field indicates that this archive was output in the
P1003 archive format. If this field contains TMAGIC, the uname
and gname fields will contain the ASCII representation of the
owner and group of the file respectively. If found, the user and group IDs are
used rather than the values in the uid and gid
fields.
For references, see ISO/IEC 9945-1:1990 or IEEE Std 1003.1-1990, pages 169-173 (section 10.1) for Archive/Interchange File Format; and IEEE Std 1003.2-1992, pages 380-388 (section 4.48) and pages 936-940 (section E.4.48) for pax - Portable archive interchange.
GNU-формат использует дополнительные файловые тип, описывающие новые типы файлов в архиве. Они указаны ниже.
GNUTYPE_DUMPDIR
'D'
size field gives the total size
of the associated list of files. Each file name is preceded by either a `Y'
(the file should be in this archive) or an `N'. (The file is
a directory, or is not stored in the archive.) Each file name is
terminated by a null. There is an additional null after the last file name.
GNUTYPE_MULTIVOL
'M'
size
field gives the maximum size of this piece of the file (assuming the
volume does not end before the file is written out). The offset
field gives the offset from the beginning of the file where this part of
the file begins. Thus size plus offset should
equal the original size of the file.
GNUTYPE_SPARSE
'S'
GNUTYPE_VOLHDR
'V'
name field contains
the name given after the --label=archive-label
(-V archive-label) option. The size
field is zero. Only the first file in each volume of an archive should
have this type.
You may have trouble reading a GNU format archive on a non-GNU system if
the options --incremental (-G), --multi-volume
(-M), --sparse (-S), or --label=archive-label
(-V archive-label) were used when writing the archive.
In general, if tar does not use the GNU-added fields of the
header, other versions of tar should be able to read the archive.
Otherwise, the tar program will give an error, the most likely
one being a checksum error.