DelphiDabbler CodeSnip
File Format Documentation

# Main Database Files

## Introduction

The are two versions of the Code Snippets Database: v1 and v2. Both versions are similar and both are described here. Where one version differs from another the differences will be noted. Where there is no such note there is no difference between the versions.

The database is organised as a set of categories, each of which contains one or more code snippets.

Meta data describing the categories and snippets is stored in a set of `.ini` files. In addition there are numerous `.dat` files, each of which contains the source code of a snippet.

There are additional files containing further information.

All the files are plain text, encoded in UTF-8 format with UTF-8 preamble (BOM).

## Encoding

### Version 1

With CodeSnip v4 all the files are plain text, encoded in UTF-8 format with UTF-8 preamble (BOM). Prior to CodeSnip v4 local database files were encoded using the system default ANSI code page.

CodeSnip v4 must be able to work with both encodings because it may need to use a database installed by CodeSnip v3 or earlier.

### Version 2

All the files are plain text, encoded in UTF-8 format with UTF-8 preamble (BOM).

## File Format

### Meta Data Files

There are two kinds of meta data files:

1. `categories.ini` lists the categories contained in the collection.
2. A set of further `.ini` files, one for each category specified in `categories.ini`. Each of these files contains numerous details of each snippet along with the name of the file containing the source code.

#### Format Of `categories.ini`

Each section in this file defines a category and has the following format:

    [<cat-id>]
    Desc=<category-description>
    Ini=<ini-file-name>

The sections in these files are named with a unique identifier that identifies a category within the collection. The section names must be alphanumeric characters. The values have the following purpose:

`Desc`

  A brief, human readable, description of the category. Any valid UTF-8 text is permitted.

`Ini`

  The name of the `.ini` file that contains information about each snippet in the category. Must be a valid file name, without path. The files must be stored in the same directory as `category.ini`.

Here is an example (this is a copy of the "Date and Time" category's entry):

    [date]
    Desc=Date and Time
    Ini=date.ini

#### Format Of Individual Category `.ini` Files

Each section in these files defines a snippet and has the following content:

    [snippet-name]
    DisplayName=<snippet-display-name>
    Desc=<description-text>
    DescEx=<description-REML>
    Snip=<snippet-file-name>
    Units=<required-units>
    Depends=<required-snippets>
    SeeAlso=<cross-referenced-snippets>
    Credits=<credits-text>
    Credits_URL=<url-used-in-credits>
    Comments=<comments-text>
    Delphi2=<Y|N|Q|W>
    Delphi3=<Y|N|Q|W>
    Delphi4=<Y|N|Q|W>
    Delphi5=<Y|N|Q|W>
    Delphi6=<Y|N|Q|W>
    Delphi7=<Y|N|Q|W>
    Delphi2005Win32=<Y|N|Q|W>
    Delphi2006Win32=<Y|N|Q|W>
    Delphi2007=<Y|N|Q|W>
    Delphi2009Win32=<Y|N|Q|W>
    Delphi2010=<Y|N|Q|W>
    DelphiXE=<Y|N|Q|W>
    DelphiXE2=<Y|N|Q|W>
    DelphiXE3=<Y|N|Q|W>
    DelphiXE4=<Y|N|Q|W>
    DelphiXE5=<Y|N|Q|W>
    DelphiXE6=<Y|N|Q|W>
    DelphiXE7=<Y|N|Q|W>
    DelphiXE8=<Y|N|Q|W>
    Delphi10S=<Y|N|Q|W>
    Delphi101B=<Y|N|Q|W>
    Delphi102T=<Y|N|Q|W>
    Delphi103R=<Y|N|Q|W>
    Delphi104S=<Y|N|Q|W>
    Delphi11A=<Y|N|Q|W>
    FPC=<Y|N|Q|W>
    Extra=<extra-info-REML>
    StandardFormat=<0|1>
    Kind=<freeform|routine|type|const|class|unit>
    TestInfo=<none|basic|advanced>
    MinVer=<version-number>

The sections in these files are named with identifiers that uniquely identify a snippet. This must be a valid Unicode Pascal identifier. The keys in a section have the following purpose:

`DisplayName`

  Human readable name of snippet. The value is optional. If not present then the snippet identifier that names the section is used as the human readable name.

  - v1: There are no restriction on the characters used.
  - v2: May be any valid UTF-8 string of up to 64 characters.
`Desc`

  Description of the snippet as plain text. Ignored if a non-empty `DescEx` key is present.

`DescEx`

  Formatted description of the snippet using REML1 code. May be empty, in which case the `Desc` key is used. `Desc` is ignored if `DescEx` has a non-empty value.

`Snip`

  Name of the file containing this snippet's source code. The file name must:

  - be a valid file name;
  - contain no path information;
  - exist in the same directory as the `.ini` file;
  - be unique within the directory.
`Units`

  Comma separated list of units required to compile the snippet. May be omitted or empty if no units are required. The `System` unit is assumed and does not need to be specified. Unit names must be valid Pascal identifiers.

`Depends`

  Comma separated list of the identifiers of snippets that are required to compile this snippet. May be omitted or empty if there are no such dependencies. Snippet identifiers must exist within the collection and must not cause a circular reference back to this snippet.

`SeeAlso`

  Comma separated list of the identifiers of cross-referenced snippets. May be omitted or empty. Snippet identifiers should exist in the collection.

`Credits`2

  Text string that notes any credits / acknowledgements. May be omitted or empty. `Credits` may optionally contain one (and only one) section of text delimited by `[` and `]` characters that indicate the presence of a hyperlink. The enclosed text is used as the text of a hyperlink whose URL must be specified in the `Credits_URL` key.

  `Credits` is ignored if a non-empty `Extra` key is present.

`Credits_URL`2

  The URL used in any hyperlink present in the `Credits` key. Must be present if `Credits` specifies a hyperlink.

  Ignored if `Credits` contains no hyperlink section or if a non-empty `Extra` key is present.

  - v1: Should be empty if `Credits` contains no hyperlink.
`Comments`

  Text string containing any additional comments about the snippet. Optional. Ignored if a non-empty `Extra` key is present.

`DelphiXXX` & `FPC`

  This related group of keys describe compilation results for the snippets on various compilers. Valid key names are:

  - `Delphi2` – Delphi 2 compiler
  - `Delphi3` – Delphi 3 compiler
  - `Delphi4` – Delphi 4 compiler
  - `Delphi5` – Delphi 5 compiler
  - `Delphi6` – Delphi 6 compiler
  - `Delphi7` – Delphi 7 compiler
  - `Delphi2005Win32` – Win32 personality of Delphi 2005 compiler
  - `Delphi2006Win32` – Win32 personality of Delphi 2006 compiler
  - `Delphi2007` – Delphi 2007 compiler
  - `Delphi2009Win32` – Win32 personality of Delphi 2009 compiler
  - `Delphi2010` – Delphi 2010 compiler
  - `DelphiXE` – Delphi XE compiler
  - `DelphiXE2` – Delphi XE2 compiler
  - `DelphiXE3` – Delphi XE3 compiler
  - `DelphiXE4` – Delphi XE4 compiler
  - `DelphiXE5` – Delphi XE5 compiler
  - `DelphiXE6` – Delphi XE6 compiler
  - `DelphiXE7` – Delphi XE7 compiler
  - `DelphiXE8` – Delphi XE8 compiler
  - `Delphi10S` – Delphi 10 Seattle compiler
  - `Delphi101B` – Delphi 10.1 Berlin compiler
  - `Delphi102T` – Delphi 10.2 Tokyo compiler *
  - `Delphi103R` – Delphi 10.3 Rio compiler *
  - `Delphi104S` – Delphi 10.4 Sydney compiler *
  - `Delphi11A` – Delphi 11.x Alexandria compiler *
  - `Delphi12A` – Delphi 12 Athens compiler *
  - `FPC` – Free Pascal compiler

  Valid values for these keys are:

  - `Y` – compiles on the compiler
  - `N` – does not compile on the compiler
  - `W` – compiles with warnings on the compiler
  - `Q` – compilation result unknown

  If any of the above compilers is not present, the compile result is assumed to be `Q`. A compile result of `W` is obsolete and treated as if it were `Y`.

  Any compiler not recognised by a given version of CodeSnip is ignored.

  * These compilers are not yet supported by the online DelphiDabbler Code Snippets Database.

`Extra`

  Optional. Provides extra information about the snippet. When present the value must be a valid string of REML1 code.

  If omitted the extra information is generated from the values of any `Comments`, `Credits` and `Credits_URL` keys.

  When `Extra` has a non-empty value the `Comments`, `Credits` and `Credits_URL` keys are ignored.

`StandardFormat`

  - v1: Flag indicating if a snippet is in "standard format". Valid values are:

    - `0` – no.
    - `1` – yes.

    Ignored if the `Kind` key is present.

  - v2: not supported.
`Kind`

  The kind (or type) of this code snippet. Valid values are:

  - `freeform` – a freeform snippet that doesn't conform to any other other supported type.
  - `routine` – a Pascal routine (function or procedure).
  - `type` – a simple Pascal type definition.
  - `const` – a Pascal constant definition.
  - `class` – a Pascal class or advanced record definition and implementation.
  - `unit` – a complete Pascal unit.

  If `Kind` is omitted or has no value then:

  - v1: If the `StandardFormat` key has value `0` then `Kind` defaults to `freeform`. If `StandardFormat` has value `1` or is not present then `Kind` defaults to `routine`.
  - v2: If `Kind` is not present then its value defaults to `freeform`.
`TestInfo`

  Testing information for the snippet. Valid values are:

  - `none` – the snippet has not been tested.
  - `basic` – the snippet has passed some simple testing.
  - `advanced` – the snippet has passed more advanced testing, usually unit tests.

  If `TestInfo` is not present then its value defaults to `basic`.

`MinVer`

  Redundant key left over from an earlier, and now unsupported version of CodeSnip.

  - v1: Ignored.
  - v2: Not supported.

Each key should occur only once. If more than one value of the same type is present the result is not defined and may change between program versions.

This file format is quite messy, with several keys having similar or overlapping purposes. This has happened because new features have been added over time while preserving backward compatibility.

#### Pre-processor directives

Pre-processor directives are supported only in v1 of the database.

A simple pre-processor was added to CodeSnip v3 and later that can process special pre-processor instructions contained in .ini files. The pre-processor symbols ignored by CodeSnip versions earlier than v3. These symbols allow the program's version number to be tested and different parts of the file to be read or ignored accordingly.

Pre-processor instructions all begin with `;`, the `.ini` file comment character, followed by a `#` and then the instruction name. Valid instructions are:

`if-ver-eq <version>`
  Checks if application version is equal to <version>.
`if-ver-neq <version>`
  Checks if the application version is not equal to <version>.
`if-ver-lt <version>`
  Checks if the application version is less than <version>.
`if-ver-lte <version>`
  Checks if the application version is less than or equal to <version>. 3
`if-ver-gt <version>`
  Checks if the application version is greater than <version>.
`if-ver-gte <version>`
  Checks if the application version is greater than or equal to <version>. 3
`if-ver-inrange <version-lo> <version-hi>`
  Checks if the application version is in the range of versions specified from <version-lo> to <version-hi>, inclusive.
`end-if`
  Ends a block started by any of the `if-xxx` instructions above. The block of text is only evaluated if the `if-xxx` instruction evaluates true.

In all cases version numbers are dotted quads of the form `X.X.X.X` where `X` is a sequence of digits. All except the first digit can be left out: omitted digits are assumed to be zero, so that `1.0.0.0`, `1.0.0`, `1.0` and `1` are all equivalent.

`if-ver-lt 3` can be used to ignore a whole snippet definition in a normal `.ini` file, leaving the possibility of having a duplicate definition for CodeSnip v3 or later stored in a matching `.3.ini` file. E.g.:

`date.ini`:

      ;#if-ver-lt 3
      [Snippet1]
      Desc="My description"
      Snip=044.dat
      ...
      ;#end-if

`date.3.ini`:

    [Snippet1]
      Desc="My second description"
      Snip=044.3.dat
      ...

#### Additional `.ini` Files For CodeSnip v3 and v4

Additional `.ini` files are supported only in v1 of the database.

From CodeSnip v3 additional `.ini` files with `.3` interposed between the file name and extension are supported (e.g. `category.3.ini` and `date.3.ini` are valid names).

From CodeSnip v4 similar files using the interposed `.4` extension are supported in addition to the `.3` files (e.g `winsys.4.ini`).

CodeSnip concatenates all `.ini` files with the same base name before analysing the file. For example if `date.ini`, `date.3.ini` and `date.4.ini` all exist then `date.3.ini` and `date.4.ini` are appended to `date.ini` before the files are analysed.

The `.3` "inner" extension was provided for category files so that constant and type definition snippets could be hidden from CodeSnip versions below v3 but be visible to CodeSnip v3 and later. The new style constant and type definition snippets are only recorded in `.3` style files.

`categories.3.ini` has been provided to add any sections that need to be hidden from CodeSnip versions before v3. The "types" and "consts" sections are recorded in this file.

`.4` "inner" extensions are there to support any CodeSnip v4 specific features that can't be hidden from earlier versions.

Finally, it is possible that a `name.3.ini` or `name.4.ini` file will exist with no matching `name.ini`. In this case `categories.ini` (or `categories.3.ini`) must still refer to `name.ini`, not `name.3.ini` or `name.4.ini`. The program works out which files to load.

### Source Code Files

There is a separate source code file for each snippet. These file names must be named exactly as specified in the related category `.ini` file's `Snip` key. They are usually numbered from `001` and have a `.dat` extension, but this is not a requirement.

For example `102.dat` is a typical source code file name.

v1 only: Sometimes there is a separate file for later versions of CodeSnip, so an occasional file named, for example, `102.3.dat` may be found.

### Credits Files

There are two credits files that list the people who have contributed code to the database or helped to test the code, respectively.

The files are named, respectively:

- v1: `contrib.txt` and `testers.txt`.
- v2: `CONTRIBUTORS` and `TESTERS`.

Each file is simply a list of names, in plain text format, with each name on a separate line. No blank lines are permitted.

The files may be empty if there are no contributors and/or testers, but they must be present.

The credits files are not referenced by, and do not reference, any of the other files in the database.

### License Files

License files are supported only in v2 of the database.

There are two files relating to license (and copyright) information: the full text of the license in human readable format and a file providing machine readable meta data about the license and copyright.

#### Full License Text

This is a plain UTF-8 text file named `LICENSE` that contains the license that applies to the source code in the collection.

This file must be present and non-empty.

The `LICENSE` file is not referenced by, and does not reference, any of the other files in the collection.

#### License Meta Information

A file named `LICENSE-INFO` is provided that contains information about the license in machine readable form. Information is in `Key=Value` format as follows:

    LicenseName=<human-readable-name>
    LicenseSPDX=<code>
    LicenseURL=<url>
    CopyrightDate=<date-range>
    CopyrightHolder=<name>
    CopyrightHolderURL=<url>

The keys have the following meaning:

`LicenseName`

  The name of the license as plain text. E.g. `MIT License` or `Mozilla Public License 2.0`.

`LicenseSPDX`

  The Open Source Initiative (OSI) SPDX short identifier of the license, if any. E.g. `MIT` or `MPL-2.0`. If the license does not have a SPDX identifier this key must either be omitted or be empty.

  For a list of OSI licenses with their SPDXs see https://opensource.org/licenses/alphabetical.

`LicenseURL`

  The URL of an online copy of the license. E.g. `https://opensource.org/licenses/MIT`. If the license has no URL then this key must be omitted or be empty.

`CopyrightDate`

  The date of the copyright or range of copyright dates as plain text. E.g. `2020` or `2005-2020`.

`CopyrightHolder`

  The name of the copyright holder as plain text. Where there are contributors either list them all or append `and contributors`4 to the primary copyright holder's name. E.g. `Joe Bloggs` or `Annie Smith, Joe Bloggs and Jessie Sharp` or `Annie Smith and Contributors`.

`CopyrightHolderURL`

  The URL of a web page where details of the copyright holder(s) or primary copyright holder can be found. E.g. `https://example.com/joe-blogs-bio`. This key is optional.

The `LICENSE-INFO` file is not referenced by, and does not reference, any of the other files in the collection.

### Version Information File

The version information file is supported only in v2 of the database.

There is a plain UTF-8 text file named `VERSION` that contains the version number of the database in the form `vX.X.X`. where `X` represents a non-negative integer. The file is required and must be non-empty.

Version numbers must be incremented according to to the principles of semantic versioning.

The current major version is v25.

The `VERSION` file is not referenced by, and does not reference, any of the other files in the collection.

## Footnotes

1. REML is a text markup language used to format text. REML version 6 is supported. The REML format is documented here.

2. Here is an example of how the `Credits` and `Credits_URL` key work together.

   If `Credits`=`"See [example]"` and `Credits_URL`=`"https://example.com"` and the `Extra` key is empty or missing then the extra text generated will be `See <a href="example.com">example 1</a>`.

3. Due to a bug going back to CodeSnip v3.0 `if-ver-gte` and `if-ver-lte` have never worked correctly: they have always been interpreted as `if-ver-gt` and `if-ver-lt` respectively.

   A fix to this could now cause more problems than it solves in older version of CodeSnip, so the implementation of the problematic directives has been removed from later versions of the program and all occurrences of the directives in the database have been replaced by suitable `if-ver-inrange` directives.

4. If "contributors" is specified as part of the `CopyrightHolder` key value in `LICENSE-INFO` then the `CONTRIBUTORS` file must contain a list of all the contributors.

5. Version tracking was not done before v2.0.0. However it is safe to assume, using semantic versioning, that the current format is the second major release. This is because all previous database versions were backwards compatible and therefore all belonged to the same major version, which, logically, must have been v1. The fact that this version of the database breaks that backward compatibility means the major version must be bumped.