DelphiDabbler CodeSnip
File Format Documentation

# User Database Files

## Contents

- Introduction
- Encoding
- File Format
  - XML File
  - Source Files
- Change Log
- Notes for File Readers

## Introduction

CodeSnip's user defined snippets database is stored in an XML file along with a number of data files located in the user database directory. There is a different user database for each logged on user.

The master XML file is named `database.xml`. It contains all the information about user defined snippets and categories except for the source code of each snippet.

The source code for each snippet is stored in separate, sequentially numbered, `.dat` data files – one per snippet. Each source code file is referenced by the XML file.

There have been several different versions of the XML file format. The differences between versions are explained below. Details of all the changes between versions are listed in the Change Log at the end of this document

## Encoding

CodeSnip 4 stores all user database files using UTF-8 encoding. Files are saved without any UTF-8 preamble (BOM). The XML processing instruction of `database.xml` has an "encoding" atrribute set to "UTF-8".

Prior to CodeSnip v4 (and database v5) source code data files were encoded using ANSI code page 1252. The XML file was in UTF-8, but its XML processing instruction had no "encoding" atrribute.

CodeSnip v4 must be able to work with all these encodings because it may inherit a copy of a user database in an earlier format.

## File Format

### XML File

There have been six different versions of the XML file formats – v1 to v6. Tags from all six versions are explained below with notes describing which versions a tag applies to. Where there is no note the tag is valid in all versions.

XML processing instruction
  Attributes:
  version
    Always set to "1.0"
  encoding
    Character encoding used for file.
    - versions 1..4: Attribute not provided.
    - version 5 and later: Always set to "UTF-8".
codesnip-data
  Parent node for whole file. Attributes are:
  watermark
    Identifies file as correct type – always set to "531257EA-1EE3-4B0F-8E46-C6E7F7140106".
  version
    Identifies major version of file. Determines which tags are valid and rules concerning content. Valid versions are 1..6.
codesnip-data/categories
  Contains list of all categories.
codesnip-data/categories/category
  Contains information about a category. Attributes are:
  id
    Internal (unique) id of category.
codesnip-data/categories/category/description
  Description of category.
codesnip-data/categories/category/cat-routines
  Contains list of name of all snippets in category. Omitted if there are no snippets in category.
codesnip-data/categories/category/cat-routines/pascal-name
  Contains name of a snippet. One per each snippet in category.
  - versions 1..4: Name must begin with an English language letter or the underscore.
  - version 5 and later: Name can begin with any character that is valid as the first character of a Unicode Pascal identifier.
codesnip-data/routines
  Contains a list of all user defined snippets.
codesnip-data/routines/routine
  Contains information about a snippet. One per snippet. Attribute:
  name
    Name of snippet.
    - versions 1..4: Name must begin with an English language letter or the underscore.
    - version 5 and later: Name can begin with any character that is valid as the first character of a Unicode Pascal identifier.
codesnip-data/routines/routine/cat-id
  Id of category to which snippet belongs.
codesnip-data/routines/routine/description
  Description of snippet.
  - versions 1..5: Content is a single line of plain text.
  - version 6.0 to 6.10: Content is formatted text encoded in REML markup. REML v4 is supported.
  - version 6.11 & 6.12: Content is formatted text encoded in REML markup. REML v5 is supported.
  - version 6.13 & later: Content is formatted text encoded in REML markup. REML v6 is supported.
codesnip-data/routines/routine/source-code
  Name of file containing snippet's source code. No path information included.
codesnip-data/routines/routine/highlight-source
  - versions 1..5: Not supported.
  - version 6:
    Flag indicating if snippet source code can be highlighted using syntax highlighter. Permissible values are:
    - "0" – do not syntax highlight source code
    - "1" – syntax highlight source code
    Omitting this tag is permitted. Value defaults to "1" in this case.
codesnip-data/routines/routine/display-name
  - versions 1..5: Not supported.
  - version 6: Display name of snippet. Can contain any characters and need not be unique. Present only if snippet has a display name that is different to the value of the name attribute of the codesnip-data/routines/routine tag.
codesnip-data/routines/routine/comments
  - version 1: Additional comments about snippets.
  - version 2 and later: Not supported.
codesnip-data/routines/routine/credits
  - version 1: Credits for snippets. May contain a single piece of text, delimited by "[" and "]" that can form a hyperlink. URL for the hyperlink is provided in codesnip-data/routines/routine/credits-url.
  - version 2 and later: Not supported.
codesnip-data/routines/routine/credits-url
  - version 1: URL required by codesnip-data/routines/routine/credits tag. Present only if codesnip-data/routines/routine/credits requires a hyperlink.
  - version 2 and later: Not supported.
codesnip-data/routines/routine/extra
  - version 1: Not supported.
  - version 2 and later: Additional information about a snippet. Content is formatted text encoded in REML markup.
    - version 2: supports REML v1.
    - version 3: supports REML v2.
    - version 4: supports REML v3.
    - versions 5 & 6.10: supports REML v4.
    - version 6.11 & 6.12: supports REML v5.
    - version 6.13 & later: supports REML v6.
codesnip-data/routines/routine/standard-format
  - versions 1 and 2: Flag indicating if snippet is in "standard format". Value of 1 indicates true and 0 indicates false.
  - version 3 and later: Not supported.
codesnip-data/routines/routine/kind
  - versions 1 and 2: Not supported.
  - version 3 and later: Value indicating kind of snippet. Permissible values are:
    - versions 3 and 4: "freeform", "routine", "type" & "const".
    - version 5 and 6: "freeform", "routine", "type", "const", "class" & "unit".
codesnip-data/routines/routine/compiler-results
  Contains a list of compile results for the snippet.
codesnip-data/routines/routine/compiler-results/compiler-result
  Entry for each known compiler. Attribute is:
  id
    Identifies compiler. Valid identifiers are are one of:
    - d2 – Delphi 2 compiler (all versions)
    - d3 – Delphi 3 compiler (all versions)
    - d4 – Delphi 4 compiler (all versions)
    - d5 – Delphi 5 compiler (all versions)
    - d6 – Delphi 6 compiler (all versions)
    - d7 – Delphi 7 compiler (all versions)
    - d2005 – Delphi 2005 compiler (all versions)
    - d2006 – Delphi 2006 compiler (all versions)
    - d2007 – Delphi 2007 compiler (all versions)
    - d2009 – Delphi 2009 compiler (v1.1 & later)
    - d2010 – Delphi 2010 compiler (v4.1 & later)
    - dXE – Delphi XE compiler (v4.2 & later)
    - dXE2 – Delphi XE2 compiler (v4.3 & later)
    - dXE3 – Delphi XE3 compiler (v4.4..v4.5 and v6.1 & later)
    - dXE4 – Delphi XE4 compiler (v4.5 only)
      Note: CodeSnip 3 used correct dXE4 id, but CodeSnip 4 did not (see dDX4 below).
    - dDX4 – Delphi XE4 compiler (v6.2 & later)
      Note: CodeSnip 4 dDX4 in error instead of dXE4 used by CodeSnip 3 (see above). The erroneous value was retained for backwards compatibility reasons.
    - dXE5 – Delphi XE5 compiler (v6.3 & later)
    - dXE6 – Delphi XE6 compiler (v6.4 & later)
    - dXE7 – Delphi XE7 compiler (v6.5 & later)
    - dXE8 – Delphi XE8 compiler (v6.6 & later)
    - d10s – Delphi 10 Seattle compiler (v6.7 & later)
    - d101b – Delphi 10.1 Berlin compiler (v6.8 & later)
    - d102t – Delphi 10.2 Tokyo compiler (v6.9 & later)
    - d103r – Delphi 10.3 Rio compiler (v6.9 & later)
    - d104s – Delphi 10.4 Sydney compiler (v6.9 & later)
    - d11a – Delphi 11.x Alexandria compiler (v6.10 & later)
    - d12y – Delphi 12 Athens compiler (v6.12 & later)
    - fpc – Free Pascal compiler (all versions)
  Values are:
  - "Y" – Compiles with the identified compiler.
  - "W" – Compiles with the identified compiler with warnings.
  - "N" – Does not compile with the identified compiler.
  - "Q" – Compile result for this compiler is not known.
  Omitting a tag for a certain compiler is permitted and is equivalent to specifying "Q".
codesnip-data/routines/routine/units
  List of required units.
codesnip-data/routines/routine/units/pascal-name
  Name of a unit within unit list.
codesnip-data/routines/routine/depends
  List of required snippets.
codesnip-data/routines/routine/depends/pascal-name
  Name of a snippet within depends list.
  - versions 1..4: Name must begin with an English language letter or the underscore.
  - version 5 and later: Name can begin with any character that is valid as the first character of a Unicode Pascal identifier.
codesnip-data/routines/routine/xref
  List of cross-referenced snippets.
codesnip-data/routines/routine/xref/pascal-name
  Name of a snippet within cross-reference list.
  - versions 1..4: Name must begin with an English language letter or the underscore.
  - version 5 and later: Name can begin with any character that is valid as the first character of a Unicode Pascal identifier.

### Source Code Files

Source code is stored separately from the main XML file. The source code of each snippet has its own file. Files are numbered sequentially and have a `.dat` extension, for example `6.dat`.

Source code files are referenced by the codesnip-data/routines/routine/source-code tag in the database's XML file.

## Change Log

This section describes the changes between versions of the file format.

There were small changes within versions, that probably should have been given minor version numbers - but weren't. Such minor numbers have been assigned retrospectively below in order to better explain when in-version changes actually took place.

File formats v4 and v5/v6 actually overlapped in the dates they were in use. This is because v4 was used by CodeSnip v3 and v5/v6 were used by CodeSnip 4. Those two versions of CodeSnip were maintained in parallel for a while.

Version 1 - 15 September 2008

  Introduced with CodeSnip v2.0.

  Supported Delphi compilers from Delphi 2 to Delphi 2007 plus Free Pascal.

  REML not supported.

  Data files were ANSI text using code page 1252. The XML file was in UTF-8 format with no BOM and no XML encoding attribute.

  Version 1.1 - 11 October 2008
    Updated with CodeSnip v2.1 to add support for Delphi 2009.
Version 2 - 31 December 2008

  Introduced with CodeSnip v2.2.5.

  Removed following tags:

  - codesnip-data/routines/routine/comments
  - codesnip-data/routines/routine/credits
  - codesnip-data/routines/routine/credits-url

  Added following tag:

  - codesnip-data/routines/routine/extra

  The version of REML supported by the codesnip-data/routines/routine/extra tag was v1.

Version 3 - 29 June 2009

  Introduced with CodeSnip v3.0.

  The following tag is no longer supported:

  - codesnip-data/routines/routine/standard-format

  The following tag was introduced:

  - codesnip-data/routines/routine/kind

  The version of REML supported by the codesnip-data/routines/routine/extra tag was updated to v2.

Version 4 - 06 July 2009

  Introduced with CodeSnip v3.0.1.

  The version of REML supported by the codesnip-data/routines/routine/extra tag was updated to v3.

  Version 4.1 - 24 September 2009
    Updated with CodeSnip v3.4 to add support for Delphi 2010.
  Version 4.2 - 23 October 2010
    Updated with CodeSnip v3.8.0 to add support for Delphi XE.
  Version 4.3 - 07 September 2011
    Updated with CodeSnip v3.9.0 to add support for Delphi XE2.
  Version 4.4 - 17 September 2012
    Updated with CodeSnip v3.11.0 to add support for Delphi XE3.
  Version 4.5 - 02 May 2013
    Updated with CodeSnip v3.12.0 to add support for Delphi XE4.
Version 5 - 21 April 2012

  Introduced with CodeSnip v4.0 alpha 2.

  The XML file's encoding was explicitly set to "UTF-8" by setting the encoding attribute of the XML processing instruction to this value.

  Snippet names, wherever they occur in the XML file, can now begin with any character that is a valid first character of a Unicode Pascal identifier. Previously the first character of the attribute had to be one of 'A'..'Z', 'a'..'z' or '_'.

  Data files changed to use UTF-8 encoding with no BOM instead of the system default encoding.

  New "class" and "unit" snippet kinds supported.

  The version of REML supported by the codesnip-data/routines/routine/extra tag was updated to v4.

Version 6 - 11 August 2012

  Introduced with CodeSnip v4.0 beta 1.

  A snippet's description is now stored as formatted text using REML v4 markup. Previously the description was plain text.

  The following tags were introduced:

  - codesnip-data/routines/routine/display-name
  - codesnip-data/routines/routine/highlight-source
  Version 6.1 - 14 September 2012
    Updated with CodeSnip v4.0 RC 1 to add support for Delphi XE3.
  Version 6.2 - 02 May 2013
    Updated with CodeSnip v4.5.0 to add support for Delphi XE4.
  Version 6.3 - 12 September 2013
    Updated with CodeSnip v4.8.0 to add support for Delphi XE5.
  Version 6.4 - 30 April 2014
    Updated with CodeSnip v4.9.0 to add support for Delphi XE6.
  Version 6.5 - 12 September 2014
    Updated with CodeSnip v4.10.0 to add support for Delphi XE7.
  Version 6.6 - 06 May 2015
    Updated with CodeSnip v4.12.0 to add support for Delphi XE8.
  Version 6.7 - 05 September 2015
    Updated with CodeSnip v4.13.0 to add support for Delphi 10 Seattle.
  Version 6.8 - 13 July 2016
    Updated with CodeSnip v4.15.0 to add support for Delphi 10.1 Berlin.
  Version 6.9 - 31 July 2020
    Updated with CodeSnip v4.17.0 to add support for Delphi 10.2 Tokyo, Delphi 10.3 Rio and Delphi 10.4 Sydney.
  Version 6.10 - 13 September 2021
    Updated with CodeSnip v4.18.0 to add support for Delphi 11.x Alexandria.
  Version 6.11 - 16 December 2022
    Updated with CodeSnip v4.21.0 to add support for REML v5, which is backwards compatible with REML v4.
  Version 6.12 - 7 November 2023
    Updated in time for CodeSnip v4.22.0 to add support for Delphi 12 Athens.
  Version 6.13 - 2 April 2024
    Updated with CodeSnip v4.23.0 to add support for REML v6, which is backwards compatible with REML v4.

## Notes for File Readers

To ensure backwards compatibility with all user database versions file reader software that works with the latest version of CodeSnip needs to be able to interpret older formats as follows.

### Handling redundant XML tags

Readers of version 1 files must convert the contents of the the following tags:

- codesnip-data/routines/routine/comments
- codesnip-data/routines/routine/credits
- codesnip-data/routines/routine/credits-url

into valid REML code that simulates the parsed content of the codesnip-data/routines/routine/extra tag.

Readers of v1 and v2 files should map a codesnip-data/routines/routine/standard-format value of "0" to a codesnip-data/routines/routine/kind value of "freeform" and a value of "1" to "routine".

Readers of v1 to v5 files must:

- Convert the plain text snippet description read from codesnip-data/routines/routine/description into the REML equivalent of a single paragraph containing the description.
- Proceed as if a codesnip-data/routines/routine/highlight-source tag with value "1" had been specified.

Readers of v2 and later files may parse REML from any file version as if it were REML v6, since all versions of REML up to v6 are compatible.

### Handling Text File Encodings

Readers of v1 to v4 files should interpret all source code .dat files as encoded in ANSI code page 1252 - the files were created using the default code page in the UK, which is 1252. The XML file should be assumed to be in UTF-8 format, regardless of the absence of an encoding attribute.

v5 and later files will always be encoded in UTF-8.