Core ML Model
==============

A Core ML model consists of a specification version and a model description,
and can be any one of the following types:

Neural Networks
  - ``MILSpec.Program``
  - ``NeuralNetwork``

Regressors
  - ``GLMRegressor``
  - ``SupportVectorRegressor``
  - ``TreeEnsembleRegressor``
  - ``NeuralNetworkRegressor``
  - ``BayesianProbitRegressor``

Classifiers
  - ``NeuralNetworkClassifier``
  - ``TreeEnsembleClassifier``
  - ``GLMClassifier``
  - ``KNearestNeighborsClassifier``
  - ``SupportVectorClassifier``
  - ``KNearestNeighborsClassifier``

Other models
  - ``CustomModel``
  - ``TextClassifier``
  - ``WordTagger``
  - ``Gazetteer``
  - ``WordEmbedding``
  - ``VisionFeaturePrint``
  - ``LinkedModel``
  - ``SoundAnalysisPreprocessing``
  - ``ItemSimilarityRecommender``
  - ``ClassConfidenceThresholding``

Feature Engineering
  - ``Imputer``
  - ``Scaler``
  - ``Normalizer``
  - ``OneHotEncoder``
  - ``CategoricalMapping``
  - ``FeatureVectorizer``
  - ``DictVectorizer``
  - ``ArrayFeatureExtractor``
  - ``NonMaximumSuppression``

Pipelines
  - ``PipelineClassifier``
  - ``PipelineRegressor``
  - ``Pipeline``

Simple Mathematical Functions
  - ``Identity``

Data Structures, Feature Types, and Parameters
  - ``DataStructures``
  - ``FeatureTypes``
  - ``Parameters``



Pipeline
________________________________________________________________________________

A pipeline consists of one or more models.


.. code-block:: proto

    message Pipeline {
        repeated Model models = 1;
    
        // Optional names given for each model
        // If not supplied it defaults to ["model0",..., "model"(models.size()-1)]
        // These names can be used to disambiguate the scope / domain of a parameter
        repeated string names = 2;
    }






PipelineClassifier
________________________________________________________________________________

A classifier pipeline.


.. code-block:: proto

    message PipelineClassifier {
        Pipeline pipeline = 1;
    }






PipelineRegressor
________________________________________________________________________________

A regressor pipeline.


.. code-block:: proto

    message PipelineRegressor {
        Pipeline pipeline = 1;
    }





FeatureDescription
________________________________________________________________________________

A feature description consisting of a name, short description, and type.


.. code-block:: proto

    message FeatureDescription {
        string name = 1;
        string shortDescription = 2;
        FeatureType type = 3;
    }





Metadata
________________________________________________________________________________

Model metadata,
consisting of a short description, a version string,
an author, a license, and any other user defined
key/value metadata.


.. code-block:: proto

    message Metadata {
        string shortDescription = 1;
        string versionString = 2;
        string author = 3;
        string license = 4;
        map<string, string> userDefined = 100;
    }





ModelDescription
________________________________________________________________________________

A description of a model,
consisting of descriptions of its input and output features.
Both regressor and classifier models require the name of the
primary predicted output feature (``predictedFeatureName``).
Classifier models can specify the output feature containing
probabilities for the predicted classes
(``predictedProbabilitiesName``).


.. code-block:: proto

    message ModelDescription {
        repeated FeatureDescription input = 1;
        repeated FeatureDescription output = 10;
    
        // [Required for regressor and classifier models]: the name
        // to give to an output feature containing the prediction.
        string predictedFeatureName = 11;
    
        // [Optional for classifier models]: the name to give to an
        // output feature containing a dictionary mapping class
        // labels to their predicted probabilities. If not specified,
        // the dictionary will not be returned by the model.
        string predictedProbabilitiesName = 12;
    
        repeated FeatureDescription trainingInput = 50;
    
        Metadata metadata = 100;
    }



SerializedModel
________________________________________________________________________________




.. code-block:: proto

    message SerializedModel {
        // Identifier whose content describes the model type of the serialized protocol buffer message.
        string identifier = 1;
    
        // Must be a valid serialized protocol buffer of the above specified type.
        bytes model = 2;
    }





Model
________________________________________________________________________________

A Core ML model consisting of a specification version, a model description,
and a model type.
Core ML model compatibility is indicated by
a monotonically increasing specification version number,
which is incremented any time a backward-incompatible change is made
(this is functionally equivalent to the MAJOR version number
described by `Semantic Versioning 2.0.0 <http://semver.org/>`_).

Specification Versions : OS Availability (Core ML Version)

1 : iOS 11, macOS 10.13, tvOS 11, watchOS 4 (Core ML 1)

    - Feedforward & Recurrent Neural Networks
    - General Linear Models
    - Tree Ensembles
    - Support Vector Machines
    - Pipelines
    - Feature Engineering

2 : iOS 11.2, macOS 10.13.2, tvOS 11.2, watchOS 4.2 (Core ML 1.2)

    - Custom Layers for Neural Networks
    - Float 16 support for Neural Network layers

3 : iOS 12, macOS 10.14, tvOS 12, watchOS 5 (Core ML 2)

    - Flexible shapes and image sizes
    - Categorical sequences
    - Core ML Vision Feature Print, Text Classifier, Word Tagger
    - Non Max Suppression
    - Crop and Resize Bilinear NN layers
    - Custom Models

4 : iOS 13, macOS 10.15, tvOS 13, watchOS 6 (Core ML 3)

    - Updatable models
    - Exact shape / general rank mapping for neural networks
    - Large expansion of supported neural network layers:

         - Generalized operations
         - Control flow
         - Dynamic layers
         - See `NeuralNetwork.proto <NeuralNetwork.html>`__
    
    - Nearest Neighbor Classifier
    - Sound Analysis Preprocessing
    - Recommender
    - Linked Model
    - NLP Gazeteer
    - NLP WordEmbedding

5 : iOS 14, macOS 11, tvOS 14, watchOS 7 (Core ML 4)

    - Model Deployment
    - Model Encryption
    - Unified converter API with PyTorch and Tensorflow 2 Support in coremltools 4
    - MIL builder for neural networks and composite ops in coremltools 4
    - New layers in neural network:
    
         - CumSum
         - OneHot
         - ClampedReLu
         - ArgSort
         - SliceBySize
         - Convolution3D
         - Pool3D
         - Bilinear Upsample with align corners and fractional factors
         - PixelShuffle
         - MatMul with int8 weights and int8 activations
         - Concat interleave
         - See `NeuralNetwork.proto <NeuralNetwork.html>`__
    
    - Enhanced Xcode model view with interactive previews
    - Enhanced Xcode Playground support for Core ML models

6 : iOS 15, macOS 12, tvOS 15, watchOS 8 (Core ML 5)

    - Core ML Audio Feature Print
    - New type of model: ``mlprogram`` (MILSpec.Program)

7 : iOS 16, macOS 13, tvOS 16, watchOS 9 (Core ML 6)

    - FLOAT16 array data type
    - GRAYSCALE_FLOAT16 image color space

8 : iOS 17, macOS 14, tvOS 17, watchOS 10 (Core ML 7)

    - iOS 17 ops
    - Scene print v2
    - ClassConfidenceThresholding model

.. code-block:: proto

    message Model {
        int32 specificationVersion = 1;
        ModelDescription description = 2;
        
        /*
         * Following model types support on-device update:
         *
         * - NeuralNetworkClassifier
         * - NeuralNetworkRegressor
         * - NeuralNetwork
         * - KNearestNeighborsClassifier
         */
        bool isUpdatable = 10;
        
        // start at 200 here
        // model specific parameters:
        oneof Type {
            // pipeline starts at 200
            PipelineClassifier pipelineClassifier = 200;
            PipelineRegressor pipelineRegressor = 201;
            Pipeline pipeline = 202;
    
            // regressors start at 300
            GLMRegressor glmRegressor = 300;
            SupportVectorRegressor supportVectorRegressor = 301;
            TreeEnsembleRegressor treeEnsembleRegressor = 302;
            NeuralNetworkRegressor neuralNetworkRegressor = 303;
            BayesianProbitRegressor bayesianProbitRegressor = 304;
    
            // classifiers start at 400
            GLMClassifier glmClassifier = 400;
            SupportVectorClassifier supportVectorClassifier = 401;
            TreeEnsembleClassifier treeEnsembleClassifier = 402;
            NeuralNetworkClassifier neuralNetworkClassifier = 403;
            KNearestNeighborsClassifier kNearestNeighborsClassifier = 404;
    
            // generic models start at 500
            NeuralNetwork neuralNetwork = 500;
            ItemSimilarityRecommender itemSimilarityRecommender = 501;
            MILSpec.Program mlProgram = 502;
    
            // Custom and linked models
            CustomModel customModel = 555;
            LinkedModel linkedModel = 556;

            // Precision Recall Curve 'container''
            ClassConfidenceThresholding classConfidenceThresholding = 560;

            // feature engineering starts at 600
            OneHotEncoder oneHotEncoder = 600;
            Imputer imputer = 601;
            FeatureVectorizer featureVectorizer = 602;
            DictVectorizer dictVectorizer = 603;
            Scaler scaler = 604;
            CategoricalMapping categoricalMapping = 606;
            Normalizer normalizer = 607;
            ArrayFeatureExtractor arrayFeatureExtractor = 609;
            NonMaximumSuppression nonMaximumSuppression = 610;
    
    
            // simple mathematical functions used for testing start at 900
            Identity identity = 900;
    
            // reserved until 1000
    
            // CoreML provided models
            CoreMLModels.TextClassifier textClassifier = 2000;
            CoreMLModels.WordTagger wordTagger = 2001;
            CoreMLModels.VisionFeaturePrint visionFeaturePrint = 2002;
            CoreMLModels.SoundAnalysisPreprocessing soundAnalysisPreprocessing = 2003;
            CoreMLModels.Gazetteer gazetteer = 2004;
            CoreMLModels.WordEmbedding wordEmbedding = 2005;
            CoreMLModels.AudioFeaturePrint audioFeaturePrint = 2006;
            
            // Reserved private messages start at 3000
            // These messages are subject to change with no notice or support.
            SerializedModel serializedModel = 3000;
        }
    }

