# Protocol Documentation

## Table of Contents

## api.proto

Top

Katib GRPC API v1beta1

### AlgorithmSetting

HP or NAS algorithm settings.

Field | Type | Label | Description
name | string |  |
value | string |  |

### AlgorithmSpec

HP or NAS algorithm specification.

Field | Type | Label | Description
algorithm_name | string |  |
algorithm_settings | AlgorithmSetting | repeated |

### DeleteObservationLogReply

### DeleteObservationLogRequest

Field | Type | Label | Description
trial_name | string |  |

### EarlyStoppingRule

EarlyStoppingRule represents single early stopping rule.

Field | Type | Label | Description
name | string |  | Name of the rule. Usually, metric name.
value | string |  | Value of the metric.
comparison | ComparisonType |  | Correlation between name and value, one of equal, less or greater
start_step | int32 |  | Defines quantity of intermediate results that should be received before applying the rule. If start step is empty, rule is applied from the first recorded metric.

### EarlyStoppingSetting

Early stopping algorithm settings.

Field | Type | Label | Description
name | string |  |
value | string |  |

### EarlyStoppingSpec

Early stopping algorithm specification.

Field | Type | Label | Description
algorithm_name | string |  |
algorithm_settings | EarlyStoppingSetting | repeated |

### Experiment

Structure for a single Experiment.

Field | Type | Label | Description
name | string |  | Name for the Experiment.
spec | ExperimentSpec |  | Experiment specification.

### ExperimentSpec

Specification of an Experiment. Experiment represents a single optimization run over a feasible space.

Each Experiment contains a configuration describing the feasible space, as well as a set of Trials.

It is assumed that objective function f(x) does not change in the course of an Experiment.

Field | Type | Label | Description
parameter_specs | ExperimentSpec.ParameterSpecs |  |
objective | ObjectiveSpec |  | Objective specification for the Experiment.
algorithm | AlgorithmSpec |  | HP or NAS algorithm specification for the Experiment.
early_stopping | EarlyStoppingSpec |  | Early stopping specification for the Experiment.
parallel_trial_count | int32 |  | How many Trials can be processed in parallel.
max_trial_count | int32 |  | Max completed Trials to mark Experiment as succeeded.
nas_config | NasConfig |  | NAS configuration for the Experiment.

### ExperimentSpec.ParameterSpecs

List of ParameterSpec.

Field | Type | Label | Description
parameters | ParameterSpec | repeated |

### FeasibleSpace

Feasible space for optimization.

Int and Double type use Max/Min.

Discrete and Categorical type use List.

Field | Type | Label | Description
max | string |  | Max Value
min | string |  | Minimum Value
list | string | repeated | List of Values.
step | string |  | Step for double or int parameter

### GetEarlyStoppingRulesReply

Field | Type | Label | Description
early_stopping_rules | EarlyStoppingRule | repeated |

### GetEarlyStoppingRulesRequest

Field | Type | Label | Description
experiment | Experiment |  |
trials | Trial | repeated |
db_manager_address | string |  |

### GetObservationLogReply

Field | Type | Label | Description
observation_log | ObservationLog |  |

### GetObservationLogRequest

Field | Type | Label | Description
trial_name | string |  |
metric_name | string |  |
start_time | string |  | The start of the time range. RFC3339 format
end_time | string |  | The end of the time range. RFC3339 format

### GetSuggestionsReply

Field | Type | Label | Description
parameter_assignments | GetSuggestionsReply.ParameterAssignments | repeated |
algorithm | AlgorithmSpec |  |
early_stopping_rules | EarlyStoppingRule | repeated |

### GetSuggestionsReply.ParameterAssignments

Field | Type | Label | Description
assignments | ParameterAssignment | repeated |
trial_name | string |  | Optional field to override the trial name
labels | GetSuggestionsReply.ParameterAssignments.LabelsEntry | repeated | Optional field to add labels to the generated Trials

### GetSuggestionsReply.ParameterAssignments.LabelsEntry

Field | Type | Label | Description
key | string |  |
value | string |  |

### GetSuggestionsRequest

Field | Type | Label | Description
experiment | Experiment |  |
trials | Trial | repeated | All completed trials owned by the experiment.
current_request_number | int32 |  | The number of Suggestions requested at one time. When you set 3 to current_request_number, you get three Suggestions at one time.
total_request_number | int32 |  | The number of Suggestions requested till now.

### GraphConfig

GraphConfig contains a config of DAG

Field | Type | Label | Description
num_layers | int32 |  | Number of layers
input_sizes | int32 | repeated | Dimensions of input size
output_sizes | int32 | repeated | Dimensions of output size

### Metric

Field | Type | Label | Description
name | string |  |
value | string |  |

### MetricLog

Field | Type | Label | Description
time_stamp | string |  | RFC3339 format
metric | Metric |  |

### NasConfig

NasConfig contains a config of NAS job

Field | Type | Label | Description
graph_config | GraphConfig |  | Config of DAG
operations | NasConfig.Operations |  | List of Operation

### NasConfig.Operations

Field | Type | Label | Description
operation | Operation | repeated |

### ObjectiveSpec

Objective specification.

Field | Type | Label | Description
type | ObjectiveType |  | Type of optimization.
goal | double |  | Goal of optimization, can be empty.
objective_metric_name | string |  | Primary metric name for the optimization.
additional_metric_names | string | repeated | List of additional metrics to record from Trial. This can be empty if we only care about the objective metric.

### Observation

Field | Type | Label | Description
metrics | Metric | repeated |

### ObservationLog

Field | Type | Label | Description
metric_logs | MetricLog | repeated |

### Operation

Config for operations in DAG

Field | Type | Label | Description
operation_type | string |  | Type of operation in DAG
parameter_specs | Operation.ParameterSpecs |  |

### Operation.ParameterSpecs

List of ParameterSpec

Field | Type | Label | Description
parameters | ParameterSpec | repeated |

### ParameterAssignment

Field | Type | Label | Description
name | string |  |
value | string |  |

### ParameterSpec

Config for a hyperparameter.

Katib will create each Hyper parameter from this config.

Field | Type | Label | Description
name | string |  | Name of the parameter.
parameter_type | ParameterType |  | Type of the parameter.
feasible_space | FeasibleSpace |  | FeasibleSpace for the parameter.

### ReportObservationLogReply

### ReportObservationLogRequest

Field | Type | Label | Description
trial_name | string |  |
observation_log | ObservationLog |  |

### SetTrialStatusReply

### SetTrialStatusRequest

Field | Type | Label | Description
trial_name | string |  |

### Trial

Structure for a single Trial.

Field | Type | Label | Description
name | string |  | Name for the Trial.
spec | TrialSpec |  | Trial specification.
status | TrialStatus |  | Trial status.

### TrialSpec

Specification of a Trial. It represents Trial's parameter assignments and objective.

Field | Type | Label | Description
objective | ObjectiveSpec |  | Objective specification for the Trial.
parameter_assignments | TrialSpec.ParameterAssignments |  | List of assignments generated for the Trial.
labels | TrialSpec.LabelsEntry | repeated | Map of labels assigned to the Trial

### TrialSpec.LabelsEntry

Field | Type | Label | Description
key | string |  |
value | string |  |

### TrialSpec.ParameterAssignments

List of ParameterAssignment

Field | Type | Label | Description
assignments | ParameterAssignment | repeated |

### TrialStatus

Current Trial status. It contains Trial's latest condition, start time, completion time, observation.

Field | Type | Label | Description
start_time | string |  | Trial start time in RFC3339 format
completion_time | string |  | Trial completion time in RFC3339 format
condition | TrialStatus.TrialConditionType |  | Trial current condition. It is equal to the latest Trial CR condition.
observation | Observation |  | The best Trial observation in logs.

### ValidateAlgorithmSettingsReply

Return INVALID_ARGUMENT Error if Algorithm Settings are not Valid

### ValidateAlgorithmSettingsRequest

Field | Type | Label | Description
experiment | Experiment |  |

### ValidateEarlyStoppingSettingsReply

Return INVALID_ARGUMENT Error if Early Stopping Settings are not Valid

### ValidateEarlyStoppingSettingsRequest

Field | Type | Label | Description
early_stopping | EarlyStoppingSpec |  |

### ComparisonType

Name | Number | Description
UNKNOWN_COMPARISON | 0 | Unknown comparison, not used
EQUAL | 1 | Equal comparison, e.g. accuracy = 0.7
LESS | 2 | Less comparison, e.g. accuracy < 0.7
GREATER | 3 | Greater comparison, e.g. accuracy > 0.7

### ObjectiveType

Direction of optimization. Minimize or Maximize.

Name | Number | Description
UNKNOWN | 0 | Undefined type and not used.
MINIMIZE | 1 | Minimize
MAXIMIZE | 2 | Maximize

### ParameterType

Types of value for HyperParameter.

Name | Number | Description
UNKNOWN_TYPE | 0 | Undefined type and not used.
DOUBLE | 1 | Double float type. Use "Max/Min".
INT | 2 | Int type. Use "Max/Min".
DISCRETE | 3 | Discrete number type. Use "List" as float.
CATEGORICAL | 4 | Categorical type. Use "List" as string.

### TrialStatus.TrialConditionType

Trial can be in one of 8 conditions.

TODO (andreyvelich): Remove unused conditions.

Name | Number | Description
CREATED | 0 |
RUNNING | 1 |
SUCCEEDED | 2 |
KILLED | 3 |
FAILED | 4 |
METRICSUNAVAILABLE | 5 |
EARLYSTOPPED | 6 |
UNKNOWN | 7 |

### DBManager

DBManager service defines APIs to manage Katib database.

Method Name | Request Type | Response Type | Description
ReportObservationLog | ReportObservationLogRequest | ReportObservationLogReply | Report a log of Observations for a Trial. The log consists of timestamp and value of metric. Katib store every log of metrics. You can see accuracy curve or other metric logs on UI.
GetObservationLog | GetObservationLogRequest | GetObservationLogReply | Get all log of Observations for a Trial.
DeleteObservationLog | DeleteObservationLogRequest | DeleteObservationLogReply | Delete all log of Observations for a Trial.

### EarlyStopping

EarlyStopping service defines APIs to manage Katib Early Stopping algorithms

Method Name | Request Type | Response Type | Description
GetEarlyStoppingRules | GetEarlyStoppingRulesRequest | GetEarlyStoppingRulesReply |
SetTrialStatus | SetTrialStatusRequest | SetTrialStatusReply |
ValidateEarlyStoppingSettings | ValidateEarlyStoppingSettingsRequest | ValidateEarlyStoppingSettingsReply |

### Suggestion

Suggestion service defines APIs to manage Katib Suggestion from HP or NAS algorithms

Method Name | Request Type | Response Type | Description
GetSuggestions | GetSuggestionsRequest | GetSuggestionsReply |
ValidateAlgorithmSettings | ValidateAlgorithmSettingsRequest | ValidateAlgorithmSettingsReply |

## Scalar Value Types

.proto Type | Notes | C++ | Java | Python | Go | C# | PHP | Ruby
double |  | double | double | float | float64 | double | float | Float
float |  | float | float | float | float32 | float | float | Float
int32 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required)
int64 | Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. | int64 | long | int/long | int64 | long | integer/string | Bignum
uint32 | Uses variable-length encoding. | uint32 | int | int/long | uint32 | uint | integer | Bignum or Fixnum (as required)
uint64 | Uses variable-length encoding. | uint64 | long | int/long | uint64 | ulong | integer/string | Bignum or Fixnum (as required)
sint32 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required)
sint64 | Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. | int64 | long | int/long | int64 | long | integer/string | Bignum
fixed32 | Always four bytes. More efficient than uint32 if values are often greater than 2^28. | uint32 | int | int | uint32 | uint | integer | Bignum or Fixnum (as required)
fixed64 | Always eight bytes. More efficient than uint64 if values are often greater than 2^56. | uint64 | long | int/long | uint64 | ulong | integer/string | Bignum
sfixed32 | Always four bytes. | int32 | int | int | int32 | int | integer | Bignum or Fixnum (as required)
sfixed64 | Always eight bytes. | int64 | long | int/long | int64 | long | integer/string | Bignum
bool |  | bool | boolean | boolean | bool | bool | boolean | TrueClass/FalseClass
string | A string must always contain UTF-8 encoded or 7-bit ASCII text. | string | String | str/unicode | string | string | string | String (UTF-8)
bytes | May contain any arbitrary sequence of bytes. | string | ByteString | str | []byte | ByteString | string | String (ASCII-8BIT)