export const metadata = {
  title: 'Chat',
  description: 'Generate chat completions',
}

# Chat

Generate text from text. {{ className: 'lead' }}

---

<Row>
  <Col>

    ## Create chat completion {{tag:'POST', label:'http://localhost:33322/v1/chat/completions'}}

    Given a list of messages belonging to a chat history, generate a response.

    ### Required attributes

    <Properties>
      <Property name="messages" type="array">
        A list of messages representing a chat history. It is essentially the context used by the model to generate a response.
      </Property>
    </Properties>

    <Properties>
      <Property name="model" type="string">
        The model used for chat completions.
        <ul>
            <li>
                If the model name is "default", the chat model from the configuration is used (see [Documentation &raquo; Configuration](/documentation/configuration) for details).
            </li>
            <li>
                If the model name follows the format repo-owner/repo-name/model-name, the indicated model is used and, if it is not present, it will be downloaded from [huggingface](https://huggingface.co/). If it cannot be downloaded, Edgen responds with an error. Example: "TheBloke/neural-chat-7B-v3-3-GGUF/neural-chat-7b-v3-3.Q4_K_M.gguf".
            </li>
            <li>
                If the model name contains just a file name, e.g.: "my-model.bin", Edgen will try using the file of this name in the data directory as defined in the configuration. If the the file does not exist there, Edgen responds with an error.
            </li>
        </ul>


      </Property>
    </Properties>

    ### Optional attributes

      <Properties>
          <Property name="frequency_penalty" type="float">
              A number in `[-2.0, 2.0]`. A higher number decreases the likelihood that the model repeats itself.
          </Property>
      </Properties>

      <Properties>
          <Property name="logit_bias" type="map">
              A map of token IDs to `[-100.0, +100.0]`. Adds a percentage bias to those tokens before sampling; a value of `-100.0` prevents the token from being selected at all.
              You could use this to, for example, prevent the model from emitting profanity.
          </Property>
      </Properties>

      <Properties>
          <Property name="max_tokens" type="integer">
              The maximum number of tokens to generate. If `None`, terminates at the first stop token or the end of sentence.
          </Property>
      </Properties>

      <Properties>
          <Property name="n" type="integer">
              How many choices to generate for each token in the output. `1` by default. You can use this to generate several sets of completions for the same prompt.
          </Property>
      </Properties>

      <Properties>
          <Property name="presence_penalty" type="float">
              A number in `[-2.0, 2.0]`. Positive values "increase the model's likelihood to talk about new topics."
          </Property>
      </Properties>

      <Properties>
          <Property name="seed" type="integer">
              The random number generator seed for the session. Random by default.
          </Property>
      </Properties>

      <Properties>
          <Property name="stop" type="string or array">
              A stop phrase or set of stop phrases.
              The server will pause emitting completions if it appears to be generating a stop phrase, and will terminate completions if a full stop phrase is detected.
              Stop phrases are never emitted to the client.
          </Property>
      </Properties>

      <Properties>
          <Property name="stream" type="bool">
              If true, stream the output as it is computed by the server, instead of returning the whole completion at the end.
              You can use this to live-stream completions to a client.
          </Property>
      </Properties>

      <Properties>
          <Property name="response_format" type="string">
              The format of the response stream.
              This is always assumed to be JSON, which is non-conformant with the OpenAI spec.
          </Property>
      </Properties>

      <Properties>
          <Property name="temperature" type="float">
              The sampling temperature, in `[0.0, 2.0]`. Higher values make the output more random.
          </Property>
      </Properties>

      <Properties>
          <Property name="top_p" type="float">
              Nucleus sampling. If you set this value to 10%, only the top 10% of tokens are used for sampling, preventing sampling of very low-probability tokens.
          </Property>
      </Properties>

      <Properties>
          <Property name="tools" type="array">
              A list of tools made available to the model.
          </Property>
      </Properties>

      <Properties>
          <Property name="tool_choice" type="string">
              If present, the tool that the user has chosen to use.
              OpenAI states:
               - `none` prevents any tool from being used,
               - `auto` allows any tool to be used, or
               - you can provide a description of the tool entirely instead of a name.
          </Property>
      </Properties>

      <Properties>
          <Property name="user" type="string">
              A unique identifier for the _end user_ creating this request. This is used for telemetry and user tracking, and is unused within Edgen.
          </Property>
      </Properties>

      <Properties>
          <Property name="one_shot" type="bool">
              Indicate if this is an isolated request, with no associated past or future context. This may allow for optimisations in some implementations.
              Default: `false`
          </Property>
      </Properties>

      <Properties>
          <Property name="context_hint" type="integer">
              A hint for how big a context will be.
              # Warning
              An unsound hint may severely drop performance and/or inference quality, and in some cases even cause Edgen to crash. Do not set this value unless you know what you are doing.
          </Property>
      </Properties>

  </Col>
  <Col sticky>

  <ButtonRow types={["Default", "Streaming"]}>

      <div>
        <CodeGroup title="Request" tag="POST" label="/v1/chat/completions">

          ```bash {{ title: 'cURL' }}
          curl http://localhost:33322/v1/chat/completions \
          -H "Content-Type: application/json" \
          -H "Authorization: Bearer no-key-required" \
          -d '{
            "model": "default",
            "messages": [
              {
                "role": "system",
                "content": "You are EdgenChat, a helpful AI assistant."
              },
              {
                "role": "user",
                "content": "Hello!"
              }
            ]
          }'
          ```

          ```python
          from edgen import Edgen
          client = Edgen()

          completion = client.chat.completions.create(
            model="default",
            messages=[
              {"role": "system", "content": "You are a helpful assistant."},
              {"role": "user", "content": "Hello!"}
            ]
          )

          for chunk in completion:
            print(chunk.choices[0].delta)
          ```

          ```ts
          import Edgen from "edgen";

          const client = new Edgen();

          async function main() {
            const completion = await client.chat.completions.create({
              model: "default",
              messages: [
                {"role": "system", "content": "You are a helpful assistant."},
                {"role": "user", "content": "Hello!"}
              ]
            });

            for await (const chunk of completion) {
              console.log(chunk.choices[0].delta.content);
            }
          }

          main();
          ```

        </CodeGroup>

          ```json {{ title: 'Response' }}
          {"id":"f403d6f4-4826-40b1-8798-77e4837e5041","choices":[{"message":{"role":"assistant","content":"Hello! How can I help you today?","name":null,"tool_calls":null},"finish_reason":null,"index":0}],"created":1708958149,"model":"main","system_fingerprint":"edgen-0.1.3","object":"text_completion","usage":{"completion_tokens":0,"prompt_tokens":0,"total_tokens":0}}
          ```
      </div>

      <div>
        <CodeGroup title="Request" tag="POST" label="/v1/chat/completions">

          ```bash {{ title: 'cURL' }}
          curl http://localhost:33322/v1/chat/completions \
          -H "Content-Type: application/json" \
          -H "Authorization: Bearer no-key-required" \
          -d '{
            "model": "default",
            "messages": [
              {
                "role": "system",
                "content": "You are EdgenChat, a helpful AI assistant."
              },
              {
                "role": "user",
                "content": "Hello!"
              }
            ],
            "stream": true
          }'
          ```

          ```python
          from edgen import Edgen
          client = Edgen()

          completion = client.chat.completions.create(
            model="default",
            messages=[
              {"role": "system", "content": "You are a helpful assistant."},
              {"role": "user", "content": "Hello!"}
            ],
            stream=True
          )

          for chunk in completion:
            print(chunk.choices[0].delta)
          ```

          ```ts
          import Edgen from "edgen";

          const client = new Edgen();

          async function main() {
            const completion = await client.chat.completions.create({
              model: "default",
              messages: [
                {"role": "system", "content": "You are a helpful assistant."},
                {"role": "user", "content": "Hello!"}
              ],
              stream: true,
            });

            for await (const chunk of completion) {
              console.log(chunk.choices[0].delta.content);
            }
          }

          main();
          ```

        </CodeGroup>

          ```json {{ title: 'Response' }}
          {"id":"e55b11e3-985b-4fbf-ba2e-5e81e6c100c2","choices":[{"delta":{"content":"Hello","role":null},"finish_reason":null,"index":0}],"created":1706718034,"model":"main","system_fingerprint":"edgen-0.1.0","object":"text_completion"}

          {"id":"010b3954-af7c-4360-9891-1d6c5e27da8c","choices":[{"delta":{"content":"!","role":null},"finish_reason":null,"index":0}],"created":1706718040,"model":"main","system_fingerprint":"edgen-0.1.0","object":"text_completion"}

          {"id":"7455343f-99bc-4d63-9d03-cc8e75fab29c","choices":[{"delta":{"content":" How","role":null},"finish_reason":null,"index":0}],"created":1706718044,"model":"main","system_fingerprint":"edgen-0.1.0","object":"text_completion"}

          {"id":"82f72e44-162d-4e21-93c6-387950a61c79","choices":[{"delta":{"content":" can","role":null},"finish_reason":null,"index":0}],"created":1706718049,"model":"main","system_fingerprint":"edgen-0.1.0","object":"text_completion"}

          {"id":"7d451531-d117-48b1-9e68-2554338e34ba","choices":[{"delta":{"content":" I","role":null},"finish_reason":null,"index":0}],"created":1706718053,"model":"main","system_fingerprint":"edgen-0.1.0","object":"text_completion"}

          {"id":"66fe6034-c97d-4c46-8c05-bf73804e557d","choices":[{"delta":{"content":" assist","role":null},"finish_reason":null,"index":0}],"created":1706718059,"model":"main","system_fingerprint":"edgen-0.1.0","object":"text_completion"}

          {"id":"56a260a8-f41b-4954-9b67-e6620fb32f0d","choices":[{"delta":{"content":" you","role":null},"finish_reason":null,"index":0}],"created":1706718063,"model":"main","system_fingerprint":"edgen-0.1.0","object":"text_completion"}

          {"id":"ccef46ce-ba8b-4ac8-8262-a66cb96832a5","choices":[{"delta":{"content":" today","role":null},"finish_reason":null,"index":0}],"created":1706718068,"model":"main","system_fingerprint":"edgen-0.1.0","object":"text_completion"}

          {"id":"0d2b9ba2-ab04-4aed-ad51-72a89acb3122","choices":[{"delta":{"content":"?","role":null},"finish_reason":null,"index":0}],"created":1706718069,"model":"main","system_fingerprint":"edgen-0.1.0","object":"text_completion"}
          ```
      </div>
  </ButtonRow>

  </Col>
</Row>

---

## Chat completion status {{ tag: 'GET', label: 'http://localhost:33322/v1/chat/completions/status' }}

<Row>
  <Col>

    Shows the current status of the chat completions endpoint (e.g. downloads).

    ### Response attributes

    <Properties>
      <Property name="active_model" type="string">
          The model that is currently active for this endpoint.
      </Property>
    </Properties>

    <Properties>
      <Property name="donwload_ongoing" type="bool">
        The model for this endpoint is currently being downloaded.
      </Property>
    </Properties>

    <Properties>
      <Property name="donwload_progress" type="number">
        The progress of the ongoing model download in percent.
      </Property>
    </Properties>

    <Properties>
      <Property name="last_errors" type="string[]">
        Errors that occurred recently on this endpoint.
      </Property>
    </Properties>

  </Col>
  <Col sticky>

    <CodeGroup title="Request" tag="GET" label="/v1/chat/completions/status">

    ```bash {{ title: 'cURL' }}
    curl http://localhost:33322/v1/chat/completions/status \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer no-key-required"
    ```

    ```python
    from edgen import Edgen
    client = Edgen()

    status = client.chat.completions.status.create()
    print(status)

    ```

    ```ts
    import Edgen from "edgen";

    const client = new Edgen();

    async function main() {
      const status = await client.chat.completions.status.create();

      console.log(status);
    }

    main();
    ```

    </CodeGroup>

    ```json {{ title: 'Response' }}
    {"active_model":"neural-chat-7b-v3-3.Q4_K_M.gguf","download_ongoing":false,"download_progress":100,"last_errors":["Custom { kind: PermissionDenied, error: \"verboten\" }]}
    ```

  </Col>
</Row>
