
## Install dependencies

system dependencies:

```
sudo apt-get update
sudo apt-get install gcc make python3-dev libjpeg-dev zlib1g-dev
```

python dependencies:


```shell
uv pip compile requirements.in --universal -o requirements.txt && uv pip install -r requirements.txt  --python .venv/bin/python
```

```shell
uv pip sync requirements.txt
```

Extra steps:

```shell
python -m nltk.downloader punkt
uv pip install torch --python .venv/bin/python
uv pip install flash-attn==2.7.2.post1 --no-build-isolation --python .venv/bin/python
```

PYTHONPATH=$(pwd) GOOGLE_APPLICATION_CREDENTIALS=secrets/google-credentials.json uvicorn questions.inference_server.inference_server:app --host 0.0.0.0 --port 9080 --workers 1 --timeout-keep-alive 180000
