# Simple Sandbox

## Problem

We can't reliably tell if code is working without executing it, but directly executing untrusted and quite-possibly-buggy code generated by a random LLM directly on our development machines is undesirable.

## Solution

[![](https://mermaid.ink/img/pako:eNp1kM2KwkAMx18l5Cw-QPe49eZe1MWD4yE20Q46mTLNCCK-u9N2ERYxgfBP-JGvOzaRBSs8JepaWK6-nDrt82HK69icJcF3VCOvkpwCwBjgxWx8kJht3rewJWtajqeJgBf7j19c6QLbIru_fvAOD1bvftVS7k24LMCy_wSL8pSMYgyD4wyDpECey3n3gXBorQRxWBXJlM4OnT4KR9ni-qYNVmWgzDB3TCa1p7JxwOpIl75Uhb3F9DP9a3zb4wk0HWJc?type=png)](https://mermaid.live/edit#pako:eNp1kM2KwkAMx18l5Cw-QPe49eZe1MWD4yE20Q46mTLNCCK-u9N2ERYxgfBP-JGvOzaRBSs8JepaWK6-nDrt82HK69icJcF3VCOvkpwCwBjgxWx8kJht3rewJWtajqeJgBf7j19c6QLbIru_fvAOD1bvftVS7k24LMCy_wSL8pSMYgyD4wyDpECey3n3gXBorQRxWBXJlM4OnT4KR9ni-qYNVmWgzDB3TCa1p7JxwOpIl75Uhb3F9DP9a3zb4wk0HWJc)

* Network and Filesystem isolation provider by Docker
* Timeout provider by a bash exec wrapper
* Eval wrapper to capture errors, format outputs and always return a consitent result

## Implementation

`sandbox.py` (intended to be used as a library)

* `extract_function_info(language, code)` perform static analysis on potentially non-working code that implements a function. returns a `{ name, args[] }` object containing the functions name and a list of its arguments.
* `FunctionSandbox(code, language)` high-level Docker sandbox class.  Use the `.call(..)` method to invoke the untrusted function.

`timeout.sh`

Bash implementation of the timeout layer

`Dockerfile.javascript`, `Dockerfile.python`

Docker implementations of the isolation layer

`eval.javascript.tpl`, `eval.python.tpl`

Eval wrappers.

## Assumptions

* `code` contains a single function in `language`
* the language we are working with supports try/catch, lists and objects
* the language we are working with can serialize JSON