# Helper programs

During the development process of Paroxython, it proved useful to create a number of additional scripts to make the final program easier to write, test or understand. None of these scripts are meant to be run by the end user: as such they are are not included by `pip install`. But if you have to modify the source code to better meet your personal expectations, or are interested in collaborating on the project, some of them are definitely a must[^disclaimer].

[^disclaimer]: However, in the current state of things, please don't expect the <s>Spanish inquisition</s> same level of code quality, documentation and testing than in the user-facing parts of the package.

## Code generators

[`make_compare_spans.py`](https://repo/helpers/make_compare_spans.py)
:   Generates the source code of the module `paroxython.compare_spans`. Must be launched manually.

[`reformat_spec.py`](https://repo/helpers/reformat_spec.py)
:   Reads the contents of `spec.md`, normalizes its formatting (Markdown and SQL) and writes it back. The goal is to ensure the consistency of the document and the validity of its numerous internal links, as well as to avoid authors wasting time with formatting details. Launched automatically by [`test_parse_program.py`](https://repo/tests/test_parse_program.py).

[`suggest_regex.py`](https://repo/helpers/suggest_regex.py)
:   The regular expression patterns of `spec.md` use a rather limited and repetitive array of techniques. This program aims to partially automate their production. Given the `flat_ast.txt` outputted by [`parse_source.py`](https://repo/helpers/parse_source.py) (cf. testing helper below), extract the lines that characterize the feature to be recognized, and save them in the `sandbox` folder under the name `flat_ast_selection.txt`. The script will then print a regular expression that matches this selection.

## Testing

[`dump_flat_asts.py`](https://repo/helpers/dump_flat_asts.py)
:   Extracts from `spec.md` all the features specified by a regular expression, use `paroxython.flatten_ast.flatten_ast()` to parse their examples into a flat AST, and outputs the results as `sandbox/flat_ast.txt`.

[`make_dummy_db.py`](https://repo/helpers/make_dummy_db.py)
:   Generates [`taxa_and_programs.txt`](https://repo/examples/dummy/taxa_and_programs.txt) and, from this file, [`programs_db.json`](https://repo/examples/dummy/programs_db.json), used as a dummy example in various tests.

[`make_programming_idioms_folder.py`](https://repo/helpers/make_programming_idioms_folder.py)
:   Populates the example directory [`idioms/programs`](https://repo/examples/idioms/programs), with the broad variety of code snippets snapshotted from the amazing [Programming Idioms](https://www.programming-idioms.org) website of Valentin Deleplace. More info [`here`](https://repo/examples/idioms/read_me.md).

[`parse_source.py`](https://repo/helpers/parse_source.py)
:   A wrapper around `paroxython.parse_program` that allows you to quickly test the labelling of a given source code. This one must be placed under the name `source.py` in a `sandbox`  directory (unversioned). The flattened version of its AST is generated as `flat_ast.txt` in the same folder, while its label table is displayed in Markdown format. All of this may streamline the work on `spec.md`.

## Documentation builders and examples

[`build_pdoc.py`](https://repo/helpers/build_pdoc.py)
:   Builds this documentation with the excellent [pdoc3](https://pdoc3.github.io/pdoc/) command line tool, and applies a number of patches to tailor the result to our needs.

[`draw_flow.py`](https://repo/helpers/draw_flow.py)
:   Generates the diagrams on the present page and `paroxython.filter_programs`. Launched automatically by [`build_pdoc.py`](https://repo/helpers/build_pdoc.py).

[`example.sql`](https://repo/helpers/example.sql)
:   An example of query on the experimental SQLite database generated by `paroxython.make_db`.

[`parse_syllabus.py`](https://repo/helpers/parse_syllabus.py)
:   This script is invoked in the first example of the [pipeline documentation](../user_manual/index.html#pipeline-documentation). This is the one that the main author of Paroxython relies on to automate the extraction of the “imparted” programs from a textual timeline he updates after each lesson. The script starts by dropping all the text following the `"# EOF` tag. In the remaining part, it matches all the Python program paths which are preceded by a `"+"` symbol and possibly a bracketed string (for instance `"+ [mandatory] homework.py"`). Of course, if you use other conventions, you will have to come up with your own custom script. It can be written in any language (e.g., Perl, bash, etc.), as long as it prints a list of program paths on [stdout](https://en.wikipedia.org/wiki/Standard_streams#Standard_output_(stdout)).

[`update_tree.py`](https://repo/helpers/update_tree.py)
:   Computes a taxonomy of [The Algorithms - Python](https://github.com/TheAlgorithms/Python) (not versioned here) and generates [`tree.js`](https://repo/docs/resources/tree.js), a JavaScript list ready to send to the [Google Chart API](https://developers.google.com/chart/interactive/docs/gallery/wordtree) used to draw [this word-tree](../user_manual/index.html#taxonomy). Launch [`build_pdoc.py`](https://repo/helpers/build_pdoc.py) afterwards.
