> For the complete documentation index, see [llms.txt](https://docs.nuvolos.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.nuvolos.com/how-to-guides/workflows-for-instructors/setting-assignments/programmatical-assignment-handling.md).

# Programmatic assignment handling

<mark style="color:$primary;">**Outcome**</mark>\
You collect, grade, and hand back assignments at the command line using nvcollect, integrated with otter-grader.

{% hint style="warning" %}
This is a new and experimental feature of Nuvolos, please provide us with feedback to make it better!
{% endhint %}

<mark style="color:$primary;">**Before you start**</mark>

* You have access to a JupyterLab + TeX Live application.
* `otter-grader` and `pandoc` are available in the application (TeX Live is available on request from Nuvolos support).
* You have installed `nvcollect` (contact support for the current install command).
* You are working in the Master Instance.

{% hint style="info" %}
To install `nvcollect`, run the following command from the terminal in your JupyterLab application:

```bash
pip install --upgrade git+https://github.com/alphacruncher/nv-collect
```

{% endhint %}

`nvcollect` is a command-line tool available in supported applications. It lets you:

* Collect assignments from all students into a folder in your instance.
* Grade assignments with the `otter-grader` package.
* Hand back assignments to all students in one command.

### Otter Grader integration

Below is a complete pipeline integrating `nvcollect` with `otter-grader`. It assumes you have already set up a JupyterLab application with `otter-grader`, `pandoc`, and `TeX Live`.

{% stepper %}
{% step %}

#### Obtain the Otter tutorial files

In a JupyterLab terminal, download the Otter Grader tutorial files. Refer to the [otter-grader documentation](https://otter-grader.readthedocs.io/) for the current download command.
{% endstep %}

{% step %}

#### Modify `demo.ipynb` for headless use

The Otter Grader `demo.ipynb` assumes a windowed environment, which is not available on Nuvolos. In the second cell, replace the windowed call with the headless equivalent:

```sql
%matplotlib inline
```

Make sure to have:

```sql
import matplotlib
matplotlib.use('Agg')
```

{% endstep %}

{% step %}

#### Run `otter assign`

In the directory where you extracted the tutorial archive, run:

{% code overflow="wrap" %}

```sql
otter assign demo.ipynb dist --v1
```

{% endcode %}

This creates a `dist/` directory containing autograding artefacts and the student-facing assignment file at `dist/student/demo.ipynb`.
{% endstep %}

{% step %}

#### Create the assignment on Nuvolos

Copy `dist/student/demo.ipynb` to a clean folder under `/files`, then stage and distribute it as an Assignment (see [Creating an assignment](/how-to-guides/workflows-for-instructors/setting-assignments.md#creating-an-assignment) above).\
For example, copy from `/files/otter-test/dist/student/demo.ipynb` to `/files/assignment_1/demo.ipynb`, stage `/files/assignment_1/`, and create an assignment named 'assignment\_1'.

{% hint style="info" %}
Keeping the assignment name in Nuvolos the same as the folder name (assignment\_1) is a useful convention - it is not required, but it makes the `nvcollect` command much easier to remember.
{% endhint %}
{% endstep %}

{% step %}

#### Collect hand-ins

Once the deadline has passed, collect submissions:

{% code overflow="wrap" %}

```sql
nvcollect collect \
  --assignment_name "assignment_1" \
  --assignment_folder assignment_1 \
  --target_folder /files/test_collect
```

{% endcode %}

This gathers everything in the *assignment\_1* folder of the *assignment\_1* assignment from all students into `/files/test_collect/`. The directory structure is:

{% code overflow="wrap" %}

```sql
/files/test_collect
+-- nvcollect_manifest.json
+-- single_user_inst_<code1>
|   +-- ...
+-- single_user_inst_<code2>
|   +-- ...
```

{% endcode %}
{% endstep %}

{% step %}

#### Grade hand-ins

Run autograding across all collected submissions:

{% code overflow="wrap" %}

```sql
nvcollect otter-grade \
  --source_folder /files/test_collect \
  --autograder_location /files/otter-test/dist/autograder/autograder.zip \
  --relative_path demo.ipynb
```

{% endcode %}

Three notes:

* `source_folder` is the same as the `target_folder` from the collect command.
* `autograder_location` is the absolute path to the autograder zip from step 3.
* `relative_path` tells the tool where to look inside each student folder for the notebook to grade.

After grading, `source_folder` contains:

* A `grade.csv` file at the top level with the score of each student.
* A `grade.csv` file inside each student folder with that student's score.
  {% endstep %}

{% step %}

#### Hand back the results

Optionally add additional artifacts to each student folder (manually or programmatically), then push results back to students:

{% code overflow="wrap" %}

```sql
nvcollect handback --source_folder /files/test_collect
```

{% endcode %}

`source_folder` here is the same folder where collection and grading happened.
{% endstep %}
{% endstepper %}

### Collecting assignments for download and archival

Starting version `1.0.0` of `nuvolos-collect`, the library provides the `nvcollect archive` command. This command collects submissions grouped by assignment and create a zip file per assignment into a target folder.