Auto-Generated Folders

← Back to Library Structure

Not shipped with the repository — safe to delete at any time and let JuSPICE rebuild them. See Fixed Structure for what ships with the repository instead.

What Gets Created

  • venvs/ — the isolated Python environments (venvs/.venv, plus venvs/.venv_iopaint and, if you use MicroNet, venvs/.venv_micronet), created the first time you run uv sync — see Installation.

  • preprocessed_data/ — denoised/normalised images, written the first time you run a preprocessing or synthetic-data notebook.

  • Synthetic_images/ — generated images and masks, written the first time you run a synthetic-data notebook (Aug, PB, PB_NonGauss, SDiff, or DCGAN).

  • Models/ — trained model checkpoints, in two subfolders: Models/synth_models (DCGAN) and Models/segmentation_models (U-Net) — written the moment training finishes.

  • Pretrained_models/ — pretrained checkpoints for SAM-1, SAM-2, MicroNet, and (in their own stable_diffusion/ subfolder) both Stable Diffusion models — each downloaded automatically the first time it’s used, and only if you’ve used it. See the disk-space note on the Linux — Install Everything at Once page for their sizes.

Four of these five — everything except venvs/ — have their own path set in notebooks/notebooks_parameters.yaml, so you can point any of them somewhere else if you’d rather keep generated output outside the repository folder.

Inside Each Folder

Several of these folders are organised around an output name — by default <method>_<dataset> (e.g. Aug_EBC1 for the Aug method run on the EBC1 dataset), one subfolder per generation run so that different methods, datasets, or model runs never overwrite each other’s output. Pass output_name= to spice.synth_generation.generate() to override it.

preprocessed_data/
└── <output_name>/
    ├── input_images/           # resized/normalised training images
    └── input_masks/            # their matching masks

Synthetic_images/
└── <output_name>/
    ├── images/                 # the generated synthetic images
    ├── masks/                  # their binary masks
    ├── aug_transform_masks/    # masks used internally by the Aug method
    ├── labeled_masks/          # instance-labelled masks
    └── input_backgrounds/      # background crops used for augmentation

Models/
├── synth_models/
│   └── <output_name>/          # trained DCGAN checkpoints
└── segmentation_models/
    └── <output_name>/          # trained U-Net checkpoints

Pretrained_models/
├── segment_anything1_META/     # SAM-1 checkpoint  — may be added
├── segment_anything2_META/     # SAM-2 checkpoint  — may be added
├── NASA_Micronet/              # MicroNet checkpoints — may be added
└── stable_diffusion/           # both Stable Diffusion models — may be added

Only the subfolders for whichever features you’ve actually used show up — none of the four Pretrained_models/ subfolders above exist until you first run the matching notebook (see the disk-space note on Linux — Install Everything at Once for their sizes).

SAM-1, SAM-2, MicroNet, Trackpy, and U-Net all save their result with save_data(), which writes three files right next to the notebook that produced them: <stem>.npy (the full result), <stem>.json (its operation history), and <stem>_history.py (a reproducibility script). By default <stem> is auto-detected from the calling notebook — reliable when run interactively in Jupyter, but ambiguous for a plain script or a headless jupyter nbconvert execution in a folder holding more than one notebook (it silently falls back to the folder’s name, and every notebook in it collides on the same three files). Since notebooks/io/, notebooks/segmentation/, and notebooks/synthetic_data/ each hold several notebooks, every notebook in those three folders passes save_data()’s output_stem argument explicitly (its own filename), removing that ambiguity entirely — see juspice.io.save_data().

See also