Developing Modules for VentOCR

VentOCR supports a secondary analysis layer called Modules. A Module is a Python script that reads archived waveform data (Frames.csv / Breaths.csv) for a completed analysis and writes additional computed fields back to those files. Any developer familiar with pandas can create a Module without touching the core platform.

1. What a Module Does

After VentOCR finishes primary OCR analysis, the results are stored in the Archive as CSV files. A Module is called with a single analyze_id and user_id, reads those CSVs, computes new values, and writes them back. 

Archive/
  users/
    {user_id}/
      Archive/
        AID_{analyze_id}/
          Frames.csv        ← frame-level raw data
          Breaths.csv       ← breath-level aggregated data  ← Module writes here
          extra_fields.txt  ← column registry               ← Module appends here
Modules never modify the primary OCR engine. They only read and extend the existing CSV output, making them safe to add or remove at any time.

2. Execution Flow

User selects Analyze targets + Module(s) on the Modules page
        ↓
POST /run_modules_batch  →  tasks inserted into Cue table (task_type = "module")
        ↓
queue_worker picks up the task
        ↓
Validates Engine–Module mapping (Module_Engine_Map)
        ↓
Loads Module_Path / Entry_Point from Modules table
        ↓
Spawns a subprocess running the entry point
        ↓
main(analyze_id, user_id) is called
        ↓
Return value (string) logged to Analysis_Module_Run
        ↓
Result shown on the Modules page

Each module is executed in its own subprocess with the current working directory set to the /app root (same as the Flask process).

3. Directory Structure

Place your module under the Module/ directory, which is volume-mounted into the container at /app/Module/. 

Module/
  <Module_Key>/
    core.py        ← required: entry point containing main()
    README.md      ← recommended: shown in the Module Selection UI

Example:

Module/
  MyAnalysis_VentOCR/
    core.py
    README.md

The directory name does not need to match Module_Key, but using the same name is strongly recommended for clarity.

4. Required Interface — main()

The entry point file must define a top-level function named main with the following signature: 

def main(analyze_id: int, user_id: str) -> str:
    ...
    return "Success message or error description"
Parameter Type Description
analyze_id int The Analyze_ID of the target record. Use this to locate the archive directory.
user_id str The user ID of the owner. Use this together with Global_Setting to resolve paths.
return value str A plain text result message. Stored in Analysis_Module_Run.Result_Summary. Any exception will be caught and logged as an error.
If main is not found in the entry point file, the module task will immediately fail. Do not rename this function.

5. Locating Archive Files

Use Global_Setting to resolve archive paths consistently across environments (Docker / local): 

import os
import Global_Setting as GS

def main(analyze_id: int, user_id: str) -> str:
    aid_dir      = os.path.join(GS.Dir_Archive(user_id), f"AID_{analyze_id}")
    frames_path  = os.path.join(aid_dir, "Frames.csv")
    breaths_path = os.path.join(aid_dir, "Breaths.csv")
    extra_fields_path = os.path.join(aid_dir, "extra_fields.txt")

Key Files

File Description Typical Columns
Frames.csv Frame-level OCR results. One row per frame. Frame, BreathNumber, Pressure, Flow, Volume, EdiPeak, ...
Breaths.csv Breath-level aggregated results. One row per breath. Write your output here. BreathNumber, Start_Frame, End_Frame, ...
extra_fields.txt JSON registry of additional columns. Append your column here so the UI can display it. — (JSON)

6. Registering Output Columns via extra_fields.txt

After writing new columns to Breaths.csv, register them in extra_fields.txt so the VentOCR UI can detect and display them.

File Structure

{
  "secondary_fields": [
    ["ModuleDisplayName", "ColumnName"]
  ],
  "custom_fields": []
}

Example Code

import json, os

def _register_extra_field(extra_fields_path, module_label, column_name):
    extra_fields = {"secondary_fields": [], "custom_fields": []}
    if os.path.exists(extra_fields_path):
        try:
            with open(extra_fields_path, "r", encoding="utf-8") as f:
                extra_fields = json.load(f)
        except Exception:
            pass

    pair = [module_label, column_name]
    if pair not in extra_fields.get("secondary_fields", []):
        extra_fields.setdefault("secondary_fields", []).append(pair)

    with open(extra_fields_path, "w", encoding="utf-8") as f:
        json.dump(extra_fields, f, indent=2, ensure_ascii=False)
If extra_fields.txt does not exist yet, create it from scratch using the structure above.

7. Database Registration

After placing the files, register the module in two database tables.

Modules table

INSERT OR IGNORE INTO Modules (
    Module_Key,
    Display_Name,
    Description,
    Entry_Point,
    Module_Path,
    Version,
    Developer_Name,
    Is_Active,
    Sort_Order
) VALUES (
    'MyAnalysis_VentOCR',      -- unique key (used in URLs and mapping)
    'My Analysis',             -- short display name shown in the UI
    'One-line description.',   -- shown as fallback if README has no Overview
    'core.py',                 -- entry point filename within Module_Path
    'Module/MyAnalysis_VentOCR', -- path relative to /app (the working directory)
    '1.0.0',
    'Your Name',
    1,                         -- 1 = active
    100                        -- display order (lower = shown first)
);

Module_Engine_Map table

A module must be explicitly mapped to one or more engines. Only mapped modules appear in the Module Selection UI for a given study. 

INSERT OR IGNORE INTO Module_Engine_Map (Module_Key, Engine_Key, Is_Active)
VALUES ('MyAnalysis_VentOCR', 'nava_ventocr', 1);
Column Required Notes
Module_Key required Must be unique. Alphanumeric and underscores recommended.
Display_Name required Short label shown in the Module card.
Description optional Fallback text if README.md has no ## Overview section.
Entry_Point optional Defaults to core.py.
Module_Path required Relative path from /app to the module directory.
Is_Active optional Set to 0 to hide the module without deleting the record.

8. README.md Recommendations

A README is not required for execution, but it is strongly recommended. The ## Overview section is extracted and shown as a one-line description in the Module Selection UI. The full README is accessible via the README button on each module card. 

# MyAnalysis_VentOCR Module

## Overview

One-sentence summary of what this module computes. (Keep this to a single line.)

---

## Purpose

Explain the clinical or analytical motivation.

---

## Input Files

- Frames.csv — columns you read
- Breaths.csv — columns you read

---

## Output

### Breaths.csv
Adds:
- MyColumn

### extra_fields.txt
Adds:
["MyAnalysis", "MyColumn"]
under "secondary_fields"

---

## Dependencies

- pandas
- Global_Setting

---

## Author

Your Name
Your Institution
Keep the ## Overview line under ~120 characters. Longer text is automatically truncated with "..." in the UI.

9. Minimal Module Template

import os
import json
import pandas as pd
import Global_Setting as GS


def main(analyze_id: int, user_id: str) -> str:
    aid_dir           = os.path.join(GS.Dir_Archive(user_id), f"AID_{analyze_id}")
    breaths_path      = os.path.join(aid_dir, "Breaths.csv")
    extra_fields_path = os.path.join(aid_dir, "extra_fields.txt")

    if not os.path.exists(breaths_path):
        return f"Breaths.csv not found for AID_{analyze_id}"

    df = pd.read_csv(breaths_path)

    # --- Compute your new column here ---
    df["MyColumn"] = 0.0

    df.to_csv(breaths_path, index=False)

    # --- Register the column in extra_fields.txt ---
    extra_fields = {"secondary_fields": [], "custom_fields": []}
    if os.path.exists(extra_fields_path):
        try:
            with open(extra_fields_path, "r", encoding="utf-8") as f:
                extra_fields = json.load(f)
        except Exception:
            pass

    pair = ["MyAnalysis", "MyColumn"]
    if pair not in extra_fields.get("secondary_fields", []):
        extra_fields.setdefault("secondary_fields", []).append(pair)

    with open(extra_fields_path, "w", encoding="utf-8") as f:
        json.dump(extra_fields, f, indent=2, ensure_ascii=False)

    return f"MyColumn AID_{analyze_id} Success"
Start with this template and replace the placeholder computation with your actual logic. Once main() returns a string without raising, the module run is recorded as successful.

10. Tips for Module Developers

  • Use Global_Setting.Dir_Archive(user_id) for all path resolution — never hard-code absolute paths.
  • Always check that input files exist before reading them, and return an informative string if they are missing.
  • Do not raise unhandled exceptions; return an error string instead so the failure is logged properly.
  • Use df.to_csv(path, index=False) to overwrite the CSV and preserve all existing columns.
  • Register every new column in extra_fields.txt even if the column already exists in the CSV — idempotent registration is safe.
  • If your module depends on another module's output (e.g. a column written by EdiCorrection), document that dependency in your README.
  • Set Is_Active = 0 in the Modules table to disable a module without deleting its records.
Modules run as subprocess Python scripts with full filesystem access. Third-party modules should be reviewed before installation.