Fixup Scripts

Fixup Scripts are MongoDB-compatible scripts that can be applied to databases for maintenance, migration, and data correction tasks. They are executed by the bot service in a defined sequence order, ensuring consistent and predictable database modifications.

Fixup Scripts are typically used for:

• Database Upgrades: Migrating data structure due to new construction kit versions
• Data Corrections: Fixing inconsistent or corrupted data
• Data Anonymization: Removing or masking sensitive information
• Cleanup Operations: Removing obsolete or invalid data
• Schema Migrations: Adapting existing data to new data models

Script Structure

Fixup Scripts are TypeScript files that contain MongoDB shell (mongosh) compatible commands. The scripts have access to the full MongoDB API and can perform any database operation.

Basic Script Example

// MongoDB script for anonymizing email addresses and phone numbers
print("=== Starting Data Anonymization ===");

// Update customer email addresses
db.RtEntity_EnergyCommunityCustomer.updateMany(
    { "attributes.contact.attributes.email": { $exists: true, $ne: null } },
    {
        $set: {
            "attributes.contact.attributes.email": "test@meshmakers.io"
        }
    }
);

// Update phone numbers
db.RtEntity_EnergyCommunityCustomer.updateMany(
    { "attributes.contact.attributes.address.attributes.phone.attributes.number": { $exists: true, $ne: null } },
    {
        $set: {
            "attributes.contact.attributes.address.attributes.phone.attributes.number": "015199999001"
        }
    }
);

print("Anonymization completed");

Data Migration Example

// MongoDB Script: Migrate OperatingFacilities to correct tree structure
print("=== Starting Facility Migration ===");

// Get target tree structure
const targetTree = db.RtEntity_BasicNamedEntity.findOne({
    ckTypeId: "Basic/Tree",
    "attributes.name": "SBEG2"
});

if (!targetTree) {
    print("ERROR: Target tree not found!");
    exit;
}

// Get all facilities to migrate
const facilities = db.RtEntity_BasicNamedEntity.find({
    ckTypeId: "EnergyCommunity/OperatingFacility"
}).toArray();

let migratedCount = 0;

facilities.forEach((facility, index) => {
    // Perform migration logic
    const result = db.RtAssociation.updateOne(
        { originRtId: facility._id },
        { $set: { targetRtId: targetTree._id } }
    );
    
    if (result.modifiedCount > 0) {
        migratedCount++;
        print(`[${index + 1}] Migrated: ${facility.attributes.name}`);
    }
});

print(`Migration completed: ${migratedCount} facilities migrated`);

Importing Scripts

Fixup Scripts are imported using the octo-cli tool. The import process registers the scripts in the system and prepares them for execution.

Command Syntax

octo-cli -c createFixupScript -e true -n <script-name> -f <script-file> -o <order>

Parameters

Parameter	Description	Example
-c createFixupScript	Command to create fixup script	Required
-e true	Enable the script for execution	true/false
-n <name>	Unique name for the script	anonymize, fix-tree-nodes
-f <file>	Path to the script file	./scripts/0_anonymize.ts
-o <order>	Execution order (sequence)	0, 1, 2

Import Script Example

# Import multiple fixup scripts with proper sequencing
octo-cli -c createFixupScript -e true -n anonymize -f ./_fixup-scripts/0_anonymize.ts -o 0
octo-cli -c createFixupScript -e true -n fix-tree-nodes -f ./_fixup-scripts/1_fix_tree_nodes.ts -o 1
octo-cli -c createFixupScript -e true -n delete-old-tree-nodes -f ./_fixup-scripts/2_delete_old_tree.ts -o 2
octo-cli -c createFixupScript -e true -n cleanup-billings -f ./_fixup-scripts/4_cleanup_billings.ts -o 4

Executing Scripts

After importing, scripts are executed by the bot service using:

octo-cli -c runFixupScripts -w

The -w flag enables watch mode, allowing you to monitor the execution progress.

Execution Behavior

• Scripts are executed in sequential order based on the order parameter
• Only enabled scripts (-e true) are executed
• Scripts run in a single transaction context where possible
• Execution stops on first error to maintain data integrity
• The bot service logs all operations for debugging and auditing

Best Practices

Naming Convention

• Use descriptive names that indicate the script's purpose
• Include order prefix for clarity: 0_anonymize, 1_fix_tree_nodes
• Use kebab-case for script names: fix-tree-nodes, cleanup-billings

Script Development

• Always include logging with print() statements for progress tracking
• Add verification queries to confirm changes were applied correctly
• Use transactions for multi-collection operations where possible
• Include error handling and early exit conditions
• Test scripts on development data before production deployment

Sequencing

• Plan execution order carefully - some scripts may depend on others
• Use meaningful order numbers: 0, 10, 20 (allows insertion of intermediate scripts)
• Group related operations: 0-9 for cleanup, 10-19 for migrations, etc.

Safety Measures

• Always backup the database before running fixup scripts
• Test scripts in a development environment first
• Use conditional logic to make scripts idempotent when possible
• Include rollback procedures for critical changes

warning

Fixup Scripts modify database content directly. Always ensure proper backups exist before execution, especially in production environments.

tip

Scripts should be idempotent when possible, meaning they can be run multiple times without causing issues or duplicate changes.