Environment JDM configs

It is possible to create environments and environment specific JDM configurations. If JDM has environment specific configurations then JDM gets executed with each environment configuration in parallel.
Environment specific configuration overrides JDM's configuration, so it is possible to save common configs in JDM configuration and only specific configs in the environment configuration.
There is no UI for the environments currently so they can only be created directly in database.

Create environment

table environment

• id autoincrement
• name required
• description optional

example:

INSERT INTO environment (`name`, `description`)
VALUES ('DEV', 'DEV'), ('STAGE', 'STAGE');

Create JDM environment config

table job_dispatcher_mapping_config

• id autoincrement
• job_dispacther_mapping_id foreign key to job_dispacther_mapping
• environment_id foreign key to environment
• config json configuration

example:

INSERT INTO job_dispatcher_mapping_config (`job_dispatcher_mapping_id`, `environment_id`, `config`)
VALUES (1, 1, '{"shopwareApi":{"host": "http://localhost","accessKeyId":"test","secretAccessKey":"test"}}');

Execute flow with JDM environment configs

If JDM has the following environment configs:

+----+---------------------------+----------------+------------------------------------------------------------------------------------------------+
| id | job_dispatcher_mapping_id | environment_id | config                                                                                         |
+----+---------------------------+----------------+------------------------------------------------------------------------------------------------+
| 78 |                         1 |              1 | {"shopwareApi":{"host": "http://dev.localhost","accessKeyId":"test","secretAccessKey":"test"}} |
| 79 |                         1 |              2 | {"shopwareApi":{"host": "http://stg.localhost","accessKeyId":"test","secretAccessKey":"test"}} |
+----+---------------------------+----------------+------------------------------------------------------------------------------------------------+

then the flow execution looks like this:

First shopware-output-6 was executed with the config of environment 1
Second shopware-output-6 was executed with the config of environment 2

Extending the environment-specific configuration

The environment-specific configuration inherits all keys from the JDM configuration. It's possible to extend the JDM configuration with additional keys in the environment-specific configuration. Configuration keys can be overridden. The environment-specific configuration has the highest priority.

Example: The main JDM configuration is a common Shopware 6 configuration:

{
  "system": {
    "key": "synqup"
  },
  "_filter": {
    "query": "{\"tags\": {\"$nin\": [\"ignore_global\"]}}"
  },
  "locales": {
    "de-DE": [
      "DE"
    ],
    "en-GB": [
      "EN"
    ]
  },
  "lastSync": "@0",
  "_lastSync": "2000-01-02T00:00:00.000Z",
  "batchSizes": {
    "cache": 100,
    "delete": 150,
    "upsert": 25,
    "batchSize_": 50,
    "deltaCount": 425
  },
  "identifier": {
    "customFieldName": "synqup_identifier",
    "customFieldSetName": "synqup_identifier_set"
  },
  "shopwareApi": {
    "host": "https://localhost:8001",
    "accessKeyId": "***",
    "secretAccessKey": "*****"
  },
  "subsections": {
    "tax": {
      "enabled": true,
      "batchSizes": {
        "cache": 70,
        "upsert": 30,
        "deltaCount": 100
      },
      "associations": {
        "taxRules": {
          "associationsMappingMode": "ignore"
        }
      }
    },
    "product": {
      "_filter": {
        "query": "{\"identifier\": {\"$regex\": \"_ex_custom_gross\"}}"
      },
      "enabled": true,
      "__filter": {
        "query": "{\"identifier\": {\"$in\": [\"_valEx_product_cross_selling_assigned_products_invalid\", \"_valEx_cross_selling_invalid_product\"]}}",
        "operator": "and"
      },
      "variants": {
        "enabled": true,
        "storefrontSettings": {
          "media": {
            "enabled": true,
            "filenames": {
              "prefix": null,
              "forbiddenCharacters": null,
              "keepOriginalFilenames": true,
              "characterReplacementMap": [],
              "defaultReplacementCharacter": null
            },
            "mediaFolderId": "019081e026c67ad28b6f289331b22eec",
            "uploadPriority": 0,
            "assetGroupTypes": [
              "CONFIGURATOR_SETTING"
            ]
          },
          "upsertStorefrontSettings": true
        }
      },
      "___filter": {
        "query": "{\"identifier\": {\"$eq\": \"_ex_suffix_price_list_example\"}}"
      },
      "deleteMode": "hard",
      "warehouses": [
        "1"
      ],
      "_batchSizes": {
        "cache": 100,
        "upsert": 50
      },
      "associations": {
        "tags": {
          "associationsMappingMode": "delta"
        },
        "categories": {
          "preMapBulkSearch": true,
          "associationsMappingMode": "delta"
        },
        "properties": {
          "preMapBulkSearch": true,
          "associationsMappingMode": "delta"
        },
        "visibilities": {
          "associationsMappingMode": "delta"
        }
      },
      "priceRounding": {
        "precision": null
      },
      "productNumberField": "identifier",
      "mapCanonicalProducts": true,
      "enableDeliveryTimeHandling": true,
      "defaultTierPriceShopwareRuleId": "018b386e03817068a76281b663a1176d",
      "deleteAnomalyThresholdPercentage": 10
    },
    "_country": {
      "enabled": true
    },
    "category": {
      "enabled": true,
      "deleteMode": "hard",
      "defaultShopwareLayout": {
        "autoDetection": true,
        "defaultLayoutShopwareId": null,
        "overwriteShopwareLayoutId": false
      },
      "rootCategoryShopwareId": null,
      "category-operations-anomaly-check": false
    },
    "propertyGroup": {
      "enabled": true,
      "deleteMode": "hard"
    },
    "extractEmbedded": {
      "enabled": true,
      "batchSize": 500,
      "extractProductInfo": true,
      "extractCustomerInfo": false
    },
    "propertyGroupOption": {
      "_filter": {
        "query": "{\"identifier\": {\"$in\": [\"83b61dc53829dbe51a390e4087876b67\", \"500d08c2a22017dc7cd66a3663afe8fa\"]}}",
        "operator": "and"
      },
      "enabled": true,
      "batchSizes": {
        "cache": 1000,
        "delete": 100,
        "upsert": 250,
        "validate": 500
      },
      "deleteMode": "hard"
    }
  },
  "customFields": {
    "fill": {
      "sets": [],
      "fields": [],
      "whitelistMode": false
    },
    "create": {
      "sets": [],
      "fields": [],
      "whitelistMode": false
    },
    "ignore": {
      "sets": [],
      "fields": [],
      "whitelistMode": false
    },
    "enabled": true,
    "autoCreate": true,
    "customFieldTypeMap": {
      "some-custom-field-name": "bool_checkbox"
    },
    "validateAttributes": true,
    "passThroughUnknownTypeValues": false
  },
  "sanityChecks": {
    "channels": {
      "ignore": [
        "_ex_invalid_channel",
        "Kaut-Bullinger"
      ]
    }
  }
}

Then you can create an environment-specific configuration for a certain shopware instance by overring the shopwareApi host. Additionally, you can set a filter to the products' identifiers:

{
  "shopwareApi": {
    "host": "https://shopware-instance-1.com"
  },
  "subsections": {
    "product": {
      "_filter": {
        "query": "{\"identifier\": {\"$regex\": \"_ex_custom_gross\"}}"
      }
    }
  }
}

Another environment-specific configuration for another shopware instance:

{
  "shopwareApi": {
    "host": "https://shopware-instance-2.com"
  },
  "subsections": {
    "product": {
      "_filter": {
        "query": "{\"identifier\": {\"$regex\": \"_ex_custom_net\"}}"
      }
    }
  }
}

Environments and Config Snippets

It is possible to use config snippets in environment configurations. The config snippet is a JSON object that can be reused in different environment configurations. The config snippet can be used to define common configurations that are used in different environments. I Both config snippets and templates are supported in environment configurations. Also override works with config snippets in environment configurations. The priority is as follows: 1. Environment configuration (highest priority) 2. JDM override configuration 3. JDM configuration or JDM with a template

Example:

Assume that we have the following config snippets:

snippet_key	snippet_value	snippet_type	description
'%%test-shop-1%%'	'{"host": "https://localhost:8001", "accessKeyId": "", "secretAccessKey": "**"}'	1	'API credentials for the first test shop'
'%%test-shop-2%%'	'{"host": "https://localhost:8002", "accessKeyId": "", "secretAccessKey": "**"}'	1	'API credentials for the second test shop'
'%%test-shop-3%%'	'{"host": "https://localhost:8003", "accessKeyId": "", "secretAccessKey": "**"}'	1	'API credentials for the third test shop'
'%%shopware-output-default%%'	'{"system": {"key": "synqup"}, "filter": {"query": "{\"tags\": {\"$nin\": [\"ignore_global\"]}}"}, "locales": {"de-DE": ["DE"], "en-GB": ["EN"]}, "lastSync": "@0", "_lastSync": "2000-01-02T00:00:00.000Z", "batchSizes": {"cache": 100, "delete": 150, "upsert": 25, "batchSize": 50, "deltaCount": 425}}'	2	'Default template for the shopware output module'

And we assume the following main JDMs: - regular JDM with a regular configuration - JDM with an override configuration, for example: {"shopwareApi": {"host": "https://localhost:8002", "accessKeyId": "***", "secretAccessKey": "*****"}} - JDM with a configured template: '%%shopware-output-default%%'

In the environment table, we have a couple of configurations: - Environment 1 - Environment 2 - Environment 3

Then you can build the environment configurations like this:

job_dispatcher_mapping_id	environment_id	config
1	1	'{"shopwareApi": "%%test-shop-1%%"}'
1	2	'{"shopwareApi": "%%test-shop-2%%"}'
1	3	'{"shopwareApi": "%%test-shop-3%%"}'