Drupal v7

The source Drupal v7 connector allows content, user, files and folders to be picked up from a Drupal v7 system and transformed into motation.

Specifics

Setup

To set up drupal 7 as a rest server for ImpulseSync to connect to there are a number of requirements to be met.

Required modules

1. Libraries

2. Ctools

3. Entity API

4. Services

5. Services_entity

   1. Apply this patch

6. Services Basic Authentication

Reference/Relationship required modules

1. Entity reference

Language required modules

Languages are a WIP, these are the modules/config currently used, but are subject to change.

1. Internationalization
2. entity_translation
3. Localization update
4. Variable

Configuration

After the required modules are installed, a number of modules must be enabled.

Chaos Tools Suite -> Chaos tools

Other -> Entity API

Other -> Entity tokens

Other -> Libraries

Services -> Services

Services -> Services Entity

Services -> Services Entity Reference

Services - Authentication -> Services basic authentication

Services - Servers -> REST Server

Services configuration

The REST API now needs to be created. Configure the services module.

Press “Add”.

Provide a name for the endpoint.

Select the REST Server.

Set the “Path to endpoint” as “api/1.0”

Set the HTTP basic authentication for Authentication.

Save.

Go to the Server tab.

Remove all response formatters except json.

Save.

Go to the Resources tab.

Allow access for entity_file, entity_node, entity_taxonomy_term, entity_taxonomy_vocabulary, entity_user, file, and node.

Save.

Language modules config

core -> locale

core -> Content translation

multilingual -> Localization update

multilingual - Entity Translation -> Entity Translation

multilingual - Entity Translation -> Entity Translation Upgrade

multilingual - Internationalization -> Block languages

multilingual - Internationalization -> Field translation

multilingual - Internationalization -> Multilingual content

Variable -> Variable

Endpoint Config

When saving an endpoint for a Drupal v7 repository via REST instead of the UI you must use the following key:value pairs in the payload.

• contentRepo:drupal

• contentRepoVersion:v7

Field types

Field types may not always be as expected. This is due to the limitations of the system and modules in the data returned via the APIs. Check the matrix for what the Motation field type is for a field.

Most fields are of type String, Array, File, or Relationship.

We highly recommend mapping all fields from Drupal v7 due to this field type limitation and not relying on ImpulseSync's auto mapping.

Fields

The connector will attempt to transform all attributes from the rest API for any content picked up from the Drupal system. Some fields have multiple attributes within the field, while some attributes are not standard drupal fields. Below are the documented rules for naming convention and the expected value of the field.

For a standard Drupal field:

• If the field has an inner attribute value this will be the "default" value of the field. This field id will match the field machine name of the field in Motation.

• If the field has an inner attribute fid this will be the "default" value of the field. This field id will match the field machine name of the field in Motation. The field type will File

• If the field has an inner attribute tid this will be the "default" value of the field. This field id will match the field machine name of the field in Motation. The field type will Tag

• If the field has an inner attribute target_id this will be a separate field with ID field_machine_name_target_id. This is different from the default field_machine_name field. The default field will be the relationship field to be used and mapped as usual. This separate field will be have the value of the target_id (usually a string value).

• All other inner attributes will have the field ID field_machine_name_inner_attribute.

• The value of a field will be the value of the inner attribute.

  • If the inner attribute's value is a json object (or array) the value will be marshaled into a string field and the value will be the JSON object (or array) as a string.

For other attributes in the rest API response:

• The field ID will be the name of the attribute.

• The value will be the value of the attribute.

  • If the value is another JSON object (or array) the value will be marshaled into a string field and the value will be the JSON object (or array) as a string.

example:

Rest API Response

{
    "title": "article title",
    "field_tags": {
        "und": [
            {
                "tid": "170"
            },
            {
                "tid": "79"
            }
        ]
    },
    "field_author": {
        "und": [
            {
                "target_id": "100123"
            }
        ]
    },
    "field_display_date": {
        "und": [
            {
                "value": "2023-12-04 00:00:00",
                "timezone_db": "UTC",
            }
        ]
    },
    "field_media": {
        "und": [
            {
                "field_file_image_alt_text": {
                    "und": [
                        {
                            "value": "some value",
                            "format": null,
                            "safe_value": "some value"
                        }
                    ]
                },
            }
        ]
    },
    "path": {
        "pathauto": "1"
    }
}

Debug Report Value

{
  "originalContent": [
    {
      "contentId": "article::100124",
      "language": "und",
      "fields": [
        {
          "fieldId": "title",
          "fieldType": "String",
          "fieldValue": [
            {
              "binary": {},
              "order": 0,
              "relations": {},
              "value": "article title"
            }
          ]
        },
        {
          "fieldId": "field_tags",
          "fieldType": "Tag",
          "fieldValue": [
            {
              "binary": {},
              "order": 0,
              "relations": {
                "categoryId": {},
                "childIds": {},
                "fieldId": {
                  "value": "field_tags"
                },
                "order": 0,
                "parentIds": {},
                "tagId": {
                  "value": "170"
                }
              }
            },
            {
              "binary": {},
              "order": 0,
              "relations": {
                "categoryId": {},
                "childIds": {},
                "fieldId": {
                  "value": "field_tags"
                },
                "order": 0,
                "parentIds": {},
                "tagId": {
                  "value": "79"
                }
              }
            }
          ]
        },
        {
          "fieldId": "field_author",
          "fieldType": "Relationship",
          "fieldValue": [
            {
              "binary": {},
              "order": 0,
              "relations": {
                "categoryId": {},
                "childIds": {
                  "value": "author::100123"
                },
                "fieldId": {
                  "value": "field_author"
                },
                "order": 0,
                "parentIds": {
                  "value": "article::100124"
                },
                "tagId": {}
              }
            }
          ]
        },
        {
          "fieldId": "field_author_target_id",
          "fieldType": "Array",
          "fieldValue": [
            {
              "binary": {},
              "order": 0,
              "relations": {},
              "value": "100123"
            }
          ]
        },
        {
          "fieldId": "field_display_date",
          "fieldType": "Array",
          "fieldValue": [
            {
              "binary": {},
              "order": 0,
              "relations": {},
              "value": "2023-12-04 00:00:00"
            }
          ]
        },
        {
          "fieldId": "field_display_date_timezone_db",
          "fieldType": "Array",
          "fieldValue": [
            {
              "binary": {},
              "order": 0,
              "relations": {},
              "value": "UTC"
            }
          ]
        },
        {
          "fieldId": "field_media_field_file_image_alt_text",
          "fieldType": "Array",
          "fieldValue": [
            {
              "binary": {},
              "order": 0,
              "relations": {},
              "value": "{\"und\":[{\"format\":null,\"safe_value\":\"some value\",\"value\":\"some value\"}]}"
            }
          ]
        },
        {
          "fieldId": "path",
          "fieldType": "String",
          "fieldValue": [
            {
              "binary": {},
              "order": 0,
              "relations": {},
              "value": "{\"pathauto\":\"1\"}"
            }
          ]
        }
      ]
    }
  ]
}

Additional Info for Fields

The field "type" for assets and users will be replaced with IMPULSE_ASSET and IMPULSE_USER respectively

Connector queries for child folders

Due to limitations of the rest API provided by the modules, the connector must manually filter out files that are in a child folder, rather than the current folder. This can cause picking up files to take longer than expected for a transaction that is picking up files with many files in child folders.

User reference field

Contents in drupal have a reference to their related user with the key name which is set as a relationship field in Motation. This field is specially created with the field id/unique name impulse_user_reference. This is the value to use to access it's value via manipulators or mappings.

Unsupported

Rest APIs other than listed above.

Entity Reference field (from Entity reference module) with non node target type. i.e., user, taxonomy, file, comment. Only entity reference to node is supported.

Comments are unsupported.

Adapter Properties

Property	Purpose
username	The username for authentication
password	The password for authentication
url	The URL the system's REST APIs are located at. i.e,. http://example.com
rateLimitPerSecond (optional)	The rate limit requests should be sent to the Drupal system. (default: 10)

Job Options

Name	Description	Data Type	Required	Default Value
contentType	The content type to pick up.	Text	true
folder	The folder path to pick up files from.	Text	false	No Default Value
query	Query params to add to the rest call to pick up content IDs with. View Services_entity doc and this patch for additional details.	Text	false	No Default Value
UserChangedParam	Determines if the user object returns the "changed" attribute in API requests.	Boolean	false	true
additionalParams	Additional parameters to add to the REST request to the system. i.e., if a module is enabled that requires a "token" parameter for every request to the system.	Text	false	No Default Value
keepFileAttribute	Determines if an asset content should keep the file attribute.	Boolean	false	false
recursiveFolder	Determines if assets in child/nested folders should be picked up.	Boolean	false	false
maxFolderDepth	Determines how many levels of child/nested folders to pickup assets for.	Integer	false	0 (no depth limit)
excludeFolder	Regex pattern for excluding folders from pickup.	Text	false	No Default Value
alternatePathAttribute	Determines if the "path" field should be retrieved/replaced with the value from the node REST endpoint, rather than entity_node endpoint.	Boolean	false	false

contentType job option

The content type job option is the content type to pick up.

IMPULSE_ASSET keyword

The keyword IMPULSE_ASSET can be used in the contentType job option to pick up files.

IMPULSE_USER keyword

The keyword IMPULSE_USER can be used in the contentType job option to pick up users as a content.

folder job option

This option is paired with the IMPULSE_ASSET content type to limit the files picked up to a specific folder.

The folder job option is the folder to pick up files from. The folder path provided should be the full path from the root.

The folder option uses the uri attribute from the Drupal 7 system to check if the folder exists. There are times when the uri attribute includes additional prepended / characters. i.e., public:////2023-10/inner-folder . These assets that have a URI with additional prepended / will be picked up for migration as well.

query job option

The value set is query parameter(s) to add to the rest call made to the system to query for contents. View Services_entity doc and this patch for additional details and options.

The value should be formatted as query parameters to be added to the rest call made to the system to query for contents. i.e., parameter[title]=info&options[parameters_op][title]=CONTAINS

The patch to the entity_services module adds the query parameter options[parameters_op][<field_name>] which will allow various operations to be applied with the parameters query param.

parameters_op values can be any of the following:

• =

• <>

• >

• >=

• <

• <=

• STARTS_WITH

• CONTAINS

• IN

• NOT IN

• BETWEEN

• LIKE

maxFolderDepth job option

The max folder depth is used in conjunction with the recursiveFolder option. This determines the depth of nested folders to pickup assets for.

Any value > 0 will determine the depth of nested folders to pick up assets from.

Any value <= 0 is equivalent to having no folder depth. Meaning all assets in nested folders will be picked up.

excludeFolder job option

The exclude folder option is a regex pattern that when it matches will exclude that nested folder.

This pattern is run against the value of the nested folder paths.

i.e.,

If the full folder path is:

/folder1/folder2/folder3

And the folder job option is:

/folder1

The nested folder path that the regex pattern will be run on is:

folder2/folder3

Thus be sure to write your regex pattern appropriately.

Motation Object Support

Object	Supported
Category	No
Definition	Yes
Domain	No
Folder	Yes
Language	Yes
Relationship	Yes
Tag	Yes