# Automating Jobs with Pipelines ```javascript { "pipelineId": "0c08370b-bf1d-11ec-b314-0242ac150009", "name": "updated name", "steps": [ { "pipelineId": "0c08370b-bf1d-11ec-b314-0242ac150009", "name": "Step 1", "order": 1, "jobs": [ "6c454a83-a4ca-4c8f-acb5-ea10521458aa" ] }, { "name": "Step 0", "order": 0, "jobs": [ "87f0e362-82a4-11eb-8dcd-0242ac130003" ] } ] } ``` ## Creating a Pipeline To create a pipeline, you will want multiple jobs created. After you have created 2+ jobs you can get started with creating a pipeline. You can use the [pipeliner API](https://documenter.getpostman.com/view/12031716/UVsSP4EP?version=latest#62b9ce3b-6491-4210-9889-33be80117d8d) to do this. ### Create an Empty Pipeline The best way to create a pipeline is to do it in iterations. Start by creating an empty pipeline using the [/pipelines endpoint](https://documenter.getpostman.com/view/12031716/UVsSP4EP?version=latest#709cf5fb-b211-4ae9-a8c1-22fca189ddeb). ``` curl --request POST '{{impulse-protocol}}://{{impulse-domain}}:{{impulse-port}}/private/pipeliner/pipelines' ``` This will create a pipeline without any steps. The response will include the `pipelineId`. You should save this value to be reused. ### Add a Job to a Pipeline To add jobs to a pipeline, you will need the: * Pipeline ID * Job ID * Step order Using all three of these values you can add a job to the pipeline by using the [/pipelines/{{pipelineID}/job endpoint](https://documenter.getpostman.com/view/12031716/UVsSP4EP?version=latest#d90f939a-53bc-444d-80f0-f2cfe2aa52fb). This example adds three jobs to the pipeline. Two at step 0 and one at step 1. ``` curl --request POST '{{impulse-protocol}}://{{impulse-domain}}:{{impulse-port}}/private/pipeliner/pipelines/{{pipelineID}}/job' \ --data-raw '{ "jobId": "87f0e362-82a4-11eb-8dcd-0242ac130003", "step": 0 }' ``` ``` curl --request POST '{{impulse-protocol}}://{{impulse-domain}}:{{impulse-port}}/private/pipeliner/pipelines/{{pipelineID}}/job' \ --data-raw '{ "jobId": "3a72ccda-c383-11ea-87d0-0242ac130003", "step": 0 }' ``` ``` curl --request POST '{{impulse-protocol}}://{{impulse-domain}}:{{impulse-port}}/private/pipeliner/pipelines/{{pipelineID}}/job' \ --data-raw '{ "jobId": "33791608-58fc-4aef-a3de-42ba71ce0e44", "step": 1 }' ``` Based on the step order provided, the job will be added to that step. When a pipeline runs, it will run all steps in their provided order. So in this example jobs 87f... and 3a7... will run in parallel before job 337... The response to this request will show the updated pipeline, allowing you to verify that it looks as expected. ``` { "pipelineId": "0c08370b-bf1d-11ec-b314-0242ac150009", "name": "0c08370b-bf1d-11ec-b314-0242ac150009", "steps": [ { "pipelineId": "0c08370b-bf1d-11ec-b314-0242ac150009", "name": "Step 1", "order": 1, "jobs": [ "33791608-58fc-4aef-a3de-42ba71ce0e44" ] }, { "name": "Step 0", "order": 0, "jobs": [ "3a72ccda-c383-11ea-87d0-0242ac130003", "87f0e362-82a4-11eb-8dcd-0242ac130003" ] } ] } ``` ### Removing a Job From a Pipeline The [same endpoint](https://documenter.getpostman.com/view/12031716/UVsSP4EP?version=latest#5e272bb8-4948-4b10-a02c-b318fb157214) to add a job to a pipeline step can be used to remove a job from a pipeline step. The payload is the same for both the DELETE and POST requests. ``` curl --location -g --request DELETE '{{impulse-protocol}}://{{impulse-domain}}:{{impulse-port}}/private/pipeliner/pipelines/{{pipelineID}}/job' \ --data-raw '{ "jobId": "87f0e362-82a4-11eb-8dcd-0242ac130111", "step": 0 }' ``` The response to this request will show the updated pipeline, allowing you to verify that it looks as expected. ``` { "pipelineId": "0c08370b-bf1d-11ec-b314-0242ac150009", "name": "0c08370b-bf1d-11ec-b314-0242ac150009", "steps": [ { "pipelineId": "0c08370b-bf1d-11ec-b314-0242ac150009", "name": "Step 1", "order": 1, "jobs": [ "33791608-58fc-4aef-a3de-42ba71ce0e44" ] }, { "name": "Step 0", "order": 0, "jobs": [ "3a72ccda-c383-11ea-87d0-0242ac130003" ] } ] } ``` ## Updating a Pipeline To update the pipeline with a new name or add multiple jobs/steps in a single request you can use the [/pipelines/{{pipelineID}} endpoint](https://documenter.getpostman.com/view/12031716/UVsSP4EP?version=latest#3e6b84e5-6f1c-45a4-be97-c3c995447405). First you will need to get the current pipeline. ``` curl --request GET '{{impulse-protocol}}://{{impulse-domain}}:{{impulse-port}}/private/pipeliner/pipelines/{{pipelineID}}' ``` The response will return the pipeline object. You can then manipulate this object to add more steps, jobs, or update the name of the pipeline. For example, taking the above response you can change the name of the pipeline. ``` { "pipelineId": "0c08370b-bf1d-11ec-b314-0242ac150009", "name": "my-headless-pipeline", "steps": [ { "pipelineId": "0c08370b-bf1d-11ec-b314-0242ac150009", "name": "Step 1", "order": 1, "jobs": [ "33791608-58fc-4aef-a3de-42ba71ce0e44" ] }, { "name": "Step 0", "order": 0, "jobs": [ "3a72ccda-c383-11ea-87d0-0242ac130003" ] } ] } ``` Once the object has be edited you can send it as a payload to the same endpoint in a [PUT request](https://documenter.getpostman.com/view/12031716/UVsSP4EP?version=latest#d1a4a965-7521-41ea-8560-77b8034d80b9). ``` curl --request PUT '{{impulse-protocol}}://{{impulse-domain}}:{{impulse-port}}/private/pipeliner/pipelines/{{pipelineID}}' \ --data-raw '{ "pipelineId": "0c08370b-bf1d-11ec-b314-0242ac150009", "name": "my-headless-pipeline", "steps": [ { "pipelineId": "0c08370b-bf1d-11ec-b314-0242ac150009", "name": "Step 1", "order": 1, "jobs": [ "33791608-58fc-4aef-a3de-42ba71ce0e44" ] }, { "name": "Step 0", "order": 0, "jobs": [ "3a72ccda-c383-11ea-87d0-0242ac130003" ] } ] }' ``` As stated in the API doc, you will need to list every value in the request. Otherwise the value will be defaulted and likely lost. ## Running a Pipeline Like a job pipelines can be run. This is done by using the pipeliner's [/pipelines/{{pipelineID}}/start endpoint](https://documenter.getpostman.com/view/12031716/UVsSP4EP?version=latest#a295fa64-21a9-4728-801d-f7ac12382c88). This request will start a `pipeline transaction` which is combination of standard `syncTransactions` . ``` curl --location -g --request POST '{{impulse-protocol}}://{{impulse-domain}}:{{impulse-port}}/private/pipeliner/pipelines/{{pipelineID}}/start' ``` The response will include the `id` of the `pipeline transaction` . This can be used to track the status of the transaction. ## Tracking Pipeline Transaction Status To track the status of the pipeline transaction, you can use the [/pipelines/{{pipelineID}}/transactions/{{transactionID}} endpoint](https://documenter.getpostman.com/view/12031716/UVsSP4EP?version=latest#8bdb0340-b664-496f-8057-08a2efaaab0b). ``` curl --location -g --request GET '{{impulse-protocol}}://{{impulse-domain}}:{{impulse-port}}/private/pipeliner/pipelines/{{pipelineID}}/transactions/9bfe48f8-bf1f-11ec-bccf-0242ac150009' ``` The response will show the list of sync transaction started by the pipeline. You can then view the status of each [individual sync transaction](https://motivlabs.gitbook.io/impulse-user-manual/using-impulse-headlessly/getting-started-with-curl/starting-a-transaction#tracking-transaction-status). The response will also show if the pipeline transaction is active and the last step that was completed by the pipeline. ## Canceling a Pipeline Transaction Similar to a sync transaction you can also cancel a pipeline transaction. You can use the [/pipelines/{pipelineID}/transactions/{transactionID}/cancel endpoint](https://documenter.getpostman.com/view/12031716/UVsSP4EP?version=latest#1604c926-1edc-49ef-8a6f-d68c50c18805). ``` curl --request POST '{{impulse-protocol}}://{{impulse-domain}}:{{impulse-port}}/private/pipeliner/pipelines/{{pipelineID}}/transactions/{{pipelineTransactionID}}/cancel' ``` This will cancel all sync transactions related to this pipeline transaction stopping the processes from continuing further. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://motivlabs.gitbook.io/impulse-user-manual/using-impulse-headlessly/getting-started-with-curl/automating-jobs-with-pipelines.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.