> This is part 1 of 2 of the full documentation (pages 1–100 of 145).
> The content is paginated: fetch every part to see all of it.
> Next part: https://glyue.docs.sandboxbanking.com/llms-full.txt/1
> Page index: https://glyue.docs.sandboxbanking.com/llms.txt
# What is Integration Gateway?
The best integration builder for banking!
Integration Gateway is a low-code [iPaaS](https://en.wikipedia.org/wiki/Cloud-based_integration) for building and running banking integrations. Banks, credit unions, fintechs, and system integrators use the platform to quickly connect bank and credit union software.
The platform can build/host web service APIs, batch ETL jobs, file/document transfer workflows, and event-triggered pub/sub integrations (e.g. web hooks, Kafka streaming data pipelines).
Integration Gateway offers several advantages over generic integration platforms:
* Numerous built-in adapters for banking-specific systems like core processors, loan origination systems, deposit application systems, CRMs, KYC/AML providers, etc.
* Extensive catalog of reference implementations (i.e. pre-built integrations) for common integration use-cases like loan booking, deposit origination, IVR/IVA interaction, banking as a service (BaaS) APIs, bulk CRM loading, bulk data mart loading, customer 360 APIs, loan renewals/mods, core processor customer/account maintenance, etc. Once imported, these integrations can be customized or extended to fit a specific institution's needs.
* Convenient UI for specifying integration logic like outbound service requests, field mappings, validation rules, etc. Its UX/DX is familiar to those experienced with Excel.
* Security, compliance, and reporting features specific to the banking and credit union industries.
While the platform offers the speed and convenience of low-code mapping, the full power of [Python](https://www.python.org/) and its ecosystem is available within Integration Gateway. Consequently, developers are never limited by the platform's built-in functionality.
**Our team at Sandbox Banking is always trying to improve our documentation!** If you're an existing customer, we welcome you to provide feedback or suggestions via your Integration Gateway customer portal. If you're a prospective user or customer, please feel free to send us an email at [sales@ncino.com](mailto:sales@sandboxbanking.com) with suggestions, questions, or concerns.
We hope you love your integration journey!
{% content-ref url="/pages/hX3YNVBszBQTT2Y7WKgB" %}
[Tutorials](/tutorials/start-here)
{% endcontent-ref %}
{% content-ref url="/pages/Rh2DxTYTlrEAW7QebFqy" %}
[How-To Guides](/how-to-guides/how-to-run-an-integration-from-integration-gateway)
{% endcontent-ref %}
{% content-ref url="/pages/RNjbFnNPeDdJr8vwbktJ" %}
[Reference](/reference/ui_walkthrough)
{% endcontent-ref %}
***
# Start Here
## Beginner Tutorials
The three tutorials presented here are intentionally sequenced, and each tutorial builds on the ones before it.
By working through these tutorials, you will learn to navigate Integration Gateway, build, run, and inspect the outputs of integrations, and connect Integration Gateway to external systems.
These skills will provide the foundation for building larger, more complex integrations on Integration Gateway. After you finish these tutorials, we recommend you learn-by-doing, using the extensive collection of How-To Guides and Reference Docs to further extend your abilities.
**Recommended Order of Tutorials:**
1. [**Building a Single-Step Integration**](/tutorials/building-a-single-step-integration)**:** Learn the basics of navigating Integration Gateway, creating an integration and service requests, and running an integration.
2. [**Building a Multi-System Integration**](/tutorials/building-a-multi-system-integration)**:** Use the results of the first integration to make changes in a second system, a mock banking core.
3. [**Building an Event-Driven Integration**](/tutorials/building-an-event-driven-integration)**:** Modify your integration to respond to requests from external systems.
## Advanced Tutorials
Once you have completed the first three tutorials above, we encourage you to keep exploring and learning with the following, advanced tutorials:
* [**Building an Email Integration**](/tutorials/building-an-email-integration)
* [**Building a RESTful CRUD Web Service**](/tutorials/building-a-restful-crud-web-service)
* [**Building a Retrieval API with FIS Codeconnect**](/tutorials/building-a-retrieval-api-against-fis-codeconnect)
# Building a Single-Step Integration
In this tutorial, we will build a simple integration that calls out to an external API and displays the response. In doing so, we will work with the **Integration Builder**, the **Integration Configuration Wizard**, and the **Run History** page.
This tutorial will use the randomuser API, a public API that returns randomly generated user information. This API does not require authentication, making it suitable for our tutorial purposes.
# 1. Creating the Integration
**Create the Integration**
To create an integration, navigate to the **Build Page** using the navigation sidebar along the left, then click the plus icon (+) in the toolbar along the bottom. This will open the **Integration Configuration Wizard**.
In the wizard, give your new integration a name and description. For example:
> **Name:** User Management (Tutorial)\
> \
> **Description:** This integration requests a user from randomuser.com and performs basic processing on it.
Create a new integration
Leave all other fields as they are, click *Save*, then click *Continue*. You will now see the *Systems* step in the wizard.
**Adding a System**
Integration Gateway uses **Adapters** to store information about the third-party systems you're integrating with. We will now add the information for the `randomuser` API.
Click on *Add System,* then search for and select `HTTP` from the *Available Adapters* dropdown list.
Add a new HTTP adapter
Beneath it, click on the *Select Adapter Config* dropdown, then click *Add New Adapter*. This will open a popup window.
In the popup window, enter the following:
* Name: `randomuser API`
* Host: `glyuedev.public.internal.sandboxbanking.com`
Leave all other fields blank / as-is and click *Save Adapter Bindings*, then click *Continue*.
Add randomuser API details to the HTTP adapter
{% hint style="info" %}
The `randomuser` API doesn't involve authentication so we can leave most of the fields blank. For other APIs that require various forms of authentication, those credentials would be entered here.
{% endhint %}
The final step of the wizard involves sharing permissions to edit and run your integrations with others. For now, we'll skip this step and click on *Go to Build*, where we will see our new integration highlighted on the **Build Page.**
New integration created
# 2. Calling the External System
**Create the Service Request**
{% hint style="success" %}
A new feature called [**Buildhelper**](/integration-gateway-platform-reference/buildhelper) is now available to help creating new Service Requests for various external Systems / APIs with pre-created fields for various request types.
{% endhint %}
Now that the integration has been created, we can begin to add logic to it. For this tutorial, we will simply call the `randomuser` API and receive its response.
Right click on the integration, click *Go To*, then click *Service Request*.
Right click > Go To > Service Request
You will see an empty table.
On the toolbar along the bottom of the screen, click the plus icon (+) to add a new Service Request. This will create a new row in the table, which will be highlighted in green to indicate it was newly created.
Newly created, empty Service Request
We need to fill out three columns on this service request. You may need to scroll your screen to the right to see all of them.
| Column | Value |
| ---------------- | --------------- |
| System | `HTTP` |
| Service Name | `n/a` |
| Formula Variable | `request_users` |
Save your changes by clicking the *Save* icon on the bottom toolbar.
Filled out Service Request
Next, we will create the Field Mappings for this Service Request.
**Create the Field Mappings**
Right click along the service request's row, then click *Go To*, then click *Field Mapping*. You will see an empty table.
{% hint style="info" %}
You can verify that you're on the correct page by checking the *Component* dropdown on the top-left of the page. At this step, it should read "Field Mapping".\
\
{% endhint %}
Create two new field mappings by clicking the plus icon (+) on the bottom toolbar twice. Fill out the first field mapping by entering the following values in the first row:
| Column | Value |
| ---------- | -------- |
| Field | `method` |
| Value | `"GET"` |
| Value Type | `str` |
Then, fill out the second field mapping with the following values:
| Column | Value |
| ---------- | ------------------- |
| Field | `url_path` |
| Value | `"/api/randomuser"` |
| Value Type | `str` |
Two field mappings, method and url_path
Save your changes.
# 3. Running the Integration
**Run the Integration**
Now we have finished building our integration and we are ready to run it and see the results.
Return to the "Integration" level of the **Build page** by using the *Component* dropdown in the top-left corner. Right-click on the integration we just made, and click *Run Integration*. This will open the **Integration Runner*****.***
The Integration Runner
The **Integration Runner** has many options for customizing the input to an integration, which can be useful when mimicking the request from an external system. For now, we will leave all the fields empty.
Run the integration by clicking the *Execute* button on the bottom-left.
You should see a green success message in the bottom-right of your screen.
{% hint style="warning" %}
If you receive an error, double check that the field mappings in [step 2](/tutorials/building-a-single-step-integration/2.-calling-the-external-system) were entered correctly.
{% endhint %}
**View the Results**
To view the results of our integration run, navigate to the **Run History** page using the navigation sidebar along the left.
You should see an entry in the left-side column corresponding to the name of the integration we just built. Open it by clicking on it, then click on the *Step* labeled "REQUEST".
Run details for the integration
In the detail pane on the right side, you will see two sections:
* `Request`, which reflects the values of the field mappings we created in step 2
* `Response`, which contains the response sent back to us by `randomuser.me`
Verify that the response object in the `payload` contains fields that relate to a person. Take note of the structure, specifically that the payload contains a field named `results`, which contains a list of objects, each representing a person. It should resemble the example below:
```
{
"success": true,
"payload": {
"results": [
{
"gender": "female",
"name": {
"title": "Miss",
"first": "Marte",
"last": "Hanssen"
},
"location": {
"street": {
"number": 3519,
"name": "Trasoppveien"
},
"city": "Kyrkstera",
"state": "Trndelag",
"country": "Norway",
"postcode": "5590",
},
"email": "marte.hanssen@example.com",
"dob": {
"date": "1948-03-02T07:02:33.244Z",
"age": 76
},
"registered": {
"date": "2002-11-07T07:03:35.787Z",
"age": 21
},
"phone": "56531940",
"cell": "41662594",
"id": {
"name": "FN",
"value": "02034819494"
},
}
]
}
```
Finally, click on the step labeled `OUTPUT`. Notice that it currently contains a generic message noting the integration completed successfully.
The generic success message in the Output
# 4. Crafting the Output
**Generate an Output**
In the previous step, we saw the response from `randomuser.me`. Now, we will extract and format data from that response, and display it in the output of the integration.
Return to the **Build page**, and open the service request of your integration by right-clicking it and selecting *Go To > Service Requests.*
Find the *After Execute Request Success Hook* (this may require scrolling to the right) and open the code editor by clicking on the expand icon in the cell. A code editor will appear on the bottom half of your screen.
In the code editor, enter the following Python code:
```python
first_name = request_users.response.payload.results[0].name.first
summary = f"The new user's name is {first_name}."
output.payload = summary
```
Code written in the After Execute Request Success Hook
Save your changes.
**Re-run the Integration**
Return to the **Run History** page using the navigation sidebar on the left.
In the entry list on the left, hover over the entry with the name of our integration. You will see the entry highlight, and a circular arrow icon appear on the right side. Click on the rerun icon, which will open the **Integration Runner.**
Leave all fields empty and click *Execute* on the bottom-left. A new entry will appear on the **Run History.**
Close the **Integration Runner** and open the new entry. Open the *Step* labeled `OUTPUT` and now see the message we wrote in the payload. It should read "The new user's name is \[name]".
Updated Output message
***
Congratulations! 🎉 You've finished building a simple integration that makes an API call to a third-party system, parses the response, and returns a message using the information in the response.
# Building a Multi-System Integration
{% hint style="info" %}
This tutorial builds upon the work done in [**Build a Single-Step Integration**](/tutorials/building-a-single-step-integration)**.** If you haven't already, we recommend you complete that tutorial first, then return here.
{% endhint %}
In this tutorial, we extend the single-step integration that retrieves a user from the [randomuser.me](https://randomuser.me) API by further processing the received data and sending it to a second sytem, a mock banking core.
# 1. Connecting to the Core
Start by opening the integration we built. Navigate to the **Build page** using the navigation bar.
Right click on your integration and click *Configure*. This will open the *Configuration Wizard* you used when you first created the integration. Click *Continue* to move to the *Systems* page, and click *+ Add System* to add a second adapter to configure our connection to the mock core.
Add the adapter for the Sandbox Mock Core
Select *Mockcore* from the dropdown. Beneath it, click on the *Select Adapter Config* dropdown, then click *Add New Adapter*. This will open a popup window.
In the popup window, enter the following:
* Name: `Mock Banking Core`
* Host: `sandbox.demo-systems.sandboxbanking.com`
Leave all other fields bank / as-is and click *Save*, then click *Save Adapter Bindings.*
Add details for the Sandbox Mock Core
After saving, close the window.
{% hint style="info" %}
The Mock Banking Core is a playground created by Sandbox Banking to represent a banking core within an integration. It can be accessed [here](https://sandbox.demo-systems.sandboxbanking.com).\
\
The Mock Banking Core is useful for visualizing the effect of integrations before Integration Gateway is connected to a real core.
{% endhint %}
Right click on the integration and select *Go To > Service Request*. You should see the service request we already created, with its `Formula Variable` set to `request_users`.
Click *+ Add Row* to create a new service request. Fill out the following columns on the service request:
| Column | Value |
| ---------------- | --------------------- |
| System | `MOCKCORE` |
| Service Name | `n/a` |
| Formula Variable | `create_new_customer` |
This service request will communicate with the Mock Core and pass it the data that we will set up in the next step.
New Service Request for the Mock Core
Save your changes.
# 2. Field Mapping
Consulting the [documentation for the mock core](https://glyue.docs.sandboxbanking.com/~/changes/sRNNcOmmdTZIw2Ha7IFP/miscellaneous/sandbox-mock-core/dna-style-fields), we see there are 5 fields of interest to us:
* `Name.FirstName`
* `Name.LastName`
* `EmailAddresses.EmailAddress[0].ElectronicAddressLines.ElectronicAddressLine[0].AddrLine`
* `Identification.DateOfBirth`
* `PersTyps.PersTyp[0].PersonTypeCode`
Additionally, we'll need to add the `url_path` of the endpoint that we're calling on the Mock Core.
Right-click on the `create_new_customer` Service Request and select *Go To > Field Mapping*. Click *+ Add Row 6* times to create 6 empty field mappings.
Five empty Field Mappings
Each of the fields required by the core can be populated using data we received from randomuser.me, so we simply need to set up the mapping between the fields across the two systems.
First, we'll populate all the fields that the core expects. In the `field` column, copy in the following JSON paths:
| Field |
| ----------------------------------------------------------------------------------------- |
| `Name.FirstName` |
| `Name.LastName` |
| `EmailAddresses.EmailAddress[0].ElectronicAddressLines.ElectronicAddressLine[0].AddrLine` |
| `Identification.DateOfBirth` |
| `PersTyps.PersTyp[0].PersonTypeCode` |
| `url_path` |
Now for each field that the core expects, we'll enter the JSON path to the value as it exists in the response from randomuser.me:
| Field | Value |
| ----------------------------------------------------------------------------------------- | ------------------------------------------------------ |
| `Name.FirstName` | `request_users.response.payload.results[0].name.first` |
| `Name.LastName` | `request_users.response.payload.results[0].name.last` |
| `EmailAddresses.EmailAddress[0].ElectronicAddressLines.ElectronicAddressLine[0].AddrLine` | `request_users.response.payload.results[0].email` |
| `Identification.DateOfBirth` | `request_users.response.payload.results[0].dob.date` |
| `PersTyps.PersTyp[0].PersonTypeCode` | `"individual"` |
| `url_path` | `"/api/integration/fiservdna/person_maintenance"` |
Notice that while the first four items are JSON Paths into the object returned by `randomuser.me`, the last item is a static value, as it will always be the same for all users created by this integration.
Finally, fill out `str` for the Value Type column for all rows.
Five field mappings for the Service Request
We have now transformed the data received from `randomuser.me` into the format that the mock banking core expects.
Save your changes.
# 3. Running the Integration
In the previous two steps, we added the configuration information for our mock banking core, and set up a Service Request to map the received data from `randomuser.me` to the format expected by the mock banking core.
Now we can run the integration and see the new user created in the mock banking core.
First, open the [mock banking core](https://sandbox.demo-systems.sandboxbanking.com) and review the list of users present.
Sandbox Mock Core. Data may be different when you view the Mock Core.
Return to Integration Gateway, right click on our integration, and click *Run Integration.* Click *Execute* on the **Integration Runner**. Verify that the integration ran successfully. Check the output of the integration and remember the name of the new user.
{% hint style="danger" %}
If your integration resulted in a failure, double check the field mappings you created in the [previous step](/tutorials/building-a-multi-system-integration/2.-field-mapping). Verify that the values are entered exactly as instructed.
{% endhint %}
Now return to the [mock banking core](https://sandbox.demo-systems.sandboxbanking.com) and refresh the page. You will now see the new user that we received from `randomuser.me` as an entry in the mock banking core.
***
Congratulations! :raised\_hands: You've now built an integration that connects two different systems and bridges the gap in their data formats.
The process shown in this and the prior tutorial form the basis of building integrations in Integration Gateway. You can extend this model to build powerful transformations, processors, and connections between an ever-growing list of third-party systems.
# Building an Event-Driven Integration
{% hint style="info" %}
This tutorial builds upon the work done in [**Build a Single-Step Integration**](/tutorials/building-a-single-step-integration) and [**Build a Multi-System Integration**](/tutorials/building-a-multi-system-integration)**.**
If you haven't already, we recommend you complete those tutorials first, then return here.
{% endhint %}
In the previous two tutorials ([1](/tutorials/building-a-single-step-integration), [2](/tutorials/building-a-multi-system-integration)) we ran our integration manually from within Integration Gateway.
In real-world use cases, we often see a different trigger model: Activity in one system causes a chain of events that results in some action in a different system. For example, a new customer created in a CRM will result in that customer being created in the banking core.
We call these **Event-Driven Integrations**, as the integration is run in response to events that call into Integration Gateway, rather than being manually run from within Integration Gateway.
In this tutorial, we will take the logic we built in the previous two tutorials and transform it into a **event-driven** architecture, triggering the integration from a third-party system.
For this tutorial, we will use a mock CRM built using Integration Gateway's [Frontends feature](/how-to-guides/how-to-build-and-deploy-a-custom-frontend).
# 1. Setting up the Mock CRM
**Preparing the Integration for External HTTP API Access**
Integration Gateway integrations are exposed via their URL. We need to make two changes to our integration to prepare it for external calls. Start by navigating to the **Build page,** right-clicking on the integration, and selecting *Configure*.
1. Remove spaces from the integration's Path Name so it can be used in a URL.\
Change `User Management (Tutorial)` to `user_management_tutorial`
2. Enable HTTP API access by checking the *Enable API Access* box.
Modifying the integration for HTTP API access
Click *Save*, then close the modal using the *X* in the top-right corner.
**Finding the Integration URL**
To get an integration's execution URL, right click on the integration and click *Run Integration.* The top of the **Integration Runner** shows the integration's URL. Copy the URL using the *Copy* button on the right side and save it.
The URL of the integration
**Setting up the Mock CRM**
Using the navigation bar, open *Frontends.* Click *Create* in the top right. Enter the following:
> Name: Mock CRM
>
> Path: mock-crm
Download this `.zip` file that contains the source code for a simple mock CRM:
{% file src="/files/QWxCVpUmnfJA4TJJwqZO" %}
Upload it to your Frontend by dragging the `.zip` file into the *Files* section on the right side.
Creating the Mock CRM using Frontends
Save your frontend by clicking *Create*.
**Configuring the Mock CRM**
Open the mock CRM by clicking the *View* button. In the popup window, enter the integration URL you obtained in the previous step and click *Save Configuration**.***
Enter the integration URL in the Mock CRM
# 2. Receiving Inbound Requests
**Previewing the Data**
The CRM is now set up to send requests to the integration we created in Integration Gateway, but the integration is not set up to read the data that's being sent.
To start, we'll trigger a request from the CRM to view the request's payload structure on a live example.
Fill out all the fields in the CRM, then click *Send to Core.* You will see an error on CRM, which is expected at this stage.
Filling out an entry in the Mock CRM
In Integration Gateway, open the **Run History** and click on the most recent entry, then click on *INPUT* in the Steps column.
{% hint style="info" %}
If you don't see a new run, re-enter the URL you entered in [step 1](/tutorials/building-an-event-driven-integration/1.-setting-up-the-mock-crm). If the URL is incorrect, the request won't show up in the **Run History**.\
\
If you still don't see the request, reach out to to verify your IP allowlist settings to ensure external traffic is allowed to reach your Integration Gateway instance.
{% endhint %}
You should see the data you entered in the CRM appear as fields in the `payload`. Notice that while the meaning of the fields are the same as those from `randomuser.me`, the naming and structure are slightly different.
Viewing the payload structure from the CRM.
Now that we have the structure of the data from the CRM, we need to adjust our mappings to reflect it.
**Removing the Call to randomuser.me**
First, we'll remove the service request that calls `randomuser.me` because we are no longer using it as the source of our user information.
In Integration Gateway, navigate to the **Build page*****,*** select our integration, right-click, and *Go To > Service Request*.
Right click on the service request with a *Formula name* of `request_users`. Click *Delete.*
Deleting the request_users Service Request
**Updating the Call to the Core**
Right click on the remaining service request (`create_new_customer`) and click *Go To > Field Mapping*.
We will adjust the *Value* column, keeping the *Field* column the same.
{% hint style="info" %}
Note that the `PersTypes.PersonTypeCode` and `url_path`fields did not change, as their value are hard-coded.
{% endhint %}
Updating Field Mapping values
Save your changes.
# 3. Triggering the Integration from the CRM
Return to the CRM.
Open the [mock core](https://sandbox.demo-systems.sandboxbanking.com) and familiarize yourself with its current contents. After running the integration, you will see the customer you entered in the CRM appear in the mock core.
Once both systems are open, return to the CRM and click *Send to Core*.
Return to the mock core and verify that the new customer was created in the core.
{% hint style="info" %}
For simplicity, this integration always creates a new user in the core when run. If you run the integration multiple times with the same input from the CRM, you will see duplicate entires in the mock core.
{% endhint %}
***
Congratulations! :tada: You have transformed your trigger-based integration into an event-based integration, and demonstrated connectivity between a mock CRM and mock banking core.
Over the course of these three tutorials, you have learned to leverage the simple but powerful framework at the heart of Integration Gateway: transforming data using field mappings and connecting systems using service requests. Using the knowledge you learned here, you can now create integrations tailored to your specific systems and needs.
As you grow into more complex integrations, we encourage you to make use of the extensive selection of How-To Guides and Reference Documentation to leverage the full power of Integration Gateway.
Happy building!
# Building an Email Integration
This tutorial aims to guide a first-time Integration Gateway user through the process of building a real integration, step by step, from start to finish. This integration will take an email address as input, then query the [Hacker News API](https://github.com/HackerNews/API) for articles, and finally email a list of article URLs to the provided address.
Upon completion of the tutorial, the reader will understand:
* integrations, as well as how to build one,
* how to use the Admin site to configure Integration Gateway components,
* how to validate the data that passes through Integration Gateway,
* how to visualize that data at every step,
* how to trigger integrations,
and more generally, the *awesome power of* Integration Gateway!
{% hint style="warning" %}
The reader will need the following before starting:
* A Integration Gateway user account with admin privileges
* A Gmail/Google account
* A basic understanding of JSON and HTTP (helpful but not 100% required)
* Basic skills in the Python programming language (helpful but not 100% required)
{% endhint %}
# 1. Create and Configure the Integration
To start, we will create the scaffolding for the integration by setting up its metadata (name, description, and settings) and its **Adapter Configs.** We'll finish this step by validating that the adapters we created can successfully contact the systems we connected them to.
{% hint style="info" %}
**Adapters** are modules in Integration Gateway that handle authentication, sending, and receiving of information with external systems. Each adapter needs some initial configuration before it can be used, which is stored in an **Adapter Config.**
{% endhint %}
**Create the Integration and Adapter Configs**
1. Log into Integration Gateway and navigate to the **Build** page (click the 🔧 \[wrench] icon on the left-hand navbar).
1. Make sure that **Integration** is selected under **Component** in the top-left corner.
2. Click the **\[➕ADD ROW]** button on the bottom.
3. In the wizard that appears, enter the following:
1. **Path Name:** `hackernews_email_digest`.
2. **Enable API Access:** Check the box to enable HTTP calls.\
3. **Description:**
```
Queries the Hacker News API and sends an email with links to the latest top posts.
Input:
{
"email": "email_address@domain.tld",
"limitQuery": 100,
"limitEmail": 10
}
email: REQUIRED | str | the email address to send the digest to
limitQuery: OPTIONAL | int | limit to the number of articles to query
limitEmail: OPTIONAL | int | max number of articles to send in the email
```
4. Expand the **Email Settings** section and uncheck **Email Digests**
5. Expand the **Run Settings** section and ensure **Store Run History** and **Store Run History Payloads** are enabled
4. Click the **Save** button to move to the next page of the wizard.
2. On the **Systems step** of the wizard, we will create two **Adapter Configs,** one for accessing the Hacker News API and another for Gmail SMTP
1. Click on **Add System** to create the Hacker News API adapter
2. In the **Available Adapters** dropdown, select **HTTP**
3. In the **Select Adapter Config** dropdown, select **Add New Adapter** and fill out the following information, then click **Save**:
1. **Name**: `hackernews`
2. **Active:** ☑️ (checked)
3. **Validation enabled**: ☑️ (checked)
4. **Host:** `hacker-news.firebaseio.com`
5. **Port:** `443`
6. **Basic auth username**: (leave blank)
7. **Basic auth password**: (leave blank)
8. **Server auth cert**: (leave empty)
9. **Client auth cert**: (leave empty)
10. **Timeout seconds**: `60`
4. Click on **Add System** again, this time to create the Gmail SMTP adapter
5. In the **Available Adapters** dropdown, select **Email SMTP**
6. In the **Select Adapter Config** dropdown, select **Add New Adapter** and fill out the following information, then click **Save**:
1. **Name:** `gmail`
2. **Active:** ☑️ (checked)
3. **Validation enabled:** ☑️ (checked)
4. **Host:** `smtp.gmail.com`
5. **Port:** `587`
6. **Tls encrypt:** ☑️ (checked)
7. **Authenticate:** ☑️ (checked)
8. **Auth username:** (the Google email address)
9. **Auth password:** (the created Google App Password)
7. Click **Save Adapter Bindings** on the top right to save the adapters we created, then click **Continue**
8. On the final page of the wizard, give your user all permissions (Read, Write, Execute, and Debug) for the integration you just created. Click **Save Permissions**, then close the wizard by clicking **Go to build.**
**Verify the Adapter Configs**
1. Return to the Dashboard on the on main Integration Gateway interface.
2. Click the **\[VALIDATE]** button in the **Adapter Config Validation Results** section and wait for results. If successful, the results should look like this:
```
Validation passed for these systems:
hackernews
gmail
```
If unsuccessful, there will be more information about what caused the failure, including a Python stack trace.
3. Navigate to the **Swagger API** page by selecting **API** from the left-hand navbar.
1. Select `/hackernews_email_digest` and click **Try it out** to expand the Swagger console for this endpoint.
2. Delete the request body and input: `{}` (an empty JSON object)
3. Scroll to the bottom and click **Execute**.
4. Navigate to **Run History** by clicking the circular-arrow-clock icon from the left-hand navbar.
5. Under the **RUNS** column (the left-most column) there should be a `hackernews_email_digest` record, along with a timestamp, username, and id. Click it.
6. Under the next column, **RUNS**, an `INPUT` and `OUTPUT` should appear. Clicking them should display the http request (input) that triggered the integration, as well as its output respectively:
```
{
"payload": "Integration hackernews_email_digest completed successfully!",
"headers": {},
"status": 200
}
```
{% hint style="success" %}
Congratulations, the integration works!
It doesn’t do much yet though, so let’s continue building.
{% endhint %}
# 2. Input Validation
You may remember that in the integration description we provided an example input that included a required `email` field and optional `limitQuery` and `limitEmail` fields. If the requestor doesn't provide an email address in their input, how would we know where to send the digest? Or, what if the requestor provides invalid limit values (which should be positive integers)?
In this case we want the integration to fail gracefully and respond to the requestor with a failure status and indicate which field(s) need to be fixed. This is where **Validation Rules (VRs)** come in to play.
#### Adding Validation Rules to the integration
1. On the **Build** page, with **Integration** selected under **Component** at the top, right-click the row with `hackernews_email_digest` and select **Go To > Validation Rule**.
2. Click the **\[➕ADD ROW]** button 2 times to add 2 new rows. Enter the following values for each row:
1. First row:
* **Type**: `INPUT`
* **Field**: `input.payload.email`
* **Rule**: (copy and paste the below Python code)
```
import re
pattern = r"^\w[\w&'\+\-\.]*\b@[a-zA-Z\d][a-zA-Z\d\-]*\b\.[a-zA-Z\d]+[a-zA-Z\d\-]*[a-zA-Z\d]$"
retvalue = _ and isinstance(_, str) and re.match(pattern, _)
```
2. Second row:
* **Type**: `INPUT`
* **Field**: `input.payload.limitQuery`
* **Rule**: (copy and paste the below Python code)
```
not _ or (isinstance(_, int) and _ > 0)
```
3. Select the second row and click the **\[DUPLICATE ROW]** button at the bottom and make the following edits to the new row:
* **Field**: `input.payload.limitEmail`
4. Click the **\[**�� **Save]** button at the bottom.
{% hint style="info" %}
If you are familiar with Python, you may have an idea of what is going on here.\
(`_` represents the value of the field named in the **Field** column)
* In the first validation rule, we ensure that the provided email exists, is a string, and a properly formatted email using a regex expression.
* In the second and third, we ensure that, the limits are either not provided or are positive integers
{% endhint %}
#### Testing the Validation Rules
Now we’re going to validate that our new Validation Rules work correctly.
1. Navigate to the **Run History** page. Select our most recent run, and click the **curved arrow symbol** on it. The **Run Integration** dialog window should appear.
1. Copy and paste this into the **PAYLOAD**:
```
{
"email": "user@email.net",
"limitQuery": 10
}
```
2. Click **RUN INTEGRATION** at the bottom. It should run successfully.
2. On the **Run History** page, click the most recent integration and then the curved arrow again.
1. This time, provide some invalid values in our payload:
```
{
"email": "invalid%email###@@123",
"limitEmail": -1
}
```
2. Re-run the integration again. It should fail.
3. Click the most recent run (once it appears- it may take a couple seconds). There should be two **VALIDATION FAILURE** records in the middle **STEPS** column. Clicking on them will provide information.
{% hint style="info" %}
It's strongly encouraged to play around a little before continuing!
* Try triggering the integration with various valid or invalid inputs.
* Practice viewing them on the **Run History** page.
{% endhint %}
# 3. Get Story IDs from Hacker News
Now we are ready to create our first external API call. Integrations can have any number of calls. In Integration Gateway these calls are referred to as **Service Requests (SRs)**. Each SR represents one API call, and will take a set of **Field Mappings** (more on this later) as input data for sending to the other system. When it executes, it will send the data and save the response.
In order to know how to compose a Service Request and its Field Mappings for any system, the user will need both the respective Integration Gateway adapter documentation, as well as any documentation of the external system. The adapter reference article will lay out the Adapter Config, Service Request, and Field Mapping requirements, while the other system’s documentation should inform the user as to what endpoints are available, what each endpoint expects and returns, error troubleshooting advice, etc.
{% content-ref url="/pages/QuyIUmeV58dnWmvA93GH" %}
[Adapters](/reference/adapters)
{% endcontent-ref %}
#### Service Request: `get_story_ids`
In our first SR, we want to query Hacker News for the latest stories.
1. From the **Build** page, with the Component set to **Integration**, right click the integration and click **Go To > Service Request**.
2. Click the **\[➕ADD ROW]** button, and provide these values:
* **System:** `HTTP` (this is how we indicate that the SR should use the HTTP adapter)
* **Service Name:** `n/a`
* **Formula Variable:** `get_story_ids`
3. Leave the rest of the row unchanged and save.
#### Add Field Mappings for SR: `get_story_ids`
Now we’re ready to tell the Service Request what data to send using **Field Mappings (FMs)**. An SR can have any number of FMs. Each FM represents a field in the data that will be sent to the other system, as well as its value (or the logic by which the value is gotten).
Now we’re going to add a couple of simple FMs to our SR.
1. From the **Build** page with Component set to **Service Request**, right click the SR we just created and select **Go To > Field Mapping**.
2. Click **\[➕ADD ROW]** 2 times to add 2 new rows.
3. In the top row add the following:
* **Field:** `url_path`
* **Value:** `"/v0/newstories.json"` (include the quotes)
* **Value Type:** `str`
4. For the next row, enter:
* **Field:** `method`
* **Value:** `"GET"` (include the quotes)
* **Value Type:** `str`
5. Save.
#### Test `get_story_ids`
We are now ready to send our first request to an external real API! Let’s test the integration:
1. Go to the **Run History** page, and re-run our integration with valid inputs.
2. Wait for this run to appear in the **RUNS** column, then click it.
3. In the middle **STEPS** column, there should now be 3 records: INPUT, REQUEST, and OUTPUT. Select the REQUEST.
4. The collapsible `⟶ Request` and `⟵ Response` sections should appear on the right, along with their payloads. The request should look like this:
```
{
"service_name": "n/a",
"payload": {
"url_path": "/v0/newstories.json",
"method": "GET"
},
"sub_requests": {}
}
```
While the response will look something like this:
```
{
"success": true,
"payload": [
36688166,
36688146,
36688107,
36688074,
...
],
"messages": []
}
```
Congratulations! The integration has successfully communicated with an external system!
# 4. Get Story Content
We have communicated with the Hacker News API, but its response contains a list of story ids while we need the URLs for our email digest. Let’s make another request (or rather, a set of requests) to retrieve each story by its id so we can later collate their URLs into a single message.
#### Service Request: `get_story_by_id`
1. From the **Build** page, with **Integration** selected under Component, right-click our integration and click **Go To > Service Request**.
2. Add a new row, enter the following, and save:
* **Sequence:** `100`
* **System:** `HTTP`
* **Service Name:** `n/a`
* **Formula Variable:** `get_story_by_id`
* **Call For Each:** copy and paste the Python code below.\
This expression evaluates to the list of ids we got from `get_story_ids`, then cut off at the `limitQuery` value (if provided). It’s ok if this code doesn’t make sense yet, so don’t worry!
```
get_story_ids.response.payload.unwrap()[:input.payload.limitQuery or None]
```
Integration Gateway will execute a SR once for every item in its **Call For Each**. Using python lists is the most common.
#### Field Mappings for `get_story_by_id`
1. Right-click the `get_story_by_id` SR and select **Go To > Field Mapping**.
2. Click **\[➕ADD ROW]** 2 times to get 2 blank rows. Enter the following values and then save:
* First row
* **Field:** `url_path`
* **Value:** `f"/v0/item/{sritem}.json"`
* **Value Type:** `str`
* Second row
* **Field:** `method`
* **Value:** `"GET"`
* **Value Type:** `str`
`sritem` is a special variable that represents the current item in the **Call For Each** of its parent SR. In this case, `sritem` will be a story id.
#### Test `get_story_by_id`
1. Navigate to the **Run History** page (or open it in a new tab to keep it handy!) and re-run the integration with this input payload:
```
{
"email": "user@email.net",
"limitQuery": 10
}
```
2. It should complete successfully. Wait for the most recent run to appear at the top of the **RUNS** column and click it to see its **STEPS** in the middle column. We should see several records now- an INPUT, the first REQUEST (`get_story_ids`), but after that there should be 10 more REQUEST records labelled with `get_story_by_id`. Click one of them to view its request and response payloads, which should look something like this:
* REQUEST
```
{
"service_name": "n/a",
"payload": {
"url_path": "/v0/item/36712566.json",
"method": "GET"
},
"sub_requests": {}
}
```
* RESPONSE
```
{
"success": true,
"payload": {
"by": "JoeMayoBot",
"descendants": 0,
"id": 36712566,
"score": 1,
"time": 1689268222,
"title": "Windows 95, 98, and other decrepit versions can grab online updates again",
"type": "story",
"url": "https://arstechnica.com/gadgets/2023/07/windows-95-98-and-other-decrepit-versions-can-grab-online-updates-again/"
},
"messages": []
}
```
{% hint style="success" %}
Our Integration Gateway integration has now made 11 requests to an external API, with 10 of them being done in a programmatic fashion. Nice Job!
{% endhint %}
# 5. Sending the Email
Now we’re ready for the final step. We’re going to take each of those story responses and collate them into an email body and then send it off to our recipient.
#### Service Request: `send_email`
Navigate to the **Build** page, go to the Service Requests for our integration, and add a new row. Enter the following values:
* **Sequence:** `200`
* **System:** `EMAIL_SMTP`
* **Service Name:** `n/a`
* **Formula Variable:** `send_email`
* **Before Prepare Request Hook:** copy and paste the code below.\
This is the most complicated Python code we have shown to the reader thus far. Basically, we are creating a standard message for each story we received, and writing it to a file. We will later read this file for the body of the email.
```
from datetime import datetime
stories = [sr.response.payload for sr in get_story_by_id]
stories.sort(key=lambda story: story.score, reverse=True)
query_ct = 500 if not input.payload.limitQuery else input.payload.limitQuery
email_limit = 50 if not input.payload.limitEmail else input.payload.limitEmail
with open_glyuefile("email_body.txt", "w") as file:
file.write(f"Here are the top {email_limit} of the latest {query_ct} articles from Hacker news:\n\n")
for story in stories[:email_limit]:
time = datetime.fromtimestamp(story.time).strftime("%m/%d/%Y, %H:%M")
file.writelines([
f"{story.title}\n",
f"{story.url}\n",
f"Posted by [{story.by}] on {time} ({story.score} points, {story.descendants} comments)\n",
f"Discussion: https://news.ycombinator.com/item?id={story.id}\n\n"
])
```
#### Field Mappings for SR: `send_email`
Right-click the newly-created SR and go to Field Mappings. Add 4 rows with these values and then save.
* First row
* **Field:** `from_address`
* **Value:** `"email@address.net"` (use the same email as was entered for the Email SMTP Adapter Config, and don’t forget the quotes)
* **Value Type:** `str`
* Second row
* **Field:** `to_addresses`
* **Value:** `[input.payload.email]`
* **Value Type:** `list`
* Third row
* **Field:** `subject`
* **Value:** `"Top HackerNews Stories"`
* **Value Type:** `str`
* Fourth row
* **Field:** `body`
* **Value:** `open_glyuefile("email_body.txt").read()`
* **Value Type:** `str`
{% hint style="warning" %}
Don’t forget to save!
{% endhint %}
# 6. Wrapping Up
#### Final tests
Re-run the integration with this input (use a real email address whose inbox can be accessed):
```
{
"email": "example@example.com",
"limitQuery": 100,
"limitEmail": 10
}
```
The integration should take several seconds or up to a minute (it is calling the story API 100 times after all). After it completes, check the email inbox and see if it arrived (it may have gone to spam)!
Hopefully the email was received and looks a little like this:
```
I queried the latest 100 articles from Hacker news.
Here are the top 10:
Boston mayor announces residential conversion program for office buildings
https://www.bostonplans.org/news-calendar/news-updates/2023/07/10/mayor-wu-announces-residential-conversion-program
Posted by [toss1] on 07/13/2023, 20:27 (131 points, 100 comments)
Discussion: https://news.ycombinator.com/item?id=36715504
Actors say Hollywood studios want their AI replicas – for free, forever
https://www.theverge.com/2023/7/13/23794224/sag-aftra-actors-strike-ai-image-rights
Posted by [mfiguiere] on 07/13/2023, 20:41 (66 points, 94 comments)
Discussion: https://news.ycombinator.com/item?id=36715686
...
```
:tada: :tada: :tada: Huzzah! The integration is now complete and functional. :tada: :tada: :tada:
{% hint style="info" %}
Try re-running the integration from the **Run History** page with various different inputs. Play around a little bit and have fun!
{% endhint %}
This concludes the tutorial. We hope that the concepts learned in this article will lay the foundation for the reader’s Integration Gateway integration building journey.
{% content-ref url="" %}
{% endcontent-ref %}
# 7. Extra credit
### Triggering `hackernews_email_digest` from outside Integration Gateway
{% hint style="info" %}
[Postman](https://www.postman.com/) (an HTTP client for testing APIs) must be installed to proceed.
{% endhint %}
1. Download and install Postman if not already done. Open Postman.
2. Import this collection: [extra\_credit.json](/tutorials/building-an-email-integration/7.-extra-credit/extra_credit-json)
3. Go to the collection variables and replace them with the appropriate values.
4. Go to the `hackernews_email_digest` request in the collection and click **Send**.
Integrations aren’t usually triggered by Postman specifically, but it provides us with a way to fine-tune http requests and test APIs. Any external application that wants to trigger a Integration Gateway integration will need the URL as well as the username and password for a Integration Gateway user who has integration execution permissions. Usually, a "service account" user is created for this purpose.
#### Extra-Extra Credit:
If the reader has access to an external system such as Salesforce, or some other CRM, POS, or LOS, why not try setting up that system to communicate with Integration Gateway, using the Postman collection as an example?
# extra\_credit.json
```json
{
"info": {
"_postman_id": "51d39973-5b27-42e7-b059-8f0d084dc7e4",
"name": "Glyue Tutorial: Build a Basic Integration",
"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
},
"item": [
{
"name": "hackernews_email_digest",
"request": {
"method": "POST",
"header": [],
"body": {
"mode": "raw",
"raw": "{\r\n \"email\": \"{{email}}\",\r\n \"limitQuery\": 100,\r\n \"limitEmail\": 5\r\n}",
"options": {
"raw": {
"language": "json"
}
}
},
"url": {
"raw": "https://{{host}}/integrations/execute/hackernews_email_digest",
"protocol": "https",
"host": ["{{host}}"],
"path": ["integrations", "execute", "hackernews_email_digest"]
}
},
"response": []
}
],
"auth": {
"type": "basic",
"basic": [
{
"key": "password",
"value": "{{password}}",
"type": "string"
},
{
"key": "username",
"value": "{{username}}",
"type": "string"
}
]
},
"event": [
{
"listen": "prerequest",
"script": {
"type": "text/javascript",
"exec": [""]
}
},
{
"listen": "test",
"script": {
"type": "text/javascript",
"exec": [""]
}
}
],
"variable": [
{
"key": "username",
"value": "put_glyue_username_here",
"type": "default"
},
{
"key": "password",
"value": "put_glyue_password_here",
"type": "default"
},
{
"key": "host",
"value": "glyue.domain.name.com",
"type": "default"
},
{
"key": "email",
"value": "destination_email_address",
"type": "default"
}
]
}
```
# Building a RESTful CRUD Web Service
In this guide, we will use Integration Gateway to create custom web service endpoints and its vault functionality as an underlying data store for retrieving, adding, updating, and deleting data when these endpoints are executed.
{% hint style="info" %}
To help you follow this guide, you can download the JSON export of the integrations that are setup here and import the file into your own Integration Gateway environment.
{% file src="/files/Mx1iqdKrojD2RUd9eo0Z" %}
{% endhint %}
### Integration
{% content-ref url="/pages/TQoJ19imSZz3Z4AOuH5w" %}
[1. Integration Setup](/tutorials/building-a-restful-crud-web-service/1.-integration-setup)
{% endcontent-ref %}
### Vault
{% content-ref url="/pages/4TND9DZ4AsJyiHRksY10" %}
[2. Vault Setup](/tutorials/building-a-restful-crud-web-service/2.-vault-setup)
{% endcontent-ref %}
### RESTful CRUD Web Services using Vault
{% content-ref url="/pages/5d3lYF1aSPJNeiTwsN5y" %}
[3. Create Web Service Endpoints](/tutorials/building-a-restful-crud-web-service/3.-create-web-service-endpoints)
{% endcontent-ref %}
{% content-ref url="/pages/uurwOFxUJ6twyUCNiDe2" %}
[4. Execute Web Service Endpoints with Vault Methods](/tutorials/building-a-restful-crud-web-service/4.-execute-web-service-endpoints-with-vault-methods)
{% endcontent-ref %}
### Code Examples and Explanation
{% content-ref url="/pages/JJXZe5BIe8eq8sd0wkED" %}
[5. Vault Code Examples and Explanation](/tutorials/building-a-restful-crud-web-service/5.-vault-code-examples-and-explanation)
{% endcontent-ref %}
# 1. Integration Setup
### Add Integration
We will setup an integration for each of the services listed below to read and/or write from/to vault. We will set up and name our vault **data** in the next guide.
* Get all customers
* Get a specific customer by Id
* Create a customer
* Update a customer
* Delete a customer
To create an integration, log into Integration Gateway, if necessary and navigate to the **Build** page. The default component for the **Build** page is the **Integration** table, therefore, click on the **Add Row** button.
This integration will retrieve all customers from the **data** vault. Fill in the path name and the description field with the following information.
* **Path Name** - get\_all\_customers
* **Description** - This integration will retrieve all customers from the data vault.
Click on the **Save** button below.
Create an integration for the remaining services as shown in the image below.
### Add Integration Config
{% hint style="warning" %}
To perform the following steps, the user must be a Integration Gateway administrator for the target environment.
{% endhint %}
Log into Integration Gateway, if necessary and navigate to the **Admin** site.
Scroll to the **Configuration** section, locate **Integration Config** and click **Add**.
On the **Add** **Integration Config** page, select the integration `get_all_customers` from the Integration dropdown menu and check the box **Store payloads in run history for x days**.
Click on the **Save** button below**.**
Add an integration config for each integration we had created.
# 2. Vault Setup
{% hint style="warning" %}
To perform the following steps, the user must be a Integration Gateway administrator for the target environment.
{% endhint %}
In this step, we will make use of Integration Gateway's vault feature as a data store for our list of customers. To learn about vault and its full suite of methods, visit [this guide for more information](/tutorials/building-a-restful-crud-web-service/5.-vault-code-examples-and-explanation).
### Create Vault
Log into Integration Gateway and navigate to the **Vault** page.
Click on the **Create Vault** button, name the vault **data**, and save by clicking on the button again or pressing enter on your keyboard**.**
You should see the newly created vault on the **Vault** page.
### Create Vault Item
{% hint style="warning" %}
To perform the following steps, the user must be a Integration Gateway administrator for the target environment.
{% endhint %}
We will create a vault item containing a list of customer information. Each customer will have an id, first name, last name, birth date, and a social security number.
To create a vault item, click on the **data** vault to highlight it and click on the **Create Vault Item** button. Name the vault item key **customers** and add a value of `[]` for the vault item key.
Save the vault item.
The **data** vault should now have a vault item named **customers.**
### **Add Vault Permissions**
{% hint style="warning" %}
To perform the following steps, the user must be a Integration Gateway administrator for the target environment.
{% endhint %}
Log into Integration Gateway, if necessary and navigate to the **Admin** site.
Scroll to the **Configuration** section, locate **Vault Permissions** and click **Add**.
On the **Add** **Vault Permissions** page, select **data** from the **Vault** dropdown menu, select the integration `get_all_customers` from the Integration dropdown menu, and check the boxes **Read** and **Write**.
Click on the **Save** button below**.**
Add a vault permission for each integration we had created.
# 3. Create Web Service Endpoints
We will create a custom web service endpoint for each integration that was setup in the previous step.
{% hint style="info" %}
All integrations can be exposed via the standard `integrations/execute/..` HTTP endpoint. However, a custom web service endpoint will allow you to choose a specific RESTful method (e.g. GET, POST, PUT, PATCH, and DELETE) so that the method reflects the type of web service being called.
{% endhint %}
### Add Web Service Endpoints
To create a web service endpoint to retrieve all customers from the **data** vault, log into Integration Gateway, if necessary and navigate to the **Admin** site.
Scroll to the **Configuration** section, locate **Web service endpoints** and click **Add**.
On the **Add Web service endpoint** page, fill in the fields with the following information
* **Path** - get\_all\_customers
* **Method** - Get
* **Prefix** - api
* **Integration** - get\_all\_customers
* **Description** - This endpoint will retrieve all customers from the data vault.
* **Response content type** - application/json
Click on the **Save** button below.
You should now see the web service endpoint on the **Swagger** page.
The web service endpoint to retrieve a specific customer requires a path parameter for the Id number. In the path field of a web service endpoint, parameters are enclosed with curly braces (e.g. `path/{parameter}`).
To create the web service endpoint to retrieve a specific customer by Id, add another integration config and fill in the fields with the following information
* **Path** - `get_customer/{id}`
* **Method** - Get
* **Prefix** - api
* **Integration** - get\_customer
* **Description** - This endpoint will retrieve a specific customer from the data vault using their Id.
* **Response content type** - application/json
Follow the same steps above and view the image below to create the remaining web service endpoints.
{% hint style="info" %}
The path name for the update customer and delete customer web service endpoints also have an Id parameter.
{% endhint %}
# 4. Execute Web Service Endpoints with Vault Methods
To execute any of the web service endpoints setup in the previous step (e.g. get all customers, get customer by Id, update customer, and delete customer), we must first write code in each of the connected integrations to open, read, and/or write to the **data** vault.
### Get All Customers
Log into Integration Gateway, if necessary and navigate to the **Build** page. Select the `get_all_customers` integration and add the following [code](https://glyue.docs.sandboxbanking.com/tutorials/building-a-restful-crud-web-service/5.-vault-code-examples-and-explanation#get-all-customers) in the **Before Hook**.
{% hint style="info" %}
Assigning a value to `output.payload` allows you to provide the desired content for the response body.
{% endhint %}
Navigate back to the **Swagger** page. Select the web service endpoint `/rest/get_all_customers`*.* Click on the **Try it out** button and the **Execute** button.
You should receive text in the response body confirming that there are no customers in the **data** vault.
On the **Run History** page, you would noticed the integration `get_all_customers` was executed because it is the integration connected to the endpoint `/rest/get_all_customers`. Also, the `input` displays the HTTP method and the full path or web service endpoint that was used to call this integration.
In the next step, we will execute this web service endpoint again after we add a new customer in the **data** vault.
### Add a Customer
In the `add_customer` integration, we will create a service request and several field mappings to insert a customer in the **data** vault. The customer information will come from the body of a request and will contain the customer's first name, last name, birth date, and social security number.
Select the `add_customer` integration to highlight it and right click it to display a context menu.
On the context menu, hover over **Go To** or **Go to (New Tab)** and select **Service Request.**
You will be redirected to the **Service Request** component on the **Build** page. On this page, click on the **Add Row** button at the bottom of the screen and fill in the fields with the following information for this service request.
* **Sequence** - 1
* **System** - ECHO
* **Service Name** - N/A
* **Formula Variable** - insert\_customer
Click on the **Save** button below.
Click on the service request to highlight it and right click it to display a context menu again. On the context menu, hover over **Go To** or **Go to (New Tab)** and select **Field Mapping**.
You will be redirected to the **Field Mapping** component on the **Build** page. On this page, click on the **Add Row** button at the bottom of the screen and fill in the fields with the following information.
* **Sequence** - 1
* **Field** - id
* **Value** - import uuid; retvalue = uuid.uuid4()
* **Value Type** - str
Click on the **Save** button below.
{% hint style="info" %}
The value of the id field contains code that generates a random UUID number.
{% endhint %}
Add the remaining field mapping rows for the first name, last name, birth date and social security as shown in the image below.
{% hint style="info" %}
We are mapping the first name, last name, birth date, and social security number from `input.payload`. The `input.payload` contains information submitted in the body of a request.
{% endhint %}
Navigate back to the **Swagger** page and select the web service endpoint `/rest/add_customer`*.* Click on the **Try it out** button and add a JSON object of customer information as shown in the image below. Click on the **Execute** button.
On the **Run History** page, you would see the integration `add_customer` have the customer information that we submitted in the body of the request within `input.payload`.
On the **Run History Item**, click on the `insert_customer` request and you would see in the response the field mappings we added in the previous step. We will use `response.payload` of the service request `insert_customer` to add a customer in the **data** vault.
To add a customer in the **data** vault, navigate to the service request `insert_customer` and add the following [code](https://glyue.docs.sandboxbanking.com/tutorials/building-a-restful-crud-web-service/5.-vault-code-examples-and-explanation#add-a-customer) in the **After Execute Request Success Hook**.
Navigate back to the **Swagger** page and execute the web service endpoint `/rest/add_customer` again. *Y*ou should receive the id of the customer in the response body.
Insert another customer as shown in the image below and execute the web service endpoint `/rest/add_customer`.
Afterwards, execute the web service endpoint `/rest/get_all_customers`. You should see both customers in the body of the response.
### Get Customer By Id
Navigate back to the **Integration** table on the **Build** page in Integration Gateway and select the integration `get_customer`. Add the following [code](https://glyue.docs.sandboxbanking.com/tutorials/building-a-restful-crud-web-service/5.-vault-code-examples-and-explanation#get-customer-by-id) in the **Before Hook**.
{% hint style="info" %}
`input.path` is the location where path parameter values can be accessed.
{% endhint %}
Navigate back to the **Swagger** page and select the web service endpoint `/rest/get_customer`. Click on the **Try it out** button and add a valid id path parameter as shown in the image below.
Click on the **Execute** button. *Y*ou should receive the customer's information in the form of a JSON object in the response body.
### Update Customer
The following steps are similar to what we did to add a customer in the **data** vault. The major difference is we omit the id field because the customer Id should never change.
Create the following service request for the `update_customer` integration.
* **Sequence** - 1
* **System** - ECHO
* **Service Name** - N/A
* **Formula Variable** - modify\_customer
Click on the **Save** button below.
Next, create the following field mapping rows for the service request `modify_customer`.
In the field mapping table, we are using the **Call If** column to omit or include a field and value in `response.payload` based on the existence of the field in `input.payload`.
Navigate back to the `modify_customer` service request and add the following [code](https://glyue.docs.sandboxbanking.com/tutorials/building-a-restful-crud-web-service/5.-vault-code-examples-and-explanation#update-customer) in the **After Execute Request Success Hook**.
Navigate back to the **Swagger** page. Select the web service endpoint `/rest/update_customer` and click on the **Try it out** button. Add a valid id path parameter and a new first name and last name in the body of the request as shown in the image below.
Click on the **Execute** button. You should receive the customer's information in the form of a JSON object with the first name and last name updated.
### Delete Customer
Navigate back to **Integration** table on the **Build** page in Integration Gateway and select the integration `delete_customer`. Add the following [code](https://glyue.docs.sandboxbanking.com/tutorials/building-a-restful-crud-web-service/5.-vault-code-examples-and-explanation#delete-customer) in the **Before Hook**.
Navigate back to the **Swagger** page and select the web service endpoint `/rest/update_customers`. Click on the **Try it out** button and provide a valid id path parameter as shown in the image below.
You should receive a confirmation message that the customer was deleted.
# 5. Vault Code Examples and Explanation
To open the **data** vault, we use the `open_vault` function.
```
with open_vault('data') as vault:
# Get, insert, update, upsert, or delete vault data
```
### Get All Customers
```
with open_vault('data') as vault:
customers = vault.get('customers')
if customers:
output.payload = customers
else:
output.payload = "There are no customers in the data vault."
```
To retrieve the `customers` list in the **data** vault, we use the vault method `get.` Afterwards, we check if the `customers` list is empty. If it is not empty, we assign `output.payload` the `customers` list. Otherwise, we assign `output.payload` a confirmation message that there are no customers in the list.
### Get Customer By Id
```
with open_vault('data') as vault:
customers = vault.get('customers')
customers = [customer for customer in customers if customer.id == input.path.id]
if customers:
output.payload = customers[0]
else:
output.payload = f'There are no customers with an id of: {input.path.id}'
```
To get a customer by Id in the **data** vault, we first use the vault method `get` to retrieve the `customers` list. Afterwards, we use a list comprehension to extract the customer from the `customers` list that has an Id that matches the `id` in `input.path`. At the end, we assign `output.payload` the customer's information in the form of a dictionary object or an error message if the customer was found.
### Add a Customer
```
with open_vault('data') as vault:
customers = vault.get('customers')
customers.append(insert_customer.response.payload)
vault.update('customers', customers)
output.payload = { "id": insert_customer.response.payload.id }
```
To add a customer in the **data** vault, we first use the vault method `get` to retrieve the `customers` list. Afterwards, we append `response.payload` of `insert_customer` to the `customers` list and use the vault method `update` to save our changes to the `customers` list. At the end, we assign `output.payload` the Id of the new customer.
### Update Customer
```
with open_vault('data') as vault:
customers = vault.get('customers')
customer = [customer for customer in customers if customer.id == input.path.id]
if customer:
customer[0].update(modify_customer.response.payload)
vault.update('customers', customers)
output.payload = customer[0]
else:
output.payload = f'There are no customers with an id of: {input.path.id}'
```
To update a customer in the **data** vault, we first use the vault method `get` to retrieve the `customers` list. Afterwards, we search for the customer via their Id. If the customer is found, we update the customer's information from `response.payload` of the service request `modify_customer`. At the end, we assign `output.payload` the customer's information in the form of dictionary object or an error message if the customer was not found.
### Delete Customer
```
with open_vault('data') as vault:
customers = vault.get('customers')
previous_length = len(customers)
customers = [customer for customer in customers if customer.id != input.path.id]
if previous_length != len(customers):
vault.update('customers', customers)
output.payload = f"The customer with an id of {input.path.id} has been deleted."
else:
output.payload = f'There are no customers with an id of: {input.path.id}'
```
To delete a customer in the **data** vault, we first use the vault method `get` to retrieve the `customers` list. Afterwards, we record the current length of the customers list and we use a list comprehension to remove the customer from the `customers` list that has an Id that matches the `id` in `input.path`. If the length of the `customers` list has changed, we use the vault method `update` to save our changes to the `customers` list. At the end, we assign `output.payload` a confirmation message that the customer was deleted or an error message if the customer was not found.
# Building a Retrieval API against FIS CodeConnect
This How-To Guide will show you how to build a Customer360 Retrieval API using the FIS CodeConnect Gateway interface.
## **Steps:**
{% content-ref url="/pages/92ergpv6Y3b56XO1Kvp3" %}
[1. Integration Setup](/tutorials/building-a-retrieval-api-against-fis-codeconnect/1.-integration-setup)
{% endcontent-ref %}
{% content-ref url="/pages/CwaOGGDCeYtNokdB22bb" %}
[2. Service Request Setup](/tutorials/building-a-retrieval-api-against-fis-codeconnect/2.-service-request-setup)
{% endcontent-ref %}
{% content-ref url="/pages/w6O0VokTx90kZZCEScI9" %}
[3. Field Mapping Setup](/tutorials/building-a-retrieval-api-against-fis-codeconnect/3.-field-mapping-setup)
{% endcontent-ref %}
{% content-ref url="/pages/JXUh1cEwrGGRVyDxgV0m" %}
[4. Integration and Service Request Hook Setup](/tutorials/building-a-retrieval-api-against-fis-codeconnect/4.-integration-and-service-request-hook-setup)
{% endcontent-ref %}
{% content-ref url="/pages/SNujp772d7ls48ySQmfq" %}
[5. Testing the Integration](/tutorials/building-a-retrieval-api-against-fis-codeconnect/5.-testing-the-integration)
{% endcontent-ref %}
{% content-ref url="/pages/AH3IdFxqtDIxKmejz9r8" %}
[6. Common Errors](/tutorials/building-a-retrieval-api-against-fis-codeconnect/6.-common-errors)
{% endcontent-ref %}
# 1. Integration Setup
**Creating the Integration**
\
The first step to building a Customer360 retrieval API would be to create a new row on the integration table of the build page.\
\
Navigate to the Build page and change the current component to the **Integration** option.
\
Once there, press the "Add Row" button to create the integration. You'll need to provide a **Path Name** as well as a **description** in order to save the added integration using the save button on the bottom of the screen.
**Configuring the Integration**\
\
\&#xNAN;*To complete this step, you'll need access to the* Integration Gateway *Admin Page.*\
\
Navigate to the Admin Page using the panel on the left hand side of the screen. Scroll down until you find the configuration item labelled "Integration Configs" and click the "Add New" button next to it.\
\
Choose the appropriate integration name that you created earlier and choose the settings you'd like configured for this integration. Check the "Store payloads in run history" box to trace the data when building and using the integration.\
\
From here, find the configuration item labelled "Adapter Configs: FIS IBS (Code Connect)**"** in the left-hand side menu and click to add a new item.\
\
\&#xNAN;*`If you are building an integration to a different CodeConnect core, choose the appropriate item; i.e. "Adapter Configs: FIS Horizon (Code Connect) " for the Horizon core.`*\
\
Fill in the corresponding credentials on the configuration page. You'll need to reference your CodeConnect portal or reach out to your FIS account representative for access to some of this information.
Once completed, scroll down the left pane until you find the configuration item labelled "Adapter Config Bindings" and add a new item. Choose the corresponding integration that you just created and the Adapter Config you just created and click save. This step essentially assigns the **Integration** to use the adapter configuration information we just provided when interacting with the core.
# 2. Service Request Setup
**Creating the Service Request**\
\
\&#xNAN;*For reference:* [*Service Request Documentation*](/reference/integration_components/servicerequest)\
\
Click on the "View Site" button at the top right of the page to return to the main Integration Gateway UI. From here, you'll need to navigate back to the Build screen.\
\
Right click on the Integration you created and Go To -> Service Requests. Add a new row to this table.\
\
Set the Sequence Number to 1, as this service request should represents the first API call to CodeConnect.\
\
Underneath the System column, enter "CODECONNECT.IBS" to determine which system we are calling out to. (Replace this with "CODECONNECT.HORIZON" or other applicable cores if necessary).
**Configuring the Service Request**\
\
To complete the next steps, you may need to reference the latest CodeConnect documentation that is available on the CodeConnect portal.\
\
For our example in this How-To Guide, we will be using the Get Customer (IBS Customer Information - v2) endpoint to retrieve information about a given customer on the IBS core.\
\
In the Build Page, enter the service name "IBSCI/v2" or whichever IBS service you want to use. This establishes which API group the service request will use within CodeConnect.\
\
Under formula variable, we recommend entering a formula name that corresponds to what the service is doing. In our example case, we use a formula variable of "getCustomer" as it accurately describes what our service is doing.\
\
Click the Save button at the bottom of the screen.
# 3. Field Mapping Setup
**Setting up the Field Mapping Table**\
\
Right click on the service you just created and Go To -> Field Mapping. This page should be blank.\
\
From here, you'll want to add 3 different rows. The number of field mapping rows that are required for a specific service vary, but for this example we will need 3 distinct rows.\
\
Number each Sequence column for each row from 1 to 3. Under the "Field" column for each row, enter the following into each one:\
\
\- `_application_id`\
\- `_path`\
\- `_method`\
\
The "Value" column of each of these rows also needs to be filled in. For each of the Field columns, use the following matrix to determine what should go inside the value column:
| Field | Value |
| ----------------- | ----------------------------- |
| `_application_id` | "CI" |
| `_path` | f"customers/{customerNumber}" |
| `_method` | "GET" |
\
Ensure that the "Value Type" column for each of these rows is set to `str` which stands for the data type "String".\
\
Save the changes that you've added.
# 4. Integration and Service Request Hook Setup
**Integration and Service Request Hooks**\
\
From here, there are only a few more things we need to do in order to query the core system for customer information and return it to the source system.\
\
From the previous step, we set the `_path` field mapping row's value to `f"customers/{customerNumber}"` . Essentially, we've established part of the URL path we will be reaching out to in this service request.\
\
The "{customerNumber}" part of that path is a variable we need to set.\
\
Navigate to the Integration Level of the Build page. Open the **Before Hook** of the integration that you've created. Paste the following lines of code into the cell and save your changes:
```python
customerNumber = "00000000001"
keep(customerNumber = customerNumber)
```
\
The first line sets the customerNumber variable to "00000000001" as an example customer number, but this value can be replaced with a reference to the input payload, such as `input.payload.custNum` as needed.\
\
The second line of code uses the `keep()` function to ensure that this variable is kept in memory and can be used in other parts of the integration, such as the field mapping table where we reference it.\
\
Next, navigate to the service request level of this integration. From here, scroll to the right until you find the column labelled **"After Overall Success Hook"**. Open this column and paste the following line of code into the cell:
```python
output.payload = getCustomer.response.payload
```
\
Essentially, this line of code will take the **response** of the service request to CodeConnect and assign it to the **output** of the Integration Gateway integration. Keep in mind that you will need to replace `getCustomer` with the formula variable you assigned to the service request if you named it differently.
# 5. Testing the Integration
Now that all of the setup is complete, you can now test the integration. During this phase, you may run into certain connection or configuration errors; if these do show up, you will need to troubleshoot the issues as necessary.
Navigate to the Swagger API page on the left. Here, you should see a list of **Integration Execution Endpoints**, one of which being the Integration you just created.
Click on the integration and to open the dropdown menu:
From here, click on the "Try it out" button in the top left of the menu. This allows us to change the **Input Payload** that we send through the integration. During the [Integration and Service Request Hook Setup](/tutorials/building-a-retrieval-api-against-fis-codeconnect/4.-integration-and-service-request-hook-setup), if you chose to set the `customerNumber` variable to something within the input payload like this:
```
customerNumber = input.payload.testNumber
keep(customerNumber = customerNumber)
```
Then copy and paste this JSON payload into the Swagger Page:
```json
{
"testNumber": "00000000001"
}
```
Click on the "Execute" button to run the integration. After executing, navigate to the Run History Page to see the results of your test. If successful, the output record in run history should look something like this:
# 6. Common Errors
Here are some common errors and resolutions you may run into when testing the CodeConnect integration.
### IntegrationConfig matching query does not exist:
{% code fullWidth="true" %}
```
Traceback (most recent call last):
File "/glyue/django_glyue_app/integration_engine.py", line 403, in _validate_integration
models.IntegrationConfig.objects.get(integration=integration)
File "/usr/local/lib/python3.8/dist-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/django/db/models/query.py", line 435, in get
raise self.model.DoesNotExist(
django_glyue_app.models.IntegrationConfig.DoesNotExist: IntegrationConfig matching query does not exist.
```
{% endcode %}
To solve this error, please create an IntegrationConfig record as explained in [Integration Setup.](/tutorials/building-a-retrieval-api-against-fis-codeconnect/1.-integration-setup)
***
### customerNumber is not defined:
{% code fullWidth="true" %}
```
Traceback (most recent call last):
File "/glyue/django_glyue_app/integration_engine.py", line 1287, in _apply_fieldmapping
value = self.eval_formula(value)
File "/glyue/django_glyue_app/integration_engine.py", line 1651, in eval_formula
return eval(formula, namespace)
File "", line 1, in
NameError: name 'customerNumber' is not defined
```
{% endcode %}
To solve this error, ensure that you've followed the steps in [Integration and Service Request Hook Setup](/tutorials/building-a-retrieval-api-against-fis-codeconnect/4.-integration-and-service-request-hook-setup); specifically, ensure that you are setting the variable `customerNumber` to a defined value and using `keep()` to make it useable within the Field Mapping Table.
***
### Failed to load adapter configuration for CODECONNECT.IBS:
{% code fullWidth="true" %}
```
Traceback (most recent call last):
File "/glyue/django_glyue_app/integration_engine.py", line 1396, in _execute_request
adapter.load_adapter_config()
File "/glyue/django_glyue_app/adapters/__init__.py", line 93, in load_adapter_config
raise ObjectDoesNotExist(f"No active adapter config bound to integration")
django.core.exceptions.ObjectDoesNotExist: No active adapter config bound to integration
```
{% endcode %}
To solve this error, ensure that you've created an Adapter Config Binding between the integration you are testing and the CodeConnect Adapter Config as explained in [Integration Setup](/tutorials/building-a-retrieval-api-against-fis-codeconnect/1.-integration-setup).
***
###
# What is Pre-Mapping?
Pre-Mapping is the first project phase after Project Kick Off in which the integration template is reviewed.
{% hint style="info" %}
The integration template is a prepopulated set of field mappings from one specific system to another. Likely many will fit your business process, but you will need to adjust others to fit your business process. The purpose of Pre-Mapping is to approve template mappings that fit your business process and flag the others that need to be changed.
{% endhint %}
Pre-Mapping differs from the Mapping phase in that it's strictly a review of the template at the [Field Mapping](/reference/integration_components/field-mapping) level. You will approve Field Mappings that align with your business process. If a Field Mapping does not align with your business process, or you're not sure, you can mark it accordingly and add a comment to be reviewed during Mapping.
Changes to mapping logic are not made during Pre-Mapping-- the fields that are not approved during Pre-Mapping are handled during the Mapping Phase.
# Before you start
Here's a checklist of prerequisites you'll need for Pre-Mapping:
* Read [What is Integration Gateway?](/)first
* Access to your Integration Gateway DEV environment
* Full permissions to the integration in your environment
* Access to your [Help Center (Jira Portal)](https://alphapack.atlassian.net/servicedesk/customer/portals)
{% hint style="success" %}
A new feature, [Buildhelper](/integration-gateway-platform-reference/buildhelper), is now available and can be used to import pre-created fields for various external Systems / APIs.
{% endhint %}
{% hint style="info" %}
If you're missing any of the above prerequisites or are unsure if you have all of them, submit a ticket to the [Help Center](https://alphapack.atlassian.net/servicedesk/customer/portals)
{% endhint %}
# Bookmarks
The **Bookmarks Page** is where you'll find the set of bookmarks for your organization. Much like the bookmarks in your browser, **Bookmarks** in Integration Gateway are simply a link to a specific location in Integration Gateway. **Bookmarks** can point to any URL, but they typically will point to a particular set of **Field Mappings** in Integration Gateway's Build page for **Pre-Mapping**.
A given integration framework will often encompass hundreds of **Field Mappings**. **Bookmarks** are a way to group them by category into manageable collections. Examples of such categories typically grouped into **Bookmarks** are addresses, interest rate fields, or collateral details. By having each **Bookmark** focus on a particular business function or category, the **Pre-Mapping** and **Mapping** phases can progress more quickly and efficiently.
Each Integration template will have a set of **Bookmarks** for the integration and new ones can be added as needed later.
In this part of the tutorial, we'll open the **Bookmarks** page and view its contents.
1. Log into Integration Gateway
2. From the first page, hover over the menu icons to expand it and find **Bookmarks**\
3. Click on **Bookmarks** to navigate to its page which will look something like this (NOTE: the specific sections for your integration may vary). This page lists the **Bookmark Sections**; a **Section** is a specific collection of **Bookmarks** grouped by a common category:
4. To the left of each **Section** with the folder icon is a triangle to expand the view. To the right are buttons for editing and adding new **Bookmarks** which will be explored another time. Click on any of the **Sections** to expand its contents.
5. Each **Bookmark** for the **Section** will be listed here with a title/link, description, and tabulation of total rows, comments, and rows whose comments are resolved. Click on one of the links to view its **Field Mapping(s)**.\
\
You'll know you've reached the **Field Mapping** portion of the **Build Page** if the **Component** dropdown in the upper left has **Field Mapping** selected.\
{% hint style="info" %}
Suppose you don't see **Field Mapping** in the **Component** menu. In that case, you can go back to Step 4 and select another **Bookmark** (The Sandbox Banking Project Manager or Solutions Engineer can assist if you have trouble identifying the correct Bookmark(s)).\
\
Any time you want to go to a particular **Field Mapping**, you can repeat the steps outlined above.
{% endhint %}
There are typically hundreds of **Field Mappings** across different objects or record types for a given integration. The purpose of **Bookmarks** in the **Pre-Mapping** phase is to break down all of these **Field Mappings** into manageable groups for a particular category or object.
That concludes this part of the Pre-Mapping Tutorial.
# Source and Targets
Now we are looking at a **Field Mapping** in the **Build** page. The **Build** page is designed to resemble and function similarly to a spreadsheet. Much like a spreadsheet, there are several headers at the top. The columns we will focus on are under the headers `Source Record Type`, `Source Field Name`, `Target Record Type` and `Target Field Name`
These columns are sort of like labels for the integration template that's been installed.
{% hint style="info" %}
The Source and Target columns are merely descriptors of the particular field mapping and do not set the mapping.
{% endhint %}
The `Source` columns relate to the system your data is coming from such as a Loan Origination System (LOS). The `Target` columns refer to the destination system your data is going to such as your banking Core or a data warehouse.
* `Source Record Type`: This is the specific record type for this **Field Mapping** such as a Loan booking, a Credit Card application, or an Account.
* NOTE: Sometimes the API name for the field is used which may be slightly different than how the field appears in the source system.
* `Source Field Name`: This is the name of the specific field such as "Account Number", "First Name" or "Street Address"
* `Target Record Type`: This is your destination system's record type which will be something like Account, Collateral, or CIF Record
* `Target Field Name`: This is the field name in your destination system which will be something like "Account Number", "First Name" or "TID"
Your primary goal in **Pre-Mapping** will simply be to evaluate these **Source** and **Target** columns and determine whether or not the referenced pairs match your business process, if they need to be changed, if they need to be added, or if you have questions about a particular mapping.
***
Through the **Pre-Mapping** and **Mapping** phases, we are going to be spending a lot of time on this page. Let's start to get acquainted with this page with a few simple exercises:
1. The columns in the build page are somewhat narrow by default and may be too narrow to see the full entries. Hover your mouse cursor over the boundary between columns, click and drag to the right to expand the width of the column to your liking
2. You can order each column's values by numeric or alphabetical order or reverse numeric or alphabetical order. Try clicking on the header of your choosing and order the values back and forth.
3. At the moment, we only care about the 4 columns mentioned plus the `Comments` and `Status` columns (we'll cover these in the next sections). There are a lot of other columns and we won't need to pay attention to most of them until later in the project. We can decide which columns to show and hide by using the **Show Columns** drop-down:
NOTE: The **Field Mapping** page(s) will be filtered by default when accessing them through the **Bookmarks**.
4. Let's click on the **Show Columns** drop-down and select `Comments`, `Source Record Type`, `Source Field Name`, `Target Record Type` and `Target Field Name`, `Status` so that it looks like this:
5. Now that we're looking at the columns we need to see, go ahead and expand each of the rows as we had done in Step 1:
6. Again, the goal of **Pre-Mapping** is to identify and evaluate the mappings for fields coming from your **Source** system going into your **Target** or destination system.
# Field Mapping Status
Since the goal of **Pre-Mapping** is to review each of the **Field Mappings**, the **Status** column is where we are going to mark each **Field Mapping** to signify that it's been reviewed. The objective is simply to take a first pass at the **Field Mappings** which doesn't mean they have to be fully approved or worked out. If you don't understand a particular **Field Mapping**, there's a **Status** option to mark it as such and you can revisit it in the **Mapping** phase with your Solution Engineer.
**Here's a list of the different Status Options:**
* **Not Started** - This is the default **Status** which means the **Field Mapping** has NOT been reviewed
* **Not Used** - This means the **Field** for the **Target System** isn't used at all and this **Field Mapping** isn't necessary
* **Change Requested** - This means your team is requesting that this **Field Mapping** be mapped differently, either to a different **Target Field** or from a different **Source Field** (or even if additional logic is needed)
* **In Progress** - This means your organization uses this **Field Mapping** but needs to decide how it should be mapped to fit your business purposes
* **Waiting for Support** - This means you have questions or you're unsure of how to move forward with this **Field Mapping** and you need assistance from our team
* **Waiting for Customer** - Our team will use this when we answer one of your questions or have further questions for your team
* **Needs Testing** - This means you believe this **Field Mapping** to be correct but that it needs testing before being ready for production.
* **Approved** - This means that mapping is complete and the **Field Mapping** has been fully tested and is ready for production. NOTE: This status will not be used until the User Acceptance Testing (UAT) phase.
{% hint style="info" %}
**Pre-Mapping** is complete when EVERY **Field Mapping Status** has been transitioned from **Not Started** to **Needs Testing, Not Used, Waiting for Support, or Change Requested**
{% endhint %}
**Some Clarifications:**
* Changing a **Field Mapping's Status** doesn't mean you have to take immediate action on it, you're just marking it for now so that all the **Field Mappings** that need to be actioned will be collated for a more efficient **Mapping** phase
* The difference between **Change Requested** and **Waiting for Support** is that **Change Requested** means you know how the field should be mapped and are waiting on our team to implement and **Waiting for Support** means you have questions about the **Field Mapping** for our team that you need answered to understand and decide how it should be mapped
* The difference between **In Progress** and **Waiting for Support** is that **In Progress** means it's your organization's action to decide how to move forward and **Waiting for Support** puts the **Field Mapping** in our team's court to answer any questions
For the following exercises, you can pick up where you left off from [**Bookmarks**](/tutorials/what-is-pre-mapping/bookmarks) or follow the steps in [**Source and Targets**](/tutorials/what-is-pre-mapping/source-and-targets) to open up the **Field Mappings** for any **Bookmark**.
1. Find a very basic **Field Mapping** with "First Name" in the **Source Field Name** and "First Name" in the **Target Field Name**. If you need help from the Solutions Engineer assigned to your project, they'll be happy to help. If your integration isn't for personal entities, feel free to select the appropriate name field for an organization entity.
In most cases, fields like First Name, Last Name, or Organization Name should be straightforward. Verify that this field from the **Source System** should arrive in the **Target System** corresponding to the **Field Mapping**.
1. If you agree with the Field Mapping, change the Status to **Needs Testing**
2. Now, look down the list of **Target Field Names** and try to find a field in your **Target System** that you don't use at all.
3. Update that Status to **Not Used**
We are only going to focus on **Needs Testing** or **Not Used** for this portion of the tutorial. For the other Statuses, we're going to use Field Mapping level Comments to communicate questions or changes as part of the **Change Requested**, **In Progress, Waiting for Customer,** and **Waiting for Support** Statuses.
# Field Mapping Comments
As we explored in [**Field Mapping Status**](/tutorials/what-is-pre-mapping/field-mapping-status), you can mark each **Field Mapping** with a simple status such as **Needs Testing** or **Not Used**. But sometimes the other **Status** options won't capture needed details or you'll need a dialogue specific to a **Field Mapping**. This is where **Comments** come in.
For **Change Requested**, the changes you want to make can be detailed in a Comment.
For **In Progress**, you can add notes indicating what your organization needs to do to progress with this particular **Field Mapping.**
For **Waiting for Support**, you can detail what support you're waiting for -- often this is simply a question about a field to be mapped or about the mapping logic in the template for that **Field Mapping**
**Comments** are the furthest left column in any Build Page
**Comments** have several properties, they have a subject, a body, a URL field, and resolved status. Also to the left is a button to create new comments as each **Build Page** row (a Field Mapping in our case) can have several Comments associated with it
The **Comments** icon will indicate how many **Comments** there are for a **Field Mapping** and how many are resolved. The icon will be dark gray if there are no **Comments**, red if there are any unresolved **Comments**, and green if there are **Comments** and all of them are resolved.
The top icon indicates no comments, the red icon indicates there are two Comments, one of which is resolved, and the last icon indicates only one Comment which is resolved.
For the final exercises, go ahead and open a **Bookmark** of your choice. You can pick up where you left off from the exercises in [Field Mapping Status](/tutorials/what-is-pre-mapping/field-mapping-status)
1. Click on the **Comment** icon to bring up the **Comment** window as pictured above
2. Write "Hello world" in the subject (or anything you like)
3. Write "This is my first comment" or anything you like in the Content area
4. Click "Submit"
Congratulations, you've written your first comment!
***
You can also tag other Integration Gateway users in your comments. Let's create another comment and tag another user:
1. Start a new comment per Steps 1 through 3 above.
2. Type the **@** symbol, a dropdown will appear of other users you can add
3. Select another user from the dropdown, this will autocomplete the user's name when selected:
4. Click "Submit" to save the Comment
Tagging another user will let them know you've mentioned them. If anyone has mentioned you in a **Comment**, you'll see a number in a red circle above the notification icon:
You can view and clear notifications by clicking on the icon:
Users may also be emailed notifications, this depends on the notification preferences which can be found in **Settings**:
On your profile, you can click **Communication Preferences** where you can Disable or Enable email notifications:
Finally, let's resolve our new comments
1. Open any of the **Comments** you just created
2. Switch the **Resolved** toggle:
3. When you exit the Comment, you should see the icon update:
Comments with questions or requests will be addressed in the **Mapping** phase after **Pre-Mapping** is complete. When they are fulfilled, this is how the **Comment** will be marked as being completed.
# Summary
We use [**Bookmarks**](/tutorials/what-is-pre-mapping/bookmarks) to group **Field Mappings** by category. We compare the [**Source and Target**](/tutorials/what-is-pre-mapping/source-and-targets) from the integration framework and change the [**Status**](/tutorials/what-is-pre-mapping/field-mapping-status) to indicate that it has been reviewed. [**Comments**](/tutorials/what-is-pre-mapping/field-mapping-comments) are used to add further details during the **Pre-Mapping** Process.
**Pre-Mapping** is considered complete when EVERY **Field Mapping Status** has been transitioned from **Not Started** to **Needs Testing, Not Used, Waiting for Support, or Change Requested**
### FAQ
Q: What if the template doesn't have a field mapping for a field I use?\
A: Use your [Help Center](https://alphapack.atlassian.net/servicedesk/customer/portals) to let your Solution Engineer know and they will add it!
Q: Does every Field Mapping need a comment?\
A: No, they don't. But Comments help add context for the **In Progress** and **Waiting for Support** Statuses that will make the Mapping Phase run smoother
Q: How much information should I provide during the Pre-Mapping Stage?\
A: As much as possible! The more information you provide during the Pre-Mapping stage, the quicker your Solution Engineer can go during the Mapping stage. Put any additional information in a Comment on the Field Mapping
# How to Run an Integration from Integration Gateway
Using the Integration Runner
Running an integration in Integration Gateway involves composing a payload that matches the integration's requirements, then sending that payload to the integration. This can be used as the primary way to run an integration, or to mimic an expected call from a 3rd party service for testing purposes.
## Creating a request
Integration Gateway makes it easy to compose the exact request that your integration expects.
1. To start, right-click on the integration on the *Build* page and click *Run Integration* to bring up the *Run Integration* modal*.*
2. If the integration has webservices, use the `Method`, `Prefix`, and `Webservices` dropdowns along the top of the page to select the webservice you wish to trigger.
3. Configure the content of the request across the *Payload, Headers, Query Parameters,* and *Path Parameters* (see below for further details)
{% hint style="info" %}
To learn more about [webservices, see this reference](/integration-gateway-platform-reference/integration_configuration#web-service-endpoints).
{% endhint %}
## Request customization
### **Payload**
*Payload* refers to information sent in the body of the request. This information can come in various forms, outlined below:
* JSON: A JSON editor will appear to input JSON-structured content
* XML: An XML editor will appear to input XML-structured content
* Multipart: Accepts a combination of uploaded files (up to 10MB) and form data
* Binary: Accepts a single file (up to 10MB)
* Empty: The request body will contain no data
The payload interface, with Multipart selected.
### **Headers**
Headers will be automatically populated based on the content of the payload and other details of the integration. When uploading multipart or binary payloads, the *Integration Runner* will suggest appropriate headers based on the file type(s) that are uploaded.
Some headers are omitted from the display and set automatically when the integration is sent. Certain headers will be ignored even if specified, for security reasons. *See the full list ignored/hidden headers in the*[ *Appendix*](#appendix)*.*
The Headers interface.
### **Query Parameters**
To set query parameters (information passed in via the url, e.g. `sandbox.com/endpoint?flavor=chocolate` ), enter their key and value. New pairs can be added using the green plus button at the top-left.
Notice the url-encoded form of the query parameters will update in the displayed URL along the top of the *Integration Runner*.
The query parameters interface.
### **Path Parameters**
Integrations with webservices may have variables in the webservice path. These are represented by `{variable}` in the path name. As you enter values for each path parameter(s), you should see the value(s) update in the displayed URL along the top of the *Integration Runner*.
{% hint style="info" %}
All path parameters must have specific values before the request can be executed.
{% endhint %}
The path parameters interface.
Note that path parameters cannot be added/edited/deleted from this page — modifications to the path occur on the webservice itself.
## Re-running an integration
Oftentimes it is useful to re-use the same request, either for simplicity or debugging purposes. This is done by hovering over an entry in the *Run History* and clicking the re-run icon.
The *Run Integration* modal will be pre-filled with the same request details as the selected run. If the selected run involved sending Binary or Multipart files, the same files will be re-used by default. Feel free to change the pre-filled values as desired.
The re-run icon within the Run History list.
## Appendix
**Ignored headers (will not be sent even if set in Headers tab)**
* `accept-encoding`
* `connection`
* `content-length`
* `host`
* `user-agent`
**Hidden headers (automatically set by Gylue on integration execution)**
* `authorization`
* `content-type`
* `origin`
* `referer`
* `x-csrftoken`
# How to Invite New Users
1. Navigate to the *Invite* page. This page is only visible for Staff users.
The Invite page icon in the navigation menu
2. Enter the new user's email and give them a username. The invitation will be sent to this email.\
\
Selecting "Staff" in the *Additional Permissions* dropdown will give the user [Staff permissions](/integration-gateway-platform-reference/permissions#user-type).
{% hint style="info" %}
If you're unsure what username to give, we recommend reusing the portion of their email before the @ sign.
{% endhint %}
3. Select the integrations that the user should have access to. You will need to grant each [type of permission](/integration-gateway-platform-reference/integration_configuration#integration-permissions) separately.
The Integration Permissions page of the User Invitation wizard.
4. Click *Save* to send the invitation.
***
Invitations are valid for 14 days from when they are created. If an invitation expires, the recipient can still follow the link to request a new invitation, which will be sent to the original email.
# How to Create a Value Mapping Set
### When to use a Value Mapping Set
Value Mapping Sets are an efficient way to set up [deterministic transformations](#user-content-fn-1)[^1] on a field in a payload. For example, two systems may have a field `state`, but the source system sends the value as the state abbreviation, while the destination expects it as a full spelling. Other common use cases include:
* Translating between internal codes and external-facing names
* e.g. a product code `400` may refer to the "High Yield Savings" product
* Reconciling different names, spelling, or formatting for the same concept across systems
* e.g. `High Yield Savings` vs. `High-Yield Savings Account`
* Performing a many-to-few transformation
* e.g. All variations on a checking account (`High School Checking`, `College Checking`, `Standard Checking`) are to be represented simply as `Checking Account`
If you find yourself writing `if-else` blocks that have more than 5 cases, a Value Mapping Set should be used instead.
### Creating a Value Mapping Set
A Value Mapping Set has two components:
* The Value Mapping Set, which a field mapping will refer to
* The actual Value Mappings that belong to the set, which detail an individual transformation case
1. On the build page, find your integration, right-click, then select *Go To > Value Mapping Set.*
2. Click + *Add Row* to create the new Value Mapping Set
3. Give your Value Mapping Set a clear name and description
The default settings on the value mapping set are sufficient for most use cases, but there are some instances where you may want to adjust them.
* If the incoming values are case sensitive (e.g. if `usa` vs `USA` are two distinct values that may both appear in the source system), set the `Case Sensitive` setting to `true`
* Keeping Case Sensitivity off results in a more robust and forgiving integration, especially if the incoming values are manually entered in the source system
* If the incoming value has many options, but the majority result in the same output value, it can be easier to set a default value and only create Value Mappings for exceptions where the non-default value should be used. To do this, set `Use Default To Value` to `true`, then enter the desired default value in the `Default To Value` column.
```
Instead of this:
From To
value1 output1
value2 output1
value3 output1
value4 output2
value5 output1
------------------------
Do this:
Default To Value --> output1
From To
value4 output2
```
* If you don't know all possible values that may occur in the source system, it can be useful to set a default value to handle these cases, assuming such a value exists
* If a default value is not set, any unrecognized values will be left unmodified.
4. Find your desired field mapping, and select the Value Mapping Set you created in the `Value Mapping Set` column.
{% hint style="info" %}
Once a Value Mapping Set is created, it can be used on multiple field mappings if necessary.
{% endhint %}
### Creating Value Mappings
Having created the Value Mapping Set, we can now create the component Value Mappings that comprise it. Right click on the Value Mapping Set row, the select *Go To > Value Mapping*. Click *+ Add Row* to create a new Value Mapping. Because we opened the page via the context-menu of a Value Mapping Set, the *Value Mapping Set* column is prefilled.
Enter the incoming value in the `From` column, then the corresponding output value in the `To` column.
{% hint style="info" %}
If you have many Value Mappings already formatted in an external spreadsheet, you can simply create empty rows for them, then copy/paste them into the corresponding cells in Integration Gateway.
{% endhint %}
### Using Value Mapping Sets Outside of Field Mappings
Value Mapping Sets are powerful because they are the source-of-truth for the translation of a particular field. There are times when that information may be useful contexts outside of a direct field mapping, such as if logic in your integration expects the transformed value of a field.
Value Mappings Sets can still be used in these situations via the `map_value()` function, which takes in the name of the Value Mapping Set and the input value, and returns the corresponding output value.
More about `map_value()` can be found in [its documentation](/reference/special_functions#map_value).
[^1]: A "deterministic transformation" is one that always produces the same result if given the same input
# How to Build and Deploy a Custom Frontend
## What is a Frontend?
Frontends are lightweight applications hosted on Integration Gateway. They can be used to support a wide range of use cases, including creating dashboards, giving easy access to run integrations, or simply as a fast way to host an internal application.
## Accessing a Frontend
Frontends can be given custom subaddresses on your Integration Gateway domain (e.g. `glyueprod.examplebank.sandboxbanking.com/frontends/example_dashboard`) making them easy to bookmark, link, and distribute.
All Frontends have built-in Integration Gateway authentication; users must first log in with their Integration Gateway account before they can load a Frontend.
## Creating a Frontend
Frontends can be created by uploading a `.zip` bundle that contains the `.html`, `.js`, and `.css` files for your web application, along with any other assets that may be required.
Frontends require a file named `index.html` to be present, which will be used as the entrypoint into the application. Relative references to pages within the frontend (e.g. `/resources/logo.png`) will work as expected.
Create a Frontend by giving it a name, path, and uploading the .zip
For details on more advanced Frontend features and sample code snippest, see the [reference documentation](/integration-gateway-platform-reference/frontends).
## Editing a Frontend
Once created, a Frontend can be modified by replacing the contents of the uploaded `.zip` with an updated version. Upon save, the new version will be deployed.
The source code of the currently-served Frontend can be downloaded from within the *Edit* page of the Frontend.
# How to Migrate an Integration
When developing integrations, Sandbox Banking Solution Engineers utilize separate `DEV` and `PROD` environments to write code and test changes before deploying that code to a production environment.
When changes to an integration are ready to be deployed to the `PROD` environment, Integration Gateway provides two methods for performing a migration:
* [Migrating a whole integration](#migrating-a-whole-integration)
* [Migrating part of an integration](#migrating-part-of-an-integration)
### Migrating a Whole Integration
The *Migrate* page (pictured below) allows a user to select integrations from a source environment (often, the `DEV` environment) to be migrated into the destination system (the current logged-in environment, often `PROD`).
To migrate an integration in its entirety:
1. Navigate to the *Migrate* page using the navigation sidebar.
2. Select the *Source* environment in the left-hand pane. Often, this is the corresponding `DEV` environment.
3. Select the integration(s) to be migrated in the right-hand pane.
4. Verify that the *Standard Migration* option is selected, then click *Perform Migration.*
5. Verify that the correct integration(s) were migrated by reviewing the migration summary (pictured below).
A successful migration summary
In the example above, we see that 1 integration, which contained 3 Service Requests, was migrated.
{% hint style="info" %}
Migrating a whole integration will replace the copy of the integration that's currently in the environment.
{% endhint %}
### Migrating Part of an Integration
Integration Gateway also supports migrating selected portions of an integration, while leaving the rest of the integration as-is. The selected changes are called a c*hangeset,* and this type of migration is called a *Changeset Migration.*
Changeset migrations are useful in situations such as:
* There is parallel development on different parts of an integration
* The `DEV` version of an integration contains development-specific logic that should not be migrated to a production environment
To perform a changeset migration:
1. Navigate to the *Migrate* page using the navigation sidebar.
2. Select the *Source* environment in the left-hand pane. Often, this is the corresponding `DEV` environment.
3. Click *View Incoming Changes.* This will open the *View Changes* page, which resembles the *Build* page, but includes two additional columns.\
\
\&#xNAN;*Migrate* indicates whether a change should be included in the migration.\
\&#xNAN;*Change* describes the type of change (add / edit / delete).\
4. Using the *View Changes* page, select the changes you wish to include in the migration changeset by setting the *Migrate* column to `true` for the selected rows.
5. When you're finished selecting specific changes, click *Save Change Set* on the toolbar along the bottom right. This will show a summary of the selected changes, along with a text field to optionally name your change set. Enter a descriptive name for the changes.
The interface reviewing the specific rows selected for a migration.
6. Click *Migrate Change Set* on the toolbar along the bottom right. This will take you back to the *Migrate* page.
7. On the *Migrate* page, reselect the source system and integration. Then select the *Change Set Migration* mode*,* select the changeset you created in step 5, then click *Migrate Change Set.*
Selecting a change set to migrate
8. Verify that the correct changes were migrated by reviewing the migration summary.
# How to Set Up Single Sign On (SSO)
[Single Sign-On](https://en.wikipedia.org/wiki/Single_sign-on) (SSO) is a demanded feature for many enterprise web applications, as it allows for centralized user access control by the business or organization. Users also do not need to remember passwords, as they only need their SSO login details to access any of their connected applications.
#### Brief Overview of SAML and Integration Gateway SSO
In a SAML SSO configuration, Integration Gateway acts as the **Service Provider (SP)** and a trusted identity/access platform (such as JumpCloud, Azure AD, Okta, etc.) acts as the **Identity Provider (IdP)**. Integration Gateway SSO is designed to be IdP-agnostic and work with any provider.
{% hint style="info" %}
While expected to work with any IdP, as of the writing of this article, Integration Gateway SSO is verified to be compatible with:
* JumpCloud
* Okta
* Azure AD
More providers will be added as their compatibility is verified.
{% endhint %}
The usual user authentication flow goes like this:
1. User clicks SSO option on the SP login screen. SP redirects the user to the IdP along with a SAML request.
2. User authenticates at the IdP. IdP redirects user back to the SP along with a signed SAML response, which contains information on the user it just authenticated.
3. SP verifies the signature, reads the response, identifies the correct user in its system, and logs them in.
Before any of that could happen, though, the service and ID providers had to establish a trusted relationship. The usual flow for that goes like this:
1. An Integration Gateway administrator configures and activates SAML SSO and then provides a metadata file to the other organization.
2. An IdP administrator at the organization adds Integration Gateway as an SP, uploads the metadata, and configures as necessary. They provide another metadata file back to the Glyue admin.
3. The Integration Gateway admin creates the IdP record and uploads its metadata.
Once done, the IdP should appear on the Integration Gateway login screen as an option for users.
## Frequently Asked Questions / FAQ
#### Does Integration Gateway support OIDC (OpenID Connect)?
> No. For a deeper explanation as to the differences between SAML and OIDC, see [this excellent article](https://jumpcloud.com/blog/saml-vs-openid) from [JumpCloud](https://jumpcloud.com/), a respected cloud-based authentication provider and IT asset management platform.
#### Does a user need an account in Integration Gateway before being able to log in via SSO?
> No, by default users signing in via SSO will have an account created at that time automatically. This feature can be toggled off if asked, which will enforce login only by existing users in Integration Gateway.
>
> On the Identity Provider record on the Integration Gateway Admin site, it can be set to automatically add users to an organization and/or mark as staff. If the Default Staff Group or Default Non-Staff Group are set on the Global Config, new users will be added to those groups as appropriate depending on staff status.
>
> This way, user onboarding can be completely automated, so new users are added to your Organization (if using Organizations) as well as added to a Group with permissions to certain integrations.
#### Can I restrict users to logging in with SSO only?
> Yes. New users can be restricted to SSO by unchecking "Allow Password Login" on the invite page. If the user already exists, go to Admin > Accounts, select the account, and uncheck "Allow password auth" and save.
# Integration Gateway Setup
This article aims to guide the user through how to set up SAML SSO in Integration Gateway.
{% hint style="warning" %}
Before continuing, this user should be a Integration Gateway administrator for the target environment.
{% endhint %}
### Create a SAML Config
Log into Integration Gateway and navigate to the Admin site.
Scroll to the **SAML SINGLE SIGN ON** section, locate **SAML Configs** and click **Add**.
On the web form that displays, scroll to the bottom (without changing any values) and click **SAVE**.
Default SAML settings are considered secure and should suffice for most use cases. If SSO customization is desired, please see this article:
{% content-ref url="/pages/wQ8FUfmZQXZH6zrH0luT" %}
[Integration Gateway SAML Config Reference](/how-to-guides/how-to-set-up-single-sign-on-sso/integration-gateway-saml-config-reference)
{% endcontent-ref %}
Note that on the resulting page, there is now a URL and a download link for Integration Gateway's SAML metadata:
Provide the URL or metadata file to the IdP admin, who now must add Integration Gateway on their end as an external SSO application (aka Service Provider) and supply their IdP SAML metadata.
### General IdP Setup
1. Create a new Service Provider (aka SSO Application, External Application, SSO Integration, SAML Application, etc. or something similar).
2. Upload the Integration Gateway metadata. Some IdPs won't support uploading a file, and will prefer a URL or even for the data to be copied and pasted.
{% hint style="info" %}
Integration Gateway always serves its metadata at `/sso/saml2/metadata/`.
Also, Integration Gateway's metadata can be downloaded from the **SAML Configs** page on the **Admin** site. The page will also display the full URL.
{% endhint %}
3. Configure as necessary, add users, map user attributes, etc.
{% hint style="info" %}
By default, a user's email address is used to identify the correct user for login in Integration Gateway. The attribute (aka field) on the Integration Gateway User object is simply `email`, which may differ from the IdP's User object (which might be`emailAddress`, `email_addr`, or something similar).
In this case, the IdP user email address attribute name needs to be mapped to Integration Gateway's. Integration Gateway supports user attribute mapping on its end, and most IdPs do as well.
See [Integration Gateway SAML Config Reference](/how-to-guides/how-to-set-up-single-sign-on-sso/integration-gateway-saml-config-reference) for more information.
{% endhint %}
5. Save/activate/enable the new SSO application.
6. There should now be SAML metadata available which should be provided back to the Integration Gateway admin.
### IdP-specific setup instructions
{% content-ref url="/pages/UfXan0BoZ10MJUPwUFpD" %}
[JumpCloud Setup](/how-to-guides/how-to-set-up-single-sign-on-sso/jumpcloud-setup)
{% endcontent-ref %}
{% content-ref url="/pages/3hkxcHQiOY1DJ5w1iqCL" %}
[Okta Setup](/how-to-guides/how-to-set-up-single-sign-on-sso/okta-setup)
{% endcontent-ref %}
{% content-ref url="/pages/wM6qlluC4TRow0lq8CpI" %}
[Azure Setup](/how-to-guides/how-to-set-up-single-sign-on-sso/azure-setup)
{% endcontent-ref %}
### Add the IdP to Integration Gateway as a trusted authenticator
{% hint style="warning" %}
Before continuing, Integration Gateway should be configured and active in the IdP, and the IdP's SAML metadata should be available.
{% endhint %}
On the Admin site, under **SAML SINGLE SIGN ON**, locate **SAML Identity Providers** and click **+Add**.
Provide a name, optionally a logo image, and the metadata.
Click **SAVE**. Integration Gateway should now allow users to log in via this provider.
### Testing connectivity
Log out of Integration Gateway and go to the main page. The login dialog should have a new section, **Single Sign On**, and the added IdP’s name should appear on a button.
Clicking the button should redirect the user to the IdP login page. After successful authentication, the user should be redirected back to Integration Gateway, bypassing the login screen and going right to the requested page.
{% hint style="danger" %}
If the user encounters a **403 Access Denied** error after authenticating, this means Integration Gateway was unable to locate a user based on the information provided by the IdP. This is most likely due to one of the following issues:
1. The user in Integration Gateway is not active or doesn't exist.
2. The user’s email address in Integration Gateway does not match exactly to the email address at the IdP.
3. The IdP has not been properly configured to include the user email address in the information it submits to Integration Gateway for SSO.
4. The name of the email address user attribute is different between Integration Gateway and the IdP. This can be solved by adding an attribute mapping on the SAML Config page. Some IdPs also have this optionality on their end.\
\
See [Integration Gateway SAML Config Reference](/how-to-guides/how-to-set-up-single-sign-on-sso/integration-gateway-saml-config-reference)
{% endhint %}
# JumpCloud Setup
{% hint style="warning" %}
Before continuing, this user should be a JumpCloud admin for the organization, and a SAML Config must be created in the Integration Gateway Environment.
{% endhint %}
### Create the Integration Gateway SSO Application in JumpCloud
From the JumpCloud dashboard, select **SSO** from the left-hand navbar and click the **Add New Application** button.
At the bottom of the page, click the **Custom SAML App** button.
Provide a **Display Label**. We recommend something like Integration Gateway **DEV,** Integration Gateway **PROD**, etc. Provide a **Description** if desired.
### Provide Integration Gateway's SAML Metadata to JumpCloud
Click the **SSO** tab.
{% hint style="info" %}
The Integration Gateway environment SAML metadata will be required for the next step.
Integration Gateway always serves its metadata at `https://`\[domain]`/sso/saml2/metadata/`. If this user is also a Integration Gateway administrator, the metadata URL will be displayed on the **Admin** site under **SAML Configs**.
{% endhint %}
Click the **Upload Metadata** button and upload the metadata provided by Integration Gateway.
Provide a custom **IdP Entity ID**. Any unique value will do.
### Add Users
On the **User Groups** tab users/groups can be assigned to the application. Only assigned users/groups will be able to authenticate with Integration Gateway via JumpCloud.
Add the desired users/groups for the organization. This can also be done later.
### Activate the SSO App
Click **Activate** at the bottom of the screen.
Back in **Configured Applications**, select the new SSO App and click **Export Metadata.**
An XML file should be downloaded. This file will be needed on the Integration Gateway side in order to add JumpCloud as an IdP, which is the last step. If not a Integration Gateway admin, please provide this file to a Sandbox Banking employee.
{% hint style="warning" %}
#### Lastly…
Don’t forget to assign users or groups to the new application- otherwise they won’t be able to authenticate with Integration Gateway via SSO.
{% endhint %}
# Azure Setup
{% hint style="warning" %}
Before continuing, this user should be an Azure AD admin for the organization, and a SAML Config must be created in the Integration Gateway Environment.
{% endhint %}
### Create the Integration Gateway Enterprise Application
From the Azure Active Directory overview page, select **Enterprise applications** from the left-hand panel.
Click **New Application** at the top.
Click **Create your own application** at the top.
Provide a name and select **Integrate any other application you don’t find in the gallery (Non-gallery)**. Click **Create** at the bottom of the page.
Click **2. Set up single sign on** under **Getting Started**, or select **Single sign-on** from the left-hand panel.
Select **SAML**.
### Provide Integration Gateway's metadata to Azure
{% hint style="info" %}
The Integration Gateway environment SAML metadata will be required for the next step.
Integration Gateway always serves its metadata at `https://` (custom domain) `/sso/saml2/metadata/`. If this user is also a Integration Gateway administrator, the metadata URL will be displayed on the **Admin** site under **SAML Configs**.
{% endhint %}
At the top, select **Upload metadata file.** Select the appropriate file and upload.
Click **Save** at the top.
### Add an email claim
We need to add a claim to the SAML configuration in Azure, so that the user’s email address comes over to Integration Gateway in a field titled “email” instead of something else.
On the application **SAML-based Sign-on** page, in section **2 Attributes & Claims** click **Edit**.
Click **Add new claim** at the top.
Enter `email` in the **Name** field. For **Source Attribute** select `user.mail`. Click **Save** in the top-left.
Depending on the organization’s Azure configuration, the user email field name may differ.
### Get Azure SAML metadata for Integration Gateway
On the application **SAML-based Sign-on** page, in section **3** under **SAML Certificates** download the **Federation Metadata XML**.
Alternatively, grab the **App Federation Metadata Url** instead of downloading the file.
This will be needed on the Integration Gateway side, to add Azure as an IdP. If this user is not a Integration Gateway admin, please provide the file/URL to a Sandbox Banking employee.
{% hint style="warning" %}
#### Lastly…
Don’t forget to assign users or groups to the new application- otherwise they won’t be able to authenticate with Integration Gateway via SSO.
{% endhint %}
# Okta Setup
{% hint style="warning" %}
Before continuing, this user should be an Okta admin for the organization, and a SAML Config must be created in the Integration Gateway Environment.
{% endhint %}
### Create the Integration Gateway application in Okta
From the Okta dashboard, find **Applications** on the left-hand panel and select **Applications** under it.
Click the **Create App Integration** button.
Select **SAML 2.0** and click **Next**
Give the app a name. We suggest something like Integration Gateway **DEV**, Integration Gateway **PROD** etc. Click **Next**.
### Provide Integration Gateway's SAML info to Okta
{% hint style="info" %}
The Integration Gateway environment SAML metadata will be required for the next step.
Integration Gateway always serves its metadata at `https://` (custom domain) `/sso/saml2/metadata/`. If this user is also a Integration Gateway administrator, the metadata URL will be displayed on the **Admin** site under **SAML Configs**.
{% endhint %}
#### **Single sign-on URL**
In the XML document, locate an element named `AssertionConsumerService` and grab the URL from its `Location` attribute (do not include the `"`).
In most cases this will be `https://`\[domain]`/sso/saml2/acs/`
#### **Audience URI (SP Entity ID)**
In the XML document, locate the first element `EntityDescriptor` and grab the value for its `entityID` attribute. In most cases this will be the same URL of Integration Gateway's metadata (do not include the `"`).
#### **Attribute Statements**
Although the Okta wizard says it is optional, it is actually necessary. Add an attribute called `email`, with `URI Reference` for its **Name format**. Select `user.email` as the **Value**.
Click **Next,** provide feedback to Okta if so inclined, and click **Finish**.
### Get the Okta metadata for Integration Gateway
From the **Applications** screen, click the newly created application and go to the **Sign On** tab.
Under **Settings** > **Sign on methods** > **SAML 2.0** > **Metadata details**, grab the **Metadata URL**.
This will be needed on the Integration Gateway side, when adding this Okta environment as an IdP. If this user is not an Integration Gateway admin, please provide the URL to a Sandbox Banking employee.
{% hint style="warning" %}
#### Lastly…
Don’t forget to assign users or groups to the new Okta application- otherwise they won’t be able to authenticate with Integration Gateway via Okta SSO.
{% endhint %}
# Integration Gateway SAML Config Reference
While the default SAML config settings will suffice for most use cases, changes may be necessary depending on the IdP and/or customer’s requirements.
{% hint style="warning" %}
If changes are made to an existing SAML Config, Integration Gateway's SAML metadata will change. All connected IdPs will need the updated metadata.
{% endhint %}
### All SAML Config options
#### Attribute map
default: `{"email": "email"}`
A mapping table (in the format of a JSON object) that maps IdP user attributes (keys) to Integration Gateway user attributes (values). If the name of the attribute for the user’s email address in the IdP is something other than `email` (such as `emailAddress`), this JSON object should be extended to include that mapping:
```
{
"email": "email",
"emailAddress": "email"
}
```
{% hint style="info" %}
The user email address in Integration Gateway is simply called `email`.
{% endhint %}
{% hint style="info" %}
Some IdPs allow for mapping user attributes on their end as well.
{% endhint %}
{% hint style="warning" %}
Be sure to add the new attribute name to **Required attributes**.
{% endhint %}
#### Http client timeout
default: `10`
Seconds to wait for a response from an IdP before giving up.
#### Logout http binding
default: `REDIRECT`
How Integration Gateway should send logout requests to the IdP (user wants to logout of Integration Gateway *and their IdP*). This is usually disabled by the IdP by default and is also not commonly used.
#### Cert file
Optional. A custom certificate and public key for Integration Gateway to use for SAML communication.
#### Key file
Optional. The private key corresponding to the above cert.
{% hint style="info" %}
By default, Integration Gateway uses HTTPS and requires that IdPs sign their SAML assertions, and this is generally considered to be plenty secure. A key/cert pair is only needed if *additional security, on top of https and SAML signature verification,* is desired.
{% endhint %}
#### Required attributes
default: `email`
A comma-separated list of user attributes Integration Gateway will require from the IdP. If a custom user attribute is added to the **Attribute map** it should also be added here.
#### Want assertions or response signed
default: `True`
Integration Gateway will accept as valid all SAML assertions (login response) from trusted IdPs as long as either the assertion or the outer response containing it, is signed. This is considered secure in SAML standards and should suffice for most cases. If set to True, this overrides the next two settings.
#### Want assertions signed
default: `False`
Integration Gateway will require SAML assertions to be signed. Automatically set to `False` if **Want assertions or response signed** is `True`.
#### Want response signed
default: `False`
Integration Gateway will require the outer SAML response to be signed. Automatically set to `False` if **Want assertions or response signed** is `True`.
#### Authn requests signed
default: `False`
Integration Gateway will sign its login requests to the IdP, using the cert/key pair. Generally not required.
#### Force authn
default: `False`
Requires IdP to authenticate the user directly rather than rely on a previous security context.
# How to Install the Integration Gateway App for Zoom Contact Center
**Installation of the** Integration Gateway **App for Zoom Contact Center will deploy a demo instance of the interface for users. For a production connection to your institution’s Core Banking System and other financial software, please contact** [**sales@sandboxbanking.com**](mailto:sales@sandboxbanking.com)
**Installing the App:**
1. Install the Integration Gateway app for Zoom Contact Center within the Zoom App Marketplace.
2. Using an admin account, navigate to your Zoom profile. On the left hand side, scroll down to the “Admin” section of the NavBar.
3. Open the “Contact Center Management” and click on “Integrations”.
4. Choose the “Zoom Apps” tab and then click on the “Integration Gateway” App within the list of apps.
5. Click on the “Manage Queues” button and add the queues that you’d like to enable the Integration Gateway App for. We recommend using a TEST queue or DEMO queue to test this out, as the initial install will be a demo instance of the interface for the agents.
6. From here, log onto the Zoom Contact Center. You can do this by opening the Zoom Workplace app and navigating to the Contact Center tab at the top of the screen.
7. Ensure that the user account you are logged into is assigned to the specific queue that you enabled the Integration Gateway App for.
8. Set yourself to the “READY” status and send in a test call to the Contact Center.
9. Once the engagement comes in, answer the call, and then you should see the Integration Gateway app available on the right hand side. You can expand the window to view the app properly.
**Removing the App:**
1. Log in to your Zoom account and navigate to the Zoom App Marketplace.
2. Click Manage >> Added Apps or search for the "Integration Gateway" app.
3. Select the "Integration Gateway" app.
4. Click Remove.
# How to use the Vault
The Integration Gateway vault feature provides a way for configuration, secrets, and state to be read and written by integrations.
More specifically, it is an API for storing and retrieving JSON in the database, as well as a page in the Integration Gateway application to manage the stored content.
Some useful vault vocabulary:
* A "Vault Item" is a JSON object with a name, it can be read or written by integrations, as well as on the vault page.
* A "Vault Secret" is like a Vault Item, except that it cannot be read on the vault page, only written.
* A "Vault" is a named collection of Vault Items and Secrets
**This guide will explain how to:**
1. Create a vault
2. Create a vault item
3. Configure your integration to have access to the vault
4. Read the vault item in an integration hook
Prerequisites:
* This guide expects you are familiar with the build page and editing integrations
* You must be a superuser or have the "can view vault" permission
## Creating a Vault
First, you must create a "Vault". This allows for your items and secrets to be logically organized.
1\) Navigate to the vault page by clicking the vault icon on the sidebar: 
2\) Press the "+ Create Vault" button 
3\) Name your vault "my-vault" in the modal , then press "Create Vault"
## Creating a Vault Item
Next, we'll create an item in the vault. This is the item your integration will read at runtime.
1\) Select "my-vault" in the leftmost column
2\) Press "+ Create Vault Item" 
3\) In the modal, provide the name "my-item" for your vault item, and in the text editor, provide some valid JSON.
4\) Press "+ Create Vault Item"
You vault item is now saved, and can be seen on the vault page
## Connecting your integration to the Vault
This step assumes you have an existing integration called "my-integration"
1\) Navigate to the admin page by pressing the profile icon on the top-right of Integration Gateway and select "Admin"
2\) On the page that appears, scroll to and click on "+ Add" next to "Vault Permissions"
3\) Fill out the form to give "my-integration" permission to read from "my-vault"
4\) Press save
5\) Return to the Integration Gateway Application by pressing "View Site" on the top-right corner of your screen
## Read the vault within your integration
First - some important concepts about the vault:
A given vault can only be opened by one running integration at a time. If another integration tries to open a vault that is already open, it will halt until the other running integration closes it.
For this reason, it is important to always close a vault when you are done using it. We make this easy by providing a vault context manager (The python 'with' statement)
Here is the code we will be putting into our integration before hook:
```python
with open_vault('my-vault') as vault:
my_data = vault.get("my-item")
# The with statement automatically closes the vault at this point
debug(my_data, label="Here's what was in the vault")
```
If you do the above, and run your integration, you should see this in your run history
If you want your vault contents to be accessible throughout the rest of your integration (meaning, not just within the before\_hook), you can `keep` my\_data:
```python
with open_vault('my-vault') as vault:
my_data = vault.get("my-item")
debug(my_data, label="Here's what was in the vault")
keep(my_data = my_data) # my_data can now be accessed in any other hook or expression
```
# How to Review Field Mappings with Quick Mapping
Quick Mapping lets you review a service request's unmapped fields one at a time through a card-based interface. Mark each field as approved, not used, or change requested — no need to scroll through the full table.
Quick Mapping automatically skips fields already marked as Approved, Not Used, or Change Requested. It only shows fields that still need attention.
## Quick Mapping Workflow
To review unmapped fields for a service request:
1. Click **Mapping** and select an integration from the drop-down menu.
2. Select the service request you want to review.
3. Click **Quick Mapping** above the field mapping table.
4. For each field, decide if you need to map the field:
* **Yes** — Continue to the value check. See [Approve or Request a Change](#approve-or-request-a-change).
* **No** — The system marks the field as Not Used and displays the next field.
* **Save for later** — Skip the field. Its status does not change.
5. After you review all fields, the system displays a summary of the number of fields saved and skipped.
## Approve or Request a Change
When you mark a field as needed, the card displays its current value and source information.
1. Review the current value on the card.
2. Choose an action:
* Click **Yes** if the value is correct. The system marks the field as Approved.
* Click **No** to describe a required change.
3. If you clicked **No**, enter a description of the correct value or source in the text area.
4. Click **Save & Continue**. The system posts a comment on the field and sets the status to Change Requested.
## Keyboard Shortcuts
| Key | Action |
| ---------- | ----------------------------------------------------------- |
| **← / →** | Highlight Yes or No. Press the same arrow again to confirm. |
| **Enter** | Confirm the highlighted choice |
| **S** | Skip the current field |
| **Escape** | Close the dialog |
## How It Works
* Skipped fields remain in their original state and appear again the next time you start Quick Mapping for the same service request.
* Progress updates immediately in the service request's completion percentage and the overall integration progress bar.
# How to Track Mapping Progress
The Mapping Progress tab on the Dashboard displays field mapping completion across service requests, with forecast data, target dates, and status breakdowns for an integration.
## Mapping Progress Workflow
To review mapping progress for an integration:
1. Click **Dashboard** and select the **Mapping Progress** tab.
2. Choose an integration from the drop-down menu. If you set a default integration, it loads automatically.
3. Review the dashboard components:
* **Overall progress bar** — Percentage of fields in a done status across all service requests, with a count of unresolved comments.
* **Forecast cards** — Four stat cards that summarize the project trajectory. See [Configure Forecast Settings](#configure-forecast-settings).
* **Pie chart** — Fields broken down by status category (Done, In Progress, Unstarted, and No Status).
* **Service request list** — Each service request with its own progress bar and completion percentage. Click the arrow button next to a service request name to open its field mappings on the Mapping page.
## Configure Forecast Settings
The four forecast cards display these metrics:
* **Project Status** — Whether the integration is on track, ahead of schedule, behind schedule, or past due. The system calculates this from the target date and the configured fields-per-day rate, which only counts weekdays (Monday through Friday). If you do not set a target date, the status displays as "In Progress."
* **Target Date** — The date the integration should reach completion. If the project is behind schedule, a "Need X/day" indicator shows the daily throughput required to finish on time.
* **Fields / Day** — The configured daily mapping rate used for forecast calculations.
* **Fields Left** — How many fields remain out of the total.
Users with admin or staff permissions can configure the target date. To set a target date:
1. Click the pencil icon on the **Target Date** card.
2. Select a target date from the date picker.
3. Click the check icon or click away to save.
To set the daily mapping rate:
1. Click the pencil icon on the **Fields / Day** card.
2. Enter the expected daily mapping throughput.
3. Click away to save.
The Project Status card updates immediately to reflect whether the current rate meets the target.
## Set a Default Integration
To set a default integration that loads automatically when you open the Mapping Progress tab:
1. Select the integration from the drop-down menu.
2. Click **Set as default** below the drop-down menu.
The star icon fills in to confirm. The default persists across sessions. To change the default, select a different integration and click **Set as default** again.
# How to Configure Integration Retry
Integration Retry enables automatic retry logic for failed integrations. When an integration run fails, the system can automatically retry the execution based on a configurable schedule, which reduces the need for manual intervention in cases of transient failures.
## Prerequisites
* Admin or staff permissions on the Integration Gateway instance.
* An existing integration with an Integration Config record.
## Configure Retry Settings
1. Navigate to **Build**.
2. Right-click the integration you want to configure and select **Configure**.
3. In the Configure Integration dialog, expand the **Retry Settings** panel.
4. Select a **Retry Mode**:
* **None** — No retries (default).
* **Fixed Interval** — Retries at a constant delay between each attempt.
* **Exponential Backoff** — Retries with increasing delays. Each subsequent retry waits longer than the previous one.
5. Set the **Initial Retry Delay (seconds)** — the base wait time before the first retry. For Fixed Interval, this is the delay between every attempt. For Exponential Backoff, this value doubles with each subsequent attempt.
6. Set the **Retry Delay Jitter (seconds)** — a random value between 0 and this number is added to each delay. Jitter prevents multiple failing integrations from retrying at the same time.
7. Set the **Retry Count** — the maximum number of retry attempts before the integration is marked as failed.
8. Click **Save**.
## Field Reference
| Field | Type | Default | Range |
| ----------------------------- | ------- | ------- | ----------------------------------------- |
| Retry Mode | Select | None | None, Fixed Interval, Exponential Backoff |
| Initial Retry Delay (seconds) | Integer | 5 | 1–600 |
| Retry Delay Jitter (seconds) | Integer | 2 | 0–120 |
| Retry Count | Integer | 3 | 1–10 |
## How nCino Calculates Retry Delay
**Fixed Interval:**
```
delay = retry_delay_seconds + random(0, retry_delay_jitter)
```
**Exponential Backoff:**
```
delay = (retry_delay_seconds × 2^attempt) + random(0, retry_delay_jitter)
```
Where `attempt` starts at 0 for the first retry.
**Example:** With a 5-second base delay, 2-second jitter, and Exponential Backoff:
| Attempt | Base Delay | Possible Total (with jitter) |
| --------- | ------------ | ---------------------------- |
| 1st retry | 5s (5 × 2⁰) | 5–7s |
| 2nd retry | 10s (5 × 2¹) | 10–12s |
| 3rd retry | 20s (5 × 2²) | 20–22s |
## Viewing Retry History
Each retry attempt creates its own entry in the Run History. Retry runs are labeled with `retry_for_run_id:`, which links them back to the original failed run.
The `retry_attempt` variable is available in integration hooks, which allows hook logic to behave differently based on the current attempt number.
## Limits
The maximum effective retry delay cannot exceed 5 minutes (300 seconds). For Exponential Backoff mode, the system validates that the longest possible delay (on the final retry attempt plus jitter) stays within this limit.
# How to Export an Integration Diagram
The toolbar above the [Integration Gateway Workbench](/integration-gateway-platform-reference/integration-gateway-workbench) canvas lets you export the current integration diagram as an image or as structured data. The toolbar displays the integration name and appears only after you load an integration on the canvas.
## Export Formats
| Action | Output | Use Case |
| --------------- | --------------------------------------------- | ------------------------------------------- |
| **Export SVG** | Vector image (`.svg`) | Presentations, Confluence, Figma, Inkscape |
| **Export PNG** | Raster image at 2× resolution (`.png`) | Documents, slide decks |
| **Export JSON** | Integration data as structured JSON (`.json`) | Auditing, version control, external tooling |
All exports capture the full integration graph, not just the visible viewport.
## Export a Diagram
1. Open an integration in the Workbench to display the canvas and toolbar.
2. In the toolbar, click **Export SVG**, **Export PNG**, or **Export JSON**.
3. While the export runs, the button shows an **Exporting...** state and stays disabled. When the export completes, the file downloads automatically.
The system names exported files using the integration name:
* `integration-{name}.svg`
* `integration-{name}.png`
* `integration-{name}.json`
## Export JSON
The **Export JSON** option downloads the integration's data as a structured JSON file. Use it to capture a point-in-time copy of the integration for auditing, version control, or processing in external tooling.
# Integration Builder Page
## Main Toolbar
Buttons on the Integration Builder Toolbar enables users to add, duplicate and clear changes within the Builder. The special tool, `deep clone` creates a full copy of the entire Builder Row selected, and all of the children associated with it.
{% hint style="success" %}
A new feature, [**Buildhelper**](/integration-gateway-platform-reference/buildhelper)**,** is now available to help create new Service Requests with pre-created fields for various external Systems / APIs.
{% endhint %}
***
## Filters
### Column
The column filter handles the display status of the Build Page. By default, all columns are shown but each column can be toggled to be hidden.
### Filter
The filter feature highlights matched Builder Rows based on the entered values. It will search in real-time and provide the number of results returned within the button.
### SQL Filter
This performs the back-half of a SQL filter for the table. This allows users to search with higher specificity than the traditional text based filter.
Example Filters
```
FROM * SELECT
```
### Comments
Comments are the main source of communication when jointly working within the Integration Builder. Comments can be created by anyone who has read-access to the integration.
### Tags
Each section of the Integration Builder has the ability to add a `tag` to Builder rows. This provides a way to identify specific pieces of an integration and how they relate. It is particularly helpful when used in combination with Bookmarks.
## Find and Replace
The Find and Replace toolbox offers a powerful way to quickly search for and update text within the cells on the build page. It will only search through visible cells.
1. The text to find.
2. The replacement text, if replacing.
3. Number of matches, by cell. Multiple instances of the text in one cell counts as one.
4. Move forward/backward through the matches. *Enter* and *shift-Enter* can also be used to move forward and backward respectively.
5. Replace the currently highlighted instance
6. Replace all found instances. If (11) *Replace within Selection* is enabled, will only replace the matches within the selected cells.
7. When enabled, hides all rows that do not have a match.
8. When enabled, looks for an exact case (case sensitive) match.
9. When enabled, only counts whole words as matches (e.g. "banker" will not match "bank").
10. When enabled, interprets the "find" text as a regular expression.
11. When enabled, *Replace All* will only replace matches within the selected cells.
***
### Integration Configuration Wizard
To create a new Integration, a wizard will take you through the required and optional configuration fields.
#### Step 1 — Configure Integration
First page of the Integration Configure Wizard – "Integration Config"
1. Name of the integration
2. Description for the integration
3. Whether services outside of Integration Gateway can trigger the integration
4. Whether the integration should run asynchronously
5. Send a detailed breakdown of every integration run alongside the Email Success / Failure messages (if enabled).
6. The list of emails to receive a notification when the integration finishes successfully.
1. Multiple emails should be separated by a comma, e.g. `david@sandboxbanking.com, sarah@sandboxbanking.com`
7. The list of emails to receive a notification when the integration fails.
1. Multiple emails should be separated by a comma
8. Whether or not Integration Gateway should [store the records of every run of this integration](/reference/integration_anatomy#run-history), and if so, for how long.
1. Note storing the payloads for those runs are controlled by a separate setting (9)
9. Whether or not Integration Gateway should store the contents of the payloads of every run of this integration, and if so, for how long.
1. Payloads will not be stored if the general run history storage (8) is not enabled
10. [Integration engine version](/reference/integration_components/integration-engine-versions), to preserve compatibility with platform functions as the platform evolves.
1. Integrations are developed against a specific Integration Engine version, so if unsure, consult Sandbox support before changing this setting.
11. [Webservice endpoints](/reference/web-service-endpoints), which allow an Integration to be triggered by user-defined URL paths.
#### Step 2 —Systems (adapters)
Second page of the Integration Configure Wizard – "Systems"
1. Clears all changes on the page.
2. Saves changes to adapter bindings. When setting up a new Integration, any Systems added will need to be saved before you can `CONTINUE` to the next step
3. The system Integration Gateway will be connecting to
4. The configuration to use for the selected system
1. If no configuration exists, you can create one using the *Add New Adapter* option. This will open a popup with fields specific to the system you selected.
2. At a basic level, the adapter configuration contains the credentials for Integration Gateway to communicate with the system.
5. Adds additional systems, if the Integration requires multiple.
#### Step 3 — Permissions
Third page of the Integration Configure Wizard – "Permissions"
This page is used to set Read, Write, Execute, and Debug permissions for each user on the new integration. A description of each permission can be found at [Admin Components](/integration-gateway-platform-reference/integration_configuration#integration-permissions).
# Integration Anatomy
Breakdown of the internal logic for an integration
## Integration Components
Integrations provide the wrapper for a set of composable logic with which to connect external systems.
## Integrations
### `Input`: Request body provided to the integration
Inputs begin with the request, but can be mutated throughout the life of the integration. For example, the `before_hook` can set additional variables on the input.payload. Inputs are also deeply integrated with Validation Rules, Masks, and can even be used with Value Mappings.
### `Output`: Integration response
Integration outputs can be mutated throughout the life of the integration.
> Integration outputs are often overwritten within calls to the service request. The `on_success_hook`, `on_failure_hook`, and the `finally_hook` are the most appropriate places to update the output.
Integration outputs are useful for error handling as well as returning specific payloads or status to calling systems.
Asynchronous integrations are a unique case and return an output immediately marking the start of the integration process. Required outputs to corresponding systems must be handled explicitly within the appropriate hooks during the integration.
#### State
Integrations have a success / failure state. The control of this state is flexible and can be triggered at will throughout the life of the integration using the special function [Special Functions](/reference/special_functions#end). Both service requests and validation rules contain the ability to set the integration into a failure state.
#### Custom Response Type
By default, it's assumed the integration response type is JSON. However, to use a custom response type, the `Content-Type` header can be set via `output.headers`. For example, if the desired response type is XML, `output.headers["Content-Type"]` should be set to `application/xml`.
{% hint style="warning" %}
If the `Content-Type` header is set, the value of `output.payload` **MUST** be a byte string.
{% endhint %}
## Service Requests
The building blocks of an integration. Service requests are fired off in order to perform operations with connected systems. Each service is associated with an adapter creating a call to the service using the provided adapter configs. Services include external API calls and internal file transformations.
* `Adapter Request` the prepared request. It is available within the integration from the Service Request's `{formula_variable}.request`.
* `Adapter Response` the adapter system response. It is available within the integration from the Service Request's `{formula_variable}.response`.
* `Adapter Success` the state of the adapter service call.
* `Adapter Messages` Optional message component of the adapter. This field is generally used for return adapter specific error messages.
Service Requests have the ability to be fired off as a list of service requests to the same system using the `call_for_each` field. Given an iterable, the `call_for_each` will generate
* sridx: index of the service request iterable
* sritem: value of the service request iterable
| call\_for\_each | formula\_variable |
| -------------------------------------- | ----------------- |
| \[ {"name": "sr1"}, { "name": "sr2"} ] | people\_service |
```python
# creates
people_service = [{
"request": { "service_name": "", "payload": { "name": "sr1"} }
"response": { "success": True, "payload": "", "messages": [] }
},
{
"request": { "service_name": "", "payload": { "name": "sr2"} }
"response": { "success": True, "payload": "", "messages": [] }
}]
```
Which means during an integration run, the current service request is equivalent to:
```
people_service[sridx] == sritem
```
## Field Mappings
{% hint style="success" %}
A new feature, [**Buildhelper**](/integration-gateway-platform-reference/buildhelper), is now available to assist in creating new Service Requests with pre-created Field Mappings for various external Systems / APIs.
{% endhint %}
Field mappings define the structure of the service request. Adapters generally have a few required field mappings such as `service_path` and `method`.
Field mappings contain the ability to be applied multiple times via an iterable provided to `include_for_each`. This exposes the following variables to be used within the field mapping code blocks.
* fidx: index of the field mapping iterable
* fitem: value of the field mapping iterable
Example:
| include\_for\_each | Value | Field |
| ------------------ | ----- | ------------------------- |
| \["test", "test2"] | fitem | fm\_body\[fidx].test\_key |
Generates the following request payload
```py
payload: {
fm_body: [
{test_key: "test"},
{test_key: "test2"}
]
}
```
## Value Mapping Sets
Value Mapping Sets define a collection of Value Mappings. These are tied to an integration and can be accessed anywhere within the integration via the [Special Functions](/reference/special_functions#map_value) special function.
## Value Mappings
Value mappings mutate outgoing request values from one type to another. Common use cases are for assigning statuses from a numeric value to a string representation.
```
0: "Closed"
1: "Open"
2: "Active"
3: "Pending"
```
## Validation Rules
Validation Rules runs validation for input and response data. If a response or requested input payload fails to pass the validation rule, the integration can be put into a failure state or have other logic run based on the `abort_on_failure` field. Similarly to Field Mappings and Service Requests, Validation rules can also be run multiple times with the `apply_to_each` field. These fields use the context of vridx and vritem for the given iterable.
| apply\_to\_each | | |
| --------------- | - | - |
| vridx | | |
| | | |
| | | |
```
vridx
vritem
```
## Masks
To hide sensitive information throughout the life of the integration. These are applied to either integration inputs or service request responses. Masked information will be replaced with the following "XXXXXXXXXXXX".
## Run History
Each integration run creates a unique Run History set for an integration which encapsulates a set of Run History Items detailing the exact steps taken within an integration. Each service request, historize statement, or other additional logging calls will generate a Run History Item.
# Integration Components
# Integration
### active
`boolean` - `required`
Defines whether the integration can be run at all. If set to `false` , the integration **cannot** be run via an HTTP request (regardless of the value of `http_api`), and the integration **cannot** be run via [the `callint` function](/reference/special_functions#callint).
### path\_name
`string` - `required`
The exposed endpoint for the integration. Integrations are called from `{host}/integrations/execute/{path_name}`
### description
Documentation field providing an overview of the integrations functions.
### http\_api
`boolean` - `required`
Determines whether or not an integration can be run via an HTTP request over the web.
When enabled, the integration can be called from `{host}/integrations/execute/{path_name}` or any configured [webservice endpoints](/reference/web-service-endpoints).
When disabled, the integration can only be called from another Integration Gateway integration, via the[ `callint` function](/reference/special_functions#callint).
### run\_async
`boolean` - `required`
Controls whether to run the integration synchronously. If `true` , Integration Gateway will immediately return a `200` response without waiting for the integration to finish executing.
Useful for long-runnings tasks that may otherwise cause timeouts on clients, or in combination with `callint` to spawn additional processes.
## Lifecycle Hooks
For a visual explanation of the timing of these hooks, see the [Integration Lifecycle diagram](/reference/integration-lifecycle#integration-lifecycle).
### before\_hook
`expression` - `optional`
A block of code that executes prior to the initialization of the integration.
### on\_failure\_hook
`expression` - `optional`
A block of code that runs only upon an integration failure, prior to the finally hook.
### on\_success\_hook
`expression` - `optional`
A block of code that runs only upon a successfully completed integration, prior to the finally hook.
### finally\_hook
`expression` - `optional`
A block of code that executes prior to the completion of an integration. This block will always run regardless of integration status.
## Swagger
Integration Gateway has an integrated swagger page to allow for easy endpoint execution from within Integration Gateway. The swagger request and response fields allow for the specification of swagger request bodies and sample responses. These must be formatted in line with the [OpenAPI Specification](https://spec.openapis.org/oas/v3.1.0). Swagger
#### swagger\_request
[Swagger Request Body](https://swagger.io/docs/specification/describing-request-body/) - This field is scoped to the `requestBody` of the OAS 3 specification
```python
{ "content": {
"application/json": {
"schema": {
"type": "string",
"example": "sample string input"
}
}
}
}
```
#### swagger\_response
The input is scoped to the `responses` on the OAS 3 specification.
```python
{ "200": {
"description": "Sample Response",
"content": {
"application/json": {
"schema": {
"type": "string",
"example":
"Example Object"
}
}
}
}
}
```
# Service Request
### integration
`relationship` - `required`
The integration that the service request is related to.
### parent\_servicerequest
`relationship` - `required`
If specified, the service request will be attached to another service request.
### sequence
`integer` - `required`
Order in which the service requests associated with the integration will execute, from lowest (first) to highest (last). Service requests with the same sequence will be run in arbitrary order.
### system
`string` - `required`
The external system being called by the service request. Banking-specific systems are identified by name, otherwise the `HTTP` adapter is suitable for connecting to arbitrary systems.
### service\_name
`string` - `required`
The API service being called within the target system. Usage specifics vary from system to system.
### formula\_variable
`string` - `required`
The name of the variable that will contain the `request`and `response` values of this service request. If the service request uses `call_for_each` , `request`and `response` will be lists of objects.
This variable can be accessed in subsequent service requests, field mappings, or lifecycle hooks of the integration as a means to pass data.
### call\_if
`code block` - `optional`
This expression must return a boolean. If it resolves to `True` , the service request will be run. If left empty, the service request will always run.
### call\_for\_each
`iterable` - `optional`
Specifies an iterable. The service request will run once for every entry in the iterable. Within other code blocks or field mappings of the service request, the specific entry in the iterable can be accessed using [`sritem`](/reference/special_variables#iterable-within-integrations-xx-item-and-xx-idx)and its index via [`srindex` ](/reference/special_variables#iterable-within-integrations-xx-item-and-xx-idx).
Resolves before the `call_if` block.
### abort\_after\_execute\_request\_failure
`boolean` - `optional`
Determines whether the integration should quit if the call to the external system fails. By default, the integration will not quit (`false`). When set to `true` , the integration's `after_overall_failure_hook` will still run, as well as the service request's `after_execute_request_failure_hook`.
### skip\_execute\_if\_no\_sub\_requests
`string` - `optional`
### message\_substitution\_name
`string` - `optional`
Replaces the name of the service with specified string in error messages. Useful to improve readility of errors for non-technical customers.
### notes
`string` - `optional`
Documentation field, internal to Integration Gateway.
### Lifecycle Hooks
These are code blocks that are run during lifecycle of the service request. They are equivalent in function, differentiated only by the time at which they are executed. See the [Service Request Lifecycle](/reference/integration-lifecycle#service-request-lifecycle) for a visual explanation of hook timing.
None are required for a service request to function.
* before\_prepare\_request\_hook
* after\_prepare\_request\_success\_hook
* after\_prepare\_request\_failure\_hook
* before\_execute\_request\_hook
* after\_execute\_request\_success\_hook
* after\_execute\_request\_failure\_hook
* after\_overall\_success\_hook
* after\_overall\_failure\_hook
# Field Mapping
### servicerequest
`relationship` - `required`
The service request that the field mapping belongs to.
### sequence
`integer` - `required`
Order in which the field mappings associated with the service request will execute. Note that this does not control the order of fields in the body of the external request — field ordering is arbitrary.
### field
`string` - `required`
The name of the field in the request body of the service request. Supports lazy-instantiation of intermediate fields. For example, if request body has the shape:
```json
{
"person": {
"address": {
"street": "123 main street",
"city": "Boston"
}
}
}
```
You can specify the `street`field using `person.address.street` directly.
### value
`expression` - `required`
The value to associate with a field. Can be a literal value, a reference to value defined elsewhere in the integration, or a code expression. If the expression does not evaluate to a value, the [retvalue variable](/reference/special_variables#retvalue) must be used.
### value\_type
`string` - `required`
Typecast to perform on non-null values. If the value is an expression, the typecast is performed on the result of the expression. Options are:
* `str` — String
* `int` — Integer
* `float` — Floating point number
* `dict` — Dictionary (also known as "Object")
Incompatible typecasting (e.g. `dict('5.0')`) will raise an error at runtime.
### valuemappingset
`relationship` - `optional`
The [valuemappingset](/reference/integration_components/valuemappingset) to apply to the [#value](#value "mention").
### include\_if
`expression` - `optional`
An expression to run to determine whether the field mapping should be included in the request. Must return a boolean. Defaults to `True`if left empty.
### include\_for\_each
`iterable` - `optional`
Specifies an iterable. Will produce a new field mapping entry on the request for each entry in the iterable. Use [fitem](/reference/special_variables#iterable-within-integrations-xx-item-and-xx-idx) and [fidx](/reference/special_variables#iterable-within-integrations-xx-item-and-xx-idx) to refer to values in the iterable to differentiate the field mappings.
Resolves before the `include_if` block.
### nullable
`boolean` - `required`
Indicator for whether the [#value](#value "mention") can be be null at runtime. If set to `false`and a null value is encountered, the integration will stop execution with an error.
### message\_substitution\_name
`string` - `optional`
Replaces the sritem with specified string.
### Documentation Fields
* `target_record_type`
* `target_field_name`
* `source_record_type`
* `source_field_name`
* `notes`
# Value Mapping Set
*To learn more about when to use a Value Mapping Set and how to create one, see* [*follow this how-to guide*](/how-to-guides/how-to-create-a-value-mapping-set)*.*
### integration
`relationship` - `required`
Integration value mapping set is applied to.
### name
`string` - `required`
Label for the Value Mapping Set
### description
`string` - `optional`
Documentation field used for value mapping description
### case\_sensitive
`boolean` - `required`
If `True` will respect string case. Otherwise, all input strings will be converted to lower case for the purposes of finding the corresponding value mapping. Non-string inputs are unaffected.
### use\_default\_to\_value
`boolean` - `required`
Flag to determine whether or not a default value should be used if no value mappings match the [field mapping value](https://github.com/AlphaPack/glyue-docs/blob/main/reference/integration_components/fieldmapping.md#value-code). If set to `False` and no value mappings match, the original value will be passed through untransformed.
### default\_to\_value (boolean | string)
`boolean | string` - `required`
The default value to use if no field mappings match the input value, and `use_default_to_value`is set to `True`.
# Value Mapping
*To learn more about when to use a Value Mapping and how to create one, see* [*follow this how-to guide*](/how-to-guides/how-to-create-a-value-mapping-set)*.*
### valuemappingset
`relationship` - `required`
Value Mapping Set that the Value Mapping belongs to.
### from\_value
`string | bool` - `required`
The original value of the field mapping, to identify which value mapping to use. Only supports String and Boolean values.
### to\_value
`string` - `required`
The transformed value that will replace the original field mapping value. Only supports String values.
### notes
`string` - `optional`
Documentation field
# Validation Rule
## Parameters
### integration
`relationship` - `required`
### servicerequest
`relationship` - `required`
### sequence
`integer` - `required`
### type
`string` - `required`
* `INPUT`: applies on the input payload received by the integration
* `RESPONSE`: applies to payload received from adapter
* `REQUEST`: applies to payload sent to adapter
### field
`boolean` - `required`
`default`: True Validation rule is assesed on the boolean value returned from `field`.
### rule
`code` - `required`
### apply\_if
`boolean` - `optional`
`default`: True. Will only apply the validation rule if these resolves to True.
### apply\_to\_each
`iterable` - `required`
### abort\_on\_failure
`boolean` - `required`
### valuemappingset
`relationship` - `optional`
### valuemappingset\_side
`string` - `optional`
### max\_length
### message\_substitution\_name
### Validation Rule Hooks
These are code blocks that are run during processes of the validation rule. See the [Integration Life Cycle Diagram](https://github.com/AlphaPack/glyue-docs/blob/main/media/integration_life_cycle.svg) for timing of the hooks.
* pass\_hook
* fail\_hook
# Mask
## Parameters
### integration
`relationship` - `required`
Integration to apply the mask to.
### servicerequest
`relationship` - `optional`
Service request to apply the mask to. Only used for types of `Response`.
### sequence
`integer` - `required`
Order in which to apply the masks for a given input or response.
### type
`string` - `required`
\[`Input`, `Response`] - Specifies which inputs to mask. If type is `Response`, the service request field must be specified.
### field
`code` - `optional`
Field to apply mask to. Field has the current context of the associated input, response or the current iterable provided by `apply_to_each`.
Fields can either use dot-notation or bracket notation to access nested elements
```py
#dot notation
a.n
#bracket notation
["a"]
```
### apply\_if
`boolean` - `optional`
`code` field that requires a boolean output.
### apply\_to\_each
`iterable` - `optional`
`code` field that should specify an iterable. `apply_to_each` resolves prior to apply\_if. Once an `apply_to_each` is applied, the `field` now has the context of the iterable.
### abort\_if\_not\_applied\_x\_times
`integer` - `required`
Forces integration to abort if the mask is not applied at least this many times. If set to 0, this field will be not be applied. By default this is set to zero.
### notes
Documentation field.
### Usage
Masked Parameters hide the incoming response data or input data to prevent sensitive information from being displayed within internal processes.
| SSN Masking |
| ------------------------ |
| 123-45-6789 -> XXXXXXXXX |
### Example:
Input Payload
```json
{
"people": [
{
"name": "Person 1",
"age": 42,
"private_info": "mask this"
},
{
"name": "Person 2",
"age": 22,
"private_info": "mask this",
"key:12": {
"special": true
}
}
],
"top:level:colon": true
}
```
The following masks the `top:level:colon`, `private_info`, and special fields `special` , in that respecitive order.
Field
abort_if_not_applied
type
apply_to_each
apply_if
["top:level:colon"]
1
INPUT
private_info
2
INPUT
people
["key:12"].special
1
INPUT
people
["key:12"]
Example exported JSON for the above masks:
{% file src="/files/1tJVcX5mMiJNZdH2yMGi" %}
# Integration Config
### integration
`relationship` - `required`
Integration the config is bound to.
### email\_on\_success
`boolean` - `required`
Determine wheter to send an email summary upon a successful integration run.
### email\_on\_success\_to\_addresses
`string` - `comma separated`
Email addresses to receive summaries on successful integration completion.
### email\_on\_failure
`boolean` - `required`
Determine wheter to send an email summary upon an integration failure.
### email\_on\_failure\_to\_addresses
`string` - `comma separated`
Email addresses to receive summaries on integration failures.
### email\_digests
`boolean` - `required`
Enables a more detailed breakdown of the Integration Run for both the [#email\_on\_success](#email_on_success "mention") and [#email\_on\_failure](#email_on_failure "mention") features.
### store\_run\_history
`boolean` - `required`
Enables persistence of run histories for integrations. With this flag disabled the Integration Run History will not track run events.
### store\_run\_history\_for\_x\_days
`integer`
Number of days to keep Run Histories. After this date the runs are deleted permanently. Defaults to 730 days.
### store\_payload\_in\_run\_history
`boolean` - `required`
Enables persistence of run history payloads for Integration Run Steps. The [#store\_run\_history](#store_run_history "mention") must be enabled for this to be enabled.
### store\_payloads\_in\_run\_history\_for\_x\_days
`integer`
Number of days to persist Integration Run History Items. After this date run history payloads are deleted permanently. Defaults to 14 days.
### retry\_mode
`string` - `required`
The retry strategy for failed integration runs. Options: `none` (default), `fixed` (Fixed Interval), `exponential` (Exponential Backoff). See [How to Configure Integration Retry](/how-to-guides/how-to-configure-integration-retry) for details.
### retry\_delay\_seconds
`integer` - `required`
The initial delay in seconds before retrying a failed run. Range: 1–600. Default: 5.
### retry\_delay\_jitter
`integer` - `required`
Randomness in seconds added to each retry delay to prevent thundering herd problems. Range: 0–120. Default: 2.
### max\_retries
`integer` - `required`
Maximum number of retry attempts for a failed run. Range: 1–10. Default: 3.
### engine\_version
Engine version used when running this integration.
```
engine_version = "1.1.0"
```
# Integration Engine Versions
### v1.2.0
* Add forced override of response content-type from the integration output.
### v.1.1.0
* Add `apply_if` to payload masking
* Prevent execution of `before_hook` when errors occurred in humanization or initial integration setup
# Integration Lifecycle
### Overview
Overview of how the layers of the Build table flow into one another
### Integration Lifecycle
Integration process diagram
### Service Request Overview
Service Request process diagram
### Service Request Lifecycle
Service Call process diagram
### Field Mapping Overview
Field Mappings process diagram
# Shared Modules
The Shared Module feature allows for importable code to be stored outside of an integration. This can be useful for authoring libraries of functionality, or generally for writing code that is useful across several integrations.
## Permissions
Users must have the `can_use_shared_modules` permission in order to access this page.
## Anatomy
Users with the permission will see the "Shared Modules" page appear in their left sidebar
The **Module Selection Pane** can be used to select an existing or new module\
Each module has a **Module Name** which is used to reference the module within the integration.\
The **Module Code** pane contains the source code for the module.\
The **Bound Integrations** selection controls which integrations can use the module.\
The **Save** button saves the changes to the Shared Module.
### Importing and Using a Shared Module
Shared modules must be bound to an integration before they can be imported.
\
Shared modules behave much like python's built-in concept of modules.\
They can be imported from within integration hooks or from within other modules using the [import\_module](/reference/special_functions/import_module) special function
For example, your integration hook might contain
```
import_module("money_utils")
debug(money_utils.format_currency(123), "Formatted currency")
```
To use your money\_utils throughout the entirety of an integration, you may want to `keep` it, `keep(money_utils = import_module("money_utils"))`
### Shared Module Capabilities
Shared Modules should generally be used for defining functions and classes for use by your integrations. They can use many of the same [special functions](/reference/special_functions) and [variables](/reference/special_variables) that are available in other integration code hooks, however `keep` is not supported within a shared module.
# Special Functions
Special functions are globally available functions available throughout the life of an integration.
# add\_run\_label
`add_run_label(label:str)`
Adds the provided text label to the integration's run history instance. Accepts free-form strings.
Run labels are visible in an integration's details within the Run History page.
{% hint style="info" %}
The recommended format for searchability is colon-separated pair, e.g. `label:value`
{% endhint %}
```python
# Saving the ID of a loan
add_run_label(f"loan_id:{payload.loan.identifier}")
# Noting a property of the integration with a static label
add_run_label("type:mortgage")
```
# callint
`callint(path_name: str, payload: Any, extended_payload: Bool = False, force_run_mode: str = None) -> Any`
Used to call an integration from within the process of the currently running integration. `callint` will return the integration output.
| Parameters | Description |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `path_name` | The integration name |
| `payload` | String or dict object to be used as the integration's `Input` payload. |
| `extended_payload=False` | See Extended Payload section below. |
| `force_run_mode=None` |
Allows overriding the called integrations run\_async field.
None: No override (default) sync: Force run\_async to be False async: Force run\_async to be True.
|
```python
payload = "input_str"
x = callint("test_integration", payload)
```
{% hint style="info" %}
Integrations triggered by `callint()` will appear in the Run History.
{% endhint %}
### Extended Payload
When `extended_payload` is `False` (default), the called integration runs as if it was triggered from a `POST` request to `/integrations/execute/[integration_path_name]` with `payload` representing the body of the post request.
When `extended_payload` is `True`, the object passed to `payload` can have the following additional keys (all optional):
* `params`
* `path`
* `headers`
* `method`
* `fullpath`
{% hint style="info" %}
Specifics on each field can be found [here](/reference/special_variables/input).
{% endhint %}
This is useful for emulating the behavior of calling an integration from a [Web Services endpoint](/reference/web-service-endpoints). For example,
```
payload_with_method = {
"payload": {
"firstName": "Sally",
"lastName": "Smith",
},
"method": "POST"
}
x = callint("user", payload_with_method, extended_payload=True)
```
# debug
debug`(value: any, label: str, capture_stack_trace: bool) -> None`
Creates a Run History item with the specified variable and optional label.
This is a functional equivalent of `print` . It exists because there is no `stdout` interface within Integration Gateway, so all output must be intentionally called to be surfaced in the Run History.
```python
a = ["SAMPLE"]
debug(a, label="Sample array")
```
# end
`end(payload: any, status: any, headers: dict) -> None`
Terminates the currently running integration at the end of the hook in which it is called, logging a failure with the provided status. Integration output is set to the provided payload, status, and headers.
{% hint style="info" %}
If you need the integration to halt at an exact point within a hook, consider throwing an `Exception` within the hook, then performing necessary handling in the [On Failure Hooks](/reference/integration-lifecycle#integration-lifecycle).
{% endhint %}
Can be used when some condition requires an integration to terminate early, or at the end of the integration to write custom values to an integration's `output` .
```python
try:
a={"item": True}
a.item
except KeyError as e:
end(payload=e, status=500)
```
# get\_namespace
`get_namespace() -> namespace`
Returns the current integration namespace.
Should only be used for debugging purposes.
```python
current_int_namespace = get_namespace()
debug(current_int_namespace)
```
# humanize
`humanize(str) -> str`
Attempts to regex match a field name to the [`Message Substitution Name`](/reference/integration_components/servicerequest#message_substitution_name) for the a given integration component.
```python
## Field mapping with:
## 'Field' of 'sample.value'
## 'Message Substitution Name' of Sample
k = humanize("sample.value")
debug(k)
# will output 'Sample'
```
# import\_helper
`import_helper(str) -> class`
Imports the specified helper function into the namespace of the integration. The helper can then be accessed in the lifecycle hook where it was imported, as well as every subsequent [lifecycle hook](/reference/integration-lifecycle#overview).
```python
import_helper("ncino")
ncino.autobooking.fire_callback()
```
The full list of helper classes is:
```
import_helper("ncino")
import_helper("salesforce")
import_helper("connectware")
import_helper("data_utils")
```
# import\_module
See [Shared Modules](/reference/shared-modules)
`import_module(module_name : str) -> ModuleType` imports a shared module and makes it available for use in the current hook.
**Example usage:**
Import a module and use it in the current hook:
```
import_module("my_module")
result = my_module.test_function()
```
To use a module throughout an integration you can either reimport it in every hook it is used, or [keep](/reference/special_functions/keep) it:
```
keep(my_module = import_module("my_module"))
```
# keep
`keep(**kwargs)`
Adds all provided arguments to the current integration namespace. Once kept, they can be accessed directly throughout the life of the integration, including in subsequent lifecycle hooks and as values within a field mapping.
The keyword given as a variable determines the variable name that's used to access the value subsequently.
```python
a = ["list", "items"]
keep(a=a)
keep(should_create_account = input.payload.create_account)
b, c = [], []
keep(test=b, c=c)
x = {"y": [], "z": []}
keep(**x)
```
Note that mutations to a variable after it is kept are **not** updated within the global namespace. Changes to a variable after it is kept must then be re-kept to persist for later access.
```
# Before hook
keep(foo = 123)
foo += 1
# Finaly hook
debug(foo) # This will print "123"
```
# list\_files
`list_files() -> [str]`
Returns a list of filenames accessible by the running integration.
```
filenames = list_files() debug(filenames, "available files")
```
Any of the returned filenames can then be used with `open_glyuefile()`
```
for name in list_files():
with open_glyuefile(name, "a") as file:
file.write("foobar")
```
# map\_value
`map_value(valuemappingset_name: str, value:Any) -> str`
Applies a Value Mapping Set to a provided value. This performs the same operation as when the Value Mapping column is specified on a field mapping.
When possible, the Value Mapping column should be used instead of `map_value`within an integration hook.
{% hint style="warning" %}
The value mapping set must be connected to the current integration.
{% endhint %}
```python
#
# Assuming the following exist:
#
# valuemappingset: 'SAMPLEVMS'
# valuemapping:
# From: 'Testing inputs'
# To: 'new output'
input.payload.test = 'Testing inputs'
map_value('SAMPLEVMS', input.payload.test)
```
# open\_glyuefile
## Overview
`open_glyuefile()` returns a `GlyueFileHandle` object that serves as an interface to an encrypted file (`GlyueFile`) in the database. It is meant to almost identically mimic the built-in Python function `open()`:
```python
# Open a new file and write to it
file = open_glyuefile("hello_world.txt", mode="x", encoding="utf-8")
file.write("Hello world!")
file.close()
```
`open_glyuefile()` also supports usage in a context manager:
```python
# Open an existing file (defaults to read-text mode and UTF-8 encoding)
with open_glyuefile("hello_world.txt") as file:
content = file.read()
debug(content, "file content")
# file is now closed
file.write("This will cause error!") # Raises UnsupportedOperation exception
```
## Programmer Interface
### **open\_glyuefile(***filename: str, mode='r', encoding=None***)**
Open a `GlyueFile` from the database matching the given filename and tied to the current `IntegrationRunHistory` and return a corresponding `GlyueFileHandle` object.
*filename* is a string representing the name of the file without any path. Since an actual file on disk is not being opened, no path is needed. Do not put slashes in your *filename*. Only one file with any given name can exist in the same integration run, but the same name can be reused across runs.
*mode* is an optional string that specifies the mode in which the file should be opened. The same modes available in Python’s builtin `open()` function are available here. Common modes include ‘`r'` for reading, ‘`w'` for writing (truncating the file if it already exists), ‘`x'` for explicit creation, and `'a'` for appending (all writes are done to the end of the file regardless of seek position). In text mode, if *encoding* is not passed, the encoding defaults to UTF-8.
Available modes are:
| `'r'` | open for reading (default), fails if file does not exist |
| ----- | -------------------------------------------------------------------------------- |
| `'w'` | open for writing, truncating the file first, opens new file if it does not exist |
| `'x'` | open for exclusive creation, failing if the file already exists |
| `'a'` | open for writing, appending to the end of file if it exists |
| `'b'` | binary mode |
| `'t'` | text mode (default) |
| `'+'` | open for updating (reading and writing) |
The default mode is `'r'` (open for reading text, a synonym of `'rt'`). Modes `'w+'` and `'w+b'` open and truncate the file. Modes `'r+'` and `'r+b'` open the file with no truncation.
### GlyueFileHandle methods and properties
| **attribute** | **type / returns** | **description** |
| ------------------------------------ | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `close()` | `None` | clears the buffer and closes the file |
| `closed` | `bool` | value representing whether the file is closed or not |
| `encoding` | `str\|None` | value of the encoding the file was opened with |
| `fileno()` | `int` | returns the database id of the underlying `GlyueFile` |
| `flush()` | `None` | writes changes (if any) to the database |
| `mode` | `str` | the mode the file was opened in |
| `name` | `str` | the filename |
| `read(size=-1)` | `bytes\|str` | returns the next `size` chars/bytes from the buffer current position, or the entire contents if no `size` is passed |
| `readable()` | `bool` | returns `True` if the file is readable in its current mode, else `False` |
| `readline(size=-1)` | `bytes\|str` | returns the next `size` chars/bytes from the buffer current position, or all characters/bytes until the next newline char if no `size` is passed |
| `readlines(hint=-1)` | `list[bytes/str]` | returns a list of the next `hint` lines or a list of all lines if no `hint` arg is passed |
| `seek(position: int, whence=0)` | `int` | sets the buffer to the current `position` from `whence` (`0` from beginning of file, `1` from current position, `2` from end of file) and returns the new buffer position as an `int` |
| `seekable()` | `bool` | returns `True` if `seek()` can be currently used, else `False` |
| `tell()` | `int` | returns the current position of the buffer |
| `truncate(size: int=None)` | `int` |
truncates the file to size chars/bytes, or at the current position if no size arg is passed
This action is immediately reflected in the database!
|
| `writable()` | `bool` | returns `True` if the file is writable in its current mode, else `False` |
| `write(data: bytes\|str)` | `int` | writes `data` to the buffer from the current position and returns the length of the data written |
| `writelines(data: list[str\|bytes])` | `None` | writes `data` to the buffer from the current position (does not automatically add newline chars) |
## GlyueFileHandle Buffering
All read and write operations are done on the underlying buffer which is either a `BytesIO` in binary mode or `StringIO` in text mode.
### Reading
The file contents from the database are not immediately loaded into the buffer on open, but rather when the data is first needed (such as calling `read()` among multiple other methods). At this point the entire contents of the file have been loaded into the buffer (and thus, memory), regardless of whether or not an amount of data to return was specified (e.g. `read(5)`, `readline()`, `readlines(10)`).
### Writing
When writing, any new data written to the buffer is not *immediately* reflected in the database, but rather deferred until a `flush()` operation is performed. This can be done explicitly by calling `flush()` directly on the handle object, but is also done implicitly by multiple other methods, most notably `close()`.
### Memory Efficiency
The data in the buffer is retained until `close()` so it is recommended to always remember to close `GlyueFileHandle` objects once their data is no longer needed. This will free up system memory, and is incredibly important when working on larger files.
### As a Field Mapping Value
Use the custom `File` type to allow the integration engine to process a `GlyueFileHandle` object as a valid **value** in a field mapping table (in this example, the `file` variable is a `GlyueFileHandle` object):
| **field** | **value** | **type** |
| ------------- | --------- | -------- |
| someFieldName | file | File |
## Viewing / Downloading files from the Run History page
Any `GlyueFileHandle` objects that are historized or appear in a service request payload will be shown underneath as clickable download links:
## Database GlyueFile Retention
`GlyueFile`s in the database are related to an `IntegrationRunHistory` and are subject to the corresponding `IntegrationConfig`'s run history payload storage settings, and will be deleted after the number of days specified. If turned off altogether, files are purged from the database at the end of the integration run and will not be downloadable from the Run History page.
#### Sub-integration behavior
A sub-integration’s config must have `Store payloads in run history` turned **on** if it will return a *new* `GlyueFileHandle` to its parent.
As part of a sub-integration, it may be desirable to output/return a `GlyueFileHandle` to the parent. However, any Glyue files *created* during a sub-integration run will be subject to the child’s integration config. If `Store payloads in run history` is *turned off* then the file in the database will be purged upon completion of the run, regardless of whether or not its handle object has been given to the parent and is still alive*.* A `DoesNotExist` exception will be raised if the parent integration attempts to perform any operations that would interact with the file (now purged) in the database. If the file handle already has the data in its buffer, it may still be readable, however.
# open\_vault
`open_vault(vault_name:str) -> Vault`
Opens a vault based on the given name.
{% hint style="info" %}
Read more about the Vault on its main documentation page.
{% endhint %}
**Get data from vault**
```python
with open_vault('data') as vault:
# Get data from a vault entry
customers = vault.get('customers')
```
**Modify data in an existing vault**
```python
new_customer = {"name": "David"}
with open_vault('data') as vault:
customers = vault.get('customers')
customers.append(new_customer)
vault.update('customers', customers)
```
# Data Manipulation Utilities
## List of Functions
* [extract\_chars()](#extract_chars)
* [extract\_digits()](#extract_digits)
* [percent\_to\_decimal()](#percent_to_decimal)
* [round\_half\_up()](#round_half_up)
* [get\_current\_date\_string()](#get_current_date_string)
* [convert\_date\_string()](#convert_date_string)
* [IndexGenerator](#indexgenerator)
## Getting Started
```python
import_helper("data_utils")
```
Import the `data_utils` integration helper in the Integration *Before Hook* to expose the helper functions to the integration’s namespace.
### extract\_chars()
```python
data_utils.extract_chars(field, start=0, stop=None, step=None)
```
Takes `str` or `Flex(None)` input and returns spliced output using typical python string splicing logic. Returns `None` for `Flex(None)`
```python
data_utils.extract_chars("test123abc", 4) -> "test"
data_utils.extract_chars("test123abc", 7, 100) -> "abc"
data_utils.extract_chars("test123abc", 0, 7, 2) -> "ts13"
data_utils.extract_chars("") -> ""
data_utils.extract_chars(None, 0, 7, 2) -> None
```
### extract\_digits()
```python
data_utils.extract_digits(field)
```
Given a `str` or `Flex(None)` input, and outputs all digits.
```python
data_utils.extract_digits("abc123") -> "123"
data_utils.extract_digits("123-456-7890") -> "1234567890"
data_utils.extract_digits("") -> ""
data_utils.extract_digits(None) -> None
```
### percent\_to\_decimal()
```python
data_utils.percent_to_decimal(percent)
```
Given an `int`, `float`, `Decimal`, or number in `str` format and returns value in decimal format.
```python
data_utils.percent_to_decimal(8) -> Decimal("0.08")
data_utils.percent_to_decimal("") -> None
data_utils.percent_to_decimal(None) -> None
```
### round\_half\_up()
```python
data_utils.round_half_up(input_num, decimal_places=2)
```
Given an input of `float`, `int`, `Decimal`, or number in `str` format, outputs a `Decimal` number rounded to n decimal places. This implements the typical round half up strategy where 0.5 rounds up.
```python
data_utils.round_half_up(0.875, decimal_places=2) -> Decimal("0.88")
data_utils.round_half_up("", decimal_places=2) -> None
data_utils.round_half_up(None, decimal_places=2) -> None
```
### get\_current\_date\_string()
```python
data_utils.get_current_date_string(timezone, date_format="%Y-%m-%d")
```
Given a time zone and datetime format input, returns the current date `str` in the specified format.
```python
data_utils.get_current_date_string("US/Pacific", "%Y-%m-%d") -> "2024-12-22"
```
All timezones in the pytz package are supported. Here are the major US time zones:
* `US/Central`
* `US/Eastern`
* `US/Mountain`
* `US/Pacific`
You can use the following Python to capture the full list:
```python
import pytz
# total number of timezones
len(pytz.all_timezones)
# list all timezones
for tz in pytz.all_timezones:
print(tz)
```
### convert\_date\_string()
```python
data_utils.convert_date_string(input_date, in_format, out_format="%Y-%m-%d")
```
Given a date string and it’s date time format, converts to the specified output format as a date string. (default output format is `"%Y-%m-%d"`)\\
```python
data_utils.convert_date_string("2024-12-22", "%Y-%m-%d", "%Y/%m/%d") -> "2024/12/22"
```
### IndexGenerator
```python
custom_idx = data_utils.IndexGenerator(start=0)
```
This class is intended to generate numbers sequentially. A typical use case would be to define field mapping paths with `next()` and `.last` so that the Field Mapping paths are dynamically generated and an *Include For Each* isn’t necessary.
```python
custom_idx.last → None
next(custom_idx) → 0
custom_idx.last → 0
next(custom_idx) → 1
custom_idx.last → 1
...
```
{% hint style="info" %}
Be sure to `keep()` the instance of IndexGenerator in the relevant hook or an *AttributeError* exception will be thrown.\
\
e.g:`keep(custom_idx = data_utils.IndexGenerator())`
{% endhint %}
Be aware that Integration Gateway evaluates the *Value* Column on the Field Mapping table before the *Field* column which can result in unintended behavior. e.g.:
```
if custom_idx == 11;
Field Path: some.field.path[next(custom_idx)] → some.field.path[12]
Value Column: f”some_value{custom_idx.last}” → “some_value11“
```
# calladapter
`calladapter(system: str, service_name: str, label: str, payload: Any, sub_requests: Any) -> AdapterResponse`
Makes a call to a configured adapter from within the current integration process.
```python
x = calladapter("ECHO", "N/A", label="Internal adapter call", payload="x")
if x.response.success == True:
historize("call_successful")
```
# Special Variables
Integration Gateway has several built in special variables that allow access to various scopes throughout the course of an integration.
# \_\_adapter\_config\_\_
`__adapter_config__ -> {}`
Exposes properties of an adapter config to be overridden using field mappings or in lifecycle hooks.
{% hint style="info" %}
Overrides must occur during the `before_prepare_request_hook` or later. See the [full service request hook lifecycle order here](/reference/integration-lifecycle#service-request-overview).
{% endhint %}
A service request's payload may optionally have an `__adapter_config__` field added to it. When present, the keys and values of the `__adapter_config__` dictionary override the corresponding entries in the adapter config that would normally be used.
{% hint style="warning" %}
Values stored within Field Mappings do not have the same encryption methods as fields set directly on adapter configs.
**Do not put sensitive information, including API secrets / passwords, in field mappings or lifecycle hooks.**
{% endhint %}
The `__adapter_config__` field can either be set directly on the payload in a hook:
```
my_service_request.payload.__adapter_config__ = {
"username" : "override_username",
"host" : "other.website.com"
}
```
Or equivalently, individual fieldmappings can be created like below
```
FIELD: __adapter_config__.username
VALUE: "temporary_username"
FIELD: __adapter_config__.host
VALUE: "other.website.com"
```
# input
A dictionary representing the request that is passed to an integration.
| Property | Description |
| -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| payload | Dictionary of payload values passed in the body of the request |
| method | The HTTP method used to call the integration (e.g. `POST`, `GET`, etc.) |
| params | Dictionary of query parameters present on the request |
| path |
Dictionary of path parameters present on the request
|
| fullpath | Full path of the integration's execution URL, regardless of whether it or a web service endpoint was used to call the integration |
| headers | Dictionary of headers present on the request |
| files | Dictionary of files present on a `multipart`request. Key is the name of the file, value is a handle to the [GlyueFile](/reference/special_functions/open_glyuefile) that represents the file with Integration Gateway. |
| user | Object containing the `id`, `username`, and `email`of the user who called the integration |
Common usage is:
```
# Retrieve value from payload
input.payload.some_value
# Check method on request
if (input.method == "POST"):
# Do something
# Retrieve customer ID from path params
cust_id = input.params["cust_id"]
```
`input` can be referenced in any formula block / lifecycle hook, including the `value` column of a field mapping.
# parentint
`Dict`
Gives access to the parent integration's namespace from a child integration during a synchronous [`callint`](/reference/special_functions/callint) integration run. The full set of special variables are available on the parent.
{% hint style="info" %}
Asynchronous integrations have no reference to their parent integrations.
{% endhint %}
```python
# Access the parent's input
parentint.input.payload
# Returns the parent integration's Integration Run History Item
parentint.run_history_id
```
# retvalue
Variables that indicates the return value from a formula block. Used when the contents of a formula block / lifecycle hook spans multiple lines, or if the single line does not resolve into a singular value.
Examples:
```python
apply_exchange_rate_adjustments(sf_collateral.response.payload.collateral)
apply_inflation_adjustments(sf_collateral.response.payload.collateral)
retvalue = sum(c.amount for c in sf_collateral.response.payload.collateral)
```
```python
if input.payload == "test value":
retvalue = input.payload
else:
retvalue = "value not provided"
```
# run\_history\_id
`integer | None`
Contains the ID of the integration run instance. This value is also visible in the Run History entry corresponding to the integration run.
The value will be `None` if integration run history persistence is disabled in the integration’s configuration.
This value can be useful for reporting and analytics purposes.
# Iterables
# fitem/fidx
Provides access to the value (`fitem`) and index (`fidx`) when iterating through a list using the [`include_for_each`](/reference/integration_components/field-mapping#include_for_each) column on a [field mapping](/reference/integration_components/field-mapping).
These variables are only populated when the `include_for_each` column is set, and are only accessible in its `field` and `value` columns. They are usually used to populate lists or dictionaries for a service request's fields.
{% hint style="info" %}
Read more about the `include_for_each` column [here](/reference/integration_components/field-mapping#include_for_each).
{% endhint %}
```
# Given
include_for_each = ["foo", "bar", "baz"]
# First iteration
fitem ==> "foo"
fidx ==> 0
# Second iteration
fitem ==> "bar"
fidx ==> 1
etc...
```
**Example usage**
```
# Given
include_for_each = [
{"name": "Sally", "email": "sally@email.com"},
{"name": "Benjamin", "email": "benjamin@email.com"},
{"name": "David", "email": "david@email.com"}
]
# To fill a theortical "all_emails" field, use
field: all_emails[fidx]
value: fitem['email']
# Results in
["sally@email.com", "benjamin@email.com", "david@email.com"]
```
# sritem/sridx
Provides access to the value (`sritem`) and index (`sridx`) when calling a [service request](/reference/integration_components/servicerequest) multiple times using the [`call_for_each`](/reference/integration_components/servicerequest#call_for_each) column.
These variables are only populated when the `call_for_each` column is set, and are only accessible in the service request's lifecycle hooks and in all columns on its [field mappings](/reference/integration_components/field-mapping). They are usually used to configure field mappings when calling a service request repeatedly.
{% hint style="info" %}
Read more about the `call_for_each` column [here](/reference/integration_components/servicerequest#call_for_each).
{% endhint %}
```
# Given
call_for_each = ["abc@email.com", "def@email.com", "ghi@email.com"]
# First iteration
fitem ==> "abc@email.com"
fidx ==> 0
# Second iteration
fitem ==> "def@email.com"
fidx ==> 1
etc...
```
**Example usage**
```
# Given
call_for_each = [
{"name": "Sally", "email": "sally@email.com"},
{"name": "Benjamin", "email": "benjamin@email.com"},
{"name": "David", "email": "david@email.com"}
]
# To send an email to each individual, set the following on the service request's field mapping
field: email_address
value: sritem.email
```
---
[Next Page](https://glyue.docs.sandboxbanking.com/llms-full.txt/1)
