Subscribe to Automation Studio notifications via SOAP API

If you have used Automation Studio to build workflows on autopilot, you are certainly aware of the importance of monitoring. Checking manually the execution status of every automation in your account is tedious and exhausting. Sometimes, you need to be notified in real time so you can fix eventual issues. That is why having email notifications enabled on your automations is necessary.

There are two types of notifications to which you can subscribe on automation settings:

  • Error or skipped run: you will receive an email if the automation’s execution fails or is skipped
  • Run Completion: you will receive an email once the automation’s run completes.

Keep in mind that at this moment, the only official way of interacting with Automation Studio via API is through SOAP. I’m saying “official” because there are actually some undocumented, but very limited, REST endpoints that can be used to play with Automation Studio. Let me know in the comments if you are interested.

Therefore, if we check the official SOAP documentation, we will see that creating an automation along with activities and email notifications in its settings is pretty easy and straightforward. However, if we look closer, we will find a comment on the notification tag saying: DOES NOT WORK. But we are not going to accept this for an answer, aren’t we?

This is the SOAP request sample from the documentation to create an automation: [see code snippet on Github]

To find the source of the problem, we should get into the tags responsible for adding email notifications to the settings. In this example, they are using a wrong value for the NotificationType tag in addition to a missing ChannelType tag.

NotificationType should be either “Complete” OR “Error” and definitely not “-1”. And ChannelType should be equal to 1.

The correct SOAP request to create an automation along with a “Complete” notification email should look like this: [see code snippet on Github]

For the curious ones, you may be asking how I found the working values. I invite you to go to Automation Studio, open the Chrome Developer Tools (F12) and click on the Network tab.

Open an automation, add your email to the settings and click done. Now go check out the Network tab. There is a REST call to the endpoint below.

Given the fact that we can only interact with Automation Studio using SOAP calls for now, we can understand why the endpoint is beta and not officially documented:

https://mc.s50.exacttarget.com/rest/beta/automations/notifications/AUTOMATION_ID/

Now let us look at the request payload:

We can see that the values used for NotificationType and ChannelType are as described above. That was worth a try and it worked: [see code snippet on Github]

I invite you to check the network tab every time you need to interact with Salesforce Marketing Cloud APIs. It can be very interesting and full of undocumented information.

If you know someone that can correct this on the official documentation, please get in touch through the comments or send me a message on LinkedIn.


About the author

Rachid Mamai is a SFMC geek and a Digital Marketing enthusiast living in France. To get in touch with Rachid, visit his LinkedIn.

Loops in AMPscript and Server-Side JavaScript

In programming languages, loops are a basic, yet a very powerful and indispensable concept. They shorten the code and reduce the need to repeat tasks. A loop is a set of instructions to be executed until a certain condition is met. In this article, we will focus on the most fundamental kind of loop, the for loop, and it’s usage in AMPscript and Server-Side JavaScript.

For loops in AMPscript

NOTE: In this article, we will focus on understanding the basic concepts of working with loops in Salesforce Marketing Cloud. For a strictly technical specification, defer to the following article: AMPscript Process Loops.

A for loop, which is actually the only loop available in AMPscript, lets you execute the same script repeatedly until an ending condition is met.

Let’s use the following example: we have a Data Extension with customer IDs and their ordered items. We want to use that dataset to send an email to each customer, listing the items each of them ordered.

Here’s our Data Extension called Orders:

CustomerIdOrderIdOrderItem
0031t000005D98UAAS4632Women’s Marine Hoodie
0031t000005D98UAAS4632Glass Love Bottle – 16.9 oz
0031t000005D98UAAS4632Eco Ballpoint Pen
0031t00000ZnTENAA33686Men’s Marine Hoodie
0031t00000YEM4ZAAX7354Leather Hampton Watch

In our example, the script contained within the loop would do the following: display a bullet point and display OrderItem name, and repeat the same action until all the OrderItems for a relevant Customer were displayed.

The most important element of any loop is the counter. The counter will define how many times the script in the loop should be executed. In order to define the counter, we must first set the criteria to find all the matching results in our Data Extension. We will use the LookupRows function to look up all rows, that match the Id of the Customer, that we want to send the email to. Then, we will use the RowCount function to determine how many results were found. The outcome of the RowCount function will be our counter and will decide how many times the script inside the loop will be repeated. Here’s the first part of our script, where we count how many records match the criteria: [click here to see the code on Github]

Now let’s think through the loop logic. This is the verbal description of the loop that we would like to build:

Start at 1 and repeat the following in a loop until you reach the number defined in the RowCount variable: display the current index and the name of the OrderItem found in the referring row from the results of the LookupRows function. Increase the index by one and repeat the process. Stop looping once the index reaches the number defined in the counter.

Here’s how this looks in AMPscript: [click here to see the code on Github]

You will notice the next keyword at the end of the loop – each time this keyword is reached, the system compares the current index to the value of the counter defined by the RowCount function. If the value is not equal to the end index, the loop will repeat until the end index is reached.

Here’s a full script that we can now use in an email, with added exception handling that will show “No items to display” in case the LookupRows function doesn’t find any matching records for a subscriber in the Orders Data Extension: [click here to see the code on Github]

For loops in Server-Side Javascript

While JavaScript offers at least five different kinds of loops, and all of those can be used in Salesforce Marketing Cloud, in this article we will focus on the most basic one, the for loop.

We will create a similar solution to the one created in the AMPscript example above. We will use the same Data Exteniosn called Orders and we will use a loop to display all the OrderItems for a Customer on a CloudPage.

The first thing we have to do in order to interact with a data extension via server-side JavaScript is to initialize the Data Extension object. Then, we will use the Server-Side JavaScript Rows.Lookup function to find all rows that match the Id of the Customer. For the purpose of this tutorial, we will hardcode the CustomerId into the script: [click here to see the code on Github]

The above script will return an ArrayList, which is a collection of elements. Arrays use numbers to access its elements and the numbering always starts at zero:

Arrays in JavaScript are zero-based. This means that JavaScript starts counting from zero when it indexes an array. In other words, the index value of the first element in the array is “0” and the index value of the second element is “1”, the third element’s index value is “2”, and so on.

https://blog.kevinchisholm.com/javascript/javascript-array-length-always-one-higher/

In AMPscript we used the RowCount function to determine the counter for our loop. In JavaScript, we will use the Array length Property. Inside the loop, we will display the current index (i) and the name of the element (OrderItem) that has this index number in the ArrayList. Remember that we will start counting at “0”, not at “1” like we did in AMPscript.

Then, we will increase the index by one and repeat the process.

In JavaScript, the ++ notation is the increment operator, which means that i++ is exactly the same thing as i = i+1. The script will stop looping once the index reaches the number defined in the counter (rows.length). Here’s the script: [click here to see the code on Github]

Here’s the full script that we can now use on a CloudPage, with added exception handling that will show “No items to display” in case there are no matching records found in the Orders Data Extension: [click here to see the code on Github]

The is how the results will be displayed on a CloudPage:

Additional resources

Here is a list of additional resources to help you understand both the general concept of loops, as well as using loops in AMPscript and Server-Side JavaScript:


Questions? Comments?

Leave a comment below or email me at zuzanna@sfmarketing.cloud.

Get Journey history by filtering the Definition ID via API

In today’s article, we will be going through a non-documented way of getting journey’s history through the API. The majority of this information is available on the UI. However, having a second way of getting this data in raw format can be helpful in some use cases. You can use free software like POSTMAN to interact with Marketing Cloud’s API and follow the steps in this article.

Let’s start by setting the correct scopes in Salesforce Marketing Cloud.

Set scope on Contacts and Journeys

When requesting our access token, we need to provide scopes used by our API calls. Do not forget to enable the Journeys & List and Subscribers scopes on the Installed package. We will be using these two scopes to get our journey’s history. Check out “Introduction to SFMC APIs” article for in-depth information about Installed packages setup and Salesforce Marketing Cloud APIs.

We need to set the scopes on the payload to get our access token: [see code snippet on Github]

Get Journey’s Definition ID

In this step, we will be getting information about a specific journey. The undocumented endpoint “interaction” provides access to a range of data, including the DefinitionID. Before getting into it, let me explain the differences between the three different identifiers of a journey:

  • Id: A unique id of the journey assigned by the journey’s API during its creation
  • Key: A unique id of the journey within the MID. Can be generated by the developer
  • DefinitionId: A unique UUID provided by Salesforce Marketing Cloud. Each version of a journey has a unique DefinitionID while the Id and Key remain the same.

Since the journey’s history is version dependent, we will be using DefinitionID to get history of a specific version.

Let us start by making a call to the below endpoint using name=JOURNEY_NAME as a URL parameter. By default, mostRecentVersionOnly parameter is set to true, therefore, the call will return information of the most recent version of the journey. Use VersionNumber=VERSION_NUMBER and mostRecentVersionOnly=false to get information about a specific version. Note that more parameters are available but not covered in this article since irrelevant: [see code snippet on Github]

The DefinitonID is located in a nested array called items, something like: [see code snippet on Github]

Get Journey’s history by filtering via Definition ID

We are finally there. In this step, we will call the endpoint below: [see code snippet on Github]

The payload should be something like below. Do not forget to set the Content-type as application/json on the call’s parameters: [see code snippet on Github]

You can apply multiple filters by setting different variables in the payload. In this example, we are selecting history data for our journey starting from October 23, 2019 to now. The journey is identified by the DefinitionID in the DefinitionIds array parameter in the payload. Other filters are:

  • Page and PageSize: For results pagination. The page starts from one.
  • Extras: represents a list of additional data to fetch. Available values are all, activities, outcomes and stats.
  • OrderBy: You can order by CreatedDate
  • MostRecentVersionOnly: To get information about the most recent versio of the journey. Accepts a true or false value.

This call will return something like below depending on the payload: [see code snippet on Github]


About the author

Rachid Mamai is a SFMC geek and a Digital Marketing enthusiast living in France. To get in touch with Rachid, visit his LinkedIn.

Introduction to Triggered Sends

After reading this article you will be able to:

  • Create a triggered email,
  • Create a triggered email interaction,
  • Track and log sent triggered emails,
  • Personalize a triggered email,
  • Trigger an email from Marketing Cloud using various methods,
  • Trigger emails from Sales and Service Cloud.

Understand Triggered Sends in Salesforce Marketing Cloud

A triggered email is a message that Marketing Cloud sends to an individual subscriber in response to a subscriber action. For example, sending a confirmation message after a customer makes a purchase is a triggered email.

A triggered email has two components that you create in Marketing Cloud: content and interaction. When you define a triggered email interaction, you provide information about the message and its behaviour that the interaction uses each time it is triggered, for example: which Sender Profile and Send Classifications to use, which Publication List to use, etc.

The definition of what triggers the interaction to send the email is maintained outside of the application using API calls. The information created in the interface is used by the API when an email is triggered.

Follow the list of step-by-step instruction in this article to create and send your first Triggered Email.

Create an email for use with Triggered Sends

Let’s start by building a very simple email, without any AMPscript or personalization. Go to Email Studio > Content and create an email, just like you would with any other email created in you’re account. Triggered Emails do not have any special requirements when it comes to templates or content blocks used, content types or even where you store them – they’re just like any other emails in Salesforce Marketing Cloud’s Email Studio.

If you need help building an email from scratch, try this link: Create a Content Builder Email.

Create a Triggered Email Message Interaction

Once the email is ready, we can jump to Email Studio > Interactions > Triggered Sends and create a new interaction, which will define all of the Triggered Send’s properties.

In the Triggered Sends section, click on the Create button.

First, let’s define all the properties:

Name is the name or title of this email message interaction. Use this name to identify the message in the application. Subscribers cannot see the name.
External Key is a unique identifier for the message. Use this value to identify the message when using the API. If you leave this field black, the system will automatically generate a key for this interaction upon saving.
Description is a brief explanation of the email’s content and purpose. Subscribers will not see the description.
Send Classification lets you choose a send classification to use for this interaction.
Override Sender Profile determines whether the interaction uses a different sender profile than indicated in the send classification. If selected, select an override sender profile to use.
Override Delivery Profile determines whether the interaction uses a different delivery profile than indicated in the send classification. If selected, select an override delivery profile to use.

In Associate to Campaign section, you can associate the triggered send with a Marketing Cloud Campaign if you have one – if not, leave this section blank.

In the Content section, find and select the email that you created earlier.

In Subscriber Management, there are two separate sections called List and “Triggered Send” Data Extensions.

The purpose of the first one, List, is subscriber management. Either choose All Subscribers or a dedicated publication list if you have one. Failing to select a list results in use of the Triggered Send Managed List, a hidden system-defined List, to track status. Avoid this scenario as the Triggered Send Managed List does not support all triggered send-based functionality.

You will also notice two checkboxes under the List section:

Add subscribers to this list determines whether the list accepts new subscriber information of subscribers who trigger this email interaction. If selected, the send adds the subscriber to a selected list if that email address does not already appear on the list. If, for example, you’re using the All Subscribers list to manage subscriber statuses, and the subscriber who triggers this email interaction is not yet there, they will be added there.

Update Subscribers determines whether an existing subscriber’s attributes should be updated if you pass in new data for that subscriber via the API call.

The “Triggered Send” Data Extensions section lets you choose a Triggered Send Data Extension for use with this interaction, which will act as a SendLog, which means it will update with data from requests to trigger this email interaction. In this section, you can only use a data extension created using the TriggeredSendDataExtension template.

Next, in the Select a suppression list section, you can include a suppression list that prevents certain addresses from receiving triggered email messages, select that list. In the Exclusion Management section, you can use a data extension or exclusion script to prevent email from being sent to selected domains or subscribers. Both those sections are optional, so you can leave them blank if you don’t plan to suppress or exclude anyone.

The last section is also optional and you can choose here any additional Send Options: CC Email Address, BCC Email Address, Track all links found within this email, Send as Multipart MIME, Suppress this Send from Reports. There are also some additional options available (create a support case if they haven’t been enabled in your account):

Priority determines how quickly a triggered email is sent:

  • High – Queued immediately. Using this setting requires an additional cost.
  • Medium – Queued every minute by default. It can take up to 1 minute to send the email.
  • Low – Queued every 5 minutes. It can take up to 5 minutes to send the email.

Keyword categorizes the interaction. If you enter a value in this field, that value is available to API calls.

Send Logging, if checked, logs the send in the SendLog data extension.

The last checkbox, Disable API calls to this triggered send when an email build error threshold is reached, sets a threshold of errors before API calls are disabled and the sending process is stopped. By default, the sending process stops when an error occurs while building an email for a subscriber. These errors can occur for several reasons, including an API call passing in bad data or an AMPscript runtime error. However, the system continues to accept API calls even after an error occurs. If you want to use a back-up system when the threshold is reached, this option allows visibility at the API layer to the error.

Once you are finished, click on the Save button located at the top of the page.

For more information on creating Triggered Email Message Interactions, visit this help article.

Start a Message Interaction

Now that you created and saved your interaction, for the whole setup to go live, we need to “start” it.

From the list of interactions, check the checkbox next to the one you just created and click on the Start/Restart button. Make a note of the External Key, as you will need it later to be able to send the emails out.

If in the future you would need to make some changes to either email content or the interaction properties, such as Sender Profile or Send Classification, pause the interaction first. When it’s paused, it will cause triggered messages to queue while you make all the changes. Publish the interaction to begin sending emails again.

Congratulations, you’re now ready to trigger the first email!

Triggering the email

In order to trigger the email, we will have to interact with Salesforce Marketing Cloud’s SOAP Web Services API. We won’t get into technical details of interacting with the TriggeredSend object, but rather use a couple of code snippets that you will be able to easily copy and paste and use in your instance of Salesforce Marketing Cloud.

Trigger the email using AMPscript

Go to Web Studio > CloudPages, create a new collection and then create a new Landing Page. You can choose to use the Content Builder when creating the landing page, and for the purpose of this demo, you can use a blank page layout. From the Content Blocks available on the left, drag and drop an HTML block onto the canvas and paste the below code into the HTML editor: [click here to see the code on Github]

In the above script, replace ExternalKey with the external key of the interaction you created earlier and replace email@example.com with your email address. Now click on Schedule/Publish in the right upper corner. The script will only take a couple of seconds to execute and a status message will be displayed. In case it’s OK, you can go ahead and check your inbox, as the email should already be there.

To get into more technical details of the above script and learn how to use a Subscriber Key other than the recipient’s email address, read the following article: Send a triggered email using AMPscript.

Trigger the email using WSProxy

WSProxy is a new object for Server-Side JavaScript, introduced by Salesforce in 2018. Using WSProxy has many advantages, about which you can read in this article, Introducing WSProxy for Salesforce Marketing Cloud. The article also explains the benefits of using WSProxy over pure SSJS for sending triggered emails.

Just like we did before, go to Web Studio > CloudPages, create a new collection and then create a new Landing Page. From the Content Blocks menu, drag and drop an HTML block onto the canvas and paste the below code into the HTML editor: [click here to see the code on Github]

In the above script, replace ExternalKey with the external key of the interaction you created earlier, replace email@example.com with your email address, and either provide a SubscriberKey or use your email as SubscriberKey for this demo. Click on Schedule/Publish in the right upper corner. Again, the script will only take a couple of seconds to execute and a status message will be displayed in JSON format. If it shows "Status":"OK", you can check your inbox for the triggered email.

To get into more technical details of the above script, read the following article: Send a triggered email using WSProxy.

Personalizing triggered emails

Triggered emails, just as any other emails sent from Salesforce Marketing Cloud, can be personalized. In order to add personalization to our email, let’s first pause the Triggered Send Interaction, and then go to Email Studio to edit the email. Let’s add a salutation to our email with the subscriber’s first name. There are two ways in which you can achieve this:

  1. In the email, include a simple personalization string, for example: %%firstname%%. This will allow you to pass the firstname value along with the subscriber’s email address and subscriber key, but you will be required to use a “Triggered Send” Data Extensions for your sends and include a column called “firstname” in that Data Extension. The benefit of using this method is that all sends will be recorded in that particular Data Extension and recipient’s first name will be logged along with their email address and other attributes you choose.
  2. In the salutation, include a personalization string using the AMPscript AttributeValue function, for example: %%=v(AttributeValue("firstname"))=%%. This will allow you to pass the firstname value along with the subscriber’s email address and subscriber key, but you won’t be required to use a “Triggered Send” Data Extensions for your sends. The downside of using this solution is that it won’t be logged unless you’re using a regular SendLog which has a column called firstname and you checked the Send Logging option in the Triggered Send Interaction.

After you’ve made all the changes in your email, go back to Email Studio > Interactions > Triggered Sends and first Publish Changes and then Restart the interaction:

Now, let’s go back to our CloudPage and add the additional property, firstname, to our script: [click here to see the code on Github]

Publish the page, and if it shows "Status":"OK", you can check your inbox for the personalized email.

Trigger an email directly from Sales/Service Cloud

NOTE: This feature will only work for Marketing Cloud Connect users and may require Administrator permissions in Sales/Service Cloud.

The process of setting up a Triggered Send in Sales /Service Cloud is similar to using Salesforce Data Entry Event in Journey Builder: we need to choose an object and filter criteria so that the application knows who to send the emails to and under which conditions. Before we configure the Triggered Send, let’s first create a test Campaign in Sales Cloud and activate it, as for the purpose of this demo we will trigger emails to Campaign Member in our test Campaign.

Now let’s make sure that Triggered Sends are enabled for use with Campaign Members in Marketing Cloud Connect. In Sales/Service Cloud, switch to the Salesforce Classic UI, navigate to the Marketing Cloud tab and click on Configure Marketing Cloud Connector. Scroll down to Triggered Sends and in the Enabled Object section, add Campaign Member to the list.

After you save the settings, once again navigate to the Marketing Cloud tab and click on Triggered Sends. In the Triggered Sends Management section, click on New. Enter a Name for your triggered send and choose an Object to use with the Triggered Sends – in this case, a Campaign Member. In Recipient Lookup, choose Lead/Contact and in Filter Criteria, provide the name of the test campaign you created earlier. Next, choose to Trigger send when a new record is created.

Now, let’s find the Email that you created earlier and choose the Send Classification, a Sender Profile if needed and a Triggered Data Extension for logging the send results. Leave the Disable Individual Level Tracking box unchecked, choose immediate Send Time and who the Error Notifications be sent to. Check the box that says I certify all of these people will have opted in and I certify all of these people will have opted in and click on Save.

Now let’s activate the Triggered Send by clicking the Activate button. You’re all set! Now add a Contact or a Lead with your email address to the test Campaign and check your inbox for the Triggered Email.

You will be able to track the results of the Triggered Send in both Sales Cloud and Marketing Cloud: in Sales Cloud, in the Marketing Cloud tab > Triggered Sends and in Marketing Cloud, in Email Studio > Tracking > Triggered Sends.

If you need more information on the above process, refer to this document: Create a Salesforce Triggered Send in Marketing Cloud Connect.

Creating an Apex trigger in Sales Cloud

If upon activating the Triggered Send in Sales/Service cloud you see the following message:

Before activating a triggered send the appropriate Apex Trigger must be created for the following object: CampaignMember

you will need to create an Apex Trigger for the CampaignMember object.

In Sales/Service Cloud, go to Setup. Under Build, click Customize and choose the relevant object – in this case, choose Campaigns > Campaign Members > Triggers. Click on New to create a new trigger, name it Trig_CampaignMember and paste the following code into the editor: [click here to see the code on Github]

Make sure, that in the Apex Trigger window, Is Active checkbox is checked and save it:

Now go back to the Marketing Cloud tab > Triggered Sends and activate your Triggered Send. Once it’s activated, you can start testing!

If you need more information about creating apex triggers for Triggered sends, read this article: Configure Objects and Triggers.


Questions? Comments?

Leave a comment below or email me at zuzanna@sfmarketing.cloud.

Update multiple non-sendable Data Extensions to be sendable

In order to programmatically update the settings of an existing Data Extension, we will interact with Salesforce Marketing Cloud’s SOAP Web Services API using WSProxy. Code snippets in this article will show you how to update a Data Extension, so that a non-sendable Data Extension is turned into a sendable Data Extension, but you can modify the script to update other properties of a Data Extension, like retention settings or field properties.

DataExtension object

The DataExtension object represents a data extension within an account. In order to turn a non-sendable Data Extension into a sendable one, we will have to update three of the object’s properties:

  • IsSendable (xsd:boolean) – Indicates whether you can use a data extension as part of an audience for a message send.
  • SendableDataExtensionField (DataExtensionField) – Indicates the field within a sendable data extension to use as an address as part of a send. Possible values include SubscriberID, CustomerKey, or EmailAddress. The application uses this field to establish a data relationship between a value specified by the SendableSubscriberField property and a value within a sendable data extension.
  • SendableSubscriberField (Attribute) – Indicates field to use as sending address. The application uses this field to establish a data relationship between a subscriber field and a value specified by the SendableDataExtensionField property.

To put it in simple words, we need to set the IsSendable value to true and establish which field in the data extension relates to the Subscriber table. If you do it from the UI, you can for example set it to “ContactKey relates to Subscribers on Subscriber Key” or “EmailAddress relates to Subscribers on Subscriber Key”. Here, we will do exactly the same thing, using the following structure:

SendableDataExtensionField relates to subscribers on SendableSubscriberField

Update Data Extension properties via WSProxy

We will use the updateItem WSProxy function to interact with the DataExtension object. Below script will identify a Data Extension by it’s External Key (CustomerKey), set the IsSendable field to true and establish the following send relationship: “email relates to Subscribers on Subscriber Key”, where email is the Data Extension field of type EmailAddress.

The possible values of the SendableSubscriberField include “Subscriber Key” or “Email Address”, depending on whether the Subscriber Key business rule has been turned on. This is the tricky part of this script, as upon retrieving the properties of a sendable Data Extension, the results will return SendableSubscriberField.Name as “_SubscriberKey”, but you actually need to use “Subscriber Key” in the update call.

Here’s the script that will update a single Data Extension:

Update multiple Data Extensions in one call

We have two possible ways to go when it comes to updating multiple Data Extensions in one call.

If you’re working with a set of Data Extensions that have an identical field name and data type assigned to the field used in the send relationship, for example email field of type EmailAddress, you can use a simple loop to iterate through an array of Data Extension External Keys:

If the field names vary across the Data Extensions, it’s best to use the updateBatch WSProxy function and define properties of each Data Extension separately:

Additional resources

If you would like to learn more about using WSProxy and the DataExtension object, I highly recommend reading Gortonington’s article, WSProxy to copy a data extension.

Retrieve tracking data since account inception using WSProxy

Whether you’re building a data warehouse or are dealing with a legal compliance issue, access to historical tracking data from your Salesforce Marketing Cloud account using the out-of-the-box features is limited.

Data Views are powerful, but only allow you to query for up to six months of subscriber and tracking information.

Tracking extracts provide granular tracking data regarding several different aspects of email send jobs, such as clicks, bounces and unsubscribes. They are not time-restricted like Data Views, but if you want to get tracking data from a specific time range, the start and end dates can only be up to 30 days apart.

The third option to access historical data is to reach out to Salesforce support and ask them for a data export for a specific time range, but this service will come at a hefty price.

The method of retrieving tracking data using SOAP API and WSProxy also has some limitations, mostly related to performance when working with big data volumes, but it can be very useful in many cases where using Data Views or Tracking Extracts is not enough.

Retrieve Tracking Data with SOAP API

To retrieve tracking data using the SOAP API, we will call the SentEvent, OpenEvent, ClickEvent, BounceEvent and UnsubEvent objects, which will allow us to create a structure similar to the one we know from Data Views. Each of those objects has a persistent set of properties, which include: BatchID, ClientID, EventDate, SendID, SubscriberKey and TriggeredSendDefinitionObjectID.

Here is an example SOAP envelope that you could use in Postman to retrieve data from any of the tracking event objects:

Interact with SOAP API using WSProxy

To retrieve data from the tracking event objects using WSProxy, we will have to specify the properties to retrieve and perform the retrieve on each of the objects we are interested in. Here’s an example of retrieving data from the SentEvent object:

We can also add a filter to narrow down the results. You can filter the results by JobID, SubscriberKey or any other object property. Here, we are going to filter by EventDate, which will show us all the events that took place in the given timeframe:

If you run the above script in an account that sends large volumes of emails, you will notice that it only pulls a part of the results and displays a status message: MoreDataAvailable. That’s because the SOAP API returns up to 2500 records at a time per retrieve call. To retrieve all available data for a given timeframe, we need to include pagination of retrieves in our script. Let’s also add an HTML table, so that data is displayed in a more accessible way:

You can do the same with the remaining objects, here are the scripts:

Write retrieved data into a Data Extension

Although this will increase the processing time of the call, it will allow you to have the data ready for further processing without any additional preparations. Below script will create a new Data Extension called 01_SentEvent in your main Data Extensions folder and insert all the retrieved data.

Here are the scripts that will do the same for the remaining objects:

Performance of the retrieve calls

The official Marketing Cloud SOAP API documentation states the following:

In most production implementations, data volume in the account requires you to include specific JobIDs in the filter criteria of the request. If requests that don’t specify a JobID time out during processing, add a JobID in the filter. If specifying a JobID is not possible, or if your implementation requires a broad range of JobIDs, use a data extract-based procedure instead.

Above is true if you are sending heavy volumes of emails each month, but you are not limited to filtering by JobID. You can include any of the object properties in filter criteria. You can, for example, retrieve all send and tracking history for a specific subscriber if you use the SubscriberKey in the filter: filter = {Property: "SubscriberKey", SimpleOperator: "equals", Value: "{{SubscriberKey}}"}

If you’re experiencing time outs when working with the script, try narrowing down the amount of retrieved data by using filters.

Security

Last but not least, always remember to put security measures into practice when setting up this kind of functionality on a CloudPage to prevent your data from being exposed in a breach. For the script to run and either return results onscreen or insert them into a Data Extension, you don’t have to actually publish the CloudPage – just press “Publish”, without further saving it. Or, to be completely safe – create a Script Activity in Automation Studio and run it from there.

Retrieve client IP address and geolocation in CloudPages

There are many reasons for checking the client IP address, most common include tracking and personalization. What can you find out about the visitors of a webpage from their IP address? You can identify their ISP, figure out approximately where they’re located and see how often they (or someone else sharing their router) visit your website.

In the context of CloudPages, we most often see IP tracking for personalization purposes. By identifying the visitor’s location, you can automatically display text in their local language and control what kind of content they see.

Identify client IP using AMPscript

The X-Forwarded-For (XFF) HTTP header is a standard header for identifying the originating IP address of a client connecting to a web server. You can easily access this header by using the AMPscript HTTPRequestHeader function, which will return a specified header from an HTTP request. Here’s how to retrieve an IP address on a CloudPage using AMPscript:

Identify client IP using SSJS

There are dedicated Server-Side JavaScript HTTP Properties Functions, that allow you to retrieve various types of HTTP Request object properties and platform application values. The client browser passes this information to the server during an HTTP interaction, so this object contains information regarding the browser and session. One of the available properties to use with the HTTP Request object is ClientIP, which returns the IP address of the requesting client as a string value. Here’s how to retrieve an IP address on a CloudPage using SSJS:

This is the preferred way to retrieve the client IP, as Request.ClientIP() is a dedicated and supported function, while using the XFF HTTP header proved to be unreliable in the past.

HTTP Properties Functions allow you to retrieve other useful information, for example, browser metadata or the URL of the referring web address.

Discover the precise physical location of a given IP address

The functionality of identifying a physical location of a given IP address requires using a third party API service. There are many IP Geolocation API providers and most of them have a free plan available, as well as paid plans for bigger enterprises. My preferred one is ipify.org, which allows you to run up to 1000 queries per month for free. Once you register, you will obtain your personal apiKey, that will be used for making the calls:

The response will contain information about the country, region, city, latitude, longitude, postal code, timezone and GeoNames Id. Additionally, it will also show autonomous system (AS) info if available.

Click here to see this script in action.

As a side-note, remember that under GDPR, IP addresses are considered personal data. Tracking the IPs of your EEA based users without their consent falls under the rules of GDPR.

Send a triggered email using WSProxy

Triggered sends allow you to automatically send personalized and timely messages to your subscribers. They are sent to an individual subscriber in response to a subscriber action. If you would like to find out more about creating a Triggered Email Message Interaction, check out my other article here.

TriggeredSend object

The TriggeredSend object represents a specific instance of a triggered email send. A typical SOAP envelope consists of the Triggered Send CustomerKey (External Key of the Triggered Email Message Interaction), Subscribers associated with the send and optionally, their Attributes. Here’s an example:

Create a TriggeredSend using WSProxy

WSProxy is a new object for Server-Side JavaScript, introduced by Salesforce in 2018. It acts as a proxy between the Marketing Cloud SOAP Web Service and SSJS. The WSProxy object is native to the platform and simpler to use than the SSJS methods, that’s why it reduces overhead and increases the speed of API calls. WSProxy uses JSON to pass object properties instead of SOAP’s native XML, which makes it much easier to read and write.

To create a new TriggeredSend, we will use the createItem WSProxy function, which has three parameters:

  • The first parameter is the object type to perform the action on – here, we will use previously mentioned TriggeredSend object
  • The second parameter is a JavaScript object which represents the fields and values to set on the object when created – in below example, we will call it tsDef and use it to define the CustomerKey, Subscribers and optionally, their Attributes
  • The third parameter is optional and includes any properties to be set using the SOAP CreateOptions object

Here’s the full script:

In the above script, you will need to provide the Triggered Send External Key and pass an Email Address and Subscriber Key.

If you are using personalization strings in your Triggered Email, you can pass them as name and value pairs in the Attributes object. Here’s an example of passing additional attributes in the WSProxy call:

The results of the call contain three properties from the SOAP CreateResult object: Status, RequestID, and Results. Here’s an example response:

To see how this script works in action, visit the CloudPage that I created and submit your email address here.

Here are additional resources to learn more about Triggered Sends and WSProxy:

Find a Data Extension and it’s folder path using SSJS

If you work with a lot of Data Extensions in multiple folders, you know the pain of using the built-in search feature. It only searches the folder that you’re in, with the exclusion of any sub-folders, which is simply impractical.

Below, I would like to show you how to find a Data Extension and it’s path using Server-Side JavaScript: which functions to use and how to iterate through them.

If you’re not interested in the technical part and want to jump straight into building the search app, click here.

SSJS Data Extension and Folder Functions

In order to find a Data Extension, we will use two SSJS Core functions: DataExtension.Retrieve and Folder.Retrieve. The first function will help us identify the Data Extension, either by its name or the External Key. The second function retrieves an array of folders based on the specified criteria. It also retrieves details regarding the parent folder of a given folder if one exists, and that’s what we will use to build the Data Extension path.

In the above script, you will need to define two variables, DEprop and DEval, depending on whether you have the name or the External Key of the Data Extension. You can paste the script on a CloudPage, define the two variables, and once you click on the “Publish” button, the script will be executed and results will be visible on-screen, without the need to actually publish the CloudPage:

Create a simple Data Extension search app

If users of your instance of Salesforce Marketing Cloud often struggle with finding Data Extensions, you can create an app on a CloudPage to help them navigate the folders. By adding a simple form, you can enable the users to perform a search from a CloudPage:

Paste the below code onto a CloudPage and publish it. Anyone with access to the link will be able to search for Data Extensions located in the Business Unit where the CloudPage was created. If the CloudPage has been created in the Parent Business Unit, they will also be able to search the Shared Data Extensions folder.

Last but not least, always remember to put security measures into practice when setting up this kind of functionality on a CloudPage, to prevent your data from being exposed in a breach.

PS. There is a free tool available called SFMC Object Finder. It’s a Chrome extension that lets you search for a variety of objects, like images, data extensions and other assets, in your instance of Salesforce Marketing Cloud. All it requires is that you have an active session of Salesforce Marketing Cloud open in your Chrome browser. However, if you cannot install this extension or need to be able to find Data Extensions programmatically, you can use Server-Side JavaScript to achieve it!

Troubleshooting Journey Builder Integration with Debug Logs

If you have a Salesforce Data Entry Event set up in Journey Builder that is not injecting records as intended, it is necessary to look into Debug Logs to determine what the cause may be. Here is a step-by-step guide to setting up and initiating Debug Logs for Journey Builder. The whole process takes part in Sales/Service Cloud, so ensure you have Admin permissions to be able to access Setup.

Turn on Marketing Cloud Connect request logs

  1. In Sales/Service Cloud, click on the Marketing Cloud tab in the Salesforce Classic interface
  2. Click Configure Marketing Cloud Connector on the right
  3. In the Notifications section, click Initiate Log

Turn on Debug Logs

  1. Go to Setup and in the Quick Find, search for Debug Logs
  2. In the User Trace Flag section, click New
  3. Set Traced Entity Type = User
  1. Set Traced Entity Name to the user that will be creating/updating the record which fails being injected into the journey
  2. Start and Expiration Dates should be set to start immediately and expire in the next hour, as the Marketing Cloud Connect activities can be logged only for 60 minutes at a time
  3. Click New Debug Level
  4. Give it a name and make sure there are no spaces
  1. In the level settings, set Workflow = Finer and Callouts = Finest, leave the rest of the settings as they are and Save
  1. Click Save and make sure your new Debug Level shows up in the Debug Level field, then click Save.

Create/Update your record

Now make sure that you are logged in as the User who was earlier selected in the Traced Entity Name and create or update your record so that it meets the entry criteria of the Journey. Document the date and time when the record was created or updated, and the ID of the record.

Parse the Debug Logs

Go to Setup and in the Quick Find, search for Debug Logs. The Debug Logs will be below the User Trace Flags section. There might be several logs, depending on the actions of the user, so having the timestamp that the record was created or updated will help narrow it down.

Use the View link to open the log(s) in a new tab. Search for JBSystemFlow_(object name) (i.e. JBSystemFlow_Contact). If you get a result, check the subsequent line with new_object in it to see if the Id lines up with the record created earlier, that should have been injected into the Journey. If so, you have found the correct log.

Next, search for a line that starts with FLOW_START_INTERVIEW_BEGIN and has JBSystemFlow_(Object). The lines between this and FLOW_START_INTERVIEW_END…JBSystemFlow_(Object) will give you all of the data in the fields for the new and old objects (old objects will only have data if a record is updated), plus all of the Decisions that the record went through in the associated Flow, plus the results for each outcome.

Here’s an example:

16:23:37.142 (150821372)|FLOW_START_INTERVIEW_BEGIN|8557980ff96eae468bac8498796162d5d1cd65-6ef6|JBSystemFlow_Lead
16:23:37.142  16:23:37.142
(151572909)|FLOW_ELEMENT_BEGIN|8557980ff96eae468bac8498796162d5d1cd65-6ef6|FlowDecision|JB_SalesforceObj2d95094362a61233d7e7d05c70c5bd3a_Decision 16:23:37.142
(151749528)|FLOW_RULE_DETAIL|8557980ff96eae468bac8498796162d5d1cd65-6ef6|SalesforceObj2d95094362a61233 (151839236)|FLOW_START_INTERVIEW_END|8557980ff96eae468bac8498796162d5d1cd65-6ef6|JBSystemFlow_Lead

Each FLOW_ELEMENT_BEGIN/FLOW_ELEMENT_END and rows in between will show information about specific Decisions in the Flow, their Outcomes, and the results of those Outcomes.

Under the FLOW_START_INTERVIEW_END…JBSystemFlow(Object) line, if everything went well with the Flow, you will see a line for FLOW_BULK_ELEMENT_BEGIN…FireJBTrigger. If you see this line, unless there are several errors directly below, Salesforce ran the APEX job to fire the Entry Event. At this point, you would check the Marketing Cloud Connect logs to see if there was an issue with completing the call (click here to skip to this section of the article). 

If you don’t see the FireJBTrigger line, copy the lines with decisions to a notepad.

Find the Flow associated with your Journey

Go to Setup, search for Flows in the Quick Find box and click Flows. Click Open next to the name of the Flow from the Debug Log (i.e. JBSystemFlow_Lead).

Once the page loads, you can drag the different nodes out to see the flow of the Decisions and Jobs. Note, that there is one Flow per Object that has at least one active Journey, which means there can be multiple Journeys referenced in one flow if more than one Journey is active on that Object. If you have a few different Journeys using, for example, Lead as the Object, you will have Decisions for the separate criteria. Sometimes a record may not pass the criteria for the Journey you are looking at but will continue through the Flow to see if it meets the criteria of other Journeys.

You will now have to find a Decision, where Name in the Decision lines up with the name at the end of the FLOW_ELEMENT_BEGIN. The second line after FLOW_ELEMENT_BEGIN, FLOW_RULE_DETAIL, will give you the name of the Outcome, and the results. The details of the Outcomes show up in the lower half of the Decision properties in the Flow. Focus only on the lines from the Debug Log that end with false, as those are the ones that did not pass the criteria. Here’s an example:

16:23:37.142 (151572909)|FLOW_ELEMENT_BEGIN|8557980ff96eae468bac8498796162d5d1cd65-6ef6|FlowDecision|JB_SalesforceObj2d95094362a61233d7e7d05c70c5bd3a_Decision 16:23:37.142 (151749528)|FLOW_RULE_DETAIL|8557980ff96eae468bac8498796162d5d1cd65-6ef6|SalesforceObj2d95094362a61233d7e7d05c70c5bd3a|false 

Once you are able to match the Decision Outcome(s) from the Flow canvas with the Decision(s) from Debug Log, you will be able to find out which criteria were not met and why the Flow stopped without injecting the record to the Journey.

Check the Marketing Cloud Connect logs

  1. Navigate to the Documents object in the Salesforce Classic interface
  2. Select the Marketing Cloud Documents folder
  3. Click Go!
  1. To view the log, select the log with the corresponding date

In the log, look out for any exceptions, here’s an example:

*****2019-10-08 09:51:24.680|LOG STARTED 
*****2019-10-08 09:51:24.680|PROCESS|SecureSettingsManager 
*****2019-10-08 09:51:24.868|EXCEPTION|Expired access/refresh token.

If you see an exception related to the access token, resolve it by clearing out User Mappings and OAuth Tokens in Marketing Cloud Connect.

If there are other exceptions or errors visible in the log which you are unable to interpret yourself, it’s best to send them over to Support and ask for assistance in further troubleshooting.

Sources: https://help.salesforce.com/articleView?id=000314336&type=1&mode=1, https://help.salesforce.com/articleView?id=000314349&language=en_US&type=1&mode=1, https://help.salesforce.com/articleView?id=mc_co_initiate_log.htm&type=5, https://help.salesforce.com/articleView?id=000320849&type=1&mode=1