Retrieve tracking data since account inception using WSProxy

Whether you’re building a data warehouse or are dealing with a legal compliance issue, access to historical tracking data from your Salesforce Marketing Cloud account using the out-of-the-box features is limited.

Data Views are powerful, but only allow you to query for up to six months of subscriber and tracking information.

Tracking extracts provide granular tracking data regarding several different aspects of email send jobs, such as clicks, bounces and unsubscribes. They are not time-restricted like Data Views, but if you want to get tracking data from a specific time range, the start and end dates can only be up to 30 days apart.

The third option to access historical data is to reach out to Salesforce support and ask them for a data export for a specific time range, but this service will come at a hefty price.

The method of retrieving tracking data using SOAP API and WSProxy also has some limitations, mostly related to performance when working with big data volumes, but it can be very useful in many cases where using Data Views or Tracking Extracts is not enough.

Retrieve Tracking Data with SOAP API

To retrieve tracking data using the SOAP API, we will call the SentEvent, OpenEvent, ClickEvent, BounceEvent and UnsubEvent objects, which will allow us to create a structure similar to the one we know from Data Views. Each of those objects has a persistent set of properties, which include: BatchID, ClientID, EventDate, SendID, SubscriberKey and TriggeredSendDefinitionObjectID.

Here is an example SOAP envelope that you could use in Postman to retrieve data from any of the tracking event objects:

Interact with SOAP API using WSProxy

To retrieve data from the tracking event objects using WSProxy, we will have to specify the properties to retrieve and perform the retrieve on each of the objects we are interested in. Here’s an example of retrieving data from the SentEvent object:

We can also add a filter to narrow down the results. You can filter the results by JobID, SubscriberKey or any other object property. Here, we are going to filter by EventDate, which will show us all the events that took place in the given timeframe:

If you run the above script in an account that sends large volumes of emails, you will notice that it only pulls a part of the results and displays a status message: MoreDataAvailable. That’s because the SOAP API returns up to 2500 records at a time per retrieve call. To retrieve all available data for a given timeframe, we need to include pagination of retrieves in our script. Let’s also add an HTML table, so that data is displayed in a more accessible way:

You can do the same with the remaining objects, here are the scripts:

Write retrieved data into a Data Extension

Although this will increase the processing time of the call, it will allow you to have the data ready for further processing without any additional preparations. Below script will create a new Data Extension called 01_SentEvent in your main Data Extensions folder and insert all the retrieved data.

Here are the scripts that will do the same for the remaining objects:

Performance of the retrieve calls

The official Marketing Cloud SOAP API documentation states the following:

In most production implementations, data volume in the account requires you to include specific JobIDs in the filter criteria of the request. If requests that don’t specify a JobID time out during processing, add a JobID in the filter. If specifying a JobID is not possible, or if your implementation requires a broad range of JobIDs, use a data extract-based procedure instead.

Above is true if you are sending heavy volumes of emails each month, but you are not limited to filtering by JobID. You can include any of the object properties in filter criteria. You can, for example, retrieve all send and tracking history for a specific subscriber if you use the SubscriberKey in the filter: filter = {Property: "SubscriberKey", SimpleOperator: "equals", Value: "{{SubscriberKey}}"}

If you’re experiencing time outs when working with the script, try narrowing down the amount of retrieved data by using filters.

Security

Last but not least, always remember to put security measures into practice when setting up this kind of functionality on a CloudPage to prevent your data from being exposed in a breach. For the script to run and either return results onscreen or insert them into a Data Extension, you don’t have to actually publish the CloudPage – just press “Publish”, without further saving it. Or, to be completely safe – create a Script Activity in Automation Studio and run it from there.

Unsubscribe and Log an UnsubEvent with a LogUnsubEvent Execute Call

The LogUnsubEvent provides a convenient way of handling unsubscribes when you create your own unsubscribe landing page or profile center functionality. This call allows you to unsubscribe a subscriber and log an UnsubEvent that is tracked against a specific Job, which means that you will be able to track from which email they unsubscribed and see the results on the tracking dashboard. You can configure the call to either unsubscribe a subscriber from a specific list, publication list, or all subscribers, which will effectively unsubscribe them from receiving any emails.

What’s more important, if you’re using Marketing Cloud Connect to connect with your Salesforce Sales or Service Cloud org, the unsubscribe will be captured on the subscriber record in Sales/Service Cloud. It will automatically check the Email Opt Out (HasOptedOutOfEmail) flag and will also be visible in the Individual Email Results for the corresponding email send.

The LogUnsubEvent Execute call uses the following parameters:

  • SubscriberID – The Marketing Cloud generated ID that uniquely identifies a subscriber.
  • SubscriberKey – The client supplied ID that uniquely identifies a subscriber.
  • EmailAddress – The email address of the subscriber.
  • JobID – The ID of the Job that sent the message.
  • ListID – The ID of the List that the subscriber belonged to. You can use subscriber or publication lists (not suppression lists).
  • BatchID – The ID of the Batch within the Job.
  • Reason – (Optional) The reason the subscriber is being unsubscribed.

The parameters can be divided into 3 sections:

  1. Subscriber context
  2. Job context
  3. Unsub reason

If you make this call from the parent unit of an Enterprise 2.0 account, ensure that you include the ClientID of the child business account to return information specific to that business unit.

Subscriber Context

The Subscriber Context is defined by the SubscriberID, SubscriberKey and EmailAddress parameters. You must supply at least one of these parameters. If you provide more than one of these parameters, we retrieve the Subscriber using one of the values and validate that the other values match the retrieved Subscriber. If they don’t match, an error returns.

If the SubscriberKey permission is turned on and you supply the EmailAddress parameter, you must supply either the SubscriberID or the SubscriberKey.

Job Context

The Job Context is defined by the JobID, ListID and BatchID parameters. These values are used to determine which Job the UnsubEvent is tracked against. The subscriber is also unsubscribed from the List that the Job was sent to. You don’t need to supply all three values. The system looks up any missing values using the following rules:

  1. If the JobID is supplied, we can lookup a missing ListID and/or BatchID.
  2. If the ListID is supplied, we can lookup a missing JobID and/or BatchID.
    1. If the JobID is missing, we use the most recent JobID that the subscriber was sent to.
    2. This may not be the Job that the Subscriber is acting upon.
  3. If only the BatchID is supplied, we cannot lookup the remaining information and the job context is not defined.

If the job context cannot be established because you did not supply any of these parameters or only supplied the BatchID, the UnsubEvent is not created. The subscriber is also Master Unsubscribed from the system of being unsubscribed from a particular list. Remove the ListID to address the All Subscribers list in an account.

Unsub Reason

This is used to specify the reason the subscriber is being unsubscribed from the system. If the reason is not supplied, the default value is used: Unsubscribed via Log Unsub Event Execute call.

Here is an example SOAP Request envelope:

We will now look at three different ways to implement this solution on a CloudPage.

LogUnsubEvent using AMPscript

This is probably the most common way to use the LogUnsubEvent call. Below script will retrieve the SubscriberKey, JobId, ListId and BatchId if you link to the unsubscribe page from your email using the CloudPagesURL function. Depending on whether a list/publication list was selected for the send, it will unsubscribe a subscriber from that particular list, or it will unsubscribe the subscriber from all emails if you used All Subscribers.

LogUnsubEvent using Server-Side JavaScript

The implementation of the LogUnsubEvent call in SSJS will be almost identical to the AMPscript solution, as the methods for accessing SOAP object data with SSJS are primarily wrappers around AMPScript functions.

Like in the previous example, the script will retrieve the SubscriberKey, JobId, ListId and BatchId if you link to the unsubscribe page from your email using the CloudPagesURL function and will unsubscribe a subscriber either from a list or All Subscribers, depending on which one you use at send time.

LogUnsubEvent using WSProxy

WSProxy is a new object for Server-Side JavaScript, introduced by Salesforce in 2018. It acts as a proxy between the Marketing Cloud SOAP Web Service and SSJS. The WSProxy object is native to the platform and simpler to use than the SSJS methods. The object reduces overhead and increases the speed of your API calls.

Therefore, the below script is the most simple, and what’s more important, the fastest way to execute the LogUnsubEvent API call. In a speed test ran using the Chrome DevTools, it proved to be the fastest one to load on a CloudPage, with AMPscript just slightly slower and SSJS the slowest one, taking twice as much time to load.

Just like in the previous examples, the script will retrieve the SubscriberKey, JobId, ListId and BatchId if you link to the unsubscribe page from your email using the CloudPagesURL function and will unsubscribe a subscriber either from a list or All Subscribers, depending on which one you use at send time.

If you would like to find out more about logging an UnsubEvent or WSProxy, check out the following articles:

Salesforce Marketing Cloud API Authentication using Server-Side JavaScript

In order to interact with Salesforce Marketing Cloud APIs you need to create an installed package in your Marketing Cloud account first. Marketing Cloud uses installed packages to help authenticate users and API requests. To create and install packages you must have the Administrator or Marketing Cloud Administrator role assigned to your profile.

Marketing Cloud has two types of installed packages: packages with enhanced functionality (v2) and packages with legacy functionality (v1). If you’re not sure which package has been installed in your account, check the “Details” tab of your installed package – all legacy packages have a banner at the top indicating that it’s a legacy package and a Licenses tab. Enhanced packages have an Access tab.

Note the Authentication Base URI , Client Id and Client Secret as you will need them to authenticate. Below code examples are meant for Server-to-Server Integrations with Client Credentials Grant Type.

Request access token for a Legacy Package

The following script will allow you to authenticate if you’re using the Legacy Package. As of August 1, 2019, Marketing Cloud has removed the ability to create legacy packages, but you can still use legacy authentication and API requests with existing legacy packages.

You will need to insert the Client Id and Client Secret and your tenant-specific authentication endpoint (Authentication Base URI). Bare in mind, that it is not a good idea to publish Client credentials on a CloudPage.

The access token will be valid for 60 minutes and it is issued with the scopes specified on the API integration in Installed Packages. With Legacy Packages, the access token can only be used in the context of the business unit that created the integration.

Request access token for Enhanced Packages

For this package, you will need to insert the Client Id and Client Secret, your tenant-specific authentication endpoint (Authentication Base URI) and define grant_type as "client_credentials" for server-to-server integrations. You can also specify two optional parameters, scope and account_id. If you don’t include the scope parameter in the request, the token is issued with the scopes specified on the API integration in Installed Packages. For a full list of permissions click here. Account ID is the MID of the target business unit. Use this parameter to switch between business units. If you don’t specify account_id, the returned access token is in the context of the business unit that created the integration. Again, bare in mind, that it is not a good idea to publish Client credentials on a CloudPage.

The access token has a lifetime of 20 minutes. In the response you will also see the scope, rest_instance_url (your tenant’s REST base URL for making REST API calls) and soap_instance_url (your tenant’s SOAP base URL for making SOAP API calls).

Token type will always be “Bearer”, regardless of the package you are using.

To learn more about Marketing Cloud APIs, visit the Trailhead module Marketing Cloud APIs and the official documentation: Intro to Marketing Cloud APIs.

Send a triggered email using AMPscript

Triggered sends allow you to automatically send personalized and timely messages to your subscribers. They are sent to an individual subscriber in response to a subscriber action. Common use cases include sending welcome emails to new subscribers, service case notifications or post-purchase thank you notes to clients. They can be personalized just as any other email sent from Salesforce Marketing Cloud and triggered in various ways, among others, using AMPscript. You can read more about Triggered Sends in Salesforce Marketing Cloud here: Triggered Emails.

Create a Triggered Email Message Interaction

In order to create a Triggered Email Message Interaction, we need all the elements that we would use for a user-initiated email: an email, a sender profile and a list for managing subscriber statuses. Once you have all of the above ready, you can create a Triggered Send:

  • Go to Email Studio > Interactions > Triggered Sends
  • Click on Create
  • Provide a name, select your sender profile, email and list used for subscriber management
  • Save your Triggered Send
  • Check the checkbox on the list of your Triggered Sends and click on “Start/Restart”

Now copy the “External Key” of the Triggered Send you just created, you will need to insert it into the script in the next step.

Trigger the email using AMPscript

In order to trigger the email, we will use the InvokeCreate function, which invokes the Create method on an API object. The following script uses the CreateObject function to create the TriggeredSend and TriggeredSendDefinition objects as @ts and @tsDef, and to create a new Subscriber record as @ts_sub. We will specify the attributes of the Triggered Send object and the Subscriber object using the SetObjectProperty function. The AddObjectArrayItem function is used to add the object attributes, and everything is then passed into the InvokeCreate function. The InvokeCreate function returns a status code and a status message, by which exception handling can be built using the RaiseError function.

In the above script, replace ExternalKey with the External Key of the Triggered Send you created earlier and replace email@example.com with your email address. You are now ready to run this script on a CloudPage by publishing it. Upon success, you should be able to see the following values:

Triggered Send status code: OK
Triggered Send status message: Created TriggeredSend
Triggered Send error code: 0

The above script is simplified, as it uses the email address of a subscriber as the subscriber key. If you would like to pass the subscriber key separately, you will need to add one more variable @ts_subkey and change the SetObjectProperty for the Subscriber record:

To see how this script works in action, visit the CloudPage that I created and submit your email address here.

The delivery speed for Triggered Send emails is usually quicker than for user-initiated emails, so if the email doesn’t make it into your inbox in a couple of minutes, check your spam folder.

Here are additional resources to learn more about Triggered Sends: