In this guide, we’ll show you how to perform a sentiment analysis of your Marketo emails using ChatGPT, leading to enhanced content that resonates with your audience and improves open rates, click-through rates, and conversions.
And if you’ve been wondering how to safely integrate AI into your marketing tasks, this is a great starting point that is relatively straightforward to set up.
For those who may not know, sentiment analysis is a natural language processing technique used to determine the emotional tone behind a text. Through text data analysis, this process can identify whether the sentiment expressed is positive, negative, or neutral.
For marketers, a deep understanding of audience sentiment can provide clues about what type of content elicits positive reactions from recipients. By leveraging these insights, you can improve your content strategy and engage your audience more effectively.
This guide is geared towards Marketo users, but if you already have email data – including body text and subject lines – from a different platform exported as a CSV, skip to step 4 to see where ChatGPT comes in.
Let’s get into it!
We’ll get things started by navigating over to the “Reports” section in Marketo. We want to export an email performance report of emails that are regularly delivered to the same database – think monthly newsletters, loyalty program updates, seasonal sale events, etc.
By using emails delivered to a similar audience over a longer period, we eliminate as many variables as possible, allowing us to measure how shifts in tone and content change engagement metrics.
We should now have an email performance report exported as a CSV file from Marketo that includes open rates, click-through rates, bounce rates, and several other fields.
In this step, we’ll be using the Marketo API to add and manipulate some of the information in our CSV file.
Why are we doing this?
In short, for each email we’re isolating the “Email Name” and “Email Program”, then using that information to fetch the “Email ID”. The “Email ID” will then be used in the next step to fetch the subject line and body text of each email, which we’ll then upload to ChatGPT for sentiment analysis.
If this sounds complicated, don’t worry. It’s relatively straightforward once we break it down.
Note: The Marketo API is a way for us to programmatically talk to Marketo to enable the automation of tasks, integrate with other systems, and in our case, retrieve data from the Marketo platform.
1) In our current CSV file, the first column titled “Email Name” has a bunch of consolidated information, including the email name and the email program. The problem is, we need to split this information into two separate, dedicated columns: One for the “Email Name” and one for the “Email Program.”
Use the following code snippets to do this:
Initial Setup
import pandas as pd
import json
import requests
base_url = 'https://MUNCHKINID.mktorest.com'
client_id = 'YOUR-CLIENT-ID'
client_secret = 'YOUR-CLIENT-SECRET'
def getToken ():
response = requests.get(base_url+'/identity/oauth/token?grant_type=client_credentials&client_id='+client_id+'&client_secret='+client_secret)
temp = json.loads(response.text)
token = temp['access_token']
return token
df=pd.read_excel('YOUR-FILE-PATH.xlsx')
df.drop(['First Activity (EDT)','Last Activity (EDT)'],axis=1,inplace=True)
Get the Program and Email Name
df[['Program','Email']] = df['Email Name'].str.split('.',expand=True)
2) Once those are split into two columns, we can perform an API call that will tell Marketo to use “Email Name” and “Email Program” to fetch the “Email ID”.
Use this code snippet to do that:
Get the Email ID
def getProgramID(programName):
token=getToken()
response = requests.get(base_url+'/rest/asset/v1/program/byName.json?name='+programName+'&access_token='+token)
return json.loads(response.text)['result'][0]['id']
def getFolder(programID):
token=getToken()
response = requests.get(base_url+'/rest/asset/v1/folders.json?root={"id":'+str(programID)+',"type":"Program"}&access_token='+token)
return json.loads(response.text)['result'][0]['folderId']
def getEmails(folderID):
token=getToken()
response = requests.get(base_url+'/rest/asset/v1/emails.json?folder='+str(folderID)+'&access_token='+token)
return json.loads(response.text)['result']
def getEmailID(emailName,programName):
emails=getEmails(getFolder(getProgramID(programName)))
for email in emails:
if email['name']==emailName:
return email['id']
df['EmailID']=df.apply(lambda x: getEmailID(x.Email, x.Program), axis=1)
Now that we have the “Email ID” for each email, we’ll use the Marketo API to download all the subject lines and body text data.
Here are the code snippets you’ll need to do this:
1) Get Email Subject Lines
def getEmailSubject(emailID):
token=getToken()
response = requests.get(base_url+'/rest/asset/v1/email/'+str(emailID)+'.json?access_token='+token)
return json.loads(response.text)['result'][0]['subject']['value']
df['EmailSubject']=df.apply(lambda x: getEmailSubject(x.EmailID), axis=1)
2) Get Email Body Text
def getEmailText(emailID):
token=getToken()
response = requests.get(base_url+'/rest/asset/v1/email/'+str(emailID)+'/fullContent.json?type=Text&access_token='+token)
return json.loads(response.text)['result'][0]['content']
df['EmailText']=df.apply(lambda x: getEmailText(x.EmailID), axis=1)
3) Lastly, we must save the final results into an updated CSV file using this code:
df.to_excel(‘YOUR-FILE-PATH.xlsx',index=False)
At this point, your CSV file should now have engagement metrics, body text, and subject lines for every email.
This is where the real magic happens!
With our data ready, we can now use ChatGPT to perform a comprehensive sentiment analysis.
ChatGPT-4o can do this because of its enhanced language understanding, improved natural language processing, and advanced data analytics feature that can create code and assess specific parts of data that you upload.
Note: You’ll need a Chat-GPT Plus subscription for 20 USD per month to upload your CSV for sentiment analysis.
1) Go over to ChatGPT in your browser, press the “Attach file” button, and upload your email data CSV.
2) Prompt ChatGPT to perform a sentiment analysis. Here’s an example:
You are a Marketing Data Analyst at company X that does X. Most of our audience is X. Your job is to analyze the data from our Marketing emails and answer the following questions: 1. For a sentiment analysis: Which type of subject lines result in a higher open rate? Which type of content leads to a higher click rate? 2. Which words in the subject line result in a higher open rate? Which words in the body content result in a higher click rate? And which words result in a lower open and click rate? Remove the URL-related words. 3. Which topics lead to higher open and click rates? And which topics lead to lower ones?
The last step is to apply the trends and insights provided by ChatGPT’s sentiment analysis to improve the effectiveness of your content.
ChatGPT does a pretty good job of contextualizing sentiment scores by explaining parameters and categories clearly, so the interpretation process should be relatively straightforward – but ultimately, it’s up to you and your team to tailor and refine your email subject lines and body content accordingly.
It’s also important to constantly measure and update your content strategy, as well as reanalyze sentiment with ChatGPT as new data and feedback come in.
Leveraging ChatGPT for email sentiment analysis is a perfect starting point for anyone looking to take advantage of AI to improve their content strategy.
You’ll gain instant insights into what kind of messaging and tone is resonating with your audience effectively, and where you need to change things up to improve engagement.
Remember to regularly reassess your content with fresh data and sentiment analysis to stay aligned with your audience’s evolving preferences – view this process as one of continuous refinement over time.
If you need help setting this up, or if you want to learn about other ways we’re using AI to enhance marketing strategies, send us a message here.
