Author: rpadmin

This blog post will show you how you can extract and analyze data from your Marketo instance using the example of email data so that we can answer the age-old question:

When is the best time to send emails?

The truth is that the best send time is not one size fits all because your email recipients are in different geographies and have different schedules. The best time to send an email to a person is unique to that person and in this blog post, we will show you how to use a person’s historic email interaction and ChatGPT to tell when is the best time to email them.

1. Introduction
2. Bulk Extract Vs Rest API
3. Gathering Historic Email Activity
• Using the Bulk Extract API
• Using the REST API
• Processing the Data
4. Finding Best Send Time Using ChatGPT4
5. Using the Best Send Time in Marketo
• Uploading Best Send Times to Marketo
• Sending Emails at the Best Send Time
• A/B Testing Email Performance
6. What Marketo Data Is Next?

 

Introduction

There are two ways to extract the historic email activity in your database. Both of these methods will use the Marketo API so if you have not used the API before then you should check out the Marketo API Quick Start Guide to learn how to:

1. Get your authentication credentials
2. Make your first requests using the free Postman testing tool
3. Transition from making requests in Postman to a programming language

As discussed below the best method for your Marketo instance will depend on the available capacity you have as well as the email activity you are looking to extract and over what time frame. Marketo’s data retention limits for email activity are as follows:

• Email Delivered: 90 days
• Send Email: 90 Days
• Click Email: 25 Months
• Open Email: 25 Months
• Unsubscribed Email: 25 Months

Obviously the more email activities in your dataset and the longer the time frame you look over the more API calls or MB you will use to extract this data from Marketo.

In the project for this blog post, it was decided to extract email delivered and email opened data since the time emails are sent will affect when emails are opened, and giving ChatGPT this extra delivery information might give more accurate results than just sending open times alone. This meant that the “Email Delivered” activity limited the furthest we could look back in time at 90 days.

N.B. While the intention was for ChatGPT to use the delivered and opened times to make its decision, as we will see later on in the “Finding Best Send Time Using ChatGPT4” section it only used the opened times when doing its analysis and gave the most frequent hour of opening as the best send time. Therefore, you can only give ChatGPT a dataset of open times or you can refine your prompt to ensure that ChatGPT’s approach also factors in the delivered times and does a more advanced analysis.

 

Bulk extract vs rest API

The first approach uses the bulk extract API since this is the best method for extracting large amounts of data from Marketo. If you have not used the bulk extract API before then check out the Marketo Bulk API Quick Start Guide to understand the job workflow used to extract data from Marketo.

In Marketo, you will usually have a 500MB daily extraction limit unless you have paid for an increase in this limit. The limitation to using the Bulk extract API is that you may exceed this 500MB limit if you have a large database and lots of email activity. If this is the case then take a look at whether you have sufficient REST API call quota to use the REST API to extract this email activity.

The default REST API quota in Marketo is 50k API calls per day so see if this is enough for you to be able to extract the email data you need.

In order to help you determine which approach is best go to Admin > Web Services. Here you will see your REST API and Bulk API limits as well as your usage for both over the past 7 days. Clicking into the hyperlinked numbers beside “Requests in the Last 7 Days” and “Capacity Used in the Last 7 Days in MB” will show you day-over-day usage over the past 7 days.

Based on what these numbers show you will see whether you have more REST API or Bulk Extract capacity or if you are already very close to your limits with both then you might have to contact Marketo support to get an increase in capacity or reduce the number of calls or jobs currently being created to free up capacity.

It is hard to predict ahead of time how many API calls or MB will be needed to extract the email activity history for your database so it will take some experimentation to find out how much you need by making some API calls or creating some jobs for a subset of your database or over a short time frame. Then once you have done these small experiments you can extrapolate the API call or MB usage expected for your entire database or time frame of interest.

 

Gathering historic email activity

Whether you intend to use the REST API or the bulk extract API you will need the activity IDs for the email activities you are interested in. To get these:

• Make a GET request to the  Activity Types endpoint
• Search the response returned to Postman for each activity and note down the id

Once you have these IDs check out the Github files linked below to see how to use these IDs in API requests to extract the email activity we are interested in.

Note that the code in all the Github files linked below is designed to be plug-and-play so that once you update the variables at the top of your code with your information then you should be able to run the code right away.

 

Using the bulk extract API

As mentioned in the docs the maximum time span for a job is 31 days so if we want to get data over a longer time frame e.g. 90 days, then we will need to create multiple jobs and then join the data from each job together. The Bulk Extract code uses a number of functions to do just that.

 

Using the REST API

The REST API code works similarly to the bulk extract code except here it is data from “pages” that are being joined together instead of data from jobs. Since only a limited amount of data can be returned in a single REST API request, the data is paginated and we must loop through all the pages to extract all the data.

To begin a request is made to the paging token endpoint, specifying the start date in the past that we would like to extract data from. The paging token that is returned is then used as the starting point for a while loop that makes requests to the activities endpoint to get the activity data from each page.

As the while loop is iterating, the data returned for each page is added to a list and then this list is converted to a dataframe and saved as a CSV.

 

Processing the data

Once all the data over the desired timeframe is gathered some processing of the dataset is done to remove bot activity and tidy up the dataset to make it easier to process later. You can see how the data is gradually transformed by going through the leftmost tab to the rightmost tab in this sheet. You can see the code used to do this processing here.

 

Finding best send time using ChatGPT4

By far the easiest way to find the best send time for each lead ID is to use the “Advanced Data Analysis” option with ChatGPT-4. In order to access ChatGPT-4 you need to have a paid subscription. Then to enable “Advanced Data Analysis” from the chat screen:

1. Click on the 3 dots beside your profile image and name at the bottom left-hand corner of the screen
2. Click on “Settings & Beta”
3. Click on “Beta features”
4. Turn the toggle beside “Advanced Data Analysis” on

This “Advanced Data Analysis” feature allows us to upload our CSV of email activity and ask ChatGPT to give us an output file containing the best send time for each lead id. To analyze a file with ChatGPT-4:

1. Click on the “GPT-4” tile at the top of the screen
2. Click “Advanced Data Analysis” in the dropdown menu that appears
3. Click on the plus icon to the left of “Send a message” to upload your file
4. Enter your prompt

Based on my testing and with the prompt I used below, I found that the best format for the input data was grouping the email activity by lead ID and email as shown in the Group by Lead & Email tab. Note that the times returned from the Marketo API are in UTC time and we need to ask ChatGPT to give us the output time in the same timezone as our Marketo instance.

 

"Given the data which contains the delivered and open times for the emails received by each lead please determine the best time of day to email these leads to maximize the chances that they will open future emails. Note that the times are in UTC time. Provide the output in a table containing 2 columns: 1 for lead Id and 1 for an optimum send time in CDT where the times should be to the nearest hour as a digit from 1-24."

 

Once you enter your prompt and the input file ChatGPT will start analyzing the data and print out the methodology that it is using to transform and analyze the data as it is going along.

As shown in this chat using the above prompt ChatGPT actually does a very simple analysis to get the best send time where it returns the hour in which someone most frequently opened emails. If this is all you are looking for you do not even need to use ChatGPT for this, you could use Python code or Google Sheets formulas to compute this best send time for yourself.

Therefore if you want ChatGPT to do a more complex analysis and take the email delivery times into account for its analysis then you will have to refine your prompt and ask ChatGPT to make adjustments in subsequent messages as you see the approach it is taking with the analysis.

As I show you can ask questions based on the input or output data, for example, I ask ChatGPT to give me a pivot table of best send times so I can see how the best send times are distributed across the output dataset. I also asked ChatGPT why so many lead IDs were missing from the output dataset and it answered that leads with no open times were not included in the output.

Once you are happy with the analysis that ChatGPT has done you can then ask ChatGPT to export the output dataset as a CSV. Next, take a look at the “Using the Best Send Time in Marketo” section to see how we can get these send times into Marketo and start using them to optimize our email open rates.

 

Using the best send time in Marketo

You might have noticed that the Marketo API associates the email activity with leadID so that the output dataset of best send times is for lead IDs. However, if we want to do a list upload to Marketo to populate the best send time for these leads we need to have the best send time for each email address instead.

 

Uploading Best Send Times to Marketo

In order to get these best send times into Marketo for each lead follow these steps:

1. The first step of getting the best send time for each lead into Marketo is to go to a smart list that contains every email address in your Marketo instance e.g. using the “Email Address” is not empty filter.
2. Then go to a view or create a view that contains only leadID and Email Address
3. Export this view with these 2 fields only
4. Return to the ChatGPT-4 window where you have the chat with the output dataset containing the best send time for each lead ID
5. Attach the exported dataset from Marketo containing the leadID and email address and then ask ChatGPT to join the best send time dataset with this dataset on lead id (as shown in this chat) so that we get an output dataset containing lead ID, Email Address, and Best Send Time
6. Download this dataset as a CSV
7. Upload the CSV of Emails and Best Send Times to a static list in the “Group Lists” folder in the “Database” section

 

Sending emails at the best send time

Once the best send time has been uploaded for each email address you are then ready to build a smart campaign to send emails to each person at their unique best time. Being honest building the flow for this smart campaign is a pain because it consists of 24 “Send Email” steps and 23 “Wait” steps.

The smart campaign is scheduled to run at midnight on the day that we want the email to be sent. Therefore the very first action is to send the email to anyone whose best send time is “0”. Then we wait for 45mins with the wait ending at 1am before sending the next email to anyone whose best send time is “1am”. These actions are then repeated for each of the remaining hours in the 24-hour clock.

 

A/B testing email performance

So long as you have this smart campaign nested within an email program in Marketo and you have the email selected in the “Control Panel” you will still be able to see the email performance in the program “Dashboard”. But what if you want to A/B test to see how sending at the best send time performs compared to sending to everyone at the same time?

In order to A/B test performance of the best send time:

1. Create a smart list containing your usual email send criteria along with an extra “Best Send Time is not empty” condition
2. Export this list with a view showing “Email Address” & “Best Send Time” only
3. Ask ChatGPT to split the list of emails with best send times into 2 (as shown in this chat) so that each list has the same distribution of leads for each send time.

Then upload these 2 lists to Marketo and then use the smart campaign outlined above to send to one list at their best send times and then a second smart campaign (or the email program) to send the email to the second list at a particular time e.g. 10am CDT as is so often recommended.

Then either using a Business Intelligence tool that has access to your Marketo data or using the bulk extract or REST APIs to extract email activity and then using Google Sheets or Python for the analysis:

• Pull the email activity you are interested in (Opens, Clicks, Unsubscribes) filtering on the email ID that we conducted the A/B test for
• Join this activity data on the lead IDs from the 2 lists exported from Marketo that were involved in the A/B test, giving 2 output datasets
• For each dataset, you can now get counts for each email activity, plot them, and compare the counts to see whether sending at the best time improved performance

 

What Marketo data is next?

Analyzing email activity from your Marketo instance is only the start. For example, you send lead activity information to ChatGPT and ask it to highlight the leads that are most engaged and you could then send these to your sales team. The sky is the limit!

Now that you know the power of the Marketo API you should take a look at the Marketo API Crash Course so that you can learn how to use both the REST and Bulk Extract APIs for so many different automation and marketing operations use cases.

TLDR: Strategy defines your outcomes, tools help you achieve them. Connect the dots between strategy and tech to influence decisions with tools that contribute positively to your goals.

Each year brings a flood of new tools onto the market. So how do you know which is right for your business?

Strategy and technology ideally work in lockstep. One defines your outcomes, the other helps you achieve them. But this relationship gets tangled when your goals aren’t strongly defined.

In this guide, we’ll cover how to connect the dots between strategy and technology. You’ll learn tips for:

a) having a productive discussion with your superiors that links goals and tools together, and

b) framing the conversation to relate your needs in MOPs back to the strategy.

 

Strategy is your foundation

Martech is going through a vendor boom that shows no signs of slowing down. As tech evolves to address more capability gaps, it’s tempting to see the purchase of a shiny new solution as a way to gain a competitive edge.

But tech doesn’t solve problems or set purpose by itself.

Without addressing a clear need for your business, any new tool is like a Ferrari without a racetrack. Powerful, expensive, and wasted.

Strategy is the foundation for all the actions you take, goals you work towards, and places you invest as a business. The outcomes that leadership targets should cascade downwards into a series of smaller goals and define the kinds of tech you consider bringing on board.

Your CMO’s top priority is continuous revenue growth. Your Marketing Manager might want to achieve that by increasing qualified leads. In that case, a solution might be a lead management tool that improves your grasp of lead behavior. The insights from which lets you tweak the customer journey in the right places to bring in more leads and create growth opportunities.

Bottom line: Whatever piece of tech you choose must complete a clear throughline between top-level strategic priorities, relevant performance targets further down, and tools with the capabilities to achieve those aims.

That alignment is how you frame the relationship between strategy and technology for your boss. Leadership wants the tools you use in MOPs to support their aims—this isn’t possible without a thorough sense of purpose.

Ask your boss to share:

• definitive priorities per quarter and year (and their boss’ too)
• how particular performance targets contribute to them, and
• the need a new piece of marketing technology should fulfill. I.e., are we trying to reduce costs? Increase growth? Work more efficiently?

With that understanding, you can advise on the right piece of tech for your circumstances.

And, if you need help defining your strategy and best practices for a technology platform, we can help.

 

Take a need-driven approach

Any decision your business makes on tech acquisition needs thorough involvement from MOPs. As the person likely to manage a new tool, you’ve got a window into the current strengths and challenges of using your marketing technology stack.

Make your case to leadership: You have the knowledge to assess whether a solution is truly the right fit for your needs. Here’s where you explain the evaluation process.

Your boss might be insistent or excited about getting a tool on board, but you need to know how the capabilities and quirks of a particular technology suit your business before you buy it. That’s going to take time.

How to buy time: Gather feedback from users and internal teams, explore trials and demos, and assess how well a tool integrates with your stack and fits your business model—these steps are all crucial for your business to make the right decision. In other words: they’re time well-spent.

Sometimes, the answer isn’t a new tool at all. If leadership wants to achieve a particular outcome, encourage them to think about how you can accomplish the same ends with your current stack.

Are you utilizing all the features of your toolset? If there’s potential to meet performance goals by using the same tech in different ways, that’s a cost-effective way to bring tech in line with the strategy.

Bring the message home: Allowing MOPs time and space to make a thorough evaluation of what technology offers your business, and really taking that assessment on board, increases your chance of spending only on the tools that advance the achievement of your strategy.

 

Speak strategic language

If you want to ensure your leadership team is on the same page as you about how strategy and tech support each other, you’ll likely have to play ‘translator.’

People at various levels of a business speak in different languages, so when approaching your boss (and their boss) about tech, frame the gritty tech details in the context of big-picture strategic objectives.

Is your reporting taking too much time? Are your processes and systems not syncing as well as they should? Could your data insights be faster or more sophisticated?

When you speak with your boss, clarify how those issues impact the top-level outcomes—revenue, growth, productivity.

Position your day-to-day tasks as part of the broader strategy, and it’ll send an important message to leadership: making tech decisions that benefit your work directly feed the overall goals of your business.

Choosing the right technology for your circumstances, and aligning your strategy and tech together, takes some important conversations.

Whatever guidance you need, Revenue Pulse is here to help.

 

P.S. Follow us on LinkedIn to never miss an update.

Are you a relatively new project manager who wants to expand their skillset?

As projects become more complex and larger in scope, it can feel like you’re outgrowing some of the simplified tools and tactics you started with.

Here’s how you can mange projects and deadlines more effectively.

When managing any project, a one-size-fits-all approach isn’t going to be optimal.

It’s important to take a look at the project and consider which management methodology will fit.

Defining your timeline is a perfect place to start when doing larger-scale projects, such as a website redesign or working with other vendors.

• Deciding how firm or flexible your timeline can be will guide the rest of your management approach.

Here’s a summary of two different project management methodologies (depending on the rigidity of your timeline) that will help you get the ball rolling.

 

1) The “Waterfall” Approach

The waterfall methodology is a linear, phased approach to project management.

It’s a traditional approach that would be suitable for your team if the project you’re working on is timeline-driven with a very defined scope and budget.

In this case, you will have to sit down with your team to establish the project’s end goal and a firm timeline to follow.

From there, you can work backwards to establish the necessary steps everyone must take to meet that deadline. It’s important to have a clear, sequential plan toward your end goal so that everyone can stay synchronized and productive.

 

2) The “Agile” Approach

Conversely, you can take an agile approach if the project scope and timeline is more ambiguous.

In this case, your team will be relying on smaller deliverables submitted on a weekly or monthly basis – for example – to make progress more quickly.

Team members and stakeholders involved with the project will be communicating and collaborating much more frequently with this approach. Agile allows for quick adjustments and iterations to the overall strategy as you go.

TL;DR

• Waterfall approach: Good if you have a single, all-encompassing end goal the entire team is working towards with a firm deadline and clearly defined timeline.
• Agile approach: Good if your timeline is less certain and you have smaller goals you want to hit to get updated results and prompt feedback.

 

Other factors to consider

In addition to your timeline and overall goal, choosing which project management approach is going to be influenced by the nature of the:

• project itself
• stakeholders involved, and
• allocated budget.

Once you’ve defined those variables, you’ll be better equipped to choose the methodology that will help your team execute effectively.

 

Know when to ask for help

Sometimes you will run into projects that involve much more:

• stakeholder coordination
• larger budgets
• longer timeframes, and
• external vendor collaboration.

So if you’re feeling overwhelmed, don’t worry – that’s completely normal.

 

“Project managers have years of experience handling these challenges.”

 

Project managers have years of experience handling these challenges and are equipped with many more processes, methodologies, and tools that you don’t have access to yet.

If you find that the projects you’re dealing with are just too big, consider bringing in an agency-side project manager to help.

Being honest about this with your boss will prevent projects from getting off track, and will give you an opportunity to learn more about project management from an expert in the field.

Do you ever feel like you’ve bitten off more than you can chew?

Many small (or even one-person) marketing operations teams are required to keep themselves organized and on track.

Without a dedicated project manager on board, things can get overwhelming fast.

But don’t worry — you’re not alone.

Although project management is quite a deep topic, here are some practical tips and tools you can start utilizing right away to stay on top of things!

1. Invest in an organization tool

The very first thing you should do is invest in some sort of project management (or task management) tool to keep yourself and your team organized.

Your projects likely have many moving parts with multiple stakeholders or team members involved. A good management tool ensures everyone is completely updated and in sync.

In addition to creating basic task lists, you should be able to:

• assign team members to tasks
• track any dependencies among tasks
• track the progress of tasks, and
• set high, medium, or low priority for tasks.

 

2. Find the right PM tools

With so many project management tools out there, it can be tricky to figure out which one is best for you and your team.

At a high level, it’s important to pick tools that have a great user interface and are intuitive to use. Some examples of this include:

• Trello
• Asana
• Wrike, and
• monday.com.

Without getting into the detailed differences between them, it’s safe to say that each of these tools are fairly easy to use and will have all the basic features you need — without becoming overwhelming.

It’s also important to keep your company’s budget in mind when choosing a tool.

While many of these options have capable free tiers, they often require paid versions if you want to unlock premium features or scale your team up to a certain size.

If your budget doesn’t allow for a full blown project management platform, it’s also possible to get creative with cost-effective choices like Google Sheets — at the very least, creating a sheet to keep track of tasks and priorities which can be shared with other team members.

 

3. Get comfortable managing people

Successfully managing projects requires open lines of communication with team members to ensure everyone stays on track and problems or bottlenecks are addressed promptly.

An effective way to achieve this is to hold scheduled meetings with your team so you can catch up on the status of each task, discuss risks or concerns, and so on.

The frequency of these meetings — daily, weekly, monthly, etc. — will largely depend on the timeline of your project and the nature of the tasks.

I also recommend you put an emphasis on reporting deadlines for team members.

Aside from your regular meetings, receiving written documentation of progress on a regular basis is an effective way to stay updated with what’s going on at all times. Reports are also useful for historical referencing, as well as communicating project status to other stakeholders in the company.

 

4. Promote collaboration

The final point I want to leave you with is: create an environment that promotes healthy collaboration. As a manager, you are responsible for keeping your team on track and ensuring they’re comfortable asking for help when they need it.

Project management tools and regular meetings are great for staying organized, but you want to avoid a situation where everyone goes off on their own to complete their respective tasks in a silo between meetings.

Encourage all team members to maintain an open dialogue and interact with each other on a regular basis.

De-siloing your teams and taking a collaborative approach will allow everyone to work more efficiently and effectively towards your goals.