Our next bonus episode of the PPC Ponderings Podcast is now live!

Come listen to Heather and Kirk chat about her take on attribution, statistics, imputation (what?! just listen to the episode...), emotional impact on attribution, and more. Whether you agree or disagree vehemently with Heather, you won't help but be challenged to think more about the the role of attribution in our digital marketing.

‍

Listen on Apple Podcasts

Listen on Google Podcasts

Listen on Spotify

‍

If you haven't already, make sure to catch the first full length episode here: Episode 2 - Attribution Concerns & Pitfalls

In the PPC Ponderings Podcast core episodes, we depart from a more traditional "interview approach", and come from more of an investigative journalism format. Enjoy!

Heather is the VP of Analytics at Seer Interactive. She has over 20 years of experience in product and data & analytics as a statistician, health economist, digital analytics product manager, systems integrator and digital analytics consultant. Heather’s expertise spans both B2C and B2B marketing analytics, web analytics and attribution measurement.  She has her BS in Mathematical Sciences from Clemson University and a MA in Economics from Duke University.

Outside of the office, Heather enjoys hiking, snowboarding, watching her daughters’ soccer games, and trying new recipes on her forgiving family. Heather is also the Co-Lead for the Salt Lake City Chapter of the Digital Analytics Association, the leading professional association for digital nerds.

‍

Episode Transcript

Heather Aeder (00:03):

AI, machine learning, those are predictive statisticalmodels. Attribution is using past data to comment on a current situation. It isnot predictive.

Kirk Williams (00:15):

The emotions is something that I think as humans, like whywe are more drawn to a certain ad than another is important.

Heather Aeder (00:27):

When should you not care about attribution is anotherimportant question, just as much as when you should care.

Chris Reeves (00:36):

Welcome to the ZATOWorks PPC Ponderings Podcast, where wediscuss the philosophy of PPC and ponder everything related to digitalmarketing. Today's show is a bonus episode of our full interview with the VP ofanalytics at Seer Interactive, Heather Aeder. We interviewed Heather for ourepisode on attribution and it ended up being a fantastic conversation. If youhaven't heard our second PPC Ponderings episode yet, give it a listen.Otherwise, enjoy our behind the scenes conversation with Heather.

Kirk Williams (01:08):

Chris is our fun tech guy, so we were on a team call theother day and he did ... I've seen someone else say this online, and he figuredout how to do it. He did this video thing where he pretended to be listening.He'd nod his head and stuff, but it was a video, and then all of a sudden hecame in and handed himself a cup of coffee. It was really funny and crazy.We're like, "What?" So can you give us name, where you work, what youdo there?

Heather Aeder (01:37):

Sure. My name is Heather Aeder. I am the VP of analyticsat Seer Interactive, a digital marketing agency. I would've said based inPhiladelphia, but now we are a remote first agency, so we're based everywhere.I live in Park City, Utah.

Heather Aeder (01:54):

And at Seer Interactive, I am responsible and lead ourdepartment, our strategy and analytics department. So I make sure that wecollect data the right way, that we analyze and understand what it's tellingus, and that we present it and tell a story back to our clients so that theycan take action either on their websites to optimize their digital properties,websites or apps or whatever that may be, or so that they can optimize theirmarketing. Primarily we focus on SEO and paid media channels like PPC andprogrammatic and social.

Kirk Williams (02:41):

So are you doing a lot of data primarily that's housed inthird party things like Google Analytics, Google Ads, that sort of thing? Orare you also doing a lot of SQL database type stuff as well?

Heather Aeder (02:55):

We're a mix of both. And I think that we're probablymoving more towards being tool agnostic and getting data down into a data warehouse.And I think that's where our clients are headed too, which is why we're headedthat way. In addition, I think a lot of the tools themselves are headed in thatdirection. So if you just take Google infrastructure as an example, that'sreally what the new GA4, or Google Analytics 4 is all about is modifying howthey collect data to be more collected in events that can more easily betransferred into a data warehouse or a table row structured format.

Kirk Williams (03:32):

Do you think that's primarily because of privacy or isthat just those things are happening at the same time? Or are a lot of peoplemoving more and more towards their own warehouse and that because of privacyand getting rid of cookies and that's going to be about all we can do?

Heather Aeder (03:47):

I think it was happening way before the privacy challengecame into play and you'll see new tools that are popping up that started thatway natively. So like Amplitude, Heap, those tools started as event-based datacollection. And they started that way because brands got tired of missing dataand not being able to tie it back to all of their other systems.

Heather Aeder (04:15):

So if you take an e-commerce brand as an example, they'reconstantly trying to tie data back to their order management system to be ableto track, oh, this order is in this stage. So it's not just been purchased, butit's been returned. So I actually don't want to count credit towards marketing channelsfor ... All my paid search keywords from one ad group are resulting in returns,that's not positive ROI for me. So I think that database component has beenaround for a long time and the reason is because it's to connect to datasources beyond just digital sources.

Kirk Williams (04:48):

So I fully admit I'm very naive when it comes toevent-based tracking especially. I think I've seen enough people scared of GA4online who have gotten into it that I'm like, "I'm going to kick that candown the road a little bit." But in terms of event-based, how would thatimpact even our discussion of attribution?

Heather Aeder (05:09):

I don't think it's going to change it too much and that'sbecause attribution itself is the event equals the session in that instance. Sofor me, the way I define attribution is, attribution is the study ofunderstanding how multiple marketing touch points on the way to a conversionimpact return on ad spend or return on investment.

Heather Aeder (05:36):

And so for attribution specifically, a touch point is a session.And so because the event equals the session, I don't actually think that'sgoing to impact attribution too, too much, other than with the industryevolving to more of the events being stored in a data warehouse. It's going tomake the data more accessible to more brands and allow for broader connectionsoffline into downstream conversions versus isolating attribution to digitalconversions. So I think that's potentially where the impact comes into play,not so much in what am I counting.

Kirk Williams (06:14):

And in those scenarios where, especially larger brands,they have their data warehouse, they're looking at everything in there, do youforesee them almost creating their own attribution models as well within thoseecosystems?

Heather Aeder (06:29):

Yeah, it's happening already. And it's not just largebrands, I think it's even starting to come down into mid-size brands or brandsin that $30 million to $50 million online range. They're creating some verybasic attribution models, mostly without the statistical modeling and more justgiving linear credit across the board. I'm seeing that already in that mid-tierbrand range versus it just being large brands.

Heather Aeder (07:00):

Large brands, there's a wide variety of large brands out thereand I think some are going to believe and be able to trust and explain themodels that they're creating internally from their own data warehouse a littlebit easier than perhaps they can explain the models that are coming out ofthird party tool sets that are a little bit more of a black box.

Heather Aeder (07:23):

And so that's often why I see brands try to build theirown in-house attribution model is because it's easier for them to explain it toexecutives internally and get buy in internally than using a black box, is thisa Bayesian model, all that scary statistical talk that goes on outside of theirown walls.

Kirk Williams (07:44):

Gotcha. So are they creating their own algorithmic-basedmodels within those warehouses as well, or are you referring to they'rehesitant to use more of the black box of obviously have a third party likeGoogle and Facebook and that, their statistical models? My assumption, againthis is just ... I don't really have access to clients who are doing this atthis point, which is fair. I would expect Seer to have a little bit more accessthan our five person agency. Are they creating statistical models at thispoint, some brands, or is it still fairly simplistic in the model?

Heather Aeder (08:20):

Some are. So I think those that are, it depends on theiramount of spend. So I'd say the complexity in the model increases when themodel can return back more dollars for optimization. So I might see brands thatare spending, I don't know, $10 million to $20 million online in media spend,that's when a little bit of a refined model can get you back a lot of dollarsto go put back in market. And so I see the models increasing in complexitybased on spend, whereas somebody who's a challenger brand just now entering themarket, they might just be using simple linear attribution or they'repartnering with a third party to quickly bring them insights that they can goaction on. So it's all a scale based on marketing dollars.

Kirk Williams (09:18):

Cool. Cool. Cool. Well, you did a fantastic job of movingus into our questions, so thank you for that. So you defined attribution, whyshould people care about attribution?

Heather Aeder (09:32):

They should care because of the increased marketing spendthat it gives them back to go put in market. But I would say there, when shouldyou not care about attribution is just another important question, just as muchas when you should care. And I think some of the biggest pitfalls that I seewith attribution are when you're not ready to action against it.

Heather Aeder (09:59):

So I've seen countless number of brands go out and spend aton of money on something like a Neustar, which those are not cheap in terms ofattribution tools. They spend six to nine months implementing that tool andthen they come out of it and they're unwilling to optimize their keyword spendbased on attributed revenue versus last click revenue. So you just wasted allof it.

Heather Aeder (10:23):

So attribution is great when you're ready to change yourbehaviors and your tactics based on the attributed revenue metric that's comingout of these tool sets, or your algorithms, or wherever you're doing it. Andthe reason to do it is to get that money back to then go optimize your channelsin a different sort of way.

Kirk Williams (10:47):

Awesome. Yeah, so that would be maybe a challenge,limitation, you noted that, when a brand's not ready to action against it. Anyother limitations or challenges to attribution that you'd call out?

Heather Aeder (10:59):

Yeah. I think with the privacy concerns coming out the ...One important thing, if we go back to what the definition of attribution is,attribution is the measurement of one or more marketing touch points on the wayto a conversion. So that timeline, or look back window of how many marketingtouchpoints do you count prior to a conversion occurring is going to be reallyheavily impacted by privacy and the deprecation of third party cookiespotentially.

Heather Aeder (11:34):

And so I do have concerns about where does attribution gofrom here. And I think that can be a pitfall is like, is it worth it still withthat investment. So for example, if you're a brand and your typical time toconversion from the first marketing touchpoint to conversion is less than aday, the attribution's not worth it because you're probably only going to haveone touchpoint, maybe two on the way to a conversion.

Heather Aeder (12:06):

Before you go spend a ton of money on an attribution toolor tons of effort and process changes and all that kind of stuff, understandingwhat your current landscape is in terms of average time to convert as well asaverage number of touch points to a conversion is a really important checkpointto go through or gatekeeper to go through prior to going down that pathway. Sothat's a common pitfall is people make this investment and then they come outof it realizing, oh, on average, I only have 1.5 touch points on the way to anorder. Okay, I'm not going to change my behavior based on that. So I thinkthat's a pretty big potential pitfall.

Heather Aeder (12:39):

And then how privacy plays into that is with thedeprecation of cookies, like the MarTech vendors themselves, all theseattribution vendors, they need to find a way around that. And some of them aredoing that by server to server data collection versus pixel-based datacollection. So they're working on methods and they're also working on thingslike augmenting their modeling with user-based surveys that are launched when aconversion happens on site to get more direct feedback on how long have youbeen in market and how many touch points do you remember engaging with througha recall.

Heather Aeder (13:18):

And then I'm also seeing attribution work around theseprivacy pitfalls. They're migrating how they measure incrementality throughactual experiments versus just a model-based or statistical approach. So withan experiment, think a test and control sort of environment. And some platformsare building the experiments within the tool itself. A Measured would be anexample of a tool that does that, or a tool like Rockerbox, is they are tappinginto MarTech vendor or a Google Ads platform or a display platform. They'rebuilding connections with those platforms. So if you're running tests in thoseMarTech platforms, you can consolidate all of your experiments within theattribution platform vendor, use the data on incrementality there to inform themodeling that then tells you the attributed revenue.

Heather Aeder (14:21):

So there's been quite an evolution. This is just reallygoing on in the last two and a half years, I'd say, of leveraging experimentsin influencing the models. So there's another way to get around it. There'scool stuff going on. It's exciting to see.

Kirk Williams (14:33):

That is, that's amazing. Yeah, exactly, it's very cool tohear what people are already working on, especially, you'll have an idea andyou'll think, oh, you know what would be cool is if this, and then you realize,oh, there's seven tools already out there working on that problem.

Heather Aeder (14:45):

Yeah. The other interesting thing is that these newertools are doing it at a lower price point for brands too. I think of it as likefirst generation versus second generation attribution tools. So firstgeneration would be Neustar, MarketShare, all the ones that have been aroundnow for seven, eight, nine years, which is really not that long, but in thecontext of attribution, it is.

Heather Aeder (15:13):

And there's other tools that have just gone by thewayside. So Convertro's gone, ClearSaleing, which I actually used to run themodeling team at ClearSaleing. They're gone, they're dead. They're beingreplaced by more next generation attribution tools, which are getting rid of alot of complexity, and these features that nobody ever used in the legacytools, replacing that with these other more innovative experiments to reallyrefine the measurement and build more trust in the measurement. So I reallylike the way these next generation tools ... I like where they're headed.

Kirk Williams (15:49):

Yeah, that's very awesome. But one of the things I've beenpondering about attribution is just the idea of the human, the human making apurchase. At some point we can get a lot of data that shows us, hey, for thisspecific store, the humans making the purchases, they purchase after 2.5touches. And yet, especially as you get into more touches on average, the why,the emotions is something that I think as humans is all a part of us. Why weare more drawn to a certain ad than another is important part of the consumerbuying cycle, and yet, we can't really get that into an attribution model.

Kirk Williams (16:41):

And so I'm just curious in some ways, what are your thoughtson that? What do you think about emotions and how that fits into attributionand attribution focusing specifically on the percentage of touches and blah,blah, blah, in a sale? Do you think that's even important to try to thinkthrough? Are there limitations with that? Do you think data can handle allthat? There was a mess of a half question in there and maybe you can make itbetter than what I asked. What do you think?

Heather Aeder (17:14):

It's an interesting question. I'll say I wouldn't considermyself to be an expert in emotions as a statistician, not my strong suit. But Ithink how I might translate your question into the practicality of howmarketers make decisions of what to put in market is what content resonates.

Heather Aeder (17:40):

So if I translate emotion to various types of content,okay, when I hear the word content, then I can think, okay, well, when I thinkcontent, I think testing because I can always test different types of contentto see which types of content either drives a micro conversion or the finalconversion that I'm looking for. And a micro conversion might just be attractsthem to the site, initiates a session event. The micro conversion could be, ifwe just think about e-com because it's an easy construct, they go view aproduct page or they do a search.

Heather Aeder (18:19):

And so if I think about how does content come into playwith attribution, I would say there's not a lot of analyses that are done inthat area today. I would say content and evaluating how content triggers areaction in a consumer, most of that evaluation is done within the construct ofA/B or multivariate testing, where you're serving up different content, ideallyto a similar set of users and then measuring the incrementality. So I'd sayemotional reaction content leads me to incrementality measurement, which ismeasuring the lift of one population or one variation compared to another. Sothat is a point in time sort of analysis.

Heather Aeder (19:15):

With attribution, you are more frequently measuring theimpact of multiple touch points over time. Now you might have an incrementalitytask for content where you're measuring path one, which has content A, versuspath two, which has content B in it. And so I can see those interplayingsomewhat with attribution. But more so than not, most folks that are in theattribution space, the business questions that they're trying to answer iswhere should I spend my next dollar. When you're talking about emotion and theuser journey and what resonates with the user, it's just a slightly differentbusiness question I think, and a different business approach.

Heather Aeder (20:05):

I don't usually combine content with the broader questionof the journey and effectiveness there, unless I'm studying does content Acause more lift in this same journey than content B. I don't think it comes upthat much. It'd be interesting to do some work in that area I think. And mostof the time content as an attribute, isn't evaluated in an attribution path.

Kirk Williams (20:33):

Yes, that for sure. And some of the question I think thatyou brought up, which is a great question is even some level of should it be ina sense, or are those two different businesses, that sort of thing. So yeah,no, that's good. I need to keep pondering that. Okay, do you think it'spossible, can attribution be overvalued at all? [crosstalk 00:21:00]-

Heather Aeder (21:00):

So 100% yes, attribution can be overvalued.

Kirk Williams (21:03):

Do you want me to-

Heather Aeder (21:08):

You know that old hype cycle graphic that has this bigbubble coming up in the chart and the period of hype cycle, and then it dropsdown into the trough of disillusionment and then it comes back up more intoreality? I would say attribution has gone through that. It's had its hype cyclea couple years ago. The trough of disillusionment probably happened a year anda half ago when all these other big shops started to close and people gave upon attribution. And now it's starting to come back because we're approaching ita slightly different way as an industry. And so absolutely, attribution can beover hyped. I think it really was a couple years ago when people were way overpaying for it. And the reason it can be over hyped is, I always go back tothis, when you can't action against it.

Heather Aeder (21:57):

So at my previous agency, prior to Seer, it was ane-commerce based agency only, so all we worked on was e-com brands. I can saythis now because that agency's shut its doors, so I feel comfortable talkingabout everything.

Heather Aeder (22:14):

So we put every single client of ours on an attributiontool, no matter their size. And it would take easily three months to get themup and running on that attribution tool. We had to do tons of work integratingtheir spend dollars into the tool set. We did a ton of work making sure thatall the parameters were perfect on every single digital marketing asset thatwas in market. We had to make sure that all of the data was coming in the rightway, everything was tagged correctly, all the pixels were in place.

Heather Aeder (22:49):

Tons of effort in that, and not just for us as an agency,but also for our customers who had to go in and do all the discovery work withus. And we were doing this for brands that were spending $2 million a year, notworth it at all in terms of what they got back. And then we would go and we'dget all that in place, we'd be so excited to go in and look at the data andthen be like, "Average number of touch points, 1.5. Oh my God, all thatwas wasted for nothing. We're not going to change anything we do."

Heather Aeder (23:19):

So I would say, when attribution is right for a brand iswhen they've got to be in more than one channel. So if you're just getting inbusiness and you're in PPC only, and maybe SEO, don't do attribution. If you'restarting to move up into up-funnel tactics and you're spending a significantamount of dollars in display, that's when it's time. People naturally gravitatetowards attribution when their display spend is multiple millions of dollarsand they need to do things like understand frequency capping. That's thenatural tipping point for me is when you move up funnel, more than one or twopaid channels.

Kirk Williams (24:05):

I love it. Let's talk maybe some models, let's just saywithin the Google Ads realm. I'm assuming most of the people listening to ourpodcast specifically are probably Google Ads people for the most part, workingwith Google Ads, doing a lot of that. So in terms of attribution modelsavailable to people who are utilizing Google Ads, maybe speak to those models alittle bit. Specifically, are there any models that you actually would notutilize, which you would avoid? Are there any that you do like? And then we cango from there with follow up questions.

Heather Aeder (24:43):

Yeah, sure. So I'll give away my first secret, which isI'm not a Google Ads expert. So my expertise lies more in measurement and myexpertise from a channel perspective is in affiliate. So yeah-

Kirk Williams (25:01):

No way.

Heather Aeder (25:01):

Way. Yeah, I know, shocker. So when I think about models,I'm just going to describe them and hopefully these translate to Google Ads insome way. So I think the most common models that are available out of the boxin any of these bigger MarTech platforms are going to be first click, lastclick, or data driven, or linear. Those are probably the four most common ones.Any that aren't that four are probably not widely used, so U-shaped or timedecay. Those types of models, I've not met a brand yet that uses either ofthose types of models.

Kirk Williams (25:42):

That's really interesting because we do utilize U-shape.

Heather Aeder (25:47):

Oh, interesting.

Kirk Williams (25:47):

And some of that is just because I, for probably a lot ofthe other reasons, typically will see, okay, first click and last click have alittle bit more of that strong presence, and then yeah, go ahead and divide upthe mid stuff based on getting some credit, but clearly it wasn't enough tomake a significant decision. So anyways. That doesn't even mean it's right, Ijust think that's super interesting to hear it [crosstalk 00:26:15]-

Heather Aeder (26:15):

Yeah, that is really interesting.

Kirk Williams (26:16):

... really focuses on that. So anyways, please go aheadand continue with your thought.

Heather Aeder (26:24):

Okay. So I would say I don't ever recommend just lookingat one model. So the reason that there are multiple models in there is so thatyou can actually get perspective depending on what you're trying to optimizeagainst. So let's say CMO comes to you and is like, "I got so muchinventory. I got all these widgets and they just got here. I'm never going tosell them. I need to push it." And so in that instance, you probably caremore about the last click view of that and what's going to convert for thatparticular widget that you're pushing in that particular campaign and you'regoing to optimize to that converting channel.

Heather Aeder (27:05):

If CMO comes to you and says, "I don't care whathappens, I just need more people in my database at this point, because I've gotanother plan. I've got this full email journey mapped out that as soon as I getsomebody signed up, I'm going to go send them 20 different things to go learnabout the brand and do all these kind of things to eventually get them toconvert." In that instance, if you run a campaign there, you're probablygoing to look at the introducing or the first click model and see how can youoptimize against that, your spend.

Heather Aeder (27:42):

If you are trying to make decisions where you're trying tofigure out across channels and over time, you're sitting down and you'replanning, I've got $100,000 to spend next month, where should I spend it, inthat instance, I want to look at either the data driven model, I usually callit weighted or attributed model too, I'm not sure what it's called in GoogleAds, or you might look at the linear model.

Heather Aeder (28:09):

So the difference between linear and data driven, itdepends on the tool. So linear is if there's three touch points on the way tothe order, each touch point gets 33.33% of the revenue credit that happenedwith that order. Data driven looks at more things, so it's not just a straightsplit or divide by the number of touch points. Data driven is where you'retaking a statistical model and layering that in. They're measuring things likeincrementality as an input into that model for predicting how much credit atouchpoint should get. They are also looking at things like recency andfrequency as part of that model.

Heather Aeder (28:54):

And then some tools even look at outside or external datasets as input into that model as well. It could be something like seasonality.If they're developing a model that's custom built for a brand, they may takeMMM, or medium mix modeling data as an influencer input into that model aswell. So the data driven models can get really complicated and they'redifferent depending on what tool you're in, what channel you're in, all of thatjazz.

Heather Aeder (29:26):

And so in terms of when you're making bigger decisions andyou're optimizing where to spend your money cross channel, cross tactic, that'swhen I think the data driven model is the most effective because it takes intoconsideration the most variables, but it's also really hard to explain. So mostCEOs are going to be like, "You said what," when you try to explain adata driven model to them. Linear, it's pretty easy to explain. Threetouchpoints divide by three, each gets a third of the credit.

Heather Aeder (30:02):

And so what model you eventually choose it's no one sizefits all. You might use different models for different things, or you might notgo down the path of data driven because it's too hard to explain when you'remaking big dollar decisions. So it really varies. There's no one size fits all.

Kirk Williams (30:21):

Yeah, so CEOs, C-suite and that, so they can struggle tounderstand what's happening within a complex algorithmic model, like DDA orthat. And that is what it's called in Google Ads, by the way, literally DDA, datadriven attribution. Do you find that overall when you are talking with themthat they like the idea of machine learning aiding the algorithms, or do theyoften just feel safer with the linear model? It's kind of interesting to me, Icould see CEOs and that being really excited about oh, sure, you're applying"machine learning," air quotes for those who can't see me. You'reapplying machine learning to it, sure, go ahead. But do you find the opposite?

Heather Aeder (31:02):

I think it's changed over time. So again, five, six yearsago, when I was in pitch meetings, when I worked at an attribution vendor, thebiggest pushback we got was, I don't understand, sorry. I don't get your model.I don't trust it. No, thank you. That was the number one thing that we had tofight against when we were in those pitch meetings.

Heather Aeder (31:26):

I would say now, if we go back to the hype cycleconversation, AI, machine learning, they're at the peak of their hype cycleright now. And so if you speak of the modeling in that way, I think CEOs wouldbe more receptive to it. But the reality is AI, machine learning, those arepredictive statistical models. Attribution is using past data to comment on acurrent situation. It is not predictive. So attribution doesn't really use AI,machine learning from a mathy, nerdy sort of standpoint. It's more likeanalyzing what happened in the past to tell me what's happening right now, tomake decisions right now. So they're actually not related from my point ofview.

Kirk Williams (32:14):

Yeah. No, that's excellent. Agree. It is interesting to mehow they are in some ways treated in every way as similar things.

Heather Aeder (32:23):

Yeah.

Kirk Williams (32:25):

But clearly there's a clear difference between predictiveand modeling and that, and just saying this is what we have, I would assume.Okay, so in a privacy-focused world, do you see us just winding up with more ofthe predictive models simply because that's about the best way that ... like wejust won't even have as good of access to data for just normal attribution?

Heather Aeder (32:53):

Yeah, I do think that's potentially where the predictivepiece comes into play is filling the gaps of where we're missing information.And that's one of the primary reasons Google's rolling out GA4 is they areincorporating ... A component of that is the ability to be predictive on sayingyeah, these are the same people and tying the journey together. So makingassumptions and modeling out that yeah, person A's journey started here, oh,and we're predicting that this is actually a person over here and tyingtogether the journey. So I do see some of that coming in there and that'sreally going to be highlighted in GA4.

Heather Aeder (33:38):

The other way it can help and where we can use statisticsto predict missing information is this statistical concept of imputation, whichhas been around for a long time. And imputation uses past data trends topredict what's next when there is missing data and they look at sample dataversus a population. And that's something that Google is really good at doingis collecting sample data. And then they know the universe size, so they canpredict up to the population. That statistical technique has been around for areally long time.

Heather Aeder (34:17):

Would I, again, call it AI, machine learning? Probablynot. But that might just be because I'm old and I'm a traditionally trainedstatistician. I don't call myself a data scientist. And so I think for me, Istill think this like data sciencey world is a very flashy terminology, AI,machine learning. It's all wrapped up into this cool thing, but in reality,it's all just statistics. But there are many kind of ... Statistics is a verybroad field and different techniques are used and applied in different ways.

Kirk Williams (34:55):

That's good stuff. Did you have anything that was justburning you wanted to make sure you communicated about attribution, we hadn'ttalked about it yet, you didn't see the question?

Heather Aeder (35:09):

I guess I'd like to know from you-

Kirk Williams (35:11):

Any last thing?

Heather Aeder (35:11):

... just conversationally, how do you think search willchange as these different way of collecting data evolves? Has search evolvedenough either within the native platforms or via tools like Kenshoo or Marin tobe able to handle different types of data coming back, to be able to handleattributed data in automatic bidding methodologies and things like that? Issearch there? I don't know enough about it to know.

Kirk Williams (35:42):

I do think you alluded to it just in terms of the sheervolume of data needed, especially as you start to get into the statisticalmodels. It'll be interesting to see what Google is utilizing for that,especially for smaller accounts, because you have a lot of small accounts thatGoogle really does want to make sure they keep paying money.

Kirk Williams (36:07):

So let's say for DDA. So DDA, they had specificlimitations on which accounts you could apply that to. I don't remember what itwas. It was like 30 conversions in the past 90 days or something like that,whatever it was. I think now they have pulled that. I don't think they'vepulled it entirely, but they've significantly reduced that number required. Andso some questions I have without just ... they haven't really communicatedwhat's going on with that. I'm just curious to know what led to that decision.Do they feel that-

Heather Aeder (36:44):

They couldn't trust them.

Kirk Williams (36:47):

Exactly. So if they feel like they have enough data tomake good decisions, in some ways then I'm curious, where are they getting thatdata? I would assume they're pulling it cross account, which also even hasinteresting questions about all of us competitors getting together, sharing ourinformation with Google, it applying those things. Those are interestingquestions even about smart bidding and automated bidding, if the one that hasaccess to all of that stuff is the one making all those decisions. So that'sone of them.

Kirk Williams (37:21):

One concern I have, and some of this stuff will be on thepodcast itself, but I'm also trying to think through, so especially Google'sattribution tools are always going to be platform specific. So we're makingdecisions and then that's making decisions on our bidding based on only GoogleAds clicks. That to me is concerning as well, not bringing in any other part ofthat buyer journey. As far as I'm aware, it still is only Google Ads, so inchannel, which I think even would change ... You would know this better than Iwith statistics, but even that alone to me would change how they arecalculating and assigning value-

Heather Aeder (38:10):

Yeah, for sure. Totally.

Kirk Williams (38:10):

... by taking part of a user journey. What about all thatother part of that user journey? Well, we're just going to look at the 10 timeswhere they have Google Ads and then make that decision. Well, okay.

Heather Aeder (38:21):

Yeah, you bring up an interesting point because I thinkit's important to understand that there's multiple types of attribution.There's within channel attribution and then there's cross channel attribution.Cross channel is where it should be, period, but MarTech vendors can onlycontrol their own channel because they don't often collect the data of all theother channels. And so they are limited in what they could do.

Heather Aeder (38:48):

You've talked about how it comes up in search a lot, butwhen I was an affiliate ... So I spent five years at an affiliate vendor priorto Seer and that was all we talked about was attribution, all we were concernedabout, because affiliate is moving up funnel now more so than it has been inthe past. But the only data we had was within channel, like within affiliateattribution, to be able to make decisions and influence things like that. Butwe would've been so more powerful had we been able to collect data in other channelsand influence commission rates in affiliate based on where they fell in thefull user journey versus just the affiliate user journey.

Kirk Williams (39:31):

It makes total sense, because your model is based on howyou're assigning value in this person's user journey. So it makes sense that ifyou're only capturing part of that user journey and yet everything about whatyou're doing is trying to assign value for where that person is in theirjourney, then only capturing part of that seems to be like it would be anissue. Yeah, so those are probably some of the things. Oh, I did think of oneother question I had actually.

Heather Aeder (40:00):

Yeah.

Kirk Williams (40:01):

I'm totally curious, do y'all see more and more brandsshifting to more of a media mix model mindset, or even using goals like mediaefficiency ratio, MER, rather than ROAS and this click attribution focus? Areyou seeing that as a little bit of a trend? Not really? What are your thoughts?

Heather Aeder (40:22):

I am somewhat seeing media mix modeling become a thingagain for some brands that can afford it because media mix modeling, it's notcheap. That's usually somewhere between $150,000 to a $200,000 engagementannually and you get quarterly models basically. So for those that can affordit, I think media mix models are really good to help you set your quarterly oryour annual budgets by channel. That's the best way to use those. And thenattribution tools or attributed data are really helpful for once you're inmarket, optimizing at a more granular level, like at the keyword level or atthe publisher level and affiliate as an example. So I think they are often usedtogether versus one or the other.

Kirk Williams (41:13):

What's your preference out of those?

Heather Aeder (41:14):

Run MMM to set budgets and MTA, or multi touch attribution,to optimize budgets.

Kirk Williams (41:23):

Gotcha.

Heather Aeder (41:24):

That's how I see them being used together.

Kirk Williams (41:26):

That makes sense. Interesting. Cool.

Heather Aeder (41:28):

And a lot of those legacy vendors that we were talkingabout, like the Neustars, the MarketShares, they were traditional MMM shops andthey bought attribution. They acquired attribution vendors to augment their MMMand they now display, or they have one UI where you can come in and get yourMMM results and your MTA results.

Kirk Williams (41:51):

There's a big world of MarTech out there. Well, thank you.Thank you so much.

Heather Aeder (41:56):

Pleasure. Thank you for having me.

Kirk Williams (41:58):

Yeah. Yeah, it was delightful. You clearly, clearly knowyour stuff, and so thanks for helping us think about it. I'm very excited to beable to share your thoughts with the world.

Chris Reeves (42:11):

This has been a bonus episode of the PPC PonderingsPodcast. Keep checking back for more interviews and our next full episode. Ifyou like what you hear, please consider sharing this with your network andleaving us a review on Apple Podcasts. Until next time, may the auctions beever in your favor.

‍