Why You’re Losing Money with a Bad Customer Experience with Marcus Merrell

Joe Casabona: “All of the world is experienced through code.” This is a statement that Marcus Merrell makes on today’s episode… and it’s absolutely true. See, even if you sell the most niche handmade thing, you probably sell it online, which means someone’s code is powering your business. And your buyers are experiencing your business through code. So the worst thing for your business is bad code because that makes for a bad experience.

And I know what you’re thinking: “What if I don’t write code?” Well, if you’re listening to this show, you’re probably creating user experiences. And today Marcus tells us how we can do it better. We should all listen, our businesses depend on it.

Look for these top takeaways in today’s episode: how even if you sell products, physical products, you’re interfacing with code. This is what the no-code movement is based on. That as products are good and improving, they’re not improving as fast as people’s expectations and what we should do about it, and how to properly test your experiences, whether or not you write code.

Now, in Build Something More, we talk about the acquisition process from the acquirer’s point of view, as Marcus’ company made an acquisition recently. Usually, we get it from the perspective of the seller, not the buyer. So I’m excited about that.

To hear that and get ad-free extended conversations of every episode of How I Built It, you can become a member over at joincreatorcrew.com. It’s just 50 bucks a year, that’s less than five bucks a month and you get those ad-free extended episodes, you get behind-the-scenes content, the weekly wrap, which is a bonus episode every week, and you get access to live stream archives, and all of my workshops, which are usually 40 bucks a seat. Again, that’s over at joincreatorcrew.com.

This is episode 274. So you can find all of the show notes over at streamlined.fm/274. Thanks to today’s sponsors, Nexcess and LearnDash. You’ll hear about them later on in the show. But for now, let’s get on to the intro and then the interview.

[00:02:30] <music>

Intro: Hey everybody, and welcome to How I Built It, the podcast that helps small business owners create engaging content that drives sales. Each week I talk about how you can build good content faster to increase revenue and establish yourself as an authority. I’m your host Joe Casabona. Now let’s get to it.

[00:02:53] <music>

Joe Casabona: Marcus, how are you today?

Marcus Merrell: Doing great. Thank you. It’s great to be here, Joe.

Joe Casabona: Thanks for coming on the show. So in our pre-show, we were talking about how you have 20 years of experience in QA and user experience specifically, but you started in the game industry. I thought that was really interesting. I was just telling somebody today that my favorite book, maybe one of the most impactful books for me, is Masters of Doom, which is all about id Software. And that kind of made me want to be a developer, which, I mean, 20 years later that book still has a big impact on me.

Marcus Merrell: Yeah, absolutely. I remember at some point, there was a rumor that Romero was going to visit the offices I worked at in Austin, Texas.

Joe Casabona: Oh, wow.

Marcus Merrell: And if he ever came, I wasn’t there that day. But he was this legendary game designer, and it would have been great to see him. But yeah, it’s a small group of folks that work in the industry, and we generally know each other but I never met Romero.

Joe Casabona: That’s funny. He was kind of like the rock star of the pair. He was kind of like… I mean, I try not to always talk about Apple, but he was kind of the Steve Jobs to John Carmack’s Steve Wozniak, I think, was-

Marcus Merrell: Yeah, that’s right.

Joe Casabona: …the brains of the operation.

Marcus Merrell: Fun times.

Joe Casabona: Yeah. Awesome. Cool. So I’m really excited to talk to you about this because like I said, I think that from my experience as a developer myself, testing wasn’t always my favorite thing to do in code because you’re trying to basically break code that you just got working, which is probably why there’s a separate QA people.

But then from a no-code perspective, as somebody who’s moved fully into the kind of creator space and helps people build their own platforms without code, I think testing might be even more important because now you’re chaining a bunch of tools together hoping that they work… and testing could be pretty difficult. Sauce Labs has a white paper called Every Experience Matters. Can you tell us a little bit about that?

Marcus Merrell: Yeah, sure. I mean, the basis for the white paper is essentially that all of the world now is experienced through code basically. If you sell a product and that product is, you know, wooden stars that you put together on your own and then you sell them online, at some point, your product, no matter what you sell, is going to interface with code at some point. And have to be packaged to be sold via code, shipped logistically via code, and then moved out to someone. So you can no longer really avoid being in this world of code. And code is just as fragile as the humans who write it. So it’s interesting.

The point that you make around this low code concept of no-code, the WordPress kind of thing, to me that’s really exciting. Because, as you said, you probably can’t just always innately trust that the components that are put together to make a WordPress site work are always tested perfectly. You can be reasonably confident, though that, you know, they’re probably in a decent state.

To me the nice thing about that is that it gives you the ability to focus on your top priorities in your business, and not just what I always had to do in my QA career, which is say, “Is that button actually a button? Does it does it work right?” You’re probably going to be right. But now you can focus on “are my users able to search for and buy my stuff. Am I showing up on Google the way I need to show up?” You could test for that stuff now.

Joe Casabona: Yeah, absolutely. I’m old enough to remember trying to integrate payment gateways in a pre-Stripe days, where you couldn’t just like plug in a button basically and have it work. You would need to do some wild handshake with either PayPal’s… I think it was called IPN or what was the… authorize.net was the big one that I’m sure big corporations still use, right?

Marcus Merrell: Yeah, probably.

Joe Casabona: Independent creators aren’t really using that these days.

Marcus Merrell: Yeah. They used to have to do the rest handshake that the code to actually manage the API requests and stuff like that, test all of that stuff at the same time. And it was a nightmare.

Joe Casabona: Yeah. It gave me agita. I remember I got a… First of all, I undercharged for this project, because I had no idea what I was doing. But this nonprofit organization wanted to raffle off a million-dollar house. So we set up PayPal to do it and I didn’t realize that raffling was considered gambling, and so PayPal shut it down.

Marcus Merrell: Oh my!

Joe Casabona: And they were like, “Well, you need to figure out a different way to do this. Maybe we could just like store the credit card numbers and do the charge on our end.” And I was like, “No, we cannot do that.” I was like, “I may not be the most experienced developer at this point but I know that much.” I think they ended up going with somebody who kind of understood payment gateways better.

Marcus Merrell: Yeah, regulation, compliance. There’s all sorts of stuff you don’t want to have to mess with.

Joe Casabona: I’m like, “I’m not PCI compliance or whatever.”

Marcus Merrell: Right. Exactly.

Joe Casabona: So I like what you said here: “All the world is experienced through code.” I don’t know, like what the expiration date on mentioning the pandemic is, but the pandemic really accelerated that for a lot of businesses, right?

Marcus Merrell: Yeah.

Joe Casabona: Businesses that still weren’t online had to go online in order to stay in business.

Marcus Merrell: There’s a famous comic. It’s just a one panel where there’s a bunch of people in a boardroom in the tall building, saying, “Yeah, maybe it’s time to start talking about our digital transformation.” And meanwhile, there’s a wrecking ball right outside the window called COVID-19. Yeah, you are going to start talking about your digital transformation now.

What’s interesting is that this white paper we put out was survey-based, and we got some pretty strong indications that, you know, products are good and they’re improving, but they are not improving as fast as people’s patience is deteriorating. So even as experiences overall get better and the market is shifting to having better quality practices and having better, you know, basically gates in place to make sure that you can’t ship bad software. But at the same time, one in five people say they won’t wait any amount of time for an error to be fixed. They’re just going to go somewhere else.

Joe Casabona: Wow, that’s so interesting, right? Because the common stat is like 80% of people will abandon a site if it takes five seconds to load, right?

Marcus Merrell: Yeah.

Joe Casabona: You’re absolutely right. I almost bought something last night and I was having trouble. And it was like, $1,000 thing. And I was like, “I mean, if this isn’t working, why would I shell out the money?” And I would have looked more into it on my computer but I forgot. I was putting my kids to bed and I forgot.

Marcus Merrell: Couple months ago, we were trying to close on a house. And closing is in three days, and I need to upload this bank statement. And I click the button to upload the documents and it just simply doesn’t open any dialogue at all. So what do I have to do? I have to try launching it in Safari and didn’t work. Try launching it in Edge. Oh, well, that worked. It’s like, “How is this still happening in 2022?”

Joe Casabona: And how are non-technical people supposed to… I mean, how many people have more than one browser installed on their computer?

Marcus Merrell: Goodness. And I’m a web tester by trade. So yeah, of course I have… But what a great point! How many people even have that?

Joe Casabona: You know, I’ll do the same thing. If something’s not working, the first thing I do is disable my ad-blocker. And then if that’s not working, I use Safari usually. So I’ll go to Chrome where things tend to work better anyway.

Marcus Merrell: Clear cache.

Joe Casabona: Yeah, clear cache. All that stuff, which is like… I’m lucky that I got my dad to watch the Yankee game a couple of weeks ago because it was on Amazon Prime. And he’s like, “I don’t know how to do that.” So I talked him through signing in on his Roku. And I’m like, “That’s a bad experience for baseball, for example. That is too many steps to get people who are used to putting on channel 11 to watch the game to then go and have to do that.”

Marcus Merrell: Yeah. And even when you’re lucky enough to have a TV that you could just dial into channel 11, for some reason, it takes like 10 seconds for it to figure out whether or not it’s working or not, an audio doesn’t come on for another five seconds after that. It’s just weird to me that, you know, we used to have this cable box with an up and down arrow and it just worked instantly. Now it has to go through multiple levels of filtering and processing in order to work at all. And now that that has gotten worse, we are also less patient.

Joe Casabona: I haven’t prepped you on this question but I am curious to hear your answer. Why do you think that is? My first thought is Amazon probably because like they made everything so fast and easy, but, you know, maybe there’s some other… Maybe people are just generally less patient because they’re used to fast internet connection.

Marcus Merrell: I think that’s gotta be it. We’re spoiled enough, I think, by the good experiences to not tolerate the bad ones. So if you’re not putting as much effort into the experiences Amazon is, or Uber, or something like that, one of those big companies where the stuff just seems to always work, then you really, really have to.

Because I mean, the statistics we see on this are just… I mean, 25% of users will leave a negative review. Half of them will tell friends and relatives. And this is all survey information from very recently. If you don’t get it right, you’re probably going to lose that customer because there is too much choice in this market not to be able to. I mean, now what I wonder is, how many apps are on everyone’s phone that have been used only once because the first time you tried to launch it you have to go through two or three-factor authentication, then you have to go through that identifying the truck on Google or… which of these tiles is a crossroad?

Joe Casabona: So much pressure, right? Is that little white spot part… Or traffic light is… Does the pole count as part of the traffic light?

Marcus Merrell: Exactly. And then the problem there is that you don’t even control that as a software developer. You’re letting Google come in and step in and step all over your user experience in order to authenticate you as not a bot. But if you don’t do it, you might have bots all over the place. So yeah, I think we’re losing patience because we do have some stuff that does work that well, but we’re limited in our choices. So we’re hoping that everyone comes along and gets that good. It’s kind of what Sauce is trying to help with.

Joe Casabona: You know, that’s so funny too, because it really does happen quickly. Now, if I download an app on my iPhone and it doesn’t have sign in with Apple, I’m not going to create an account. Like you want me to type my name and email address and a password on my phone when I should just be able to push a button? Like six months ago that’s just how everybody did it. But then Apple was like, “No we’re making it really easy.”

And to your point, that is something that is available via API just like I make sure to have like the Apple and Google Pay buttons on my shopping carts because I get annoyed when they don’t exist on a shopping cart I try to use.

Marcus Merrell: Punch in a credit card number by hand or something like that.

Joe Casabona: Yeah, right. Again, apps try to make it easier, right? Apple has their keychain and one password stores credit card information. But again, how many non-technical users trust or know that?

Marcus Merrell: Right. And really where it gets to be bad is, I mean, we’re here talking about stuff like I can’t order a rake on time. But when it gets to stuff that’s critical, government services, health insurance, that’s where… You know, if something goes wrong, you might be at real risk of losing something or not getting something accomplished that you desperately need to get accomplished.

And then you add to that the fact that 15% of people who use the internet require assistive technology for disabilities that they’ve got: visual, hearing, cognitive, neurodivergent. And 60% of those people who need assistive technology—I’m actually one if you consider glasses to be—they kind of have no recourse because they are unable on a daily basis to get a given task done.

So, to me, if you want to talk about how bad code costs you, one of the great frontiers of revenue growth would be invest in making your site accessible. Because right now there’s chance. If you haven’t put any thought into it, you’ve already excluded something like 15% of your audience. You add that in and your competitors don’t, suddenly your audience grows.

Joe Casabona: I’ve been beating that drum with podcast transcripts a lot because there are people who cannot listen to podcasts. And by adding the trans… Yes, that’s a real tangible cost, sure. But I’ve increased my consumership, my listenership? I’ve increased the people who can consume the podcast by adding transcripts. And I definitely saw a bump in strangely downloads when I added transcripts several years ago. So I think that’s a great point.

I talked at length with Amber Hinds about accessibility, so I’ll link her episode in the show notes, which you’ll be able to find out streamlined.fm/274.

There’s one more point I want to make here before we kind of move on to… Hopefully, we’ve convinced people at this point that they need to make sure that they… I mean, when we say bad code, we are talking about a bad experience, whether it’s your code or someone else’s code-

Marcus Merrell: Or design, honestly.

Joe Casabona: Yeah, yeah. That’s exactly right. You mentioned when something’s critical. My brother-in-law, his wife is self-employed. When he got married, they had to go through some sort of rigmarole to get her on his insurance. And the website failed and the paperwork never went through, and she was without insurance for an amount of time because of it. That is critical. Like it could be life-threatening.

Marcus Merrell: Yeah, absolutely. And what if you don’t have a smartphone? What if you don’t have a laptop? You cannot assume that everyone has the access to these kinds of technology to be able to get that. You need to be able to do it from a library or some terminal or something like that.

Joe Casabona: That’s exactly right. Have you read Eric Meyer’s book? The title is escaping me right. Now. I probably should have brought it up.

Marcus Merrell: I’m gonna say no based on the name of the author.

Joe Casabona: It’s basically a book about designing in crisis. It might be called that.

Marcus Merrell: Interesting.

Joe Casabona: But his daughter, sadly, passed away from a rare form of cancer. And before they knew, they had to get to the hospital. And so they’re like in the car looking for critical information on a hospital site and they couldn’t find it. Like the site was so poorly designed that they couldn’t even find the address to get to the hospital. Design for Real Life is what it’s called. I’ll link that in the show notes too. But you know, it’s not something you necessarily want to have to think about.

And maybe my membership most would not consider that life-critical. But if you’re making websites or you’re helping people make websites, those are things you need to think about. What’s the most important information for somebody who is going to the hospital and maybe they’re distressed?

Marcus Merrell: I mean, we always encourage people to take their app on the road. Take it out. We have one customer, a prominent bank in America, who tells us that they routinely send people into New York City subway to see how their app acts under duress in constant cell phone tower changes and connections disconnections.

It’s not that the app is expected to always work perfectly every time, but that under duress, it will at least let you know “can’t do this right now. There’s something in the way. There’s an interruption.” At least give you the feedback to know, “Okay, we either we can’t process this right now, or we process this in a queue and we’ll send it later.” Something like that.

Because I’ve worked on some apps, one theme park app I’ve used before where, you know, you’re walking around and you don’t really understand if the thing you’re trying to do is working or not. You feel like the app is responding but it’s not actually telling you. And then you figure out that your food order got lost in some package level because you were on a ride and lost your cell phone tower.

Joe Casabona: That’s so interesting. I love that. Because it’s also about like kind of failing gracefully. Right? I think there’s the common story of the Instagram app where if you like something it’ll store that like on your device if it’s not connected to the internet. So it’s like still positive reinforcement. And then it’ll handle it in the background.

Marcus Merrell: Queue for later.

Joe Casabona: I like to do this. I love going to Disney World. My brother works there. And I will like to open up the app and look at like the wait times it’s saying and then compare it to the wait times around the park. Really accurate. So nice work Disney, I guess.

Marcus Merrell: They’re fantastic.

Joe Casabona: Yeah, right. It really is. And on that same token, I was developing like a Star Tours tracker because you can get like… I think it’s over 60 combinations now.

Marcus Merrell: That’s right.

Joe Casabona: And I wanted to keep track. But it was like a web-based thing and I was like, “What if people don’t have internet connection?” So then I started looking into local storage using JavaScript and stuff like that. Like you said, if you are using the app… I would have never thought about that if I hadn’t tried to log a ride right after getting off the ride, right?

Marcus Merrell: One of the things I’m really excited about is that we… You know, bug reports are really hard for, you know, for a non-QA person, especially to say, “Here’s what I was doing. Here’s the steps I went through before this thing didn’t work.” That’s why I’m really happy with one of the companies that we acquired last year that sort of allows you to put an app in the field and then submit that kind of a bug report with all this extra information and telemetry and steps to reproduce all that stuff. It’s really great.

Joe Casabona: That’s awesome. For those who are interested in Build Something More, our members-only portion of the episode, Marcus and I are going to talk about acquisitions from the acquirer side. On this show I’ve talked to acquired people but I haven’t talked to an acquirer. So if you’re interested in that, you can join the Creator Crew. You’ll get ad-free extended episodes of this in every episode of the podcast. And that’s going to be over and streamlined.fm/274.

Bug reports are really hard. I created a template when I was doing client work. That’s basically “I was using this device with this browser at this time. I tried to do this, I expected this, and this happened.”

Marcus Merrell: What we try to do is say, “Here are the 10 things the user did on the device before they submitted a bug report. And then here is the log from within the device itself. And here are all the HTTP requests that it tried to make at the time and maybe there was an API failure in there.” All that sorts of stuff about status around where the battery was and just to all that. So they could fill in all the blanks that developers naturally going to ask once you submit a bug report. Also it includes like a 10-second video, I think of the last 10 seconds before the bug was submitted. So, yeah.

[00:24:05] <music>

Sponsor: This episode is brought to you by LearnDash. Look, I’ve been making courses for a long time, I’ve taught at the college level and I’ve created curriculums for several different organizations, including Udemy, Sessions College, and LinkedIn learning. When I create my own courses, there’s no better option than LearnDash.

LearnDash combines cutting-edge eLearning tools with WordPress. They’re trusted to power learning programs for major universities, small to mid-sized companies, startups and creators worldwide. What makes LearnDash so great is it was created by and is run by people who deeply understand online learning, and adds features that are truly helpful for independent course creators. I love the user experience.

And now you can import Vimeo and YouTube playlists and have a course created automatically in seconds. I trust LearnDash to run my courses and membership. And you should too. Learn more at streamlined.fm/learndash.

[00:25:12] <music>

Joe Casabona: Let’s see if we can apply this information. As a creator, as a small business owner, maybe they’re thinking, “Well, this is great, but I’m not a developer. I’m using a tool like ConvertKit to sell my products. I assume they’ve tested it. So what should I be doing?”

Marcus Merrell: I think that it’s possible that, you know, everything’s fine and they have tested it. But there are a couple of things that I would watch out for. So I think in general there’s some stuff I would watch out for, then there’s some stuff that I would think, “Yeah, that probably is covered.” And then there’s the stuff that you definitely need to make sure you keep testing.

So what I think you should be comfortable with is that if you have a, you know, just a simple web form element, button, text field, you know, and you’re trying to put together a workflow or some sort of a list of items that you’re trying to sell… I don’t know typical example of the kind of flow you’re talking about here. But if it’s like an online retail site… I don’t know if that’s a good example or not.

Joe Casabona: Yeah, totally. Let’s say somebody’s signing up for a membership.

Marcus Merrell: So there’s a really good chance that the login is going to work. You should test it. But like, once you’ve tested it a couple times, I don’t think I would build like a whole suite of automation around making sure that the login is working. Stuff that you are… I would say that if it’s anything that you’re confident that 90% of that provider’s other customers are doing, it’s probably in good shape.

If you’re trying to combine things in a way that you think is probably net new or if you have to customize any of it, then that’s where you would probably think, You know what? These things change in the background, and they release new versions of it all the time. It’s a cloud provider, it’s a software as a service, which means you don’t control when the versions get shipped, it don’t get packaged up, they’re gonna change stuff in your background all the time.

So I want to make sure that my stuff continues to work. So once in a while I’ll run an automated test, especially if I have any amount of customization in there. Because if I have customization, there’s a chance that what we call the API contract might be broken with the back end service. And I could explain that if you think I should. The main thing…Go ahead.

Joe Casabona: Oh, yeah, I was just gonna say, yeah, let’s go through this and then maybe we could talk a little bit about that. Because I also want to ask you about some automated testing tools.

Marcus Merrell: Sure. Great. The thing that I would definitely test at least a few times is what I think we agreed was called the user analytics. So at my previous employer, it was an ecommerce site. Very, very large eCommerce site. And we relied extensively on user analytics to do three different things.

One of them was essentially tell us, describe to us what the user is actually doing on our site. The second one was to help us understand the SEO, Google Search Engine Optimization success and failures that we were having so that we could get a heads up on problems with SEO before Google could tell them about us. And then the third one was attribution for affiliate links.

We made our money by sending our customers to websites like Macy’s or a retail site. And we would send them with a coupon code, we would drop a cookie that would give us attribution so that we got 6% of the sale or whatever the commission was.

And then one day, I did all this work and kind of I was thinking, “Okay, I’ll just test to make sure all these analytics events are being sent when we think they’re being sent.” And then I found out that something like one out of seven because it was in this position on the page and it was in Chrome, and if it was on a Tuesday at 7:30 in the morning, you know, if you clicked on it a certain way, it simply wouldn’t get sent. And like one out of seven of our code paths simply were not dropping the right attribution. And the funny thing is, it was providing a fine user experience. The user did not notice a difference but we weren’t getting paid-

Joe Casabona: Wow.

Marcus Merrell: …on something like 1/7 of our transactions. And it was shocking.

Joe Casabona: That literally cost you money.

Marcus Merrell: Yeah, literally cost us money. So yeah, it’s not just protecting the user experience because… In a way it is protecting the user experience because those user events couldn’t feed into our sorting algorithms, prioritization algorithms, SEO algorithms. We were not able to actually accurately describe what was going on with the customer at the time.

So testing analytics, even if you’re with WordPress or no-code, a low code site, making certain that those analytics events are exactly what they should be is crucial.

Joe Casabona: I mean, I wasn’t losing like one-seventh of my revenue, but I have a similar story where I chained a couple of tools together. I was using a form builder in WordPress, because I didn’t like ConvertKit’s native forms. So I was using a form in WordPress to send the form that was filled out to Zapier. And then Zapier would send that information to ConvertKit.

So if anybody wants to join my mailing list, they would fill out this native WordPress form, which I thought looked nicer, and then presumably, their information would get sent to ConvertKit. And I noticed that for like, two weeks, not a single person signed up for my mailing list.

And I’m not getting a ton of people on my mailing list every day but nothing over a two-week span raised a flag. I think I was testing the zap or there was something in the WordPress side that got turned off, so people were filling out the form, and it looked good to them. The form was quote-unquote, “submitted,” but the ZAP was never firing.

And I, because I’m a big dummy, didn’t have a backup where the information was also getting submitted to WordPress. I was like, “This works.” Right. So checking those analytics and testing to make sure is really important. And I would also say, if you’re making forms in WordPress, always store the form submission. I’m not a lawyer, I don’t really understand GDPR, but do the things that you need to do for GDPR if you need to do things for GDPR. But yeah.

Marcus Merrell: Good point. Yes, absolutely. Storing PII.

Joe Casabona: Yeah, absolutely. So you mentioned some automated… Well, actually, let’s back up for a minute. One thing that probably should always be tested is payment gateways. I guess my question around this is, what is a good way to test kind of payment processing? Like Stripe offers the test mode? Is that good enough? Should I do like a real $1 transaction with an actual credit card? Because that’s like people are submitting… That can also literally cost you money if the payment gateway is not working.

Marcus Merrell: Right, right. Or if you do something… Like I can imagine making a mistake and saying, “All right, we’re gonna run our test automation on every single commit.” And people do a thousand commits and you realize, “Oh, I didn’t turn off that $1 test.”

Joe Casabona: Yeah, right. Yeah. I mean, this answer might be a little bit nuanced and maybe a little bit complicated. But what I would say is, your job is not to test PayPal. Their software either works or it doesn’t. It’s either down or it’s not. PayPal has published an API that is a contract to you. That contract has a list of endpoints, a list of data types that it accepts, and then a body that it returns to let you know whether it’s succeeded or not. Your job is not to test that. Your job is to understand whether your software knows how to communicate with that contract or not.

So what I would recommend in those cases is, yes, do your test mode. Testing, you know, once in a while. But mostly for I’d say for 90% of the test cases you need, test against a mock endpoint. Make sure that you know that your software can absolutely communicate with that payment gateway 100%. And if that gateway is taking eight seconds to respond one day and you have 500 tests that use it, then that’s going to be 500 tests times eight extra seconds for each test that has to run.

If you mock out the endpoint, then it’ll return in 0.1 seconds and you’ll be certain that your software knows how to deal with PayPal. And then you have a few tests that just say, Okay, we know that our software knows how to communicate with that contract. Now, let’s make sure that it’s actually working right now with the contract. So that’s what I would say. It’s kind of a complicated answer, but it’s the way you can reasonably ship software with confidence and not have to worry about, “Oh the payment gateways test site is down right now. I can’t do anything.”

Joe Casabona: Right. Right. I mean, that makes sense. And that’s like all the more reason to use tools that… You know, like WooCommerce for example, you can reasonably assume that WooCommerce and eCommerce software for WordPress knows how to communicate with PayPal and Stripe. Like it’s built-in.

But then to your point earlier, if you’re doing more nuanced stuff, right? Like one example is I was using WP Simple Pay, which allows you to add these really simple Stripe buttons to your site for simple one off transactions. It’s not a shopping cart. So people would just click the button and pay the amount. And I had some metadata that I just assumed was getting sent to Stripe so that it would kick off, again, Zapier automation to get them into like this customer flow.

But that metadata, I had to explicitly send it to Stripe using the plugin or something like that. Like it wasn’t getting sent at first. So that’s the kind of testing… Like I knew that the money would get captured and that’s like maybe the most important part, but like, the second most important part or the most important part to the user is their good experience. Right? If they’re not getting that initial, like for a coaching program, that initial email, saying, “Hey, thanks for paying. Grab a time on my calendar,” because the automation associated with the payments is broken, that’s on me. And now the person who just paid me like $1,000 is already having a bad experience.

Marcus Merrell: Yeah, that makes sense. I mean, there’s where you need kind of a confederacy of tools. But I think making sure that the metadata gets sent. It’s a little bit difficult to do that in an automated fashion, but it certainly as possible. In the past, you would always have had to use something like a proxy or network tools in order to be able to make sure that the HTTP request is sent properly.

Joe Casabona: Right.

Marcus Merrell: At Sauce, we have tooling that allows you to sort of download all the HTTP activity generated by the app that you’re trying to test.

Joe Casabona: Nice.

Marcus Merrell: We have that now on mobile devices. In fact, if you just run a test session with the right things enabled, you’ll get a hard file at the end of it that tells you everything that happened.

Joe Casabona: Nice.

Marcus Merrell: That feature is only three weeks old. So I’m super excited about it.

Joe Casabona: That’s awesome.

Marcus Merrell: In the past, you would have to hook up something like Charles Proxy, or BrowserMob or something like that. And now we just sort of offer it to you natively, which great.

Joe Casabona: That’s great. There’s also.. What’s the… There is like a Mac app that tests API’s. I want to say it’s called Mailman or something like that. But that doesn’t make a lot of sense.

Marcus Merrell: Gosh, I don’t remember like.

Joe Casabona: But it tests API’s. I’ll try to find it for the show notes. I can’t remember right now.

Marcus Merrell: And there’s a bunch of free tools on the market. I mean, if you’re into coding, REST Assured is a really nice Java library for testing API’s. It’s what I used before I discovered some of the commercial tools that I think make me have to do less work.

Joe Casabona: Nice. So let’s talk about maybe, let’s say, there’s a creator who’s not really technical. Are there tools for them, where they can kind of get an idea of like a user experience without kind of digging into like these API contracts that we’re talking about?

Marcus Merrell: Well, I mean, it’s tough. The API contract stuff, I don’t know of an easy way unless maybe… I mean, so small creators probably wouldn’t be using Adobe analytics. But I know that Adobe Analytics has a lot of good tools for making sure that these things are done right.

It definitely is a little bit tough for this kind of thing. That’s where I would probably try to get maybe a few hours of consulting time from some professional tester. Because I don’t know if I know of an easy way to just do that.

Joe Casabona: Yeah, that’s tough. That almost feels like it needs real person. What about the front end stuff? There are certainly front end testing tools. Hotjar maybe. It’s not really testing but it gives me insight into how the user is using my site.

Marcus Merrell: I don’t know that tool, honestly.

Joe Casabona: Oh, it’s pretty neat. It’s like you install a little bit of JavaScript on your site, and then it records user… assuming they accept cookies. It kind of records their session. So you could see like how much time they spend on the page, and then you can see what they click and where they’re going to, and then where they exit. It’s pretty neat though. And then it can generate a heat map for you of the most…

Marcus Merrell: All right. Sold on this tool. I’m already looking at the website.

Joe Casabona: It looks really good. Hotjar, if you’re listening sponsor this podcast.

Marcus Merrell: Yeah, right. It’s pretty cool. I mean, it would really compete with us. That looks pretty great.

Joe Casabona: Nice.

Marcus Merrell: I am all about user behavior, actual user behavior. I’m definitely going to be looking at this. And I know that there’s a couple of others. There’s one called… is it a Datadog?

Joe Casabona: That sounds like-

Marcus Merrell: There’s another one. FullStory is the other one I’m thinking of. But I think is more of an enterprise level. Enterprise level. One thing I’ve observed in my shift over the last 20 years in QA is I used to spend all of my time trying to say, “Have I covered all the use cases? Have I covered everything? Have I exercised every piece of the software to make sure that I’ve executed every single line of code to make sure that I know I can say that this product works as designed?”

Over the last 20 years as I’ve evolved, and I think the industry is very much evolved to a point of saying, rather than let’s make sure that the product works as designed, let’s make sure the product works the way it should work for the betterment of our users to the point of Hotjar. But furthermore, we’re shifting from a concept of asking, “Does it work?” to a point of saying, “Have we covered our risk the most?”

So what I’m encouraging people to do these days… And I can tell you a story about this also. I was very proud at one point. I had a team of offshore developers and they did a great job. I went on vacation, and I said, “Automate everything to do with the user profile page while I’m gone.” And I went away, and I came back a week later, and they said, “We did 200 Selenium test cases dealing with the user profile page.”

So now if you can edit your user profile, you can change your avatar, you can change your name, you can change your zip code, and the analytics all work. We run 200 Selenium tests. This stuff works. It’s stable, they’re green, and everything.” And I was so happy about this.

I went back and I was talking to someone in business intelligence to BI Group at lunchtime, and I said, “We just spent a week auditing, many making sure that we could do 200 use cases.” And the person from bi said, “Less than 1% of our users ever go to that area of the site.

Joe Casabona: Oh, wow.

Marcus Merrell: “Why in the world would you spend that much effort on that part of the site that nobody cares about, that will not generate a dime of revenue?”

Joe Casabona: Wow.

Marcus Merrell: That conversation changed my career.

Joe Casabona: I mean, spend your time wisely is kind of the thesis of this whole episode. You want to make sure that you’re doing the things to create the best experience for both you and your users.

Marcus Merrell: That means if you’re a retail site, you need to test to make sure that your search engine lands you at the right place, that the top coupons are at the top, that the experience of getting someone through a checkout flow is flawless. And if you do that, that’s probably 90% of your revenue.

Joe Casabona: Oh, man, I love that. I mean, I have a story about that where for a while on my membership site… this is one of the things like I just assumed the software worked as I expected it to work. But I had it when someone signed up for a membership, they would get access to all of my courses. And then separately, I had an automation that would email them whenever they were enrolled in a course.

And since the membership plugin was different from the LMS at the time, the membership plugin would auto-enroll them in all the courses which would trigger that automation for every course. And it wasn’t until, I don’t know, like 10 or 12 members did it that someone finally emailed me it was like, “Hey, did you know I’m getting like 12 emails?” And I’m like, “No, had no idea.”

So I basically killed all those auto-enroll emails and set up one through ConvertKit, which is now where all of my emails come from that says, “Hey, you have access to all the courses. You can access all of those courses here.”

Marcus Merrell: Yeah, you got to protect that experience. My friend Titus Fortner, he’s a big test influencer. He might be someone to talk to one day.

Joe Casabona: Nice.

Marcus Merrell: He says, “Test what the money wants.”

[00:43:22] <music>

Sponsor: This episode is brought to you by StoreBuilder from Nexcess. When it comes to setting up an eCommerce site, you have a choice between easy but limited or a limitless platform that you need to manage yourself. Until now. StoreBuilder is ECommerce made easy for everybody. It saves you time and delivers a storefront that lets you get to selling.

As someone who set up multiple eCommerce sites, I can tell you that StoreBuilder has been a much easier experience than anything else. Answer a few questions, add your content, and sell. StoreBuilder was created and is supported by eCommerce experts at Nexcess. Get the speed, security, and support you need when you need it.

Are you ready to launch your perfect online store? Head over to streamlined.fm/storebuilder for a special offer. That’s streamlined.fm/storebuilder.

[00:44:19] <music>

Joe Casabona: Let’s wrap up with maybe this question. You have your own set of tools. I’ve talked about kind of building my own WordPress tools. And maybe this has already been covered. But let’s say somebody is using a site like Etsy to sell. Is there something that they can do to make sure they’re offering the best experience to their users?

Etsy is a pretty well-established store, but maybe, you know, I think that’s something that usually falls to the wayside is like the emails that go out? Should people be like customizing those? Should they be mindful of something that they might not see as the store owner that the customer will always see?

Marcus Merrell: It’s hard for me to say that. That actually falls into the realm of, you know, the real business expertise. I would just encourage people to really pay attention to the reports that you get out of Etsy around how things land. And there’s probably all sorts of stuff you could do to make sure that you show up in search a little bit better and everything.

I don’t know if there’s really, you know, technically QA to be done on something like that. But certainly, there’s ways you can go about optimizing the experience and making sure that… The best Etsy experiences I’ve had had been ones where the owner has been very responsive and very interactive and engaging and positive and that kind of thing. That goes under I would think common sense.

So I would think there’s a lot more variability and something like WordPress where it’s basically a blank slate. And that’s where there’s a lot more variability. And you do control the workflows. Even though you probably have a template, there’s probably at some point going to be some way in which to diverge from what other people are doing. That’s where you really want to make sure that you have…

You know, hallway testing is a phrase that I think maybe has survived the pandemic. But grab someone from the hallway, grab, you know, your mother… I always try to have my mother or father check over something like that to say, “Look, can you get through the flow? Can you do this?” Because if you can’t, I am not objective. I’ve been doing web stuff for 20 years. I can’t just, you know, assume that my experience is the only experience. So hallway testing… grab someone.

Joe Casabona: That’s such a great tip. I usually will test on behalf of my wife. But testing on behalf of my wife is not my wife testing, I should say. So that, I think, is such a great point no matter what system you use. But yeah, that email example definitely came from my experience with WordPress, where again, I have WooCommerce, I had LearnDash, I have ConvertKit, all of these things are sending emails. And so how do you control that experience?

Well, WooCommerce doesn’t really do a great job of helping you control those emails. You could get a paid or premium plugin. Or you can turn those emails off and manage it all through ConvertKit, where ConvertKit integrates with WooCommerce. You send purchase data and email, and then you could tag them as such, and you can add them to a welcome purchase sequence and stuff like that. But that all goes to thinking about the experience.

Marcus Merrell: Well, I can tell you. One little technical tip I’ve learned somewhat recently that I think could help at least with testing email flow, is there’s a trick in Gmail. I don’t know if you know about this trick. That you can use a plus sign inside your alias. Do you know that trick?

Joe Casabona: I know that trick. But I don’t know if I’ve said it on the show.

Marcus Merrell: What I’ll do is, you know, my email address—I don’t mind giving it out, whatever—is at mmerrell@gmail.com. But if I say, mmeril+1234, on WooCommerce that would take me as a separate user. But Gmail treats everything before the plus sign as the alias. So you can have roughly infinite number of different email addresses for the site that will always go to your inbox. So you can test the flow as many times as you want using the plus sign hack and then all you need to do is make sure you keep track of which test case was which thing that came after the plus sign.

Joe Casabona: Yeah, that’s great. That’s really cool, right? Because you could do like, you know, to test and then a timestamp or whatever if you’re testing across multiple data. That’s really-

Marcus Merrell: You can test today and then test your multi-day workflow for something like a Marketo, something like that.

Joe Casabona: Oh, that’s super cool. I’m gonna treat that as the trade secret. I usually ask if you have a trade secret for us. I’m treating as that. That’s a really good tip. Let’s wrap up here with… I mean, we haven’t really… I mean, if someone’s listening to this, I’ve mentioned what Sauce Labs does in the beginning. But let’s kind of get it in context. Because I think you mentioned Selenium. You mentioned in our pre-show that you would do user interface testing, which I know of tools I think like Casper that might do that sort of thing. That’s like a JavaScript library where it like simulates user interactions. And you can write those tests. But what exactly does Sauce Labs do, and how can that help creators?

Marcus Merrell: So Sauce Labs is trying to sort of help people enter the field of test or ensure a good user experience every time. So we’re trying to provide testing at all stages of the software development lifecycle. We started off with Selenium. So if you had a set of Selenium tests that did automated UI testing, then we would say, Well, you can only run those once at a time on your own machine. You know, if you have 50 tests, that means you have to run all 50 one at a time on Chrome, and then you have all 50, one at a time on Firefox, and all 50 one at a time on, you know, Safari.

What we do is we provide the infrastructure to say, you can actually run all 50 of those at the same time in parallel on Firefox, Chrome, Safari, and Edge all at once. So instead of taking, you know, four hours test, you could test theoretically in about five minutes, because you could just do all of it in parallel.

And then as the company evolved, we started offering other kinds of testing platforms. Our frameworks like Cyprus, Puppeteer, Playwright, TestCafe, the JavaScript frameworks. We also offer mobile devices. So if you want to do your app testing web or native app, we offer any number of iPhones, Android devices, all sorts of stuff. Like if you need to make sure that you’re supporting iPhone seven users, because you… You know, I think there’s an interesting thing to be said about knowing your audience. If you work for something in healthcare, there’s a fair chance that you have maybe an older population that might have older technology. And you can’t always get a hold of an iPhone 6.

Joe Casabona: Right. Right.

Marcus Merrell: We’ve got plenty that you can use on older versions of iOS. So that’s the kind of thing if you need to test for device diversification that is hard to get a hold of. You don’t want to buy 50 devices and have them in a rack and have to charge the batteries and the power supply. My brother borrowed that one the other day and he logged and it’s his own… So you don’t want to mess with that. So we provide that kind of infrastructure. And lately, we’ve been getting into other kinds of testing as well like API testing in this field beta testing, like I talked about.

Joe Casabona: That’s great. And such a great resource too. When I when I wrote my book, Responsive Design for WordPress, I had a whole chapter on testing. And I was basically like, well, device labs exists, depending on where you live. BrowserStack exists if you’re willing to pay. You could go to the AT&T and load your website up on all of the phones there.

Marcus Merrell:

Joe Casabona: Yeah. I’ve done that before. And so like I left and like a bunch of fonts just had my personal website up on it. But I mean, being able to do this in an automated way is a) saves you time, b) makes it more thorough and makes it more consistent, right?

Marcus Merrell: Mm-mm.

Joe Casabona: Because, you know, maybe I phone in the last couple of tests because I’m tired of testing and I’m just like, “Clickety, click right. Yeah, I tested this button a million times. Surely it also works in Edge.” Right?

Marcus Merrell: Yeah, right. That happens so much. Especially once you start getting into analytics testing, oh, my God, it’s the most tedious thing in the world. But it’s so important you can’t skip it. When I was in my last job, the eCommerce company, we had rooms full of people tapping and clicking and hooking up to proxies and watching HTTP logs and saying, “Okay, the events all went out as scheduled.” It’s like, “Okay to check all the key value pairs?” “No.”

Joe Casabona: No.

Marcus Merrell: So yeah, automation… You know, there’s certain things you should not automate but there’s certain things you should absolutely automate.

Joe Casabona: Yeah, absolutely. I’ll end on this lot because I tried to sign up for a big companies service recently. Like they offer like a very expensive enterprise option and they’re very well-known, and they didn’t accept American Express. So this was a disconnect. I had to reach out to support for confirmation. Right?

So two failures here. One was that the error code I was getting was payment cannot be processed. It wasn’t like, “You’re using an Amex and we don’t accept Amex.” I think any place… Maybe I’m like calling you out. I hope I’m not. I think any place that offers an enterprise solution should absolutely accept American Express. Because I feel like at least in the United… Maybe this is a very United States-centric thing to say. But I’m gonna say most big companies have a company card that is an Amex card. I think that’s a fair assumption.

Marcus Merrell: My experience, the only places I feel like I’ve run across that haven’t taken it recently are government-related. Like license plate registration, stuff like that. But I mean even then they’re coming around. So that’s surprising to me.

Joe Casabona: My former web host didn’t accept it on their website. So every time I came up for renewal, I’d have to reach out to them and do like some weird… I’m like, “This is annoying.” And then I realized-

Marcus Merrell: What points of friction you put in the way if people getting to do their job, man. They want to give you their money. That’s the thing. They’re sitting out there waiting for you to give them the money.

Joe Casabona: That’s exactly right. This is what I say. I’m like, “How can I pay you? However, you prefer to pay me is how you can pay me. If people are trying to give you money, make it as easy as possible for them to do it.

Marcus Merrell: Yeah, absolutely.

Joe Casabona: Awesome. Well, Marcus, this has been fantastic. As I mentioned, we’re going to talk about acquisitions from the acquirer’s point of view in Build Something More. You can find everything we talked about, including a way to become a member over in the show notes at streamlined.fm/274. Join the Creator Crew, by the way. It’s 50 bucks a year. That’s less than five bucks a month. I paid $6 for an iced coffee today. So less than five bucks a month $5 or $6 for an iced coffee that I finished in a few minutes. You determine the value.

But Marcus, this has been an absolute pleasure. Thanks so much for joining us today.

Marcus Merrell: Sure, thanks. I’m glad to be here. Thanks, Joe.

Joe Casabona: And last question, because I lost it in my massive notes document here. Where can people find you?

Marcus Merrell: Find me I am @mmerrell on Twitter. I don’t do much on Twitter. I mostly retweet other people. And also I’m on LinkedIn somewhere as just Marcus Merrell, my name. And then saucelabs.com.

Joe Casabona: Fantastic. I will link to all of that as well in the show notes again over at streamlined.fm/274. Thank you so much for listening. Thanks to this week’s sponsors. And until next time, get out there and build something.