Better Website Staging and Testing with Nathan Tyler

Joe Casabona: This episode of How I Built It, is brought to you by two great sponsors. The first, is our season-long sponsor. Liquid Web has been best known as a managed hosting company with tons of options. It’s also designed a managed WordPress offering that is perfect for mission-critical sites. If you’re looking for improved performance, maximized uptimes, and incredible support, Liquid web is the partner you’ve been looking for. Every liquid web managed WordPress customer has ithemes synced integrated into their managed portal allowing them to update several sites with a single touch. Liquid web hosts all of my critical websites and I couldn’t be happier with them. If you Sign up today, using the discount code ‘howibuiltit33’, you get 33% off for the next six months. Visit buildpodcast.net/liquid to get started. That’s buildpodcast.net/liquid.

It’s also brought to you by the WP CRM system. WP CRM system is a customer relationship management system that is designed for digital marketing professionals unlike other CRM solutions on the market, WP CRM system believes that you should always be able to access your data. That’s why the WP CRM system doesn’t charge more for extra users, or lock in to expensive contracts just to access your customers information. WP CRM system is run on your WordPress website, not a third party website so you own all of your data. It integrates with many popular plugins such as Gravity Forms, WooCommerce, and Easy Digital Downloads as well as third party apps, like Slack, MailChimp, and Zendesk. in addition to customer records, WP CRM system can also be a project management tool for your growing business. Display progress reports to clients and their customer client portal and collect payments via Stripe through the BoltOn invoicing system. Go to builtpodcast.net/crm to learn more and to get 20% off any plan with the discount code “How I Built It”.

Now, today, my guest is Nathan Tyler. I met Nathan at WordCamp DC over the summer and we got to talk about this really cool project, Staging Pilot. And so I asked him to be on the show, he also sponsored an episode awhile back, and it’s a very interesting conversation because we get pretty developery which I totally love to do. So we talked about his smart algorithm to figure out differences between the production in the staging site and a whole bunch of other things. So make sure that we get a developer hat on for this one because it is so doozy. And without further ado, on with the show!

Hey, everybody. Welcome to another episode of How I Built It, the podcast that asks “How did you build that?” Today, my guess is Nathan Tyler of Tyler Digital and Staging Pilot. Is that about it, right? I should have asked you that a moment ago. Awesome! Well, Nathan, welcome! Thanks for being on the show.

Nathan Tyler: Yeah. Thanks for having me.

Joe Casabona: So we met at WordCamp DC over the summer. I was very interested to hear about Staging Pilot because staging sites are like having an environment that is safe from production but mimics production well. It can be very difficult, right? Like managing a bunch of different changes so I’m always interested to hear about that stuff. But let’s start at the beginning. Why don’t you tell us a little bit about who you are and what you do with the idea?

Nathan Tyler: Sure. So I’m Nathan. I’ve been doing consulting work since 2004. That’s when I started doing client work and I’ve worked on a variety of little projects and plugins and product Staging Pilot being the most recent, trying to do a little bit of both which has obviously been a challenge that I think a lot of people have tried to take on.

So a Staging Pilot kind of came about because my maintenance business, like my consulting building websites for clients business, we were having a lot of success with maintenance plans. And finding that was our kind of gateway into recurring revenue and updating peoples plugins and testing everything worked and sending them the updates of what happened. And we got up to about 50 maintenance plans which was really giving me more freedom in time to pursue products and projects. But it was just a really overwhelming amount of time that it was taking. One of the guys that I hired on to help me with development tasks and building out sites like probably a capital month was going to just deal with all of these updates and the issues that would pop up. And definitely the one David would be marked to decide to update this site would blow up. And then they went the rest of the day or the week and it was always an emergency. So the very beginnings were like how do we automate little pieces of this? And I of course estimated it would take me maybe a month to hack out some shell scripts and WP CLI thing that we get automated all this stuff and it’s kind of grown and grown and grown over the last three years to where it is today.

Joe Casabona: Nice. So estimated a month and that was three years ago.

Nathan Tyler: Yeah, yeah, exactly.

Joe Casabona: That sounds like most software projects, right? Well, actually kudos to you right? Because I am about to do something and then we’ll hit that month and I won’t be done and I’ll be like, “Well, I guess this has never seen the light of day.” So kudos to you for sticking with it. And I’m always really interested in this especially now because I’m in the product space fully. Like finding it very hard not to take on client work. What’s it like kind of straddling the line between freelance work and selling a product because there were two very different things, right?

Nathan Tyler: Yeah. It was very difficult, especially at the beginning when you just don’t have the time to dedicate to it. I think the first thing that got me enough freedom to spend time on the product was the maintenance plans of the recurring revenue because I had that base even if we’re not doing these projects work that was the one thing that was you know, over the years was building up to the point where we have like 50 of those ongoing plans like it gives you enough to cover all basic senses and start stepping in that direction. So that was definitely a big help and then honestly, it didn’t work so well like you know, when you’re trying to split your time between clients and projects, products like the clients always wins, you know? It’s always an angry email and it’s always very clear what needs to be done versus like your product is this big abstract wonderful thing. You have to think about a lot and it’s always easier and more demanding and more clear to have to work on that urgent priority from a client. So, the thing that I found is that it was getting neglected for quite awhile. And then actually hired someone and then this just said like you’re not touching client work like only working on products. And that was probably the biggest moment for me. I was like psychologically, I was happier at least working on you know, this client stuff I wasn’t like the grudging and angry that like, “Oh, my product isn’t getting anywhere”, and I knew that it was making progress every day, you know? Incrementally even if I wasn’t actually writing that code. And also it was just like a consistent effort you know, and overtime, I’ve ramped up my time. At the beginning I was spending 3 or 4 hours a week, and then we had full time work going on it, and then I’m kind of increased my time. So at this point, I’m like, I’m down to only a couple actual client projects and I’m very close to getting all the way to products.

Joe Casabona: Nice, that’s awesome. I mean you raised a couple of really good points there which is, you know, when you’re splitting your time, it’s like…for me, it’s a lot easier to just choose the freelance work, well this is guaranteed income. And it’s not necessarily guaranteed that it’s a lot of work to market your product and stuff like that. But it sounds like you made time by hiring somebody who you decided this person would have the time or focused on nearly guaranteed income and you focus on this product that I want to build. So that’s interesting for anybody out there who wants to kind of do both and wants to move from freelancing to products. You probably need to make a decision about how you want to spend your time, and if you are able to hire somebody to help you get there.

Nathan Tyler: Yeah. And if it was just me, like I think if I was like solo, I probably would have just made that switch and started reallocating my time. But you know since I’ve been building the consulting site for years like I had like a team and so I just like pulling the plug and saying “Oh! Yeah, we’re just doing products now”, like you know, nobody has any work to do, like it wasn’t really something I wanted to do or would do. So, I was slightly in different situations. I mean there are different ways you can go about it but it did kind of help get the ball rolling you know? Then I could kind of start switching over my time. It was a little further or longer.

Joe Casabona: Nice, that’s awesome. So it sounds like part of the research was just figuring out the things that you do that possibly could be automated. But did you do any other research to see what other projects are out there? Or what kind of features do you want?

Nathan Tyler: I’m sorry if I’m plaid that there wasn’t any research process. No, actually as we were building all that stuff, I was simultaneously trying to pitch a different product to beat. It was just absolutely nowhere. I obviously had that self-rationalization that many entrepreneurs do all this…you know, I’m going to build this for myself and other people might want it but it was primarily like starting out is just something that we were going to use internally and to save time and automate all these processes away. I figured there would be other people in my position but I was not like that was not going to be the main thing that I was trying to promote or sell. And as I was talking at WordCamps and trying to pitch them my other thing, I would get that glazed over look in their eyes and you know like, flat responses. But then, I did start to get questions about this from people that I meet in that group at WordCamp and try out the Staging Pilot thing on a couple of my sites. So that kind of fell backwards into my research process as well. Actually some people are actually asking about this proactively and there might be some interest there.

Joe Casabona: That’s funny, I mean especially on this show, I’m talking to the people who have the products that they’re just scratching their own itch and it became a popular thing. And it seems this seems to be the case for Staging Pilot too but you never hear about all of the projects that are there. Well I thought this would be a useful site, I thought other people would find it useful too but they don’t. I find a lot of, I mean, I guess the same thing could be said about a lot of things I do and then the podcast is the thing I backed into. And it sounds like “Oh, I’ll do a podcast to help promote my online courses.” And the podcast is way more fun than online courses. So that’s really interesting. So conversations with people at WordCamp some things like that, do you have a mastermind group or contemporaries that you talk too?

Nathan Tyler: Yeah. I’ve gone for a couple. So you know I have a couple guys that we talk like once every month. And you know that’s kind of helpful just to kind of bounce ideas around. And what’s inside the WordPress space, what’s outside of WordPress is just kind of like online business which is it’s always good to get a slightly different perspective too.

Joe Casabona: Absolutely. And that was my follow up question. You know, I’ve been in mastermind groups where it’s like vastly different backgrounds and I’m currently in one where we’re all kind of course developers for different people. And you know, I think that finding a good mix or finding a couple of mastermind groups is important. You want people who understand your business but it’s also great to have a person who doesn’t at all understand the space here and to ask you basic questions or leading questions that could help you come to a different answer, right?

Nathan Tyler: Yeah.

Joe Casabona: Awesome. So it’s been awhile at least in the order in which I’m recording my episodes that I’ve talked to a developer. So I’m very excited to ask you this question which is, “How did you build it?” I know you mentioned shell superscript and WP CLI but I’m excited to get a meet and greet text up here because I haven’t talked to tech seriously with somebody in awhile.

Nathan Tyler: Okay. Well, yeah. There’s a lot of it. Yeah, sure.

So the very basic setup, you know, we set some WP CLI scripts to automate the actual plugin updates and then we needed an environment. A staging copy of a site to run all those updates and then we implemented our tests. Actually I don’t even know what we carry at Staging Pilot. I think we jumped right in. But I probably should have covered that into the intro.

Joe Casabona: We both definitely made an assumption that people understand what staging environments are. So yeah. Let’s rewind a little bit and say what the Staging Pilot does, first?

Nathan Tyler: Okay, so basically if you’re selling maintenance plans for your clients it helps you automate all of those steps. So creating…because everybody knows you are not supposed to click update in production and cross your fingers and hope. But, yeah, it’s supposed to be done in a local development or staging environment. But everybody knows that keeping those staging environments In Sync and up to date with the live site is slow and tedious and a lot of times we just end up clicking in production. So Staging Pilot probably creates a copy of the site, does all of the updates there. It will actually take screenshots of all your pages before and after. And they do a pixel by pixel comparison and figure out if anything broke as a result of these updates. And then, if everything passes then it goes and applies those updates to the production site. It also generates an email report that you could send to your client at the end of the month. So basically, all those different steps you know fulfilling a maintenance plan or taking care of. And if everything is going smoothly, it’s all 100% automated. If something breaks you get a notification and you can look at the floor in the after view of the page that broke and tell you which plugin was updated when it broke and you know all the updates get halted there and you get a chance to address it. And that’s how we differentiate ourselves from, you know, full-service companies like WP Sitecare or something like they’re actually fixing the problems for you. We’re a tool for the agency and for the developer who you know they built the site and you know they’re offering the maintenance plan and they’re going to get that profit 90% of the time where we didn’t know nothing to do at all. But you know when there’s something to fix, we’re learning with them before it goes live and so they have time to work it into their schedule. But then, they are responsible for fixing it. So that’s what Staging Pilot does.

And then under the hood, I would say a lot of tech to do that. So we have the actual updates themselves you know are updates that are run by WP CLI to set up that staging environment. We have like one click integrations with a couple hosts you know like Pantheons, Cloudways and working with WP Engine so that we can actually use the staging sites provided by those hosting environments like the managed hosts. But even if you don’t have, even if it’s just the Vanilla FTP connection or GoDaddy server or something we can, we spin up a staging copy of the site. We synchronize all the files in the database before we run the update. So we’ll use FTP commands on the command line and database. We migrate everything over and then we take screenshots of the staging site versus your production site to make sure before anything starts that they’re identical.

Joe Casabona: Nice. So that somehow sounds like sending the dirty bit or whatever, right? Like if we’re getting super sad, like you’re making sure that these things are definitely the same before you even get started.

Nathan Tyler: Yeah. And then we run the updates with WP CLI that would take another round of screenshots. We see if they’ve changed again, and then you either deploy those changes or create an error notification, and you get that test result view where you can see them before or after. So that’s kind of a really quick rundown of the workflow. And then we obviously have to have, so that’s the stuff doing the actual work.

And then we also have the whole customer-facing application like where you log in to look at your sites, and the status of them, and add new sites. So we have that kind of pretty different separate efforts. You know we’re using the rest API and everything and that communicates to both sides. So you know the engine doing all of the work and the updates in the tests and also just the application where the customer goes to login and locate.

Joe Casabona: Nice, that’s awesome. So I have several follow up questions and we’ll just work backwards. So you mentioned the rest API, what language are you using to talk to the rest of the API?

Nathan Tyler: On which side?

Joe Casabona: Oh let’s go both. Because you know there’s been a lot of discussion at least since the time of this recording like, what JavaScript thing did you use or whatever?

Nathan Tyler: Sure. So we’re using Vue.js in our customer-facing applications. We actually have a few react components that we started with but we’ve actually switched to Vue. I haven’t actually migrated those pieces over there kind of independently so we have a little bit of both on our website. But it’s actually not a full headless like it’s not a full react app. We’re actually using the WordPress template API and you know, the user authentication system and page templates that have like view and reacting components inside of them which originally I thought it was going to do a whole react, you know? Acts from all headless and all the routing and everything and as we were just starting to get into that we were just running into issues and there was so much work going into the actual platform and execution itself that you know I said, “Okay! Well, let’s get something quick up for now. Let’s use the tools that we know”, which is also what we use most of the developers I have on my agency side and we were able to get up quicker. And you know, all in all, I think it’s then a lot easier to manage, especially the technology that is changing so quickly and then like we switch to view it was much less of a problem so embedded in the react router setup.

Joe Casabona: Nice, that’s awesome. and reassuring because I’ve decided that I’m going to start, I have chosen vue.js framework or whatever that I’m going to learn. So hopefully I’m not isolating myself or whatever. So you mentioned that if a hosting company or if a client shows does not offer staging, you spin that out of yourself. So do you kind of have to bring your own infrastructure?

Nathan Tyler: Yeah. So our whole platform runs an Amazon Web Services so we spin up like EC2 instances that actually handle the staging sites.

Joe Casabona: Gotcha. So you’re taking screenshots? So you’re kind of verifying, validating that they are very similar. But do you try to do anything to match the steps like, “Okay! This site is running 5.6. Do you try to spin up an EC2 box that has 5.6 on it?”

Nathan Tyler: Yeah. So for like the managed host obviously, on their own staging environments or practically identically but ours, we are trying to match, so we can match the PHP version. And the couple status is, fairly, it is not going to be an exact match if you have some enterprise server or something that’s super specific configurations. But we match the PHP version and we try to maximize the SQL version as well. And that’s usually good enough especially because WordPress plugins or you know meant to work on every version and like.

Joe Casabona: Cool. So this is the question I think I’m most interested in because machine learning is an interesting thing to me like my masters thesis was doing like predictive traffic patterns in 2008 like before GPS devices just did not. So when you take screenshots of each page, how do you determine what is close enough, right? Because I mean, you might have small changes that are not necessarily affected by the plugin, maybe. It’s like, just the way that page happened to be rendered at the time. So how do you determine whether this is definitely a thing that’s broken, or this is close enough that it’s probably just a weird thing?

Nathan Tyler: So, that is the million dollar question which is the…have been one of the…I think that’s taking the most time. Like you know, to get our first version of we’re, “Okay, we’re taking screenshots”, and we’re taking a screenshot before and after we’re doing pixel comparison. You know, after a 0% difference like we’re going to push it live like that was pretty easy and we were done with that. And then followed like months and months and months of how we handle the edge cases and what confidence do we have this is the same. But we’ve actually landed on it’s a pretty sophisticated combination of things like that’s actually a lot of our custom code and logic is built around that. But one of the things we do is we try to normalize the page before we take the screenshot so we will actually manipulate the Dom and the HTML before we take a screenshot . So if you have an autoplaying video, we actually rewind it to the first frame and weak points of positive video. If you have a slider, we rewind the first slider because sometimes, like the load time, you end up taking a screenshot when you’re on slide 2 instead of slide 1. We script out like if you have a light box that pops up with, like OptinMonster, is something we can have an option to dismiss as sometimes they show up and sometimes they don’t. And then that’s a huge difference.

Joe Casabona: Right, yeah. That’s going to say something’s broken.

Nathan Tyler: Also advertisements. So if you got, you know, Google AdSense on your site even if nothing has changed it can be just a different size banner or ads, and that throws the pixels way off, that flags it like, “Oh! it’s changed like 40%”, even though now I just have a skyscraper ad instead of a smaller banner ad. So basically we do all of those steps to normalize it. We strip out those things, we have options to strip out social feeds from Twitter and Instagram, Facebook that kind of stuff then we take the screenshot. So that eliminates a lot of stuff off the bat. But on top of that, you might have just natural content that’s random like your homepage might have testimonials than on every page load you’re loading a different testimonial.

So what we do is when you load a site on the Staging Pilot we actually scan the navigation of your site and figure out what are the main pages of this site. You know, we will get ‘Home’, ‘About’, ‘Blog’, ‘Contact’, so that we have some default pages to take screenshots of. And then we take screenshots of each of those pages like 10 times in a row without changing any code or doing anything. Let’s just see how much it changes by default. And we might discover that the contact page is really boring. There’s nothing on it, it never changes so 0% is you know the threshold that we want for that. But your homepage has a testimonial and maybe it’s got a little sidebar thing and every time we load it’s changing 10%, 15%, 16% we take the highest percentage of there. And then we set that as the threshold. So if your home page changes 18% on average, when it is not changing anything then that’s your threshold. So when we take the screenshots before and after to do that evaluation of whether it breaks or not, we’re actually comparing it. Each page has its own set threshold which you can adjust later like if you find that it’s flagging too often or not enough, you can adjust that. But between those two things we’ve got some really good confidence wise. So my agency runs maybe like eighty sites a month through the platform and we probably have maybe five or ten false positives. You know, the updates are running every day on 80 sites and there’s like five or ten times a month where we log and like, “Oh, that’s not right!” and then it’s usually sold but okay. Yeah, we did the client change some content on that page and now it’s a little bit more random, and we need to increase the threshold. But you know that’s a lot of screenshots and a lot of checks that are pretty confident now.

Joe Casabona: That’s awesome. So do you have a background in Computer Science out of curiosity?

Nathan Tyler: No, my degree is in Economics and Business so I’ve been working with computers forever but I’m taking some computer science classes, but not a major, not a professional.

Joe Casabona: Because this seems like a very interesting, academic problem, right? Just like how do you determine what’s a regular change on the website, and what’s like a broken website? That sounds very cool. And I can talk to you for a lot longer but we are coming up on time and actually more questions that I like to ask right? We talked a little bit about the transformations right? So that’s usually a question I’d like to ask but I think we’ve sufficiently covered that unless there’s something like other major transformations we haven’t talked about.

Nathan Tyler: No, that’s good.

Joe Casabona: All right, cool. So what are your plans for the future of Staging Pilot?

Nathan Tyler: So I think you know we’ve spent a lot of time getting the product ready and working out a lot of these education cases you know, handling different kinds of plugins and all those kinds of different scenarios I was describing with the screenshot. So at this point you know, we’re just focusing on our integrations with hosting platforms and making it easier because one of the challenges that we found is that, you know people, I can talk to them and they’re excited and definitely want to do it. And if they aren’t on one of our, like, one click partners it still just takes some time, right? Like they’re going to get their credentials in order, they gotta like, it doesn’t take that long but takes half a day to get all your sites and getting kind of set up. And you know we’ve been assisting people with that process which has helped a lot. But I think the main thing to kind of help us grow is like how can somebody say, “Yes, I want to do this and try it out” in like 5 minutes. So our focus is kind of building out those features and yeah. I just know working with more customers and seeing what else needs to be there. But that’s kind of where we’re going.

Joe Casabona: That’s awesome. And definitely something to certainly look forward to. So I will make sure to link Staging Pilot in the show notes of course to check it out. If you do, if you are a freelancer who does a lot of maintenance work or a developer, and agency, this sounds like a tool that can save you a lot of time in the long run. It sounds like, one hour billable work or two hours of billable work you can kind of automate this process. Awesome! So with that, I would ask my favorite question which is, “Do you have any trade secrets for us?”

Nathan Tyler: I guess, I’d say that glazed overlook I referenced earlier, to be watching out for it when you’re talking to people about your project or your product or whatever you’re working on. Almost everyone will be too plaid to tell you that they don’t care at all or it’s not interesting or it’s fatally flawed even though that would actually be helpful to hear. No one would say that you know, in the past, this is probably my 12th, or 13th you know product or project of some kind that I’ve…then my exit from client work. If I had lots of different ideas over the years and I’ve always worked on them for a very long time and then revealed them to the world to you know, no interest or whatsoever. So being able to watch for those reactions or lack of reactions and then take those into account earlier has really helped I’d say twelve or thirteen and probably six of those have been in the last like year and a half or two years. You know, like basically from when I started Staging Pilot three years ago I was getting a lot faster at listening to that feedback and saying, “Okay! Time to move on”. You know, and maybe you scrap the whole project or maybe you just realize that maybe I need to look at a different market. You know if you’re really convinced that it is the product, maybe you’re just looking or trying to sell them to the wrong people. But people can be very excited and enthusiastic and still not like actually taking the steps to use your product. So if you don’t have overwhelming enthusiasm it’s like, when you’re in person and they’re trying to be polite to you, then you know, you’ll have a pretty hard time.

Joe Casabona: Gotcha. I love that, because I think a lot of us are probably guilty of just…”I think this is a great idea and I’m going to do it”, and certainly, surely people will buy it. And even if you say to somebody “Hey, if this exists would you buy it?”, “Yeah, sure! I want to buy it right now”. So that’s just great advice. And on that, Nathan, thank you for joining me today. I really enjoyed the conversation.

Nathan Tyler: Yeah. Thanks for having me.

Joe Casabona: Lots of great stuff in there, might even be worth a second list of parts everything out. And definitely check out Staging Pilot, it’s a really really cool tool. And if you’re a freelancer, it could save you time, money, all sorts of things. Other things that can save you time and money are sponsors for this week. So Liquid web has been a season long sponsor. I am eternally grateful to them for that because the season likely wouldn’t have happened without their support and they offer incredible hosting. I host all of my websites on Liquid web and they have a great offer for How I Built It listeners if you go to buildpodcast.net/Liquid.

And then a new sponsor for this week is, you heard at the top of the show, WP CRM. So if you’re looking for a CRM that you host and own all the data definitely worth checking out and they are offering a 20% discount to How I Built It listeners as well so make sure to check them out at buildpodcast.net/crm.

Now, we’re entering the home stretch of Season Three, it’s been a great year. The next couple of episodes are fantastic, I’m really excited for them. We’ve got the folks from Sitelock coming up . and then we’re closing the season out with Morten Rand-Hendriksen and that’s going to be an hour long season finale bonanza. So make sure to keep an eye on whatever he did you use to get the latest episodes. And if you’re enjoying the show, please, please, please, head over to apple podcasts and rate and review the show. If you do, I will perhaps read your review on the air.

So until next time. Thank you so much for listening. Thank you, Nathan, for coming on the show. Thanks to Liquid web and WP sharing system for sponsoring.

And until next time, get out there and build something.