Video Transcript
TERRY: Kevin for those who don’t know you, can you give us a brief introduction to who you are and your story?
KEVIN: My name is Kevin Ohashi.
I run ReviewSignal and most recently launched wphostingbenchmarks.com
Review Signal is the largest web hosting review site on the internet.
It aggregates what people say on social media, mainly Twitter, what they’re saying about different hosting brands and analyzes it automatically.
It uses a purely data-driven way to compare web hosting companies. It started around 2013 as basically a bet with an executive at GoDaddy. He said, “I bet our hosting platform will perform as well as any other WordPress-specialized platform out there.”
And I kind of raised my eyebrows and asked if he’s really willing to test that.
So the WordPress Hosting Performance Benchmarks were born out of essentially a bet with GoDaddy.
And they actually did quite well that year and it was wildly popular and companies wanted to get in left and right, (they wondered) how do we stack up?
And over the past, what is it, eight years now, those tests have evolved to become more complex, testing more different aspects.
And more and more companies keep signing up every year to see what an unbiased look at WordPress Hosting Performance is.
TERRY: Great.
KEVIN: That’s what I do.
TERRY: Sure. At the moment I’m working on a few books that I’m writing because writing is my professional background among a few other things.
And one of my books is about when people or companies pivot from one thing to another.
I’m quite curious about what you were involved in online before Review Signal. Was there some other big project and then you moved into Review Signal?
KEVIN: The history of it is that I had just finished my second master’s degree.
I was living in Lund, Sweden. That’s where I did both of the master’s degrees.
I decided not to do a PhD and wanted to start my own business.
And it came down to joining another startup in Copenhagen or going home and doing my own thing.
I opted for the latter obviously.
I moved back to Washington DC and played around with a few ideas.
I had just written my master’s thesis on movies, and I had collected Twitter data and predicted box office sales for movies.
it worked incredibly well, but nobody paid me for that, sadly.
I had been involved in web hosting on the internet since I was a kid.
10 years on, I still couldn’t tell you where to get an honest web hosting review.
And I somehow put two and two together and Review Signal was born out of the idea that I could take this technology I built for a master’s thesis, looking at the wisdom of the crowd and putting that as something consumers could see.
Because it’s hard to just search and create that, but I can collect millions of pieces of data and automatically analyze them.
So, I put it together. Review Signal was born and that’s kind of the history of it.
TERRY: During the lifespan of Review Signal so far, has that led to spinoff projects or invitations, or other things come out of that?
KEVIN: Spinoffs not really, this is the first, WP Hosting Benchmarks is the first real spinoff from the company itself.
But Review Signal, it was full time for the first few years, getting it running.
Afterwards, I started doing consulting part-time as well.
Because of what I’ve done at Review Signal, there’s been a lot of interest in hiring me as a software developer data guy, doing different things.
I worked with the Washington Capitals, ice hockey team, doing some sports analytics for them.
And if you watched “Moneyball“, there was Brad Pitt, who was the manager.
There was Jonah Hill, who was the guy who analyzed the data.
And then I was Jonah Hill’s minion that never got any screen time that did some coding.
It was fun. I was really interested in it.
TERRY: You don’t have to be a baseball nut to appreciate that movie. It’s so universal, the lessons in it.
KEVIN: Oh yeah. It was a great, great movie. And I’m not a baseball fan at all.
Actually hockey is my sport.
Opportunities like that have come up because I’ve been able to work with large amounts of data and actually build and deliver things.
So I’ve had a lot of interesting projects like that and also in the WordPress space.
TERRY: Kevin, just tell me a bit about the philosophy of your annual WordPress hosting performance tests.
And also as part of that two-part question. I told you it’d be tricky, this interview, difficult questions all the way.
How has your philosophy evolved over the time you’ve been doing the testing?
KEVIN: Sure. The philosophy behind it is I wanted to create an unbiased look at performance because performance is measurable.
One of the trickiest things doing reviews is pulling yourself out of it.
And also trying to resist all the temptations for the piles of money that companies will shower on you if you say nice things about them and pump them.
How do you create a system that is as unbiased as humanly possible to get real data?
And that’s one big reason I focused on performance.
The other is performance matters.
I mean, we see it pounded into our heads every day by, well, probably Google’s the biggest one at this saying, you need to make your website load faster or Amazon saying, 200 milliseconds increases your sales by X percent.
Performance, performance, performance.
And after all these years, people really care. It was kind of a natural fit.
How do we get unbiased performance?
And so the philosophy has always been, let me measure and create repeatable (tests).
And I try and use third-party tools as much as possible because there is a trust issue who, if I create a test and run it from my own servers, that’s not repeatable, that’s potentially corruptible.
I like to partner with as many other companies that can help measure this kind of stuff.
That’s the philosophy. I’m not sure if I answered the second part.
TERRY: Here’s a quick fun side note.
I was working on a Core Web Vitals blog article a little while ago and you know that obviously Google Page Speed Insights is the big tool that Google is using for a lot of this.
But Google’s own properties do not do well on that tool.
YouTube does extremely poorly.
The Google Play Store does very poorly, even google.com, which is basically a logo with one field doesn’t get a perfect score and it shoots out a whole list of recommendations for them to change that page.
I had a bit of fun seeing how Google’s own tool measured their own properties.
KEVIN: Is that a case of do, as I say, not as I do?
TERRY: Exactly. And particularly if you look at something like YouTube, which is, I think the second most visited site in the world probably, depends on the country.
It loads instantly on any device yet it has an extremely poor performance on that tool.
You can make of that what you will.
Kevin, while you’ve been doing Review Signal, and that obviously embedded you a lot more in hosting, what have you learned and seen about the hosting industry over the time that you’ve been doing the testing?
KEVIN: It’s been a real interesting look being this involved in the industry for this long now.
I haven’t done a new one in a while, but back in the day I did a blog series.
I think it was titled Dirty Shady Secrets of The Web Hosting Industry.
And I think I only did three or four of them, but it was exposés on the different kind of fraud and scaminess and underhanded tactics that companies both web hosting and on the review side would take and do so there would be spam networks exposing, fraudulent connections.
There was one infamous company that I don’t want to mention by name because I don’t want to give them any extra promotion, but basically it ended with their CEO publicly admitting in a big WordPress hosting group, yeah, it’s perfectly okay that we tell our employees to go write fake reviews.
They use our services.
That’s legitimate.
And I don’t think anyone in their right mind thinks paying your employees to write fake reviews is a legitimate thing.
And they defended it, this was the hill they wanted to die on for morality.
They got banned from that group and many others, because they were a terrible company.
But that’s the sort of behavior at the extreme end.
But at the subtle end, there’s a lot of money.
And that’s the problem.
The web hosting customer locks in for years and is worth, even for GoDaddy, which people associate as a cheaper brand, kind of a budget friendly one.
And they’re public so you can look at their financials and their customers are worth over a hundred dollars a year, every single one of them.
And you’re thinking, how are they making that when they offer something for a few dollars a month and they lock them in for years?
And so the payouts are so huge and that subtle influence is pernicious to everyone and everything in this industry.
And so it’s been an interesting learning experience trying to promote honesty and transparency because I try and get a lot of attention for these performance benchmarks.
There’s a lot of WordPress blogs out there.
You’d be hard pressed to find one that doesn’t have a web hosting affiliate or somebody that they’re pushing and probably making a good amount of money on.
And the competition for putting your name versus their own affiliate is probably an easy decision for many to say, no thanks. I don’t care about, maybe the quality of good work being published because this is financially driven.
It’s really eye opening and not eye-opening at all that people, businesses are self-interested.
Mixed feelings on it, but I’ve met a lot of wonderful people too.
The other side of that is there’s a lot of great companies and people that are really trying to improve in so many areas and for consumers.
Some of them are even fighting for broader internet issues. One of the most recent examples – are you British?
TERRY: Australian.
KEVIN: Australian?
Okay.
Have you been following, there’s this issue with Nominet in the UK? Nominate is the registry for .uk.
One hosting company there called Krystal, their CEO basically launched a campaign to oust half the board because of what’s been happening at the registry. It was set up as a public benefit corporation.
It’s supposed to do good.
And over the past many years, their salaries have gone like this (way up).
Their public giving has gone like this (way down).
And they’ve tried to become a tech company and failed miserably.
They’ve tried to start new stuff, all this stuff you go, why as a registry are you getting involved with that?
Their new ventures all failed.
They’ve given themselves a tremendous amount of money and pats on the back and completely at the cost of the membership who are hosting companies, registrars, et cetera.
And that money was supposed to be going towards helping people.
It’s a public good. I watched that campaign happen.
There’s some very good people who do care as well.
I don’t want it to paint it entirely in a negative light.
I want to highlight some of the great stuff going on from the industry.
TERRY: Good job.
You’ve just finished now the huge amount of work on the 2021 tests, Kevin, and you probably thought about suicide a few times while you’re assembling all of that information.
Not sure.
KEVIN: Not yet. Give it time.
TERRY: It’s a monstrous amount of information there across so many categories and stuff there.
Did anything in 2021 testing surprise you, something new popped up this year?
KEVIN: It’s a good question.
And often it takes me a little bit of time to even really process.
I know I’ve published all the data, but the analysis and publishing, again, I try and almost step back from it.
I have pretty defined criteria about how I evaluate companies.
There are certain thresholds, like you cannot be awarded anything if you don’t maintain 99.9% uptime.
That I think is an industry minimum for anything.
That’s one thing.
For load testing there can’t be an error rate exceeding 0.1%.
Latency, I can’t have responses taking over a second.
There’s very strict things and I go through all the data.
I look at all the graphs and try and understand what happened.
But really what I’m trying to understand is did this company meet the threshold.
Are they that close?
That’s what the honorable mention is.
Or did they miss that?
And oftentimes I’m not even really thinking so deeply into it, beyond that.
I’m just trying to critically and fairly evaluate everything.
And I just have a big spreadsheet marking.
Yes, yes, no, close.
As far as anything really, really insightful yet-
TERRY: -That’s something we can work for our next interview, Kevin.
KEVIN: There’ll be a blog post if I notice it.
TERRY: Yeah.
KEVIN: I’d like to do some analysis between years, which is maybe going to be more simple to do now that I have this new website and it’s all in a database, actually graphing those.
I found that visualizing things helps me and I think many others actually understand data.
Because I used to publish just tables.
Now that I’m building better and better visualizations and the tools to pull them, create these different views that may be new insights will show themselves and reveal.
That will be interesting.
TERRY: Good stuff.
Which brings me on to my next question about data.
And you obviously love data. I’m curious, I’m not a data guy.
I’ll be quite honest and confess on that one.
I’m not a big data guy necessarily (I’m a maths idiot).
What is it about data for you that’s interesting, that’s cool, that’s exciting, that’s fun to explore?
KEVIN: To me the beauty of data is that we can look for truth in it, whereas it’s that qualitative versus quantitative view.
I think humans, most of us, or most people are naturally qualitative.
We go on narrative, we go on storytelling.
We try and understand from a more human perspective and I don’t have any actual numbers to back this up, but I think some people skew on data where that quant view is, I don’t believe it until I see the numbers, the science.
I need fact.
And we treat data as fact.
Of course there’s potentially flaws with data.
But if you have good data, the belief is you can find truth in it.
And I’m one of those people.
And finding truth in data, I think is A, hard and B, if you can do it, it’s rewarding.
I imagine scientists trying to unlock the why about anything.
I look at data and want to understand, why and how.
Using data to figure out what is truth.
And that’s the appeal.
TERRY: Yep. Gotcha.
A while back I was reading a book, I think the title was Everybody Lies.
And it was basically comparing what people actually typed into Google compared to what they said in surveys.
These surveys, people wanted to present themselves as doing the right thing, very respectable and all of that.
But the stuff that we’re typing into Google, dear God, it was pretty, pretty crazy.
KEVIN: Is this in a GDPR lawsuit right now, this book?
TERRY: Yep. Kevin, let’s come back to WordPress for a minute.
What do you think is in the future for WordPress and the WordPress ecosystem around it?
KEVIN: One interesting thing from my point of view for a lot of this stuff is I straddle a lot of industries, including WordPress.
But I don’t really live inside the ecosystem.
It’s the same for WordPress, I’m also tangentially in the Drupal community, in the web hosting community.
I’m not a web host or customer really.
I straddle all these worlds and not really sitting deep in any of them.
I work with WordPress, I have clients that are on WordPress.
But I don’t have any deep philosophical or financial ties to it.
And looking at the future of it, it’s hard because I don’t have that deep connection or vested interest in it, or even trying to push it.
I have friends that are, and people talk to me a lot.
One of them is pushing this idea of it needs to go towards more of an app store model.
People I know, Gutenberg was a big push, not too long ago. I wasn’t terribly happy with that because it launched right in the middle of my testing.
And I had to decide what to do with all these updates and how to measure my … what am I measuring in the middle of this.
But the future of WordPress, I think it’s getting hard to predict.
And because of the scale, many things can be true.
And how do you acquiesce to 40% plus of the internet?
And I’m not sure you can.
I think you’re going to see these more, maybe the core is the same, but there’s already entire ecosystems living in themes.
Are we going to see more of that or are we going to see a push back towards the core?
There’s the commercialization and there seems to be a little bit of consolidation with bigger companies picking up more and more critical plugins and whatever.
ACF was just acquired, big theme companies get acquired like consolidation seems like a natural thing and I don’t know if that’s good, bad or healthy.
I look at data and try and see, what can I tell from this rather than predicting the future?
What are your thoughts on that?
TERRY: I think to come back to an earlier point you mentioned.
I think WordPress needs to be viewed as a tool, getting to an outcome that you want.
Let’s say that for example, well, I have a dog foundation, Every Dog Matters EU.
If we have a website, what matters is a reliable, flexible vehicle for us to communicate with an audience, to get dogs rehomed, to attract volunteers and all of that kind of thing.
Does that need WordPress necessarily?
Probably not.
But at the moment, that’s a good option because I know it, we know it very, very well.
But the outcome is what matters.
And it’s a bit like if you go from a certain kind of tool before there were electric tools and you’re a carpenter or something, and then one day there are electric tools.
Well you go, it’s a no brainer, I’ll use the electric one.
It’s a lot easier.
I think WordPress is very, very useful and an incredibly successful platform.
In the future maybe Google will acquire it.
They’ve been sniffing around a bit on WordPress.
I see a big shift to those page builders like Elementor, Thrive Themes, Beaver Builder, and Divi, and all of those.
Whereas it used to be back in the day much more, you would just buy templates on ThemeForest or something like that for your site, hire some guy to do some changes on it or whatever.
But there’s been a big shift to those page builders.
I think I’ll answer one of my own later questions here for you, Kevin, to take a bit of a load off for you.
I think that if you’re kind of a non-technical entrepreneur and by entrepreneurial, I don’t just mean making money.
I mean, you have some project or mission that you really believe in that you want to do.
There are certain problematic issues with WordPress around plugins, and security, and the PHP version, and the WordPress version, all kinds of getting along.
We deal with that on a daily basis here.
There are tons of badly coded plugins out there.
A lot of good ones as well.
Often they will conflict with other plugins, or the PHP version, or the WordPress version and things like that.
And that does add complexity for a non-technical person to manage or get expertise to do it.
And you see some closed platforms.
If you look at Shopify for example, or Wix, and those kinds of things.
On one hand they’re not perfect and they have their limitations as well.
But they’re closed ecosystems so you have to worry less about that stuff if you’re not a very confident technical person.
I think that the future is going to be pretty interesting for WordPress.
Maybe my homework is to think about that some more for my next interview.
KEVIN: I actually think those are the biggest threat to WordPress personally, Shopify, Wix, Squarespace.
I mean, we see people pushing towards those Elementor and whatever.
Those are basically the same thing wrapped up with none of the underlying parts.
And I had to do a Squarespace site for a client, not that long ago and realized they’ve even cut off, it seems like the templating system that I couldn’t even get in there and make custom templates anymore, it’s just entirely in the builder.
And from a development standpoint, I’m like, that’s amazing for them.
They’re maintaining hopefully close to one system or millions of websites, whereas a hosting company would have to maintain, like you said, God knows how many PHP versions and different configurations.
And just an absolute nightmare of different options.
So yeah, I can see that.
TERRY: One of our frustrations in WordPress, and if you look at something like Core Web Vital it’s on Google.
If you put an empty default WordPress installation, it gets a hundred out of a hundred, it gets a perfect score.
But then the owner, and we don’t do this with other products in our life, we don’t go home at night and take the back off our TV, not most of us and tinker around trying to improve the performance, or the microwave oven, or the fridge.
But we do it constantly with our WordPress sites.
We’ve seen for example, at the extreme end, a customer with 200 plugins on one WordPress site.
And many will have 50 or 60.
And the irony is that when we go in and start to audit and try to fix problems, they’re not even using most of them anymore.
And often the non-technical people will not understand that the way that they modify a site can potentially have a lot of performance implications, let alone security vulnerabilities.
In a way, the flexibility and the plugin capability of WordPress is simultaneously the best thing and the worst thing about it.
We constantly have to educate customers when our secondary teams were working on sites and say, look at the waterfall result here in Pingdom or GTmetrix or whatever.
This plugin here is taking 15 seconds to load anything.
But their initial reaction is just to blame us or the host that they’re with.
With all of that stuff, we’re engaged in a constant education process about impact of their choices.
KEVIN: It’s one of those catch 22s for you, where you are going to be responsible no matter what.
And educating is expensive and it’s much cheaper to say, well, I’ll just switch to another company, they’ll be faster.
And sometimes that’s true.
There are some companies that are slow and switching is the easiest gain.
And then sometimes you’re like, well, you do have 6,200 plugins and I’ve had to go in on, for clients on those sites.
And you’re like, well, there’s no saving this.
You were doing entirely way too much.
Do you really need 18 analytics plugins?
TERRY: Yeah, exactly. And I will guarantee they’re not checking them anyway.
That’s all the shiny object syndrome, the new plugin is going to change everything, which it doesn’t.
And then tomorrow there’s a new one and a new one, and the old ones weren’t removed as well.
Kevin, I’m just getting to the end now, because you’ve been very generous with your time.
With future testing have you considered in coming years to look at things like Core Web Vitals or support quality testing, for example?
KEVIN: I’ll separate those two out.
I think they’re very different.
Core Web Vitals, that’s Google’s new, whatever metric that they like to push.
It’s kind of already in there.
It’s not explicitly called that.
But the webpage test is essentially running Core Web Vitals with those load times, the first content full paint, whatever.
I’m not measuring them separately, they have two or three metrics that they call Core Web Vitals. Instead of separating it out, it’s really just encapsulated on the webpage test.
But on the actual webpage test, it is measuring those.
I think maybe next year I will have to use Lighthouse as well and put an explicit number on all those, maybe.
TERRY: One of the things we see with a lot of customers with Core Web Vitals is because Google has said it will be a ranking factor, they have assumed it will be the dominant ranking factor when of course there are at least 200 ranking signals.
We have a lot of neurosis to deal with over Core Web Vitals.
And I’m not convinced when I look at Google’s own properties that I mentioned earlier, among the sea of other metrics, a few points here or there is not going to make a massive difference to your ranking.
KEVIN: Well, when you’re the monopoly and put yourself at the top, does it really matter?
I think unless they change it, because Core Web Vitals is the successor to Page Speed Insights.
Which I was looking at them recently and I’m not sure Page Speed Insights does anything that Core Web Vitals does, or Lighthouse doesn’t.
For next year I would say there’s a good chance that I’ll measure those things unless they decide that there’s the next thing.
Then I don’t know.
I try and adapt to what’s out there.
As far as support quality. That’s a question I get asked a lot.
It’s which one has the best support.
That’s a very important question.
How do you answer that?
How do you measure on a data-driven way what does good support look like and how do you compare it, apples to apples between companies?
And especially with different business models and value propositions too.
Because if I sign up for a company that has 24/7 phone support versus a company that just does tickets.
How do I compare that apples to apples?
How do I measure quality?
And I don’t have a good answer for that.
TERRY: I think the way I describe it to people is hosting is a bit like electricity or water at your home.
It’s kind of a utility and if you don’t have an issue, you don’t notice it.
You don’t make a fuss.
And it may well be that an electricity company has, let’s say, a hundred thousand customers.
And for the last 20 years, they haven’t had a single problem.
Then there’s a problem with a bushfire or whatever.
And power goes out for two days.
So, you now have 30,000 people on social media saying that those bastards at that electricity company, they screwed up.
But maybe it could be like that.
Or it could be, you don’t hear about the people who are happy with the service because it’s working perfectly for them and they have no need to complain.
That kind of thing.
It’s a tricky one.
KEVIN: It is. And I hear this a lot because people … because I run Review Signal people talk to me and I do have this live chat on there.
I installed that thinking, I wonder if anyone will use it.
I don’t think I’ve had a contact form since. It’s actually really interesting to talk to people and try and help them and hearing these kinds of questions.
I try and tell them, I do measure support quality via proxy monitoring social media.
That’s what the main Review Signal does.
And what I like to point out to them is nobody comes close to a hundred percent.
Web hosting as a service, it relies on physical hardware, physical hardware will break.
Things will go wrong.
We saw the Fastly outage yesterday, which impacted, I don’t know how many millions, if not billions of people trying to access some of the largest properties on the internet.
Things can go wrong no matter who you use and at what scale you are.
What might be different is how the companies handle it.
Fastly seems to have fixed it, I think within the hour and also seem to have been fairly communicative about the issue.
Whereas Amazon historically is pretty, they’re status pages will say, yes, we’re completely online when you can’t reach them.
But also that kind of treatment of yes, we failed and here’s how we’re going to make it up to you and make sure this doesn’t happen again versus not our fault.
How they handle failure is a big deal.
But sometimes you just don’t know until you do.
TERRY: Yep. Here’s a fun sidenote, Kevin.
Because we’ve been around since 2013, when we only had one server, it died.
We only had one, it died.
And we had to rent a horrible old machine as a temporary backup.
But because we were very open about it and constantly talking to the customers at that time some many years ago, I don’t think we lost even one customer because we were constantly talking to them, apologizing a lot.
And working on it as fast as we physically could.
And the fact that we did that and put our hand up and said, look, we suck at the moment.
We’re sorry about that.
We’re working as quickly as we can.
And people do appreciate that.
So, I think you’re absolutely right. It’s how the company handles it.
It matters hugely.
KEVIN: And I can have a personal anecdote there where I had a similar issue and there was some issues with a dedicated machine that I had got and they admitted they screwed up with the setup and something went wrong.
And that could have been the end of the relationship very easily if, okay, you don’t know what you’re doing.
This is terrible.
But instead it became, before I even asked for anything, it was like, we’re sorry, we screwed up.
As an apology, we’re going to give you 10X the time that we were down.
And we’re talking days. I think I got like three months of free hosting because, we’re at fault.
And I went, I can’t say no to that.
TERRY: Yeah, exactly.
KEVIN: I think I stuck around with them for many, many years since because they treated me with respect and acknowledged failure.
And if there’s going to be failure in the future, that’s what you want in a partner.
Which was what a hosting company is.
TERRY: Agreed.
Kevin we’re at the end now.
And I just wanted to share where people can catch up with you.
If you want them to, apart from reviewsignal.com.
Are you coming up with an Oprah interview soon or some book deal, big book deal coming out, speaking tour, what’s happening there?
KEVIN: I don’t think the speaking tour will be too popular at the moment.
The travel situation’s a little difficult.
But you can find me on all sorts of social media.
I am Kevin Ohashi on Twitter.
I think there’s an underscore on Instagram if you want to see pictures of sunsets and fish.
I’m pretty available just one Google search away.
You can find, I think everything about me, all the LinkedIn, the whatever.
The only thing I don’t really talk to strangers on is Facebook. Every other social media, I’m pretty easy to reach out to and chat.
TERRY: Kevin, appreciate your time today. And the amount of work that goes into those tests is just phenomenal. I hope people appreciate it.
KEVIN: Thank you very much. Thank you for having me.
TERRY: Sure. And we will talk more soon. Kevin, have a good day there.
KEVIN: Sounds good, you too.
To see how WPX performed in Kevin Ohashi’s independent WordPress hosting performance tests, results are here for 2019, 2020 and 2021.
