Ep. 010 George Natsvlishvili

George Natsvlishvili, Growth Product Owner at Lingokids, sits down with Adam to discuss monetization strategies as well as thoughts on how mobile app ratings impact ASO.
 
 To connect with George directly, catch him on his website: www.3minutesbullshitwithgeorge.com, LinkedIn @GeorgeNatsvlishvili, or on the MAU Vegas website, MAUVegas.com.

Transcript

MAU[Talk]  00:01

Hey guys, welcome to MAU [Talk]. A podcast from MAU Vegas, the premier mobile acquisition and retention summit. Adam sits down with George Natsvlishvili, growth product owner at Lingokids to candidly discuss some topics around mobile app ratings as well as App Store Optimization. Take it away Adam.

Adam Lovallo  00:20

Welcome back to MAU [Talk]. I took a bit of a hiatus but we're now back into the swing of things. I am joined by George and now George keep me on this Natsvlishvili? Pretty close?

George Natsvlishvili  00:34

Pretty close!

Adam Lovallo  00:36

George um, spoken at MAU before. Very well known in the industry I think, just in general, especially on ASO and other topics. Now, you know, you're a product owner growth guy at Lingokids, and I know you're doing other stuff. So tell us a little bit about Lingokids.

George Natsvlishvili  00:55

Actually, actually thanks Adam, to invite me for your show and I'm super happy to be here. Very interesting, very interesting podcast you have and variety of people and themes man, and topics you discuss. So I'm happy to be here and to join you guys. So regarding my existing full time job at Lingokids, as a product owner of growth. Lingokids is English apps for kids. It's not only English shares, but it's like entertainment. And because of this app, kids can learn English very well: pronunciation writing, and spelling skills.

Adam Lovallo  01:42

It's English learning and then it's presumably localized into how many local languages or is the whole application all languages?

George Natsvlishvili  01:51

Exactly, yeah, so it's, for me for many languages, but the main focus is English. So you can be like, I don't know, Chinese or from Latin America but of course, the main focus is English--English learning through games. So imagine, even in especially right now, during this whole situation people scared to go out. They are inside their apartment, together with their kids. So it's not only the app that through which you can, your kid can learn English, but it's also an entertainment app. So you can through our app, your kids can entertain and play games, but also educate, you know, educate your kids. So it's not just like a simple game that you play and doing nothing. Your kid increase intelligence, through learning English.

Adam Lovallo  02:50

Is it a freemium model? Like, is there a premium level? Is our advertising an app? What's the monetization?

George Natsvlishvili  02:56

It's a subscription base. So we have a subscription model. We are running in four main countries, right now we have one subscription model, but we are trying to develop the different models based on needs of our customers, what they need, what are their kids needs? And based on that we are developing on that direction and offering them a different packs, game packs, you know, spelling, grammar, etc, etc, that they can choose they pay for that.

Adam Lovallo  03:32

Got it. Okay. All right, great. Now, George, I have not. I had Andy from Phiture. So he's a good ASO guy, but I really haven't had a podcast episode like that, where we went super tactical, on ASO. So I want to start there. We'll talk about some other stuff as we have time. So first of all, I mean, MAU was cancelled last year, as you know, we were gonna have you speak you were going to find out about ASO and specifically, we're going to talk about the importance of ratings. So whether it's Lingokids, prior experience, your other projects, like could you just talk a little bit about how you've gone about trying to improve ratings in apps, like what have you done? And then secondly, when you've done so what kind of impact have you seen on conversion rates or potentially even on just like ASO in general as a function of the ratings improvement?

George Natsvlishvili  04:28

Exactly. And besides my experience, in Lingokids to improving the app and operating I had, in my past company, also, I did a huge improvement from 3.0 to 4.2 on Android on Google Playstore. And there are like several tactics that you can improve app ratings. The main idea how you can do this is of course to write, to ask people to get rating on your app on.

Adam Lovallo 05:05

A ratings will prompt right?

George Natsvlishvili  05:05

Exactly man, on a best moment, you know, when they are super satisfied with your app, they are super happy. So first of all, you have to find the best moment when people, when your customer is happy. And then you have to ask a very simple question about the rating here. Hopefully, hopefully, both of these stores right both of the systems, Android and iOS, they are right at once the latest in past only iOS has this prompt popup message of asking people inside the app not to go to App Store and on Google Play Store, you didn't have this chance. But from November of last year, you have this chance if you implemented also from popup message or on Google Play Store you're going to have a chance to increase your ratings a lot. And the rating is very important here for, for customers because people are deciding downloading your app in most cases based on reviews on other people's and ratings. And whenever they see there are different mindsets of the people, if your ratings are below four, it's your app sucks. If you are three, it's like junk app, or whatever from customer perspective. When your app is four, and between four and 4.5, it's okay app is good app. And when your app is about 4.5, they're thinking that this is really good, good, app, and they have to download it. Also, app ratings are visible, not only for App Store and Google Play Store, but also just the universal app campaigns during during the UA performance. So one of the stuff that people are seeing on ads, of course, together with title, and icon of course, ratings as well.

Adam Lovallo  07:00

Yeah, yeah. Yeah. Okay. And so when you think about optimizing when to show the ratings prompts, I mean, obviously, that's unique to every app, and no doubt unique to every app that you've worked on. Have you seen significant variants? And when to show those prompts? Like based on country, for example at Lingokids, or is it a kind of a uniform thing? 

George Natsvlishvili  07:00

Yeah, it's mostly a uniform, because people are in most cases satisfied, you know, within all games, and their entities, switching people, people here and more or more or similar, you know, for across all countries. So it's kind of universal stuff. If you find in one country when this stuff, that's it's win to win situation. Yeah.

Adam Lovallo  07:50

Right to roll out. Okay. All right. So we know, we know app ratings are important. We know, we want to use incremental prompts, we know that you can now get ratings in line on both platforms, which I think is actually quite powerful. Um, I want to talk a little bit about conversion optimization of the actual, you know, pages, essentially, in the app stores, the listing pages. So firstly, on the Google Play side, obviously, you can use their native tools and run A/B tests there. Curious how often you do that, like, is that something that you've got running constantly? Or, given how many countries you have, I assume you actually have a lot of different assets and different descriptions and titles in the app. So how do you manage Google Play optimization? Just starting with that.

George Natsvlishvili  08:37

Actually there like a different approach, how to you can test it, yes. So in Google, of course, only Google Playstore has a testing tool. But unfortunately, the confidence rate of this testing is so low 90%. In statistical, it's not it's in statistic, if you're statistic guy, you shall know that at least 99% of confidence level, it's okay to do A/B testing. Yeah, and Google has only 90%. So it's very low confidence rate. So during this testing, it's very hard to say that okay, one of the tests are okay, really good. You have to retest it and retesting again. Moreover, within different countries, different assets works, different visuals works, for instance, for Eastern Asian countries, completely different testing works versus European countries, you know, for you US is also kind of different, but mostly like Asian and the European and US parts have completely different behavior. And sometimes the visual that you see that you show to your customer is very crucial. And if you're gonna win on one country, because it doesn't make any sense in some countries to do, to implement this, you know, because behavior is different. They also, also another approach, except, I recommend this guys except A/B testing you can do like you can risk and take a risk, you can do one country and do implementation of these assets and do it two weeks before in two weeks after implementation. Now, if you have the same stuff in terms of performance marketing paid UA is the same, you know, similar you have to choose, it shouldn't be the differences and it should be within one season, you know, and you can, you can easily find out the winner version. Of course, in terms of if we talk about conversion rate, the most impactful part stuff is in terms of increasing the conversion rate and among assets. Of course icon then on Google then on Google Playstore, is this feature graphic, feature burner. And we use for sometimes for testing videos for Google Play. It is also we do yeah, because sometimes, Google Play testing itself is out of play that sometimes for some devices for somebody was not in the app, it's not out to play it was feature graphic is very important. If it's uploaded the visuals, important. If you don't have a window, the first important the important stuff is three, three main screenshots or three and a half main screenshots,

Adam Lovallo  11:18

Generally, I mean, again, provided someone has the time, would you say rule of thumb, you should have a video asset on the Android side? I mean, do you see that consistently work better? Or do you have any teasers where you don't video at all?

George Natsvlishvili  11:31

It's on, it's very hard to test it. Because, you know, most of the people, if it's not auto played, they don't click on it. So it's very hard to test. If this video works or not, if it's not out to play, they only see the feature burner and not visible itself. So it's very hard to say if it's very important for Android, is for iOS. For iOS is very important, especially the horizontal window, you have a horizontal window, it's, it's different. First of all, during the search, it shows in a different play in placement during the search ads. If you have a brand keywords and your app is on top as search ads. So on a search ads you see, people see the assets of horizontal video. And below on organic search, they're going to see this first three screenshots or first, which could be video and two screenshots as well. So it's on all devices, on device type, it's updated. And so it's very important to you to have it's very important versus Android.

Adam Lovallo  12:37

On our side. Okay, so let's talk about, let's talk about the iOS side for a second. So, I mean, as you well know, there are at least a handful of, quote-unquote, like third party testing tools, you know, that more or less recreate the App Store in the mobile web. What's your general without, I don't believe out, but what's your general impression of those?

George Natsvlishvili  12:58

We do respect this guy, they are really good. And I know them. Because first of all, whenever you test through this certain tool, it's a fake behavior. So people see first, the test version. Whenever they click, they redirected to the real store, and they see the different version. So most of the people they even think that this is bad, you know, and if, for the first attempt this, they see that, okay, it's nothing wrong with this app, they don't trust, you know, the trust level is very low. And even the cases are bad they might not download your app because of that. So you're gonna lose a lot of stuff. Also, the test version itself, on these tools are very tricky, because they are not testing the difference between percentage, but they are testing the difference between percentage of percentage, so to get a more impact, you know, to show you exactly to show you the more impact and this is not the case, you know, because people in online marketing, they are not a math approach, guys, so in most cases, they believe in this figures, and they say, oh, my God is like 10%, 30% increase of convention. It's cool stuff, you know, and it's not the case, man. It's not a percentage difference is percentage point different. This is different, you know?

Adam Lovallo  13:14

Yep. So, okay, so we don't like, I knew you'd say that, but we don't like a third party testing tools. Fair enough.

George Natsvlishvili  14:33

No, actually if you have a money and if you have, like, additional resources as an additional tool test to prove your concept. You can use it.

Adam Lovallo  14:44

Okay. All right. That's, that's a good con then. Yeah. So how do you test or think about what assets to quote-unquote test on the iOS side? Do you find that you can take stuff from learnings from the Google Play and apply them directly? Do you do your pre-post analysis and treat them as they're totally two separate things?

George Natsvlishvili  15:04

So like, in most apps, are focused on the US market or English speaking market. So, hopefully, iOS has a different English speaking languages app pages so it's like Australia, Canada English. Also US English, UK, English and New Zealand English or whatever. So how, how I test it, whenever the main I am I'm, it depends on you know, CEO and it depends on the guy who made the decision here. But usually I'm there to approach or take a risk and in the switch for instance, for two weeks switch within the new assets and then calculate analysis doing as before and after, on the main market. Or, for instance, if you are in US very strong, and in UK, not you can just test the UK market, because it's kind of similar behaviors kind of similar versus US. And if in UK, it will be kind of winner version, you can just say, okay, that my next step will be to test it in US, you know, and you just test in US without any risk because in 98% of cases UK,  US. People are having similar behavior.

Adam Lovallo  16:34

Yeah. Okay. Okay. Now, more topical, so we're recording this in early February. We'll release it soon. Obviously, you've got all of this IDFA deprecation, stuff like that. So I'm just curious. Do you think any of the above significantly impacts the organic traffic side of things, obviously, seismic shifts to paid but do you think any of that spills over in any way to changing how people are thinking about ASO and organic and you know, that side of this equation?

George Natsvlishvili  17:16

Not really, it's mostly reflect on a paid UA. You know, because in paid UA there is no, you can't say okay, like you can say 100% what is the cap on? What is the impact of the certificates? Yes. So that's mostly impact on paid, of course, but on organic so you can't say 100% what's going on here. But of course, iOS 14 is mostly impacts on paid because the main source of paid was like IDFA, they're going to be removed, you know, and that's mostly impact on paid UA, not organic. Organic still, through organic, you can see like in Google Play and iTunes analytics, nothing gonna change, you know, at the end of the day, and you can see the okay, what can I do? What am I going to do if I switch the assets, blah, blah, etc, etc. So there will not be huge impact on iOS 14, in my opinion on organic versus paid. Of course, if you want to track incremental impact of paid on organic, that's going to be very tough and very complicated in that case, because you have to take okay, what impact what brought paid UA and it will be super sophisticated to calculate and super like probabilistic variable approach.

Adam Lovallo  18:45

Estimated, um, okay, and also on this IDFA topic. I mean, I know you were even talking about organic and I'm sure with paid too. What's your take? Or have you looked closely at this Apple Search Ads specific API, like the form of measurement that Apple Search Ads is making available to themselves that's different than the full ATT framework? Like have you read much about that or?

George Natsvlishvili  19:17

Yeah, actually, yeah, there are the materials that are available and you know, mostly our tech team is doing the reading all this one thousand stuff info. Yeah, but you know, like, I mean, in my personal opinion, there is no maybe not with reality. Before removing something Apple show offering some alternative and they are not ready to offering some alternatives and they are removing they're cutting their hands to be honest, you know, and you know, without the alternative solution, because there is no clear solution that they offer everything and they are cutting their money, you know, they are cutting themselves. 

Adam Lovallo  20:00

You know, I was talking about this morning with a friend and he said, well, if I see, you know, first of all if x percentage of Facebook and Instagram users opt out, and then I, myself in my own app have to implement Facebook's prompt per the SDK and then have a healthy percentage of those people opt out, like I get, what am I gonna do? I'm like, yeah.

George Natsvlishvili  20:23

We are even worse situation, Adam in Lingokids because we can't as a kid app is not allowed to use SDK. So for us even it's like what? Okay, if we don't, can't use SDK, please tell us what to use, you know, to track somehow what's going on? And it's super complicated. No one knows the answer so far, and there are like

Adam Lovallo  20:48

Pre this IDFA deprecation, were you allowed to use an MMP and track through the MMPs? Was that permissible or no?

George Natsvlishvili  20:55

Oh, no, not really, like through SDK, we can't use since July, because of kids apps, there was like a rule. Right now, with this completely removed IDFA, that's only stuff that we can move forward is scan campaign, scan network to do our own scan network shows, you know, like and to see the difference between other solutions or tools as well, to see how data is different between our scan networks in between, you know, their solution that they offering.

Adam Lovallo  21:36

Yeah, interesting. Okay, and what about, I don't know if this is from your time at Glovo or maybe Lingokids, but, you know, three or four years ago, it might have been longer there was a lot of excitement that you could now index in app content in Google's index for organic search, you know, conventional SEO, and that'd be really great. And you know, apps would start to rank organically. Haven't heard a lot about that. You have indexing of content for the purposes of Spotlight Search. I don't know if anyone on the planet uses Spotlight Search, I actually do on my phone. So occasionally, I'll take a look. Um, so I'm just curious, like, to what extent in the mobile app kind of ecosystem have you really dealt with or thought about SEO? Or was it you wanted to rank pages as per usual in the mobile web and desktop web and, you know, you use the right app, install prompts, you know, in the right browser to make sure people get to the right app? Like, what's your intersection of SEO and app install stuff?

George Natsvlishvili  22:36

Actually, that's a process that you mentioned. Yeah, like, many apps, I was super skeptical, because in that case, you have to give all codes to Google, you know. And people are very strict because Google's built their own apps as well, you know, like, and one day they can, they can just copy paste the codes from your app. And then in one day, they create their own app, you know, and that's it, because they are Google. Yeah, they can stop indexing. So people are like, usually, developers are super skeptical regarding this API indexing stuff. So this API indexing is very, very good. As you mentioned, if you have like music, you know, music listening, or if you are an e-commerce stuff, too. For instance, if someone search for, I don't know, some branded shoes, and your apps appears at the end of the day, and they can directly, you know, click on it. So this is very useful. But in general, developers are not ready to share their code, you know, because of that feature. So if we want to increase the visibility, they are going the old fashioned style with pure SEO, through web or web to app, or completely developed the web. So to be honest, web initiatives is super, like, in my opinion, is one of the stuff that app industries will develop more and more, because most of the app they have websites, and SEO, anything, you know, paid with if it's app and mobile oriented. In most cases, even if they have a website, even the cool apps they suck with web, you know, and if even, instead of spending millions to fix that or to buy traffic through mobile apps, this thing in wasting money on web traffic, that will give a really huge benefit there. The issue with that, of course, it's not there you will not see a result as fast as the through app, you know, but whenever you see the results whenever you get this SEO on this level, That's it, you're gonna just do some small iteration, and you're going to receive a lot of much more traffic because web traffic is so huge. And there are so low competition in web. So I mean, especially during this situation that you can track, iOS14, and all this going on, and they are like cutting themselves, you know, cutting ahead themselves, in my opinion, is the right way to move forward for most of the apps, to move forward to web development, because there are huge room for improvement and huge opportunity in terms of customer. There are a lot of people. Especially right now, because they are shifting from desktop to mobile so there is a huge opportunity. If you have a good mobile website. That's it, you know, like you are good. Because many people either don't want to download app because they don't like to download a specific app for this, specific app for that, they want to just like one-click and get results done. Yeah. And in that case, this is the good user experience for them.

Adam Lovallo  26:11

Okay, I got one more question for you. Now, this one you may not have an answer for, I'm not sure. But also a couple years ago, there's a lot of excitement about like some of these app streaming experiences like app streaming on Google, you know, well, it doesn't fully require the install, you can start interacting with the app more quickly. You know, I would argue a little bit different use cases with App Clips on iOS, but kind of similar idea like exposing more of the functionality pre install so that it's a little bit less intimidating to like, actually add this into your phone. I'm just curious, what's your general take on that? Do you think that's a thing or not really?

George Natsvlishvili  26:51

Not really, in most apps and game data I was working, we implemented it. Mostly the games, it's useful for games to pre install, for gamers to see all the new games because they are super highly addictive person. So most, if you want to do this preinstall stuff. That's the best way only in games, in my opinion, in apps industry is more complicated, because there are low awareness of what's going on. There are lower addicted persons initially, you know, before you're ever gonna launch, if you are not a huge brand. Yeah, of course. But, you know, the pro-install in my opinion, all of these initiatives that Apple and Google as well offered during the last year, they to be honest, not the results from there is not as expectations are high and the reality is really good. Because it's a poor implementation, poor user experience. You know, they are just okay, launching something without proper, you know, like, proper care. They don't build the case. They just, oh, okay, we get a good idea launch, you know, yeah, this is not the case. This should not be the case in this company in these two companies, you know.

Adam Lovallo  27:28

Yeah, yeah. Okay, this is spectacular. My last question for you. If people want to, you know, seek you out, what do you do prefer LinkedIn or do a newsletter or Twitter like, what's your where do you live online?

George Natsvlishvili  28:27

Yeah, so you can reach me through LinkedIn, but I have all my own channel as well. Two Minute Bullshit With George. So you can just type. I have my like, my show, but it's not sponsored the rotary just for fun. Because I know a lot of people like inside this online marketing and most of them don't like to, you know, to speak or whatever. But within my company, they are okay to speak. You know. 

Adam Lovallo  29:01

That's great!

George Natsvlishvili  29:02

That's funny, that's funny, man. So through LinkedIn, you can find me if you prefer.

Adam Lovallo  29:10

All right, George, this was spectacular. And I look forward to seeing you hopefully, at an MAU event in the future.

George Natsvlishvili  29:17

Me too man, me too. That was my key destination to be honest for 2020. 

Adam Lovallo  29:26

Alright, well, thank you again George. 

George Natsvlishvili  29:28

Thank you, Adam for everything. 

MAU[Talk]  29:51

Thanks for joining us. You could find George's contact information in this podcast's description or at mauvegas.com. Make sure to subscribe wherever you get your podcasts and we'll catch you on the next episode of MAU [Talk].