Overcome launch anxieties and ship apps with ease (Firebase Summit 2019)

what’s a great feeling? Launching an app. You know what’s really scary? Launching an app. Today we’re here to
answer the question, what do I need to know before
I launch my Firebase app? My name is Rebecca
and I’m an engineer on Firebase app distribution. ROB DIMARCO: And I’m Rob
DeMarco, engineering lead for Firebase build products. Let’s start with a story. Together, we’re
building a gaming up. It’s a really cool concept. We’re excited about
it, and we want it to be used by people
all over the world. We start by building a
functional prototype. We’re testing it out with our
team, we’re iterating on it, we’re fixing all of the
bugs we find as we go along. We think we’re almost ready
so we set a launch date. Still, we’re feeling
a bit anxious. We fixed all the
things that we do know but what about all the
things that we don’t know? Even so, launch day is rapidly
approaching and we go for it. Early signs look promising. We start seeing downloads,
we’re generating some buzz, people are talking about it. We’re feeling really excited. REBECCA HE: But then the bug
reports start to come in. And on the app store,
we’re seeing mixed reviews. We see five star reviews that
we’re really excited about. But we also see one star reviews
and we feel anxious and scared. Some customers are saying
that they really don’t like a certain feature of our app. And a handful of
others are even saying that the app crashes for them. Then to make matters
worse, a few days later we get a security
vulnerability report. Apparently, users can
easily cheat their way to the top of the
global scoreboard. It becomes this big joke
and people start trying to poke holes into our app. All eyes are on us. ROB DIMARCO: At this
point we’re obviously feeling pretty bummed. We put a lot of work in. We really wanted that
launch to go well. But all of these issues
were preventable. So what did we miss? That is what we’ll be
talking about today. We’ll give you the tools
to gear up for launch and launch with confidence. REBECCA HE: So looking
back, what happened? We built an app and
then we launched it. But then we started
seeing unexpected bugs and bad customer reviews. It seems like our testing had
missed something critical. So now, we probably
wrote some unit tests. We probably also tested on a
few emulators and everything looked fine. But deep down we
were still wondering, what would happen
after we launched? So knowing this, how
can we gain confidence in the quality of our app? The key is
understanding that users have different backgrounds
and use different devices. Our users are diverse. So our testing needs
to be diverse too. Given that, what can we do? ROB DIMARCO: First, we’re
going to need real devices. Testing on an
emulator simply is not enough to tell us
the whole story. Next, consider–
what if our users speak a different
language than we do? Or they use our phone
in landscape mode and we use it in portrait? We need to test
different situations too. We also can’t forget about
having a very diverse tester base. We need more users,
more perspectives. Would be awesome if
we could get our app in the hands of real
users before going live. Now, we know what we need to do. But it’s a lot of work. We’re really focused on building
something that our users love and we don’t want
testing to introduce any additional stress. This is where Firebase comes in. Firebase Test Lab
makes automated testing so much easier. Instead of having to go out and
buy physical iOS and Android devices, Firebase Test
Lab stores those devices in a Google data center
so you don’t have to. Instead of configuring
emulators for things like different screen
sizes, different languages, different locales, and the many
combinations of those factors, Test Lab will do it for you. Going back to that
example at the beginning, how could we have used
Firebase Test Lab to figure out why our app was crashing
for certain customers? REBECCA HE: So one
way we could have used Test Lab is by
running a robo test. The robo crawler does a
depth first search analysis of our app. It will go to different screens,
click on different buttons, and really try to test
all the user pads. So let’s go to the demo
and see how this works. Thanks, Rob. All right, so now we’re in the
Firebase dashboard on the Test Lab page for our demo app. We’ll start by
hitting Run a Test and we’ll select Robo Test. So an error is occurring
while loading this page so we’ll try uploading it here. What we would do
is upload our EPK. It would validate. And then the next page we
would see is a device matrix. So on this page,
we have the option to select so many
different types of physical and virtual devices. And the default
selected a few Pixel 2s. But since I’m trying to figure
out why my app was crashing I want some variety. So I can preload a template. And so I will select
the Pixel 2, a Pixel 3, and let’s just get an older
device just shake it up a little bit. And then at the
bottom of the screen, we also have the option to
select a different orientation and also different
locales if we want to test different languages. We would start by hitting
Start Three Tests. This will take a
few minutes to run so let’s just look
at the results from a previous example. Can we switch back to
the slides, please? All right, so
after a few minutes we would have results
that look like this. We immediately can see
that two of our tests passed and one failed. Let’s go through the
results one by one. Let’s first look at the
oldest device, the Nexus 5. So here on the test
results screen, we see that our test passed. Our robo was successful
in crawling our app. But there are also
a few different tabs that we can dig into
with more information. Here on the first page, we
have the crawler metrics. And then if we scroll
down to the bottom, we’ll see a crawl graph. And this gives us a sense
of all the different paths that the crawler took. This is really useful
for understanding the coverage of our robo test. But a robo test
doesn’t just tell us if a crawl was successful,
if it crashed or not. On the screenshots tab, we
also have the opportunity to really visualize our app. And we can even see where
the crawler interacts with user inputs. On the Videos tab, we get
a similar but more detailed view where we can actually
walk along the entire crawler experience. Both the screenshots
and the video give us a really great sense
of how our app looks and feels across different devices. There are two more tabs,
Log and Performance, that we won’t dig into
but those are really useful for debugging issues. All right, so I’m feeling
good about the Nexus 5. Let’s look at the Pixel 2. So here we see that the
Pixel 2 crawl passed. But interestingly, we see a
new tab that says Test Issues. Apparently we’re using an
API that will no longer be supported after Android P plus. API level 28 is Android P
so we’re just safe for now. But doesn’t this mean it will
be a problem for newer devices? We’re feeling concerned. And then if we go back, and this
time we look at the Pixel 3, let’s figure out
why it’s failing. So our concerns are confirmed. If we dug into the
logs a bit more we’d find that there was
a No Such Method exception because that API was
no longer supported. We realize that this is why
we’re having mixed reviews. Customers on older
API levels were fine but customers on
the newest API level were running into this crash. In just a few minutes
with little setup we found the reason why our app
was crashing for our customers. ROB DIMARCO: Robo tests
are really powerful. But we can also run highly
customized iOS and Android instrumentation tests. Firebase Test Lab
lets us perform end to end tests for all
our apps in one place. Now that we’ve run
Firebase Test Lab, we’re feeling a
lot more confident about the quality of our app. But no matter how many
devices we test on, automated testing
doesn’t have context. The crawler doesn’t understand
what your users actually want to do. We need real users
to be able to tell us if the user experience is
confusing, or unintuitive, or if the app is not behaving
how they might expect it to. We need some level
of manual testing. That’s where Firebase App
Distribution comes in. Let’s walk through it. Instead of having to manually
send all of your APKs in IPAs to testers,
App Distribution lets you manage iOS and Android
distributions in one place. In one central location you
can upload distributions, attach release notes,
and send to testers. And what’s more, you can get
start it instantly with no SDK. It also provides an
intuitive UI for you to manage testers
and create groups. For example, you might want
to create two internal tester groups, one for iOS
and one for Android. For the tester experience, when
you upload a new distribution testers get emailed instantly. App distribution also
provides a tester UI so your testers get to
choose which version of the app they want to install. This gives them the ability to
easily test different variants of your application. During the keynote, we
demoed how app distribution can be used in the Firebase
console and with Fast Lane. But you can also distribute
iOS and Android apps with the Firebase CLI or
with the Gradle plugin. We understand how important
automation is for you, and we want to give you the
choice and the flexibility to use Firebase App
Distribution with the tools that you’re already
familiar with. Let’s take a look look at Gradle
plugin and see how it works. REBECCA HE: So going back
to that example, when we had one star reviews
and customers didn’t like a certain feature
of the app, how could we have used App
Distribution to get ahead of that feedback? Let’s check out a demo. So here I’m on the Firebase
console for my app. I’ve already distributed
two versions of my app to testers using Firebase
App Distribution, but now I’m ready
for version 1.2 because I’m releasing
a new feature. So to distribute with
the Gradle plugin, I’m in my build up Gradle. The first thing I need
to do is apply the plugin and add the dependency. Then, in the specific
build variant, you can specify a
Firebase App Distribution block for the parameters
that you care about. In this case, we’re specifying
a few parameters including a service credentials file which
lets us authenticate, release notes to let testers know what’s
included in this new version, and groups and testers, the
people who we want to receive the distribution of our app. So now that we’re ready,
we’ll go to the command line and run a command. We’re going to first assemble
the debug version of our app and then run App
Distribution, Upload Debug. So what it’s doing is, it
will upload the debug version of my app to Firebase
App Distribution and then make sure
my release notes and testers are also included. I also want to point
out that it doesn’t have to be the debug command. App Distribution is
variant specific. We want to give
you the flexibility to upload a Firebase
App Distribution and have different
parameters for different build types and product flavors. So it looks like it
uploaded successfully so let’s go to the console
and check it out! So if we refresh
the page, we should see a new distribution for 1.2. And we do. We verify that it
has the release notes and the right testers. And then, if I’m a tester
and I refresh my inbox, I should get an email. But in the meantime, let’s
look at a previous release. I would get an email
looking like this and I’d be able to open
it on my mobile phone and start testing the app. And here the email just came in. Can we go back to the slides? Awesome. So we successfully
distributed our Android app. But I want to point out that
we can distribute iOS apps too, using the CLI tools. Using App Distribution you can
address feedback before launch and ultimately be more
confident that your app will work in the wild. Also, as you may have seen
earlier in the keynote today, App Distribution is
now in public beta. So go to the console
to get started. ROB DIMARCO: Now that
we’ve tested our app, let’s move on to
validating its security. Think back to that example
at the beginning this talk. That’s terrifying, right? If your app goes
live, it’s growing, it’s met with initial success. But that growth and
that success also invites additional scrutiny. We all want to get
application security right. But this task can seem daunting. For the next few
minutes, we’re going to give a brief overview of
the application level security model in Firebase
and then show you how to use Firebase tools
to validate the security of your app before going live. For many of these
might be a refresher. But let’s start with how
to think about security for apps built on Firebase. Usually when you build an app,
your mobile or web clients connect directly to
services that you control. And then those services
perform authorization and authentication checks before
they change or mutate data. But when you build
an app with Firebase, we abstract much of that
infrastructure away. So you don’t need
it think in terms, or keep up with, OS level
or network level security. With Firebase, we ask only
that you think in terms of application level security. But we still need to ensure
that only authorized services, clients, and users, are
able to read or write the correct bits of data. So how do we secure
those requests from untrusted client devices? The answer is with
security rules. Security rules are the tool
used to define whether or not authorization should be granted
for requests to your database. We’re going to walk
through the tools that Firebase offers to
help you learn, write, and test security rules. REBECCA HE: First, let’s
see an example of how security rules can be helpful. Let’s say I’m working on my
new game called Goldmine. In the game, users
collect gold as they go through each game play. They also have the ability
to compete against each other and place in a
global scoreboard. Each of these data types– the users, the game
play, the leaderboard– has a different security need. Let’s dig deeper into the
security need of users. So we want user records to
be publicly readable but user editable. This means that only the
user themself should be able to edit their own profile. Makes sense. So how can we enforce
this using security rules? Well, writing rules
can be complicated, especially if we’ve
never done them before. This is where the Security
Rules Simulator comes in. The simulator is an in browser
playground that lets you safely experiment with security rules. You can simulate
reads and writes without actually needing
to modify your database or having to set up a
development environment. Let’s go to the console and
see a demo of how this works. So here we’ve started writing
security rules for our app. The first thing we need to do
is match a specific document. In this case, we’re
matching the user document. And then inside
this block, we can write rules to match read
and write operations. Let’s think about reads first. We want user profiles
to be publicly readable. Anyone should be able
to read it regardless of if they’re logged in. So we’re going to write a rule
that’s allow read if true– pretty straightforward. Now let’s move on to writes. So for writes,
remember we only want the user themself, the
user who is logged in, to be able to edit that profile. So let’s try taking
a stab at it. All right, given my limited
security rules knowledge I think this is right. But how do I validate it? Let’s go to the simulator. So here in the simulator I’m
logged in as myself, Rebecca. And I’m trying to
update Rob’s profile. This shouldn’t be allowed. So I’m going to click
Run and the simulator tells me that the
write is denied. All right, that makes sense. So now let’s try to
edit my own profile. If I hit Run again,
the write is allowed. This looks good, so I’m going to
publish my rules and save them. Let’s go back to the slides. ROB DIMARCO: Now, the
Security Rules Simulator is useful for learning
and experimenting. But once you start building
an app for production, you want to start developing,
testing, and deploying rules following the same best
practices you apply to the rest of your code base. And for that, we turn to
the Firebase Emulator Suite. The Firebase Emulator Suite lets
you run our production services on your local machine,
including Cloud Firestore, Cloud Functions, Firebase Hosting,
and the Firebase Real Time Database. During the keynote, we saw
how the emulator suite can be used for local development. But even more importantly,
the emulator suite is essential for
thorough testing. Running locally it’s
super fast, it’s free, and it doesn’t introduce
the risk that you in any way harm a running
production application. Going back to my
Goldmine game, let’s move onto securing the global
leaderboard where users can post their best scores. Using rules, I’m trying to
prevent fraud in this game by making sure that users can
only share scores for games that they themselves played. And those scores need to match
the official gameplay result– no spoofing. Let’s use the Firebase
Emulator Suite to validate this behavior with
thorough integration tests. Switch over to demo. Here in my ID, I’ve started
writing some integration tests for rules to secure
that leaderboard. First, I’ll start the
local emulator suite with a single command
using the Firebase CLI. With that, I have
a local instance of Cloud Functions, Firestore,
the Realtime Database, and Firebase Hosting all
running on my local machine. As you can see, each time I save
my security rules the emulator suite is noticing the
change, picking them up, hot reloading those rules, and
applying them to my databases. Now take a look at my tests. Compared to what the
simulator is capable of, this is much more advanced. This is far more rigorous. This test, for example, will
clear the Firestore database, bootstrap it with
some gameplay data, and then I’ll write
a test that is going to check to
see whether or not a user can share that result
with a mismatched score. It passes! Great. But remember, when
testing security always test both code paths– both the access granted
and the access denied. Think of this just
like code coverage. If we test the if
branch, we also need to test the else branch. Adding the complimentary
test and rerunning, we’ll see that we’ve got a bug. It looks like I’ve got a
typo in my security rule. I’ll fix this typo,
save my rules. The emulator suite will
hot reload the change and I’ll rerun my test. All right, it passes. That’s what the Firebase
Emulator Suite is all about– rigorous, comprehensive
security testing for your apps before launch. Let’s go back to the slides. REBECCA HE: So at
this point, we’ve corrected all the problems
with the initial launch. We’re feeling confident. But how do we make sure
we launch with confidence every single time? The answer is with automation. Automating lets us maintain
stability and security without sacrificing velocity. Building great apps is hard. We’ve all got things to do. Speed matters. ROB DIMARCO: That is
why all of these tools were built from the ground up
with continuous integration in mind. We understand how important
automation is for you, and how important
speed is for you, and we want to you succeed. Let’s see an example now of how
you can integrate Firebase Test Lab, Firebase App Distribution,
and the Emulator Suite together in your CI/CD pipeline. REBECCA HE: So here we have an
example continuous integration configuration. First, I’ll need to download and
install the command line tools for Firebase and Google Cloud. ROB DIMARCO: Next,
we’ll invoke our unit and integration tests using
that Firebase Emulator Suite. This lets our security
tests run locally, quickly, and in a safe,
isolated environment. Running the tests in
this way also ensures that the emulator suite is spun
up fully before tests are run and cleanly shut down after
running tests is complete. REBECCA HE: Then we can
run our Test Lab robo test. In this example, we’re
testing it on two devices. ROB DIMARCO: If all
of our tests pass, we deploy our changes to
Firebase and distribute the latest version of our
app to trusted testers using Firebase app distribution. If we wanted to, we could
customize this further, doing things like distributing
different app versions or variance to different
tester channels. REBECCA HE: So let’s summarize. We’ve built our app, we’ve
tested it, we secured it, we’ve added automation, and
now we’re ready for lunch. Remember, these app stability
and security problems are preventable. Firebase is here to
give you the tools you need so you can launch
with ease and confidence the first time and every time. ROB DIMARCO: Go back to your
teams, try these tools out. Let us know what
feedback you have and how we can help you further. Thank you. [MUSIC PLAYING]

Leave a Reply

Your email address will not be published. Required fields are marked *