Frictionless Android testing: write once, run everywhere (Google I/O '18)
[Music] my name is Jonathan and I work on the mobile ninjas we're a team within Google passionate about testing and testability if you've ever written test for Android before it's likely you've used some of our products espresso Robo electric or the Android testing support library and together these tools combine for
billions of testing vocations both within and outside of Google so today there's a general consensus within the software development community are the virtues of writing tests sure there's a cost to writing tests but it's accepted one and is one that quickly pays dividends throughout the lifecycle of your project tests provide fast feedback on failures like a bug caught early on in development is far cheaper to fix than after you've deployed your application they give you a safety net for making changes to your code you're free to refactor cleanup and optimize safe in the knowledge that you're not going to break any of the existing functionality and what's more a suite of readable tests provides the living breathing specification of your applications behavior now in software development there exists the concept of the testing pyramid it's made up of three layers unit integration and end-to-end tests and as you go up the pyramid you're gaining fidelity but this comes at the expense of test execution time it also gets harder to maintain and debug these kinds of tests but just like a rock band where you need a perfect blend of musicians to create that great track each layer in the pyramid is equally important you're leveraging the Vantage's of one layer to compensate for trade-offs in others to produce a holistic automated testing environment we recommend a 70-20-10 split as a general healthy guideline and now while the rules of these pyramids still apply some of the unique characteristics of Android development have introduced some difficulties along the way so unit tests due to their need to be fast will run on the local workstation an integration and end-to-end tests due to their need to run in a really faithful environment tend to run on a rail or a virtual device and so separate tools are evolved at each layer of the pyramid robolectric or the mockable android framework for your off device unit tests espresso and the Android testing support library for those on device tests now androids got some really familiar core concepts such as getting a handle to your application context or maybe driving your activity lifecycle and each of these tools has its own distinct api's and ways of doing things for achieving these exact same tasks now this has led to something of a test writing crisis as a developer it's hard sometimes to know what tools are available for use and which of those are recommended having multiple tools at each level has led to an explosion of styles each with their own distinct patterns and api's and this in turn leads to a lack of mobility between layers tests can't easily be refracted or reused between layers in a pyramid without being completely rewritten for a new tour to discuss this further let's orientate ourselves with what constitutes a well-structured test so there's some following common patterns that define a well-structured test and so we generally break tests down into three clear sections given when and then and I like to separate them with a blank line to clearly demarcate them so given some predetermined state of the system when you execute an action that you wish to test then verify the new state of the system or some behavior has occurred be sure to name the test after both the condition you're testing and the expected outcome keep the test focused on very specific behavior and then test all your behaviors independently unless it's always more with tests these guidelines will help you keep each test understandable in isolation use common setup methods only for scaffolding this is maybe creating the object that's under test and wiring up some of its dependences let's take a look at the problem of the explosion of styles and we'll highlight this using a simple test case we'll have a single activity with one button that responds to a click sending an intent to the Android system and I'm going to walk you through at this test case comparing the different styles of mockito Robo electric or espresso so first up let's consider mojito now mocking is a really powerful tool but is one that's often overused and sometimes used inappropriately mocking your own classes is great but while mocking the android framework may seem like a great idea at first it consumed lead down a path of difficult problems many Android classes are stateful or they have complex contracts and these are really difficult or even impossible to satisfy with marks so even though we don't recommend it let's just start by taking a look at one of these tests and walk through it using the mojito framework to mock the android framework so in the given section we can you a power activity under tests but look we first of all have to stub out some of the android framework behavior in the superclass activity so that it responds as we expect in our tests now this introduces some problems first of all were partially stubbing the class under test and this means we're not testing the true behavior of that object on the test and furthermore it brings with it excessive stubbing which introduces all this undesirable boilerplate and that quickly becomes distracting from the true intention of your test for the when section of the test to execute your code under test you'd have first had to need to register an argument captor earlier to get a handle on that click listener which you will then vote manually to to call the code that you're wishing to test now with this approach you can soon descend into a undesirable mess of argument captors stubbing calls and answer invocations finally in the then section to obtain ten intent sent to the Android system you can have to use another argument captor in mocking the Android frameworks in this way tends to force you into a testing implementation details when you want to be testing behavior instead and furthermore these drawbacks have tended to lead to developers to build their own abstractions to isolate Android this little turn leads to its own set of problems firstly you're introducing another layer of Krofft into your application and secondly you're introducing text in gaps where bugs can hide and we believe that while you should architect your application very thoughtfully the limitations of the tools shouldn't dictate your application architecture either so let's see how this looks like with robolectric robolectric is the popular open source testing framework it allows you to follow the best practices surrounding marks as you're able to use real Android objects in your tests rather than having to program your own stubbing behavior in each test it runs on your local host which means it's very fast making it ideal for unit tests robolectric tends to create tests that a lot cleaner so let's walk through each section in turn in the given section we can simply bring an activity into the resume state for our test just by calling robe electrics setup activity API in the when section we're able to use real Android SDK api's such as fine view by ID to get ahold of the view and then robolectric s– click on api safely click on that view and invoke the code that we wish to test and finally in the vent section we again use robe electrics own testing api's to check that the intent was sent to the system see how much cleaner this version is we're focusing on the items that really matter in the test and were free of all those extra pieces of distraction now espresso is a UI testing framework and it runs on a real or a virtual device it provides you with a really realistic environment the trade-off here is a much slower execution speed you're building up your entire apk deploying it to the device and instantiating the test run waiting for the results and then collecting those back on your local workstation all of this is adding valuable development cycles now the exact same android concepts exist here we're just getting hold of an activity clicking a button and then verifying an intent was sent the Android system here though as you'll see the api's are very different let's step through this one together so in the given section we'll use the activity test role which comes from the Android testing support library this can be used to start an activity bring it to the resume state and provide us with a handle to it in our tests for the when section we can use the espresso view matcher api's they'll find the view in question and then safely click on it to invoke the code that we wish to test and finally in the vent section we use the espresso intents library to capture that tent and verify that it was the one that we wanted sent to the system notice here but a Twila test has many similar structures to robolectric test the example that we saw earlier the syntax is very very different so while each of these testing frameworks have contrasting strengths and weaknesses it's this explosion of styles that's really become the big problem for writing tests who in the audience has been using robolectric to write tests raise your hands and what about espresso and who's using both we often hear developers talking about I need to write a robolectric test or I need an espresso test but we'd much rather you be thinking about writing an Android test instead we feel as a developer that no matter what kind of test you're writing you shouldn't first have to think about environments and tools and libraries that you'll need we believe that you shouldn't have to suffer the mental load of having to learn multiple sets of api's for doing exactly the same thing so and of course you should have the freedom to refactor and reuse your code no matter where you choose to run it so what if there was only one set of API is that you needed to learn and now imagine also being freedom to focus on writing your tests rather than considering those tools and libraries and environments well to make this a reality today we're launching android test it's part of jetpack with jetpack testing is now a first-class citizen of the Android tool chain will unifying the development experience around a canonical high quality set of api's that will reduce the boilerplate and eliminate the number of tools that you need to learn naturally Kotlin in support is included allowing you to write beautifully concise tests and of course all of this will be open sourced we love contributions from our community members we're going to satisfy developers needs in each of the four key sections of a test remember scaffolding given when and then scaffolding encompasses the configuration and control api's think getting a hold of that application context for your test scaffolding Android j-unit foreigners used to execute your tests you can use the instrumentation registry to get a handle to that application context well today we're excited to announce for the very first time you can now use these api's for both your on and off device tests the next section of a test is the given section and here we're going to provide two key categories of api's for you firstly the under Jamiat rules from the Android testing support library will become part of jetpack and will soon adding more api's to help drive the component life cycles for you in tests as you've seen previously the activity tests for all is used to start up your activity and make it available to your tests in the resume state you probably use this API running on a device many times before well today this API too will be available for tests that run off device as well secondly we'll be providing you with a set of Android test data builders these will help you construct Android objects that your code on the test will interact with many of the android framework classes that you need to have for setting up your test date are difficult to create often there's no public constructor so they can't be instantiated for tests perhaps their final so mocking is out of the question either and sometimes they just plain clumsy to instantiate with any degree of brevity at all so we're including Android test builders within jetpack to give you a concise way to set up your test environment they produce readable code they're fluent way to create the Android components that you need to interact with and of course they're portable Android test data builders work for both your on and off device use cases the third section of a well-structured test is the when section exercising the code under tests usually this is just simply a case of calling your own code directly but when you're writing a UI test it's likely that you'd write for the reach for the espresso view match your api's today we're happy to announce that espresso 2 is joining Android test jetpack espresso view API is a fluent and they read beautifully you've used them for you're off on device tests for a while now and today we're providing preliminary support for these tests in the off device use case also the final part of a well structured test is a Ven section this is where you make assertions on the state of the system in response to an action so firstly espresso intense two is going to be joining jetpack as Android test intense those intense ap is that you've been using for your on device testing great news today they too will run in your off device tests and finally we're also releasing an assertions library to help reduce the boilerplate in your tests using traditional Jay Manette assertions can lead to tests that are not immediately readable see here how easy it is to get the actual in the expected arguments mixed up an Android uses a lot of integer constants for efficiency but this makes it difficult to comprehend the error messages in tests at Google we love to use truth it's our own open source fluent testing assertions library using a fluent assertions library is a great step to producing more readable code and writing the tests become much more easy too because you can lean on the built-in support of your ideas Auto completion feature so to help you write concise tests against Android code we'll be releasing a set of truth extensions for Android which reduces the boilerplate read beautifully and gives immediately meaningful error messages of course these assertions will work across all environments both neon and off device tests so with jetpack Android tests we'll be bringing you the tools that you need so that you can concentrate on writing beautifully concise easy to read tests without first worrying about libraries or tools or environments a single set of canonical api's for common tasks that will reduce the boilerplate leaving your test clear and readable and a very environmentally agnostic allowing you to run the test either on device on your local workstation or perhaps in a cloud test lab and now that I've shown you how to use this unified set of api's that decouple the act of actually writing a test from where it's going to run I'll hand you over to my colleague Stefan who's going to show you how you can run these tests in the new simplified world Stefan [Applause] Thank You Jonathan welcome everyone ten years entered quite amazing I also want to say a big welcome to everybody who is joining us at the livestream today at Google we believe that testing should be a fundamental part of your app development strategy let's bring back the pyramid as we can see our friend with the chat pack has solved the API dilemma now that we got one API to rule them all it's really easy to start writing tests for Android no more excuses we can start with a simple test for our business logic usually a unit test and over time we can add more and more tests once we start implementing the UI for our application it might be worth adding an integration test this chest became a fluent experience since both layers support the same api's no more context switching great but that's not the only use case where the new API became really handy let's imagine we have an integration test I got two large and complicated I have seen many of those tests it's just too convenient to test business logic and UI flow in one single test at Google we don't like those tests they are hard to read and they tend to become flaky small well focused tests are much better so let's refactor this large complicated test refactoring a large test can be a very painful task Queeny audience has experienced this recently raise your hand here's some good news for you if the original test was written with the new API this test become much simpler we still have to test our UI and we'll keep this part in the integration test this layer provides high fidelity and we don't want to lose that for our UI test the rest mainly business logic can directly go into the unit test layer here we gain speed since we can run off device on the local java vm so let's verify the refactoring together we still have high fidelity for the UI test that's good we gain speed even better and the tests are decoupled and less complex nice job your co-workers will thank you so let's run those tests but wait we still have to choose a random environment it's not obvious trivial to pick the right one we also have to work in multiple source sets the combination of runtime environments plus source sets full lead to an explosion of test configurations just imagine how hard it will get to choose the right configuration should I run on a device or offer device and don't we need to run the entire test suite on our continuous integration server before we submit and you all know the feeling I have to be kicked off the test run waiting eagerly for the results Oh No how to submit one of the test is flaky i'm pretty sure many of you have been in this situation flakiness is one of the biggest pain points for developer productivity so what if there was a better way to set up your test harness execute your tests in a reliable environment with unified test results today we are proud to announce project nitrogen the new single entry point for all Android tests nitrogen is fuel for your jetpacks with our expertise of running billions of tests a year we had to build the fastest and most reliable test executor nitrogen covers the entire testing lifecycle from set up to test execution and reporting nitrogen provides deterministic behavior across different pool systems it will be fully integrated in Android studio at Google we already used nitrogen to test our own apps such as Gmail google maps photos YouTube and many more nitrogen is highly extensible it provides api's for test authors to customize the test invocation at any point nitrogen will be fully open sourced later this year but let me give you an overview starting with setup natural chain connects your test to any execution environment on device of device or in the cloud it installs required test artifacts and if necessary it can run custom fixture scripts for you let me show you how this works first natural chain finds and provisions a device this can either be a simulated device a virtual device or real device once the device is ready nitrogen installs the application under test and the test matrix n stages any test data dependencies and sets device properties if additional setup is necessary it can all be done here let me give you examples setting up a network tunnel starting an emetic server caning granting permissions and much more and finally it prepares the device for the next step test execution there are many ways how we can run an Android test it's a little bit like running in the real world here we have Sprint's with distance and long distance runs we also have different environments direct running Road running cross-country running and mountain running a unit test on your local workstation is like a sprint as fast as possible to the finish line the tests read on your continuous integration server is the marathon the goal is to get to the finish line and not to fail along the way today these tests are executed in different ways from the command line from Android studio or triggered automatically when you submit with project nitrogen this will become consistent we provide a valid affiant protocol and our unifying all the Android test offerings natural chain uses our own device infrastructure to run your tests Android Orchestrator and enrich a unit runner you can already enable the own device infrastructure today it has been available since Android studio 3.
0 the orchestrator collects your tests and kicks off test execution by running each test in a separate process shared state is minimized and crashes are isolated moreover your tests are executed in a familiar j-unit environment provided by an original runner the orchestrator collects all test results additional artifacts and streams it back to nitrogen-nitrogen provides a unified reporting format for these test results in addition it provides a huge selection of test output data such as logcat screenshots video profiling data battery performance and much more all artifacts are scoped per test this means that for example a Lockett snippet is reduced to the test method no more digging through hundreds of lines of Lockett so let me show you how nitrogen improves the entire testing flow on Android first nitrogen finds the device and configures it for the test run second it runs tests in isolation using the orchestrator and finally while tests are running nitrogen host-side infrastructure will be streaming the test results from the device pulling down all test output data and feeding it back to you with nitrogen all this complexity is hidden from you whether you are running a test on Android studio or on a CI server nitrogen is the single entry point for all Android tests it works from a sprint to a marathon natural Chen can regress the device in a fire base test lab reliably running your tests returning unified test results nitrogen supports Google Cloud from your workstation you can deploy and run tests on the real device or on a virtual device nitrogen works seamlessly with Rob Electric now we can treat Rove electric as a simulated device with raw electric 4.
0 we have made big improvements in start-up time and memory consumption raw electric for bond oil is released today and if this is not enough nitrogen supports your custom needs for example if you have an in-house device let allow me to summarize previously you had to learn multiple approaches for doing the same thing tools like the mobility to move between the layers of the pyramid you had to choose wisely we have reduced the cognitive load by providing you a single set of api's that work across environments for both on and off scenarios and a single entry point for Android tests with the flexibility to customize any point in the test invocation jetpack with nitrogen is a child leap forward in test automation for Android write your test once run it everywhere [Applause] this is just the beginning now you have both the tools and the knowledge to accelerate your testing experience I strongly encourage you to check out our collabs especially our latest edition building Android apps with basil basil is the open source version of our internal build system allowing you to build large Android apps at Google scale if you have further questions or if you would like to discuss your testing strategy come find us tomorrow morning at 11 a.
m.
in the office our stand in section 8 we hope you enjoyed our session and we will love to hear from you so please take a moment to submit your feedback and with that happy testing [Applause] [Music].
Nhận xét
Đăng nhận xét