E2E Code Coverage, is that even possible?
The concept of code coverage is used to ensure that the main application code has been tested through Unit Tests. End to End (E2E) tests do not run code in isolation, they run on the UI. If E2E tests run on the UI level and have no view on the application code, is it possible to see how much code is covered by E2E tests? Also, should E2E tests have any level of code coverage? Is E2E code coverage even possible?
E2E Code Coverage – Should UI Cover Code?
Why do we use Unit Tests to cover code? We do it because it is easy to do, we gain confidence in our code and it allows us to make changes knowing that if anything breaks, our tests will catch it. However, an E2E test can not do this since it cannot see the code. An E2E test only see’s the outer shell of the application, it can only see the UI.
This almost sounds like E2E tests are not designed to see the code! Is this a good thing or bad?
Being able to see the code from an automation perspective is great. We can be very careful, delicate but more importantly precise about what we test. However, Unit Tests cannot test an application on a large integration level. This means Unit Tests, although they can see code, they cannot ensure that two or more components are working together. E2E tests on the other hand are able to test the interactivity of all components of an application, in one go.
With this said, should an E2E test cover code coverage. No, they should not. They should sit away from code. This is not a disadvantage, this is an advantage. E2E tests have that flexibility of knowing how an application works in its entirety. To do this, E2E tests need to sit outside the application and observe it (just like a real customer/client).
E2E Test Can Do Functionality Coverage
Let’s assume for a moment that your E2E tests need to provide some level of coverage for your main code. If an E2E test cannot see your main code base, how can it conceptualise the concept of covering code?
Well, we may not be able to do it directly, we may be able to do it explicitly.
Let’s assume we have captured all the functionality of an app through the medium of documentation. Let’s also assume that the functionality captured is 100% of the functionality of the app. Finally, let’s assume that we have written a test covering each of the functional points in our documentation. Does this now mean that we have 100% explicit code coverage of our application through E2E tests?
You may not agree, code coverage dictates that we actually touch the code, in this instance we are explicitly touching the code. But, if you agree that explicitly covering the code is the same as touching the code itself then perhaps we can consider functionality coverage via E2E tests is the same as code coverage through Unit Tests.
What do you think?