It’s easy to build an automation framework that works. It’s tough building a framework that works well.
What is an Automation Framework?
The need to have an automation framework should always come from a ‘need to a problem’, not a ‘want to a solution’.
The concept of a framework is to provide a foundation which can be extended for further use. When talking about frameworks in the context of Automation, the same rule applies. Let’s take a real world scenario:
Manager: We need to write some tests, what do we have so far? Developer: Depends on what tests you want. We have Unit tests which give us a good coverage, not many UI tests though. Manager: Ouch, does not sound very good. Can you write some tests? Developer: I can, I will get back to you sometime next week. Manager: Why not know? Developer: Not sure what the requirements are? I need some time to figure that out. Not sure if our current structure supports UI tests.
So, what is actually happening in the conversation above? Firstly, the Manager is asking the Developer to write UI tests. The Developer wants to firstly understand the requirements, check for support and then write UI tests.
- How many tests should the developer write?
- Actually, should the developer write tests right away?
- Should the Developer instead take a step back and think about what sort of tests to write?
The answer to the above questions will most likely help paint a picture of the problem. When building a framework, it is important to understand what the test are expected to achieve. For instance:
- Where should the tests run?
- How should the results be reported?
- How often the tests should run?
- What language should the tests be written in?
Assuming you have an answer to the questions, this will help identify the requirements. Achieving and implementing the list of requirements results in a framework.
In short, a framework is a list of features and mechanics which are needed in order to achieve something, in this instance writing automated tests.
With all of this said, let’s take a moment to discuss some of the most key features of a framework. Here is a list:
- Error / Debug
Let’s take each of the above in turn and quickly discuss:
Writing one test that takes 10 seconds to run can be seen as a good use of time. Writing 100’s of tests which take hours to run can make the framework very redundant. The speed factor here is ‘how long do all your tests take to run’.
Ideally, your framework should support mechanics which can help to reduce the ‘total execution time’ of all your tests. Parallel execution of tests is a great way of decreasing the total test run time.
Implementing patterns such as the Page Object, writing clean code and writing code that everyone understands helps to maintain the test framework. Writing dead code, duplicated code, confusing code converts maintenance into a nightmare.
It is important that your framework is scalable. For instance:
- Being able to extend your test to run against any given environment
- Parameterizing your tests
…make your code more scalable. For instance, if your tests are only able to run against a given environment, this decreases the value of your tests. This also means your tests can not run against other environments.
Being able to:
- Extend your framework to ‘support different technologies’
- Run your tests in any CI environment
- ‘Plug in / out technologies’
…makes your framework more extendable. Not being able to do this would result in your framework relying on specific technologies which can lead to security, reliability and general ‘out of date’ issues.
Error / Debug
What value does a test provide if you are not able to identify the issue if it fails? What value does a framework provide if you are not able to cleanly exist in the instance of systems errors? These are very important questions to answer since they help to aid when errors occur. This also helps to debug your current framework’s code base.
Theory and Principle
With all this said, what’s the point of trying to build a good framework? Why not just build a framework that does the job ‘good enough’?
Well, assume you were handed a framework with no room to change it. You had to ‘work around’ the framework in hopes of writing stable tests. How would you feel if you felt that you could enhance the framework but actually did not have the ability or authority to make the changes. Not a very nice place to be in.
Now let’s assume you had the privilege of building a framework which others would consume. If you build a framework that is very rigid, there is a chance you will be requested for many changes. No one wants to write tests in a framework which itself is fragile.
Building a framework which is:
- Able to adapt to changes
- Easy to write and debug tests
- Easy to add or remove technologies
….makes everyone’s life easier.
I have had the pleasure of working with both good and poor frameworks. One has made my job easier and pleasant. The other, not so much.