AUTOMATION FRAMEWORKS

A Test Automation Framework is a set of Standards, Assumptions and Process to come up with Automation Testing for a specific application for better management and maintenance of testing activities.

Points to be Remembered:

1.      Is it possible to automate 100% of the Application?

Yes, it’s possible. But it’s mostly for products, not for projects.

For Products, most of the features will be standard. Even any changes come across means, very limited. So it’s possible to automate 100% of the application.

For Projects, it’s not like we can’t automate 100% of the application. But client requirements will be frequently changing and not advisable to go for automation development for these kinds of areas. Hence we pick up the stable functionality related test cases and try to come up with automation development.

2.      Is automation really meeting the Clients Expectations?

Yes, automation meeting clients’ expectations.  Only the thing is, have to come up with expertise framework with skilled people.

Types of Automation Frameworks:

            There are 5 types of Automation Frameworks available.

1. Record and Playback Framework or Linear Framework.
2. Modular Framework.
3. Keyword Driven Framework.
4. Data Driven Framework.
5. Hybrid Framework.

Record and Playback Framework or Linear Framework:

            This is the framework used to work in starting days of Automation Testing Development. Those days automation was not developed and used to do like just recording the test cases and playing them back using Record and Run buttons, also called Capture and Replay Framework.

In Record and Playback Framework, we follow the below steps.

      Design a Test Case.

      Using the Record Button in the QTP Tool, Capture all the User Actions.

      Enhance the Recorded Script with Verification Points, Synchronization Points, etc…

      Run the Scripts and Analyze the Results.

Drawbacks:

      The major drawback of this framework is, the scripts are not reliable, even if the application has not changed, and often fail on replay.

      If the tester makes an error entering data, or test data got changed, or Application Functionality got changed, etc, the test must be re-recorded.

modular Framework:

            Modularity is well known programming strategy for reusability.  For repeatable kind of task, we come up with making modules and reuse them depending on the requirement.

When working with test scripts (in any language or proprietary environment) this can be achieved by creating small, independent scripts that represent modules, sections, and functions of the application-under-test. Then these small scripts are taken and combined them in a hierarchical fashion to construct larger tests.

Advantages:

      Scripts can develop while application development still in progress.

      Test Automation should be easily manageable and maintainable, when Application Changes happened.

Drawbacks:

      Requires technically proficiency personnel in the scripting language used by the tool.

Folder Structure Followed for Modular Framework:

v     Master Driver: If company is not using Quality Center for managing Automation Framework Development, this is the driver script uses for running the scripts in batch mode un-attended.

v     Main Modules: In this folder we try to place all the main scripts. Depends on the modules available in the Application, we can come up with Sub Folders with Module Names and put in Main Modules folder.

v     Actions: All Reusable Actions of the Application will be placed in to this folder. Again like Main Module Sub-Folders, in Actions folder also we can come up with Sub Folders.

v     Object Repository: Shared Object Repositories will be placed in to this folder and these repositories will be associated in to scripts during Automation Test Execution. We can come up with multiple Shared Object Repositories depends on the Modules Available in the Application.

v     Recovery Scenarios:  If your framework is managing Un-Expected Events, all those Recovery Scenarios should be placed in this folder and associate them to the scripts during Night Execution.

v     Results: During Automation Test Execution, can create 2 types of reports like Summary and In-Detailed reports can generate and will place them in Results folder.

v     Help: All support kind if documents like Automation Process, Automation Standards, Automation Training, etc… will be placed into this folder for Quick Reference.

Note: This folder structure can be customizable depends on the Types of Application, Client Requirement.

Data Driven Framework:

            These frameworks are driven by Data, that’s why these frameworks we are calling as Data Driven Framework.

For this framework, input and output values read from external files like Excel Files, CSV Files, INI Files, ODBC Sources, and Such and are loaded in to variables during run time.

Advantages:

      Scripts can develop while application development still in progress.

      Data input/output and expected results are stored as easily maintainable text records.

      Functions return “TRUE” or “FALSE” values to the calling script, rather than aborting, allowing for more effective error handling, and increasing the robustness of test scripts.

Drawbacks:

      Requires technically proficiency personnel in the scripting language used by the tool.

      Multiple data files require.

      Tester must not only maintain Detail Test Plan with specific data, but must also re-enter the required data in various required data files during each Automation Execution.

      If a simple "text editor" such as Notepad or Excel file is used to create and maintain the data-files, careful attention must be paid to the format required by the scripts/functions that process the files, or script-processing errors will occur due to data-file format and/or content being incorrect.

Folder Structure Followed for Data-Driven Framework:

v     Master Driver: If company is not using Quality Center for managing Automation Framework Development, this is the driver script uses for running the scripts in batch mode un-attended.

v     Utility Functions: Framework Supported Functions like Reading Data from External Files like Excel, Text files, Generating Customized Reports in the form of HTML or Excel or XML or Text, etc put here in this Utility Functions folder and use these functions in the Framework depending on the requirement.

v     Inputs: For passing input data to your automation framework. Depends on the type of the application or client requirement we can pass Keywords through Excel Files, or Text Files, or XML Files, or Environment Variables etc…

v     Main Modules: In this folder we try to place all the main scripts. Depends on the modules available in the Application, we can come up with Sub Folders with Module Names and put in Main Modules folder.

v     Object Repository: Shared Object Repositories will be placed in to this folder and these repositories will be associated in to scripts during Automation Test Execution. We can come up with multiple Shared Object Repositories depends on the Modules Available in the Application.

v     Recovery Scenarios:  If your framework is managing Un-Expected Events, all those Recovery Scenarios should be placed in this folder and associate them to the scripts during Night Execution.

v     Results: During Automation Test Execution, can create 2 types of reports like Summary and In-Detailed reports can generate and will place them in Results folder.

v     Help: All support kind if documents like Automation Process, Automation Standards, Automation Training, etc… will be placed into this folder for Quick Reference.

Note: This folder structure can be customizable depends on the Types of Application, Client Requirement.

Keyword-driven Framework:

Keywords like Browser, Page, Push Button, Check Box, Click, Set, etc drives the framework hence calling this framework as Keyword Driven Framework. Like these words we can pass through Data Table or Excel files.

Means, here in Keyword-Driven Framework, the functionality of AUT is documented in a Data Table or Excel File as well as in step by step instructions for each test. Once creating the test tables, a driver script or a set of scripts is written that reads in each step executes the step based on the keyword contained the Action field, performs error checking, and logs any relevant information.

Example:

Test Table for Opening a Window

Class Name	Properties	Operation	Arguments/Value
Window Name	Menu	Click	File, Open
Window Name	Menu	Click	Close
Window Name	Pushbutton	Click	Folder Name
Window Name		Verify	Results

Advantages:

      Incorporates all the Features of Data Driven Framework.

      The Detail Test Plan can be written in Spreadsheet format containing all input and verification data.

      If "utility" scripts can be created by someone proficient in the automated tool’s Scripting language prior to the Detail Test Plan being written, then the tester can use the Automated Test Tool immediately via the "spreadsheet-input" method, without needing to learn the Scripting language.

      The tester need to know only required “Key Words” and the specific format to use with in the Test Plan (Data Table or Excel File).

Drawbacks:

      Development of "customized" (Application-Specific) Functions and Utilities requires proficiency in the tool’s Scripting language.

      If application requires more than a few "customized" Utilities, this will require the tester to learn a number of "Key Words" and special formats. This can be time-consuming, and may have an initial impact on Test Plan Development.

Folder Structure Followed for Keyword-Driven Framework:

v     Master Driver: If company is not using Quality Center for managing Automation Framework Development, this is the driver script uses for running the scripts in batch mode un-attended.

v     Utility Functions: Framework Supported Functions like Reading Keawords from External Files like Excel, Text files, Generating Customized Reports in the form of HTML or Excel or XML or Text, etc put here in this Utility Functions folder and use these functions in the Framework depending on the requirement.

v     Inputs: For passing input data to your automation framework. Depends on the type of the application or client requirement we can pass Keywords through Excel Files, or Text Files, or XML Files, or Environment Variables etc…

v     Main Modules: In this folder we try to place all the main scripts. Depends on the modules available in the Application, we can come up with Sub Folders with Module Names and put in Main Modules folder.

v     Actions: All Reusable Actions of the Application will be placed in to this folder. Again like Main Module Sub-Folders, in Actions folder also we can come up with Sub Folders.

v     Object Repository: Shared Object Repositories will be placed in to this folder and these repositories will be associated in to scripts during Automation Test Execution. We can come up with multiple Shared Object Repositories depends on the Modules Available in the Application.

v     Recovery Scenarios:  If your framework is managing Un-Expected Events, all those Recovery Scenarios should be placed in this folder and associate them to the scripts during Night Execution.

v     Results: During Automation Test Execution, can create 2 types of reports like Summary and In-Detailed reports can generate and will place them in Results folder.

v     Help: All support kind if documents like Automation Process, Automation Standards, Automation Training, etc… will be placed into this folder for Quick Reference.

Note: This folder structure can be customizable depends on the Types of Application, Client Requirement.

hybrid Framework:

Hybrid Framework is nothing but, combination of Modular, Data Driven and Keyword Driven Frameworks.

This is most successful automation framework most of the companies and projects developing these days.

Folder Structure Followed for Hybrid Framework:

v     Master Driver: If company is not using Quality Center for managing Automation Framework Development, this is the driver script uses for running the scripts in batch mode un-attended.

v     Utility Functions: Framework Supported Functions like Reading Data from External Files like Excel, Text files, Generating Customized Reports in the form of HTML or Excel or XML or Text, etc put here in this Utility Functions folder and use these functions in the Framework depending on the requirement.

v     Inputs: For passing input data to your automation framework. Depends on the type of the application or client requirement we can pass Test Data through Excel Files, or Text Files, or XML Files, or Environment Variables etc…

v     Main Modules: In this folder we try to place all the main scripts. Depends on the modules available in the Application, we can come up with Sub Folders with Module Names and put in Main Modules folder.

v     Actions: All Reusable Actions of the Application will be placed in to this folder. Again like Main Module Sub-Folders, in Actions folder also we can come up with Sub Folders.

v     Object Repository: Shared Object Repositories will be placed in to this folder and these repositories will be associated in to scripts during Automation Test Execution. We can come up with multiple Shared Object Repositories depends on the Modules Available in the Application.

v     Recovery Scenarios:  If your framework is managing Un-Expected Events, all those Recovery Scenarios should be placed in this folder and associate them to the scripts during Night Execution.

v     Results: During Automation Test Execution, can create 2 types of reports like Summary and In-Detailed reports can generate and will place them in Results folder.

v     Help: All support kind if documents like Automation Process, Automation Standards, Automation Training, etc… will be placed into this folder for Quick Reference.

Note: This folder structure can be customizable depends on the Types of Application, Client Requirement.