Ideal Test Execution Summary / Report

What the ideal Test Execution report should include, is always a question of debate. It should be crisp and contain the infomration required by Test Management Team OR all the stake-holders. Below are the important aspects which Test Execution Summary should include.

Level Of Testing :- Unit/System/System Integration/UAT

Type Of Testing :- Functional/UI/Compatibility/Non-Functional (e.g. Performance, Security etc)

Enviornment Details :- Build Details, Server Details on which execution is carried out, Database details, Database package details

Test Suite Details :- Details about the test suite executed i.e. e.g. Regression Pack 1.1, SIT 2.2 etc.

Total # of Test Cases:-

Total # of Test Cases Executed :- No Test Cases should be in "No Run" (ideally)

Total # Test cases Passed :-

Total # Test Cases Failed :- # along with Defect# raised during this run of Test Execution

Total # Test Cases NA :- # along with reasons

Total # Test Cases as Deferred :- # along with reasons
% of Test Cases Passed : (Total#Test Cases Passed)/(Total#TestCases Executed)*100

% Test Cases Failed :-  (Total#Test Cases Failed)/(Total#TestCases Executed)*100

These are the "MUST" of any Test Execution Report

If we collate this information in tabular format, then this will give exact picture of our Test Execution.

Sample Test Execution report is as below, please not all data is fictious any resemblance with other is purely co-incidance.

Software testing - Is it different in Agile projects from traditional projects?

"Yes", There are lots of differences in testing an application in agile methodologies and in traditional projects. To have a comparative study lets consider Agile Scrum as base

"Sprint team should ship a workable product at the end of each sprint" is the purpose to adopt the Agile methodology.

Documentation Challenges:

Agile project follows minimum documentation so each time test team have to liaise with Dev, Product, System Analyst to get their queries clarified. (Depending on nature of queries). This slows down the test prep phase and sometimes understanding gaps. So testers have to be a quick learner to overcome these kinds of hurdles.

Estimation:

Estimation used in Agile projects is Poker planning. This requires knowledge of application under test. This imposes a barrier on new team members in participating in estimations, poker planning and sprint planning sessions.

Review of Test Deliverables:

In Waterfall or other project methodology, we allocate special time frame for Review, Rework and Sign Off of Work Products. But in Agile, we follow generally a sprint cycle of 2 weeks or 4 weeks. In such a short span of time, it’s very difficult to follow a detailed review process. This may impact in missing some test scenarios. So tester should be competent enough to not to miss any scenarios and should give a full test coverage and assure a quality product.

Solution to this review process could be selective review of functionalities. Business critical functionalities should be reviewed irrespective of time crunch issue.

Retesting after Demo:

In Agile Scrum, QA Points provides the demo of functionality to product owner and sometimes product suggest the changes after demo is completed successfully. This creates additional burden to QA Team to complete the testing. (retesting the complete functionality)

Regression Test Efforts:

In each sprint, some enhancements are delivered. Regression testing is required to show that these enhancements are not impacted the current functionality. In turn regression testing is required irrespective of 2 week / 4 week sprint. If regression suite test count is too much then this will delay the test certification and project delivery.

To reduce the regression testing window, automation of regression suite is necessary.

Updating of Test Documentation / Regression Suite:

In 4 week sprint cycle, it’s very challenging to update the existing test documentation (master test plan, test strategy) and regression suite. So at any point of time these are not updated and team need to follow a sperate plan to keep these test deliverables.

This achieved by adding a tech deb task in, in between sprints.

Resource Management:

If a resource goes on leave for more than 3-4 days in a single sprint, then this creates a burden on other testers. Because of this there is risk of not meeting the commitments by the sprint team.

As sprint cycle is very short so this kinds of issues are very frequent.

To summarize, to perform the testing in Agile Projects: - Testers have to be quick learner, 'can do' attitude of work and a good team member with excellent work ethics.

"It’s not every one's cup of Tea :)"

Civil Services Aspirant

Excellent blog to get motivated to prepare for Indian Civil Services / Competitive Exams and lot more

http://chasingthebeacon.blogspot.in/2009/05/v-behaviorurldefaultvml-o.html

Technical - Android Emulators

Below are few tips while installing Android emulator in CMMI company environment where security instructions are placed :-

1.0 Make sure that you have full internet access . i.e. download access

2.0 Please add your proxy server details to settings section of Android emulator

3.0 Disable your antivirus.

4.0 If your company policy doesnt allow you to disable Antivirus then you can add your host to exception list of Antvirus. File type to add to exception list should be media files .

These are what i can recollect at the moment, will add as i get more.

Trouble shootings -

1] Few times you launch your AVD and observes that there is no signal , work around for this issue is just relaunch AVD.

2] In ADB, few times your device doesnt get listed.i.e. if you run command adb devices then you get output as List of Devices as nothing, though you have few AVD's running.

Just run below commands to start your local server again -

adb kill-server

adb start-server

These should start your daemon sucessfully and devices should get listed.

Technical - Updating your iOS

Thought of sharing the experiance of updating iOS on apple device.

I updated my iPod Touch to 4.2.1 from iOS 2.1.2. Its very easy, steps/tips/tricks as below. I performed all these things on my Windows machine.

1.0 Connect your iPod touch to the machine and launch your iTunes.

2.0 Go to your device i.e. iPod touch and verify all the details viz device name, serial number, firmware version i.e. iOS version etc.

3.0 You can see the update and restore two options in the device menu.

4.0 Click on update.

5.0 iTunes will connect to app store and determine the latest version of iOS compatible for your hardware.

Note :- While updating, latest version available in market is iOS 5 but my hardware was not compatible with this, so smart app store determined to install iOS 4 on my iPod touch :).

6.0 iTunes will start to download the setup under iTunes store section in download tab.

7.0 This download should take around 3 hours of time. [ mine took 3 hours ]

8.0 This setup doesnt get stored at any location and as soon as download completes, it will start applying these files to your devices.

9.0 In the process of installation of iOS, iTunes will show two messages.

One as pop-up window - "Back-up iPod name>> "

And other

Processing software files....in the iTunes upper tab.

10.0 In whole process, i never saw a message as "Updating your iPod touch" its always shown as Restoring your iPod...not sure why this was.

11.0 Once the application of files is done, message of iPod touch being restarted is shown.

12.0 Your iPod touch is restarted and you are done with updating.

Key to successful updation is be patient , most of the time you will think nothing is happening but its not the fact.

Keep your iPod touch pluged in and you can see the new Version of iOS installed.

This is the best , what i can recollect. Please correct if i am incorrect at any of the steps.

Best Regards,

Vaibhav