CSTE-Common Book of Knowledge-Version 6.2 – Skill Category 1

Factors Affecting Software Testing:

—People relationships
—Scope of testing
—Misunderstanding life cycle testing
—Poorly developed test plans
—Testing constraints

Ref : 1.4 p 1-29

People Relationships :

—The top ten people challenges have been identified as:
—Training in testing
—Relationship building with developers
—Using tools
—Getting managers to understand testing
—Communicating with users about testing
—Making the necessary time for testing
—Testing “over the wall” software
—Trying to hit a moving target
—Fighting a lose-lose situation
—Having to say “no”
*According to the book “Surviving the Top Ten Challenges of Software Testing, A People-Oriented Approach” by William Perry and Randall Rice.

Ref : 1.4.1 p 1-29

Scope of Testing :

—Among the broader scope of software testing are these responsibilities:
—Software testing can compensate for the fact that the software development process does not identify the true needs of the user, and thus test to determine whether or not the user’s needs have been met regardless of the specifications.
—Finding defects early in the software development process when they can be corrected at significantly less cost than detected later in the software development process.
—Removing defects of all types prior to software going into a production state when it is significantly cheaper than during operation.
—Identifying weaknesses in the software development process so that those processes can be improved and thus mature the software development process.

Ref : 1.4.2 p 1-30

Misunderstanding Life Cycle Testing:

—The traditional view of the development life cycle places testing prior to operation and maintenance. See Table 1-5 on next slide
—All too often, testing after coding is the only method used to determine the adequacy of the system. When testing is constrained to a single phase and confined to the later stages of development, severe consequences can develop.
—It is not unusual to hear of testing consuming 50 percent of the development budget. All errors are costly, but the later in the life cycle that the error discovered is made, the more costly the error.

Ref : 1.4.3 p 1-31

Life Cycle Testing Activities

—Studies show the majority of system errors occur in the design phase. Approximately two-thirds of all detected system errors can be attributed to errors made during the design phase.
—The recommended test process involves testing in every phase of the life cycle.
—Requirements phase: emphasis is on validation to determine that the defined requirements meet the needs of the organization.
—Design and program phases: emphasis is on verification to ensure that the design and programs accomplish the defined requirements.
—Test and installation phases: emphasis is on inspection to determine that the implemented system meets the system specification.
—Maintenance phases: the system is retested to determine that the changes work and that the unchanged portion continues to work.

Ref : 1.4.3 p 1-32

Life Cycle Testing Activities (cont.)

—Study sections 1.4.3.1 through 1.4.3.6 starting on page 1-33 for details about the testing process for each phase of the life cycle.
—Requirements
—Design
—Program (Build/Construction)
—Test Process
—Installation
—Maintenance

ref : 1.4.3.1 p 1-33 through 1.4.3.6 p 1-34

Poorly Designed Test Planning

—Variability in test planning is a major factor affecting software testing.
—A plan should be developed that defines how testing should be performed.
—Four steps must be followed to develop a customized test plan:
1.Select and rank test objectives.
2.Identify the system development phases.
3.Identify the business risks associated with the system under development. Place risks in the matrix.

Ref : 1.4.4 p 1-35

Testing Constraints:

—Anything that inhibits the tester’s ability to fulfill their responsibilities is a constraint.
—Constraints include:
—Limited schedule and budget
—Lacking or poorly written requirements
—Changes in technology
—Limited tester skills

Ref : 1.4.5 p 1-36

Budget and Schedule Constraints

—The cost of defect identification and correction increases exponentially as the project progresses.
—A defect encountered during requirement and design is the cheapest to fix.
—Assume that it costs x to fix a defect in the requirement and design phase. Fixing that same defect during the system test phase costs 10x. A defect corrected after the system goes into production costs 100x.

Ref : 1.4.5 .1 p 1-36

Lacking or Poorly Written Requirements

—Test objectives enable the test manager and project manager to gauge testing progress and success.
—Each test objective should contain a statement of the objective, and a high-level description of the expected results stated in measurable terms. The users and project team must prioritize the test objectives.
—Usually the highest priority is assigned to objectives that validate high priority or high-risk requirements defined for the project.
—In cases where test time is cut short, test cases supporting the highest priority objectives would be executed first.
—Test objectives can be easily derived from using the system requirements documentation, the test strategy, the outcome of the risk assessment, and the test team assignments.

Ref : 1.4.5 .2 p 1-38

Changes in Technology

—Technological developments that can cause organizations to revise their testing approach:
—Integration
—System Chains
—The Domino Effect
—Reliance on Electronic Evidence
—Multiple Users

Ref : 1.4.5 .3 p 1-39

Limited Tester Skills

—Testers should be competent in all ten of the CSTE Common Body of Knowledge skill categories to be effective.
—Lack of the skills needed for a specific test assignment constrains the ability of the testers to effectively complete that assignment.

Ref : 1.4.5 .3 p 1-39

Life Cycle Testing

—Life cycle testing involves continuous testing of the solution even after software plans are complete and the tested system is implemented.
—Life cycle testing cannot occur until you formally develop your processLife cycle testing is best accomplished by forming a test team. The team is composed of project members responsible for testing the system. They must use structured methodologies when testing; they should not use the same methodology for testing that they used for developing the system.

Ref : 1.5 p 1-40

Creating S.M.A.R.T. Goals

—Specific
—Measurable
—Attainable
—Realistic
—Timely

CSTE-Common Book of Knowledge-Version 6.2 – Skill Category 1

Skill Category 1 (Part 1)Software Testing Principles and Concepts :

Preview of Software Testing and Principles :

1.1 Vocabulary

1.2 What is Life Cycle Testing?

1.3 Reducing the Frequency of Defects in Software Development

1.4 Factors Affecting Software Testing

1.5 Life Cycle Testing

Testing is a Quality Control Activity.

# There is often confusion regarding the difference between quality control (QC) and quality assurance (QA).

# Quality methods can be segmented into two categories: preventive methods and detective methods.

# Quality has two working definitions:

# Producer’s Viewpoint – The quality of the product meets the requirements.

# Customer’s Viewpoint – The quality of the product is “fit for use” or meets the customer’s needs.

# Both quality assurance and quality control are necessary.

REF : 1.1.1 page 1-2 (CSTE CBOK)

Quality Assurance (QA) vs. Quality Control (QC)

# Quality Assurance- a planned and systematic set of activities necessary to provide adequate confidence that products and services will conform to specified requirements and meet user needs. Quality assurance is a staff function.

# Quality Control- the process by which product quality is compared with applicable standards, and the action taken when nonconformance is detected. Quality control is a line function.

Ref : 1.1.1.1 p 1-2 & 1.1.1.2 p 1-3 (CSTE CBOK)

Quality Control (QC)

—QC relates to a specific product or service.

—QC verifies whether specific attributes are in, or are not in, a specific product or service.

—QC identifies defects for the primary purpose of correcting defects.

—QC is the responsibility of the team/worker.

—QC is concerned with a specific product.

Ref : 1.1.1.2 p 1-3 (CSTE CBOK)

Quality Assurance (QA)

—QA helps establish processes.

—QA sets up measurement programs to evaluate processes.

—QA identifies weaknesses in processes and improves them.

—QA is a management responsibility, frequently performed by a staff function.

—QA is concerned with all of the products that will ever be produced by a process.

—QA is sometimes called QC over QC because it evaluates whether QC control is working.

—QA personnel should never perform QC unless it is to validate QC.

Ref : 1.1.1.2 p 1-4 (CSTE CBOK)

CSTE Body of Knowledge – Knowledge Category 9

Testing Software Controls and the adequacy of Security Procedures

The software system of internal control includes the totality of the means developed to ensure the integrity of the software system and the products created by the software. Controls are employed to control the processing components of software, assure that software processing is in accordance with the organization’s policies and procedures, and according to applicable laws and regulations. Software systems are divided into two parts, the part that performs the processing and the part that controls processing. The control part includes a system of controls as well as the means employed to assure processing cannot be penetrated by outside sources. This category addresses all the components of the software system of internal control and security procedures.

Principles and Concepts of a Software System of Internal Control and Security:

1. Vocabulary of Internal Control and Security – the vocabulary of internal control and security which includes terms such as risk, threat, control, exposure, vulnerability and penetration.

2. Internal Control and Security Models – includes internal control and security models. The current model that is most accepted is the COSO model. (Committee of Sponsoring Organizations, COSO, is comprised of five major U.S. accounting associations.)

Testing the System of Internal ControlsThe test process for testing the system of internal controls in software is:

1. Perform risk analysis – determine the risks faced by the transactions/events processed by the software.

2. Determine the controls for each of the processing segments for transactions processing including:

a. transaction origination

b. transaction entry

c. transaction processing

d. data base controle. transaction results

3. Determine whether the identified controls are adequate to reduce the risks to an acceptable level.

4. When all components of the control system are present and functioning effectively, the internal control process can be deemed “effective.”

Testing the Adequacy of Security for a Software System

Testers need to evaluate the security for an individual software system.

The tests should include:

1. Evaluate the adequacy of management’s security environment.

2. Security Risk Assessment – determining the types of risk requiring security controls.

3. Identify the most probable points where the software would be penetrated.

4. Determine the controls at those points of penetration.

5. Test/assess whether those controls are adequate to reduce the security risks to an acceptable level.

These tests should include:

a. Security awareness of the software stakeholders

b. Adequacy of management’s security environment.