Quality Frog Details

Bad Messages
2007-06-09 02:30:00
I regularly spend a great deal of time tracking down the root cause of software errors -- both on and off the job. Much of the investigation effort could have been avoided if I were not presented with incomplete or incorrect error messages. The text of error messages appears to be commonly overlooked by software developers and testers.One of my early test automation development tasks was to fix the error messages in a test tool. The tool was used to validate the structure of data exchanged between computer systems. In nearly all cases, this tool displayed an error dialog window stating that the data was not valid. This tool gave the user (a tester) no information about what was wrong with the data. The lack of detail in these messages required that the user manually examine the data -- bit by bit -- with a protocol analyzer. This turned investigation of errors reported by an automation tool into a tedious manual task. The tool encountered a problem but did not report the source of the ...

Model-Based Test Engine Benefit #4: Generate and execute new tests – and find new bugs
2007-06-08 07:49:00
The last -- and perhaps the best -- major benefit of implementing a Model-Based Test Engine (MBTE) is automation that is capable of generating and executing tests that have not previously been executed manually. Traditional regression test automation simply retraces test steps that have already been performed manually. This may find new bugs that show up in a previously tested path though an application but will not find bugs off the beaten path. In his book "Software Testing Techniques", Boris Beizer compares the eradication of software bugs to the extermination of insects. Every method you use to prevent or find bugs leaves a residue of subtler bugs against which those methods are ineffectual.- Boris BeizerSoftware Testing TechniquesWhen we apply any method to finding bugs, we will find bugs that are found by that method. However, other bugs will remain. Finding new bugs requires variation in testing, not repeating the same thing over and over. Repeatability is often advertised ...
Engine Generate Model

ERROR!
2007-06-08 02:56:00
Sometimes you just gotta read the instructions.Here's a short student film directed by a friend.Error!What's the message here? Which character do you relate to?I think I relate to the person that broke the computer in the first place. I could'a done that. Ben Simo QualityFrog.com Questioning Software℠ ...

Stupid Questions
2007-06-06 15:23:00
Why do we drive on parkways and park in driveways?Why do noses run and feet smell?If Jimmy cracks corn and no one cares, why is there a song about him?Why do we call them restrooms when no one goes there to rest?Why do you have to click the "Start" button to stop Windows?Most of my life, I have been told that there are no such things as stupid questions. This was usually said to encourage me, and others, to not be afraid to learn. However, I am beginning to think that there is such a thing as a stupid question. I don't mean questions like the above. Coming up with the questions above requires some thought and I suspect they all have reasonable answers. The above questions are more silly than stupid.So what do I consider to be a stupid question? A stupid question is a question that has little basis in intelligent thought. A stupid question is a question without the context required to provide an answer. A stupid question is one that the questioner would have realized has no ans ...
Questions Stupid

Poka-Yoke
2007-06-02 18:30:00
Poka-Yoke is not a dance. Its not an event at a rodeo. Its not what my kids do to each other in the back seat of the car. Poka-Yoke is Japanese for "mistake-proofing". Poka-Yoke was developed by Japanese industrial engineer Shigeo Shingo. He realized that people cannot be expected to work like machines and consistently do everything the same way every time they do it. People make mistakes and poorly designed processes can make it easier for people to err. Poka-Yoke's goal is to make it difficult for people to make mistakes through mistake prevention and detection.PreventionApplied poka-yoke gives users warnings about incorrect behavior and directs users towards the correct behavior. Computer PS/2 keyboards and mice share the same physical connector design but the connectors are usually color-coded to indicate which device goes into which port on a computer. Some computing hardware is shipped with warning stickers on top of connectors telling users to read a manual or install ...

Model-Based Test Engine Benefit #3: Automatic handling of application changes and bugs
2007-06-02 03:05:00
Automated tests based on models have one important feature that scripted testing cannot: automated handling of application changes and bugs. I do not mean that model-based automation can think and make decisions like a human tester does when they discover something unexpected. Instead, the automated selection of test steps supports working around the unexpected without special exception handling code for each situation.For example: If there are two methods for logging into an application and one breaks the test engine can try the alternate option to get to the rest of the application. If a traditional scripted automated test encounters an unexpected problem it will not be able to complete.The model-based test engine (MBTE) can be coded to not try an action after a pre-defined number of failures. The MBTE's selection algorithm can then seek out other options that have not yet been found to fail. This also results in the MBTE reattempting failed actions and exposing failures that only o ...
Application Engine Model

When testers create bugs
2007-05-31 07:38:00
How come dumb stuff seems so smart while you're doing it?- Dennis the MenaceDebasis Pradhan's blog entry Testers don't make Bugs. Oh Really? got me thinking about a time that I as a tester actually introduced a bug into a system. Debasis' post is about bugs that slip by testers and escape into the wild. This is not the case in my story. I asked developers to put a bug in the software and they followed my instructions.I was testing a data mastering system that assembled and converted data from a data repository's format to a variety of other formats for distribution to customers and inclusion in a variety of software products. I created a data validation tool that was used to inspect the huge volume of transformed data: comparing the actual output of the mastering system to the expected format and presentation. The validation tool also performed some heuristic-based tests that alerted testers and developers to data that may require manual inspection.Over the course of many months, ...

Where No Confabulation Goes Untested
2007-05-30 00:56:00
conferverbhave a conference in order to talk something overThe Conference of the Association for Software Testing (CAST) is coming this July.I missed last year's conference but have heard great things about it. Based on all the wonderful things I've heard from those that were there, I am looking forward to this year's conference.The CAST is different than most conferences where people sit and listen to someone present to an audience without public questioning of what is presented. AST encourages testers to test the presentations. Time is allowed for discussion at every presentation. Challenging ideas is encouraged. I could go on. However, I don't think I can push this conference any better than David Gilbert. Therefore, please take a look at David's blog post: CAST in stone.CAST early bird registration ends this week. Register athttp://associationforsoftwaretesting.org/conference/registration.htmlI hope to see you there.Ben Ben Simo QualityFrog.com ...

Don't Ignore The Little Bugs
2007-05-25 02:03:00
And that's how it happened.Believe me. It's true.Because . . .just because . . .a small bugwent KA-CHOO!One of my favorite children's books is also one of my favorite testing books. (Thanks go to Rob Sabourin* for alerting me to the testing connection.) Because a Little Bug went Ka-CHOO! tells the story of a multitude of problems that cascade from a little bug's sneeze. A worm gets mad. A turtle gets bopped. A bucket gets stuck. A policeman takes flight -- in a motorcycle sidecar. A boat nearly sinks. Pandemonium ensues.This book illustrates how things that seem to be insignificant can have substantial lasting impact on a larger system.Software bugs that appear to be trivial can be a sign of a larger problem. When we testers encounter bugs, we are usually looking at a symptom of a problem and not the underlying error that produced the bug. This requires that we do some investigation to determine if a bug is more serious than it first appears and if it is more wide-spread t ...

Driving for quality
2007-05-23 07:30:00
Years ago, I taught defensive driving classes to people cited for violating traffic law in Arizona. The central theme of my classes was that our attitudes behind the wheel often have a bigger impact on safe driving than our skill as drivers. I would start each class by having each student describe what they did to get in my class and what other drivers do that annoy them. We then reviewed the two lists (which usually ended up being identical) and the class discussed whether each item was mostly due to driver skill or attitude. Nearly everything on the lists could be traced to an attitude problem.We are likely guilty of the same faults that we find in others. I believe the attitudes of those involved in software projects can impact the quality of software more than the skills of the team. And when skill is lacking, the right attitude fosters learning. A little patience and a good attitude can go a long way.I used to teach the SIPDE mnemonic to my driving students to help improve safety ...
Driving

Don't forget to think
2007-05-22 04:30:00
This past week at STAR East, James Bach presented a number of questions, magic tricks, games, and riddles to testers that volunteered to be tested. I feel like I did fairly well on some of them and failed miserably on others. James uses these tests to teach testers to think. I thought I had learned some valuable lessons until I was presented with two riddles from children today. If only I could learn to think more like a child. :) I think that I sometimes let my search for hidden meanings keep me from seeing the obvious.A riddle from my children:You are blindfolded, placed at the start of a maze, and told to get to the other end. How do you navigate the maze?You can feel your way through.or a better answer isTake off the blindfold.It can be easy to blindly feel our way through though a challenge -- one obstacle at a time. At times it can be good to isolate problems but problems taken out of their context can be misleading. We need to look at problems in the context of their en ...

Model-Based Test Engine Benefit #2: Simplified test result analysis
2007-05-21 15:00:00
Automation is of little value if it does not report useful information that can be quickly reviewed by testers.Reported results should contain enough information to answer the following questions:What happened?What is the state of the application?How did the application get in that state?What automation code was executed?What automation data/parameters were used?Some failures reported by automated tests will be errors in the system under test and others will be errors in the automation model or code. It is important that results point the reader to both.I have found logging of the following information to be useful:Test (test configuration information)TitleStart TimeScript File(s)Model FilesTest SetSeverityEnvironmentObject MapAction Table(s)Oracle Table(s)Computer NameOperating SystemTesterActions (controlling the application)Source (where is the action defined?)TitleStart TimeAction DetailsDurationState TransitionAutomation CodeResult DetailsSnapshot (screen capture, saved files, etc ...
Analysis Engine Model

STAR East 2007 Conferred
2007-05-19 22:01:00
I am sitting in the airport waiting to fly home from STAR-East. The conference was great. It was not great due to the many wonderful presentations. It was great because of what happened outside the scheduled activities. I got to confer with colleagues from around the world.The best part of conferences such as STAR-East is the opportunity to confer with peers and thought leaders in our industry. It is an opportunity to discover that we are experiencing common problems and share possible solutions. It is an opportunity to learn from the best. I often learn more over dinner and in the hallways than I learn in the presentations.I was amazed at how quickly the conference attendees disappeared once the scheduled activities were completed. I know that many of us computer geeks are introverts. We may not be the most social bunch of people, but I believe a conference without conferring is a wasted opportunity. See y'all at CAST. Ben Simo QualityFrog.com ...
Star

Model-Based Test Engine Benefit #1: Simplified automation creation and maintenance
2007-05-19 17:17:00
Model-Oriented DesignProcedural automated test scripts may be easy to record or script. However, they are difficult to maintain when applications change. They are also difficult to adapt to new test ideas. Maintenance is simplified by automating the procedure generation in addition to the execution. New actions, validations, and data can be added to existing tests. This allows testers to spend more time thinking up new test ideas instead of maintaining procedural scripts.Simplified GUI Interaction CodingMost GUI automation tools contain complex vocabularies for controlling objects and retrieving information from those objects. There are usually different methods for interacting with different classes of objects. This requires that toolsmiths learn a class-sensitive vocabulary and be aware of the class as they code tests. There is an easier way: create functions that automatically detect an object's class and apply the appropriate method. This allows for the same command to be ...
Automation Engine Model

How Doctors Think
2007-04-27 07:09:00
Michael Bolton recently brought Dr. Jerome Groopman’s newest book, “How Doctors Think”, to my attention. Michael suggested to the software-testing list that this book contains information that is relevant to software testing. In this book, Dr Groopman explores how doctors diagnose medical problems and what can be done to reduce bad judgment.Michael Krasny recently interviewed Dr. Groopman on public radio about this book. You can listen to the interview here. Dr Groopman made a number of statements during this interview that I believe apply to us software testers’ search for and diagnosis of software bugs. I expect to find more in the book.Here are some gems from the radio interview:About computer-assisted diagnosis software.It’s a two-edged sword. On one hand, it can be useful if you feed in the most important and relevant symptoms, but the problem is that you need to be able to illicit them effectively from the patient and so you return to language and you return to some of ...
Doctors

Finite State Machines
2007-03-24 23:54:00
Software behavior can be modeled using Finite State Machines (FSMs). FSMs are composed of states, transitions, and actions. Each state is a possible condition of the modeled system. Transitions are the possible changes in states. Actions are the events that cause state transitions. For example, the following FSM shows the expected behavior of a car keyless entry system.Images like the above are great for human use, but not machines. State transitions and the actions that trigger them can also be defined in a table format that can be processed by a computer. The above FSM can be represented using the table below.The requirements for each state can also be defined using tables. The table below contains sample requirements for example keyless entry system. Ben Simo QualityFrog.com ...
Machines State

Best Practices Aren’t
2007-02-13 05:03:00
The first two of the Seven Basic Principles of the Context-Driven School of software testing are:1. The value of any practice depends on its context.2. There are good practices in context, but there are no best practices.As a former quality school gatekeeper, I understand the value of standards – in both products and processes. However, I am concerned by the current “best practices” trends in software development and testing. The rigidity that we demand in product standards can hurt in process standards. Even the CMM (which is often viewed as a rigid process) has “Optimizing” as the highest level of maturity. A mature process includes continuous learning and adjustment of the process. No process should lock us into one way of doing anything.Nearly 100 years ago, the industrial efficiency pioneer Frederick Taylor wrote “among the various methods and implements used in each element of each trade there is always one method and one implement which is quicker and better than any ...
Practices

[First] � Prev 1 2 3 4 Next � [Last]

3806 blogs in our database.
Statistics resets every week.