Take-home lesson #1 : Testing gives you confidence in your code and application.
Basic of Testing
Hierarchy of testing:
1. None.
2. Ad hoc testing - depends on people, not reproducible.
3. Unit testing
4. Bodies - help desk, users, managers.. have people beat on it.
5. Bodies + test plan - directs people on what to test.
6. Automated test plans.- gives the computer the boring, tedious parts.
Types of Testing: Low Level Code
Low-level testing done by developers to make sure an object behaves the way the spec says it should.
- Done by programmers
- Unit test are best examples
- Specific functionality testing in isolation.
Types of Testing: Application Level Testing (also Functional or Integrated Testing)
Done by non-dev people... QA and UA types.
Also may involve automated testing.
Examples are browser interactions.
Types of Testing: System Level
Includes Load, Performance and Stress tests.
Types of Testing: User Level
Testing the stories from the cards - usability and acceptance testing.
See also: conformance, security, and failover testing.
Who does testing: You. Dev team, QA team, Help Desk. Not users / customers.
Take-home lesson #2: Do not let your users do your testing.
Getting started? If you're starting from ground zero (lotsa code, no tests) you can add tests as you write new code, add tests that demonstrate reported bugs, and add test instead of clicking through the app yet again.
There's also (from http://use.perl.org/~amoore/journal/30215)
1) boiling the frog - start slowly
2) play ping-pong - write a test for some code that someone's else wrote
3) maybe a rachet? - keep improving, and make sure you keep bettering your test standards.
Take-home lesson #3: Good programmer write tests.
Functional Testing with Selenium
Easy to use, runs in many browsers, exposes browser specific issues, straightforward to automate.
Speaking seleneese: you build tests in html as 3-column tables. The command language has Actions, Accessors, and Assertions. And there are locators and patterns.
Actions: anything the user can do, there's a selenium action for.
Accessors: examine the state of browser/application, usu storeSomething each has several (often 6) related assertions.
Assertions:
assertSomething - aborts on a failure
verifySomething logs failures and continues
waitForSomething waits until a timeout or condition (ajax)
There are also the inverse of all these.
Seleneese locations: it can find things by id, name, identifier, link, dom, and xpath.
Basic test structure is a html 3-column table: command | argument | argument, where the second argument is often blank. The first line is often a comment.
There's also an IDE wich can record sessions show test, and allow editing of existing / recorded tests.
Where this gets cool is that you can have something (Java, Rails, perl, etc.) generate the 3-column HTML that is your test. Doing this, you can build test that contain decisions, loops, and database referencing.
Take-home lesson #4: Selenium will save you time.
Continuous Integration and Automation
Integration with Cruise Control and your testing is a good and worthwhile thing
to do.
Use dbUnit to set up your database before tests and reset it when you're
finished. It's also useful for testing stored procedures and the like. You
know, the smarts you put into db code.
Take-home lesson #5: If you repeat it, automate it.
Load testing with Grinder
Load tests - how does the app behave with more than one user using it.
Performance tests - load it and measure performance metrics at typical usage
levels.
Stress tests - crank up the load until it breaks.
The Grinder (Use V3 and not V2. The Beta label is bogus and is really only
related to documentation):
Agent: that run the tests
Console: coordinate the tests in a central location
TCPProxy: records the test using a browser.
Personal note: Grinder does indeed rock - we've used it for some load-related
troubleshooting. But I realize we could be using it for lots more.
Talk slides are at http://www.ashenfelter.com/ or http://transitionpoint.com/
Apologies for this one being so erratic and unedited. This is pretty raw dump of my notes from the session.
-Bill
Tags: OSCON06, testing
No comments:
Post a Comment