Charging around

8km odd at McLean's Island

53minutes hard out! Brilliant run and I still had enough in the tank for a kick at the end!

In it to win it!

Good times. I was hungover on Saturday so I was a blob of laze.

Yesterday I managed a decent 2 mile walk at a good clip, then a 14mile cycle (59minutes!)

Felt pretty average during the bike ride but once I hit the second wind I was flying.

$km run at lunchtime today...will be aiming to make it in sub 25 minutes

Woohoo

Well, there we go. Second run completed, managed to increase the running time on the same lap as day 1.

Finished it in a shade over 26mins. Target time for that is approximately 19minutes

So I can't back out!

Well, here we go!

I've committed to doing a cross country 25km race in 14 weeks time. I reckon I'm keen to do it and that I will be able to manage the distance.

So here's the line in the sand. I'm going for it. This is also going to be my training diary.

So, Day 1 was yesterday. 4km jog and walk (95%:5%)

Today, 3 km walk and hopefully a cycle tonight.


CheersD

Systems engineers...argh

What a day. There's been a battle royale at a training course and I've been party to the fall out. Dan, one of our test leads is learning about system design and he's been sent on a course with a load of systems engineers, requirements authors and principal software designers.

It sounds like he was ambushed. The lecturer proposed that customer requirements be refined into functional goals and that the test cases should test against these. As Dan rightly pointed out, this is little more than a MacJob. There's no skill required, just a bit of patience. It turned into a holy war and I feel that a lot of ground gained by the introduction of RT has been lost.

My opinions are fairly well known but I want to write them here so that I can distill my thoughts and be ready to defend my testing approach.

I believe that there is NEVER a case where customer requirements and engineering specifications belong in the same place. A customer requirement, to me, is the road map. It's a description of the customers' desires and needs, and the next big thing. It gives people an idea of how the product will be used, what is defined as quality and points out risks and opportunities. It should be written by the people that meet the customers day-in, day-out. It should be as useful to a marketeer or a salesman as a designer.

A functional spec should never take these things into account. It should be a description of how, as an engineering entity, we are going to fulfill these customer requirements. It is a storage space for the decisions we make in meeting these goals and the reasons behind those decisions.

If a customer is asking for too much or the impossible, there must be a decision made by the 2 concerned parties (the representatives of the customer and the system engineer). With agreement, both documents must change. Without this synergy (apologies for the buzz word) neither document is remotely useful.

The approach to testing the both these "documents" and their outcomes are completely different. Functional requirements must be tested using either unit or function testing techniques. Customer requirements should be tested using more innovative techniques. Exploratory, claims, use-case, context or risk based techniques must all be applied.

At the meeting, Dan faced a body of people that believed that customer requirements could be tested using functional, tick-the-boxes techniques. He was faced with a body of people with a deep and complete misunderstanding of the outputs we aim to provide. They failed to respect that within the department we have some of the most intelligent members of the company. One of them went so far as to rib him about "reaching the dizzying heights of test team lead."

As is so common, faced with this ignorance, the old motivation takes a dent. It's incredibly hard to bust hump testing a product for people that display that level of respect. I'm hoping that we can help each other over the slump and keep raising our game and pursuing the goals we've set ourselves.

In the meantime, the first person to give me grief tomorrow is likely to receive the "look", patented by my mother. God help them.

Blink and you WON'T miss it.

Rapid cognition. I knew I was going to say that ;-).

Well, I've finished the book Blink by Malcolm Gladwell. It's an interesting read, fairly light and easily digested. It was recommended as a helper to understanding some aspects people's ability t know instinctively that something is wrong. Some key lessons I learned.

Rapid cognition is powerful: Already at work I've managed to use blink tests on 2 occasions and the hunches I gained allowed me to raise 2 brilliant issues. The first: I had a massive logfile (~20k packets of data) but I knew that the order was out in some of them. With a couple of well thought out examples I was able to hold down scroll and the logs and find 2 instances where the packets were out of order.

Another was a case where I was trying to get to the bottom of a log file to see the last event that occurred. As I "PgDn"ed , my attention was grabbed. I had already pressed the key a few more times before I managed to consciously decide to investigate. Sure enough, when I traced back through the preceding 100 or so lines I found the entry that pointed to a bug.

The key to the use of the blink is not to know every possible line or packet but to know vaguely what the log should look like. Examining too closely will lead to too much conscious examination and become labourious. Having a fair idea of what to expect and scrolling through the log once in a while will take a minute and may show up an error. It's a fast, cheap test that is pretty fallible but the possible value can be astronomical.

Blink testing is not an easily defended concept.
It's a hell of a lot easier to say "I just happened upon this in the logs, is it a bug?" that "Well, we blink tested the logs and we didn't see any errors so we thought it was ok". It's a shallow test method, that becomes more successful with experience and domain knowledge, but it's not the pass/fail criterion by which products should be released!

All in all, I think the main thing I've gained from the book is a new approach. I've started to look at logs in a different light. Instead of reading lines, I'm looking for patterns and refining my understanding of what each entry means. In time, I reckon that this will enhance my testing and, hopefully open up a few more of those interesting little bugs!

Later!

Test-ify!

Well, a few weeks have passed since the rapid testing course and things have started to settle down. Some things have stuck and others appear to have been lost in the maelstrom of new ideas that accompanied the learning.

Personally, some really important developments have occurred. Firstly, in the field of communications I have managed to generate a rapport with some of our developers that has made my life so much easier. I think the reasons are two-fold.

Firstly, they both attended the course and, while disagreeing with some things James said, have come to understand our position. They are no longer aloof and all suspicion of our motives and agenda has dissipated.

Secondly, I have taken a leaf out of James' book and become a facilitator. I ensure my questions are clearly thought out and well presented. Unfortunately, my turn of phrase and my accent hinder things somewhat but I'm Irish first and a tester second.

In the field of test execution we have a large amount of ground to cover. Due to the fact we were entering the final fortnight of a release cycle it was difficult to implement as many new ideas as possible. However, things are already becoming clear as options.

Charters and Sessions: I think we've been floundering with definitions somewhat. The things I've seen called sessions are closer to charters and this is where we're floundering. The one-many notion of their relationship was somewhat missed. As a result, test leads were assigning sessions to people instead of charters and they were acting as lazy scripted tests.

From my point of view, I have implemented the following concept. When I am given a charter I try to analyse it using some of the tools that JB suggested. I generate a list of sessions and propose them to my test lead. In the space of 5minutes, he can act as oracle, sanity test and director to refine the initial trust of the testing.

While this is at odds with the dynamic nature of RT, it allows Matt a clearer picture of where I intend to go with my testing and helps me utilise the heuristics that I have yet to gain.

Notes Review: To my mind, this is still an area where we fall down a lot. I'm no saint and my note taking is abysmal compared to the ideal. I'm still plodding along, it'll come to me slowly I hope.

My main bugbear, however, is that we've fallen into the trap of reporting our charter headings rather than submitting our sessions for review.

A typical review will go as follows: "Today, I did some guerilla testing of voting and closed some issues on channel groups. "

I would envisage a more productive statement being something like: "Today I was testing the voter, to do this I did a load test on a channel group, I tried a couple of unfriendly configurations, I examined some trace logs for instances of incorrect packets. "

The latter opens the testing to much closer scrutiny and allows a peer to say "hey, did you see what happens when x or y?"

We'll get there in time.

Cute Heuristics

When I was told about exploratory testing by Matt and Dan, they spoke a fair bit about heuristics. Somewhere along the line I got the impression that these were ephemeral things one had to learn to become a good tester. Without them we were lost.

Listening to James Bach speaking about them it became clear that the term meant something slightly different.

I'm going to try and distill my idea of what they mean to a tester at the present time and hope that any errors I make will be highlighted by someone or other!

So, looking at the info I've garnered thusfar, a few common themes are obvious.

Heuristic as a problem solving tool:
To my understanding, a heuristic is a replicable method (?) or approach that is used to direct one's attention when in learning or discovering. (From wikipedia) Another term you hear a fair bit is the "rule of thumb".

The concept is that you use a heuristic to give yourself an idea of where your testing is going to take you. This could be something like "Software has failed before when I pressed the enter key repeatedly" so I try that in this case. It's neither hard nor fast but it's useful in the absence of any other directive.

Heuristics as fallible:
A point that was emphasised by James was that these tools were loose and suggestive rather than directive. They suggest a plausible path to take while testing and give some indication of a possible outcome (I'll revisit this later and work through my reasoning and see if this statement is correct).

The brilliant thing is that the heuristic is a dynamic thing so using one that ultimately fails to give you an expected result is not a disaster. While it hasn't given you any information about the product under test, it can give you some useful feedback into your preconceptions and expectations.

One point worth noting: I asked a question about why I used 32, 63,64,65 and 65535 as inputs when testing number input boxes. James made some good points about it and also pointed out that many higher level languages wouldn't fall into these traps because the compilers and interpreters would handle them. It was only when I spoke to Josh after the lesson that I realised that my bias towards these numbers was based on the numbers of ECs on an SNT and the numbers of SNTs to RPs in Ericsson AXE10s. The thought that I was using a 7 year old heuristic that had never been subject to any questioning or modification gave me pause for though.

Heuristics as a library of experiences:
It strikes me that something that I've been bad at in the past is consciously analysing my observations to see if I can take something more from it and apply it in a future situation. I may have been makins some inferences based on half formed heuristics but I never thought "Hey, I should think about this the next time I start testing"

That's me for the moment, I'll revisit these blogs frequently to see if I still believe the stuff I've typed. Feel free to disagree with my thoughts and gimme some feedback. Later

Heuristics, models and heaven knows what else.

Well, I created the blog to cover the rugby and maybe throw in a few bits and pieces about other stuff and a chance has just presented itself.

I attended a course at work this week in "Rapid Software Testing" delivered by James Bach (his company). I went into the course expecting to be guided by a tester in the arcane movements required to deflect hostile designers and managers and I ended up having my thoughts on my own ability to learn and my present career fundamentally challenged.

Since I left college I have often claimed to be finished with formal learning and I've tended to restrict my studies to the odd article in Better Software and the odd stickyminds post. A nice comfy rut for a tester to get into, I'm ashamed to say.

I've got a lot of interesting thoughts running around my head these days, opinions and instances where I think ET and RT will work but it's going to take months to distill them into decent blogs. I'm afraid that I'll lose them before I get them committed to...urm...paper (?) so I'm going to add a couple right off the bat in freeform and think about them later.

So brace yourselves, this could get interesting