CITS5501 lab 5 (week 6) – logic-based testing  – solutions

Solutions

1. \(((f \leqslant g) \wedge (x > 0)) \vee (M \wedge (e < d +c))\)

   There are 4 clauses:

   1. \(f \leqslant g\)
   2. \(x > 0\)
   3. \(M\)
   4. \((e < d +c)\)

2. \(G \vee ((m > a) \vee (s \leqslant o + n)) \wedge U\)

   There are 4 clauses:

   1. \(G\)
   2. \(m > a\)
   3. \(s \leqslant o + n\)
   4. \(U\)

In the solutions, we explain our reasoning, and then give a table with test values. This is a good way to format your answers, if you’re asked to come up with test values.

Almost always, there are multiple possible solutions (because there are many possible choices for the test values), but we show only one.

For (a):

This one should be simple. Our predicate is \(A \vee (B \wedge \neg C)\), and so our clauses here are \(A\), \(B\) and \(C\).

Note that strictly speaking, we don’t need the parentheses in \(A \vee (B \wedge \neg C)\), since “\(\wedge\)” is considered to bind more tightly than “\(\vee\)”, but we add them for clarity. (This is reflected in most programming languages, where “&&” has higher precedence than “||”.)

1. To make \(A\) active: Set \(B \wedge \neg C\) to false.
   So, suitable test values might be: \(A \in \{ \text{true}, \text{false} \}\), \(B\) = false, \(C\) = true.
2. To make \(B\) active: Set \(A\) to false and \(\neg C\) to true.
   So, suitable test values might be: \(B \in \{ \text{true}, \text{false} \}\), \(A\) = false, \(C\) = false.
3. To make \(C\) active: Set \(A\) to false and \(B\) to true.
   So, suitable test values might be: \(C \in \{ \text{true}, \text{false} \}\), \(A\) = false, \(B\) = true.

Our table of test inputs would then look like this:

For (b):

Our predicate here is \(x > 0 \; \vee (M \wedge (e < d +c))\), so our clauses are \(x > 0\), \(M\), and \(e < d +c\).

If you look at the logical structure of this, it actually has a similar logical form as in (a): \(A \vee (B \wedge C)\). But here, some of the variables are of some integral type, so our test values will be any integers we choose that make the relevant clauses true or false.

1. To make \(x > 0\) active: Set \(M \wedge (e < d + c)\) to false.
   So, suitable test values might be: \(x \in \{ 0, 1 \}\), \(M\) = false, \(e = d = c = 0\).
2. To make \(M\) active: Set \(x > 0\) to false and \(e < d + c\) to true.
   So, suitable test values might be: \(M \in \{ \text{true}, \text{false} \}\), \(x = 0\), and \(e = d = c = 1\).
3. To make \(e < d + c\) active: Set \(x > 0\) to false and \(M\) to true.
   So, suitable test values for \(x\) and \(M\) might be: \(x = 0\), \(M\) = true. We then have to make \(e < d + c\) true and false, each in turn. To make it true, we could select \(e = d = c = 1\), and to make it false, we could select \(e = d = c = 0\). (Note that any values which make \(e < d + c\) true then false would be fine; but the values we have chosen happen to be easy to write down.)

Our table of test inputs would then look like this:

For (c):

Our predicate here is \(G \vee (m \geqslant a) \vee H \wedge U\), so our clauses are \(G\), \(m \geqslant a\), \(H\) and \(U\).

Note that because “\(\wedge\)” binds more tightly than “\(\vee\)”, this predicate is equivalent to

\[ G \vee (m \geqslant a) \vee (H \wedge U) \]

Our process for making each clause active is as follows:

1. To make \(G\) active. We need to make \(m \geqslant a\) and \(H \wedge U\) both false.
   So, suitable test values might be: \(G \in \{ \text{true}, \text{false} \}\), \(m = a = 0\), and \(H = U = \text{false}\).
2. To make \(m \geqslant a\) active. We must make \(G\) and \(H \wedge U\) both false.
   So suitable values for them could be \(G = H = U = \text{false}\).
   Then we need to make \(m \geqslant a\) true and false each in turn; to make it true, we could use \(m = a = 0\), and to make it false, \(m = 0\) and \(a = 1\).
3. To make \(H\) active. We need to make \(G\) and \(m \geqslant a\) both false, and \(U\) true.
   So suitable inputs would be: \(H \in \{ \text{true}, \text{false} \}\), \(G = \text{false}, m = 0, a = 1, U = \text{true}\).
4. To make \(U\) active. We need to make \(G\) and \(m \geqslant a\) both false, and \(H\) true.
   So suitable inputs would be: \(U \in \{ \text{true}, \text{false} \}\), \(G = \text{false}, m = 0, a = 1, H = \text{true}\).

Our table of test inputs would then look like this:

Answer:

We have two predicates in our component: the first is “the lever is pulled and the chair is occupied”, and the second is “the button is pressed”.

We will define three variables to represent the clauses in these predicates:

• Let \(L\) represent “the lever is pulled”
• Let \(C\) represent “the chair is occupied”
• Let \(B\) represent “the button is pressed”

Using these definitions, the set of logic expressions (i.e. predicates) in our component is therefore:

• \(L \wedge C\)
• \(B\)

Incorrect answers

Note that it’s incorrect to try and represent “open the trap-door” as a clause.

This is because from the requirement we’re given, “open the trap-door” is clearly not a condition we have to detect, but rather an action our system must take (and something we can hopefully observe as part of seeing whether we get the expected outcome).

To see why this is so, and why we shouldn’t model “open the trap-door” as a clause, imagine writing pseudocode to represent what our component does. It might look something like this:

if lever pulled and chair occupied:
  open trap-door
  return
if button pressed:
  open trap-door
  return

Then recall that doing logic-based testing means to test the logical expressions in our system – the conditions following the “if” statements – and to ensure we’ve thoroughly tested the clauses that make them up.

Now imagine we are writing JUnit style tests to see if our component (let’s call it TrapDoorController) behaves as expected. We’ll assume we have mock objects called lever and chair and button created in a setUp method, and that our controller has a control() method to which we pass the lever and chair and button. For one of our tests, we might have something like:

@Test
/** Test the case when lever pulled, chair not occupied,
  * button not pressed */
public void testA() {
  // prepare the test environment
  lever.setPosition("pulled");
  chair.setOccupancy("occupied");
  button.setPosition("unpressed");
  // invoke the method under test
  TrapDoorController c = new TrapDoorController();
  c.control(lever, chair, button);
  // assert that the behaviour is as we expect
  assertEquals( c.getStatus(), "closed",
                "trapdoor should be closed");
}

And we’d have a bunch of other tests to test other scenarios. So – what if “open the trap-door” were a clause, rather than an action to take? Then it would become part of the test environment. But if that were the case – what would be left to be the component under test? And what could we possibly assert in order to find out if the system behaved as expected or not? There are no sensible answers to these questions; hence it makes no sense to make “open the trap-door” a clause.

Note that for the exam and assignment, student answers that make this sort of mistake will generally be awarded very few (if any) marks – it will be taken to indicate a poor understanding of logic-based testing.

a. Requirement clarity and readability

This requirement would probably be more readable if the various conditions were given as bullet points, rather than a run-on sentence (together with a little re-phrasing):

You may have other suggestions for how the requirement could be made clearer.

b. RACC coverage

There is no one correct answer to this question.

It’s certainly true that we should probably test the login feature thoroughly. However, once we have portions of the system implemented and at least a partial test suite in place, we may discover that we in fact already have achieved an RACC level of coverage.

So our colleague is not incorrect in saying that thorough testing is warranted, and they’re not wrong to suggest that RACC is a good level of coverage to aim for. However, if they are suggesting that the tests need to be designed right now, that doesn’t necessarily follow. The coverage our colleague wants may arise naturally out of applying techniques like Input Space Partitioning and graph-based testing.

For instance, suppose that when we come to implement the login feature, we end up with a method that looks something like this:

public void handleLogin(HttpRequest request) {
  String userID   = request.getParameter("userID");
  String password = request.getParameter("password");

  UserRecord userRecord = userDb.lookup(userID);
  if (userRecord == null)
    throw new NoSuchUserException("no user called " + userID);

  if (! userRecord.getPassword().equals(password) )
    throw new InvalidPassword();

  if (userRecord.getDisabledStatus == DISABLED)
    throw new UnauthorisedAccess("account is disabled");

  // if still here, all is OK
  renderWelcomePage();
}

(Note that this is simplified from how a real login handler method would work – amongst other things, we shouldn’t be storing user passwords in a database in their raw, original format.)

We will probably write unit tests (using the ISP method) to make sure this method does the right thing. We might also check what level of graph coverage we have of the method, and perhaps write more unit tests and/or integration tests in response to that.

By the time we’ve done all that, it’s quite possible we will have achieved an RACC level of coverage anyway, and don’t have to write additional tests.

(If you’re familiar with what HTTP requests look like, you might like to consider how you’d write ISP-based tests for this method. You would probably decided to use partitioning characteristics like “HTTP request contains a valid userID” and “HTTP request contains a password matching the userID”, amongst other things. Once you apply something like Base Choice Coverage as criterion for your ISP tests, it’s highly likely RACC will be satisfied for the requirement.)