CITS5501 lab 1 – Testing introduction

We will be using the Java language for the bulk of the labs, so you should make sure you have access to a platform where the Java Development Kit (JDK) is installed, and you can make use of a good Java IDE (Integrated Development Environment) or editor.

Using either VS Code on the desktop, or the VS Code–like environment, you should see a message about “Java projects being opened”. Wait for that to pass, and there’ll now be a “Testing” icon on the left sidebar, in the shape of a conical lab flask (like this:

Click on the testing icon, and then on the button “Enable Java Tests”. Select “JUnit Jupyter” tests from the options that pop up, and you should see a message saying that “Test libraries have been downloaded into ‘lib/’”.

If you open the CalculatorSimpleTest.java file, then after a short while, a green triangle icon should appear to the left of the line “public class CalculatorSimpleTest”. Clicking it will run all @Test methods in the class.

(If the icon doesn’t appear: select “View” / “Problems” from the menu, and see if there have been any compilation problems. If there have: select “Java Projects” from the left sidebar, “Reference Libraries”, and then the “+” (add) button. Add the lib/junit-platform-console-standalone-1.9.2.jar file that should be in the lib directory, then hit the “refresh” icon under “Referenced Libraries”. Ideally, the “Problems” should now disappear and the green triangle icon appear. If it doesn’t: click somewhere in the CalculatorSimpleTest.java source code, hit the ctrl-shift-P key combination, and type “java: run tests”.)

If you are able to run the tests, you should see some tests pass (for instance, testAdd), but most tests fail. This is expected!

As a first step for today’s lab, ensure you can install and/or access at least one of these.

After downloading and opening the lab code from https://cits5501.github.io/resources/#lab-labs, you may need to instruct your IDE to add the “JUnit 5” libraries to the project; typically, viewing the project properties in your IDE will reveal some way of doing this.

2. JUnit tests

Download and compile lab code

Take a look at the Calculator class, in Calculator.java – this class has trivial functionality, but is useful as an example of a class under test.

Take a look at the CalculatorSimpleTest class, in CalculatorSimpleTest.java. This class defines a number of JUnit tests for our Calculator class.

Test classes can be called anything, but by convention, unit tests (which are written to test a single class) usually start with the same name as the class under test, followed by a description of the test (or just the word “Test”).

Run the JUnit tests

In many IDEs, this can be done by right-clicking on the class (after compiling) and selecting “Test All”. In VS Code, it can be done by clicking the green triangle icon in the left margin of the code (see previous section).

You should see that some tests “pass” (with green ticks) and some “fail” (with red crosses) – see if you can work out what the failing testSubstract test is telling you about what the problem is.

Inspect the JUnit tests

Consider the following question: if you want to get all the tests passing, how do you determine what each method is supposed to do, and when it is correct? (After all, someone writing the test could have made a mistake in the test code.)

Sample solutions:

To find out what each method is supposed to do, the only reliable option is to read its documentation. If there is no documentation for the method, then you can never really know what the developer’s intention was.

3. API documentation

In most IDEs and editors, there is no need to explicitly run the javadoc tool in order to view API documentation for a method, class or annotation.

Instead, you simply need to “hover” your mouse icon over some spot in the code where the item you’re interested in is defined or used, and a “pop-up” message should appear showing the API documentation.

If you edit the Javadoc comments, the IDE or editor will normally use the edited, most up-to-date version when showing the documenttion.

If you wish, it’s also possible to run the javadoc tool from a terminal window, and the tool will generate HTML files from the source code.

In VS Code, you can open a terminal window by selecting “Terminal” / “New Terminal” from the menu.

Type javadoc -help to ensure you can invoke the command, and to see what the options for it are.

To generate documentation for the .java files in the src directory, and put that documentation in a directory doc, run:

4. Fix the code

See if you can fix the code in the Calculator class so that all the tests pass.

For the subtract method – aside from other changes you might need to make, you might want code something like the following:

Try creating your own new tests. In BlueJ, if you right click on a class, there should be an option to create a test class. Use the existing tests as an example – can you think of other tests we might add?

Sample solutions:

Fixing errors:

The code for subtract has the numbers in the wrong order.

The code for subtract also needs to throw an exception when the result would be negative.

e.g.

  if (num2 > num1) {
    throw new ArithmeticException("can't return a negative result");
  }

5. Concepts review questions

Answer the following questions to test your understanding of concepts introduced in the lectures and prescribed reading.

For each of the following scenarios, explain whether you think a failure, a fault or an erroneous state (or none of these, or more than one) has occurred, and explain why. If it is a failure – is it non-conformance with a functional or a non-functional requirement?

Possible solutions and discussion:

To come to a conclusion for these questions you will need to make reasonable assumptions and state what they are. Your answer can vary, depending on what assumptions you make.

The answers below are one possibility; but yours may be different.

We are told that users can specify that their email and date of birth should not be displayed publicly. So if those are displayed publicly, then that’s a failure: the system is not behaving in accordance with requirements.

We don’t know in this case what the exact requirement is, but most likely, it is something like:

“If the user specifies that their email and address and date of birth should not be displayed publicly on their profile, then those details should not appear on the user’s public profile.”

So the failure is non-conformance with a functional requirement.

Presumably, there’s also a problem somewhere in the code for the Witter site that causes this failure, but we aren’t given details: this problem constitutes a defect. And once execution hits that point in the code, the system will go into an erroneous state.

[This scenario is loosely based on a security failure that occurred to Google’s social networking site, “Google+”, in 2018. See the article by Newman (2018) reporting on this here; there is also a Wikipedia page on the incident.]

In this scenario, we aren’t told exactly what the requirements for the Habari system are; but presumably one of them is that communications between the mobile app and the Habari company servers should be secure. (Even if this is not an explicit requirement, it’s a reasonable expectation of drivers, users, and Habari management. Drivers and users presumably don’t want details of transactions to be visible to third parties, and management presumably would like people to pay for rides rather than getting them for free – else the company will go out of business.)

If the requirement is something like “All communications between the mobile app and the Habari servers should be made securely” then this would be a non-functional requirement. (We’ll discuss later in the unit whether that requirement is precise enough.) In this context, “functional” means “treating the system as if it were a function, a black box that takes in inputs and spits out outputs”; and “non-functional” means “the manner in which some system function executes”, rather “not functioning”.

For this unit, you aren’t expected to know the details of how secure computer communications work, but if you do, you are welcome to discuss what you can infer about the defect in the source code which causes the above failure. (In the scenario given, we probably could point to specific spots in the code where things should have been done differently – spots in the code where the programmers developing the Android app and the Habari servers should have used appropriate techniques to ensure that communications were secure. These would be defects.)

[This scenario is loosely based on one described by Shubham Paramhans (2015), who identified flaws in the security for the “Ola” ride-sharing app, as described in a Medium.com article, “Fooling the Startup of the Year”.]

When Mila’s method returns the result “INFINITY”, we’d need to know what the method’s specifications were in order to know if this was a failure or not (i.e., we’d need to look at the method’s documentation).

As a matter of mathematics, the arithmetic mean for \(n\) numbers \(a_0\) through \(a_n\) is defined as \(\displaystyle \frac{1}{n}\sum_{i=1}^n a_i\), or equivalently \(\displaystyle \frac{a_1+a_2+\cdots+a_n}{n}\) (Weisstein 2005).

So when n = 0, the result is undefined, because dividing by zero is not defined. If we were writing a specification for this method, what are some reasonable things we could require the method to do in this case?

Here are a few reasonable options:
1. The method should return the special floating-point value “INFINITY”. This is Java’s default behaviour when floating-point numbers are divided by zero. The advantage of this approach is it requires no extra work; but “INFINITY” perhap isn’t the best choice, since it doesn’t mean exactly the same thing as “undefined”.
2. The method should return the special floating-point value “Nan” (“Not A Number”). This will require a little extra code, but reflects the correct mathematical answer slightly better than option (i) does.
3. The method should throw an exception (e.g. something like Java’s ArithmeticException class). Throwing an exception is usually what maps best to a mathematically undefined result; but the caller having to handle the exception may not be very convenient.
We’d need to know more about the business requirements here to know which is the best option.

In any case: if the behaviour of Mila’s method does match its requirements, then there hasn’t been a failure; but if it doesn’t match them, then there has.

6. Moodle sign up

There are two Moodle quizzes (not assessed, but it’s recommended you attempt them) available for you to test your understanding of prerequisite knowledge and week 1 topics in CITS5501:

Visit https://quiz.jinhong.org and sign up with your UWA email address, then attempt the quizzes (in your own time, if you don’t finish them in the lab).

CITS5501 lab 1 – Testing introduction – solutions

1. Accessing required software