|
From Testing JavaScript Applications by Lucas da Costa In this article, I will teach you techniques to help you write better assertions. You will learn how to make them catch as many bugs as possible, without having to update tests too often, lessening your maintenance burden. |
Take 40% off Testing JavaScript Applications by entering fccdacosta into the discount code box at checkout at manning.com.
It takes a unique baker to recognize a unique cake. When examining batter’s consistency or a cake’s texture, an excellent pastry chef knows what to look for. Without rigorous quality control, you can’t bake tasty desserts.
In the same way, excellent engineers know what to look for in the software they write. They write robust and precise assertions, catching as many bugs as possible without significantly increasing maintenance costs.
Assertion counting
A test without assertions only fails if the application code can’t run. If you have a sum
function, for example, you must add assertions to ensure it does what it must do. Otherwise, it might as well be doing anything else. Without assertions, you simply ensure that the sum
function runs to completion.
To ensure that your tests contain assertions, Jest provides you with utilities which make your tests fail in case they don’t run the number of assertions you expect.
Consider, for example, an addToInventory
function which adds items to the store’s inventory and returns the new quantity available. If the amount specified is not a number, it should fail and should not add any items to the inventory.
Listing 1. inventoryController.js
const inventory = new Map(); const addToInventory = (item, n) => { if (typeof n !== "number") throw new Error("quantity must be a number"); const currentQuantity = inventory.get(item) || 0; const newQuantity = currentQuantity + n; inventory.set(item, newQuantity); return newQuantity; }; module.exports = { inventory, addToInventory };
When testing this function, you must be careful not to create an execution path which could lead to no assertions ever running. Let’s use as an example the following test:
Listing 2. inventoryController.test.js
const { inventory, addToInventory } = require("./inventoryController"); beforeEach(() => inventory.set("cheesecake", 0)); test("cancels operation for invalid quantities", () => { try { addToInventory("cheesecake", "not a number"); } catch (e) { expect(inventory.get("cheesecake")).toBe(0); #1 } });
#1 An assertion which only runs when the addToInventory
call throws an error.
The test above will pass, but you won’t know whether it passed because the addToInventory
function didn’t add an item to the inventory or because it didn’t throw any errors.
If you comment the line which throws an error and re-run the test, you will see that, despite the function being incorrect, the test still passes.
Listing 3. inventoryController.js
const inventory = new Map(); const addToInventory = (item, n) => { // Commenting this line still makes tests pass // if (typeof n !== "number") throw new Error("quantity must be a number"); const currentQuantity = inventory.get(item) || 0; const newQuantity = currentQuantity + n; inventory.set(item, newQuantity); return newQuantity; }; module.exports = { inventory, addToInventory };
To guarantee that your test will run assertions, you can use expect.hasAssertions
. It will cause your test to fail if the test doesn’t run at least one assertion.
Go ahead and ensure that your test will run an assertion by adding expect.hasAssertions
to it.
Listing 4. inventoryController.js
const { inventory, addToInventory } = require("./inventoryController"); beforeEach(() => inventory.set("cheesecake", 0)); test("cancels operation for invalid quantities", () => { expect.hasAssertions(); #1 try { addToInventory("cheesecake", "not a number"); } catch (e) { expect(inventory.get("cheesecake")).toBe(0); } });
#1 Causes the test to fail if it doesn’t execute at least one assertion.
Now, consider that you also want to add an assertion which ensures that the inventory only has one item.
Listing 5. inventoryController.test.js
const { inventory, addToInventory } = require("./inventoryController"); beforeEach(() => inventory.set("cheesecake", 0)); test("cancels operation for invalid quantities", () => { expect.hasAssertions(); try { addToInventory("cheesecake", "not a number"); } catch (e) { expect(inventory.get("cheesecake")).toBe(0); } expect(Array.from(inventory.entries())).toHaveLength(1) #1 });
#1 An assertion which is always executed.
The test above could still pass even if the catch
block was not executed. The expect.hasAssertions
call within the test will only ensure that any assertions run, not that all of them run.
To solve this problem, you can use expect.assertions
to explicitly determine how many assertions you expect to run. If you want, for example, two assertions to run, use expect.assertions(2)
. Using expect.assertions
will cause your tests to fail whenever the number of assertions executed doesn’t match what you determined.
Listing 6. inventoryController.test.js
const { inventory, addToInventory } = require("./inventoryController"); beforeEach(() => inventory.set("cheesecake", 0)); test("cancels operation for invalid quantities", () => { #1 expect.assertions(2); try { addToInventory("cheesecake", "not a number"); } catch (e) { expect(inventory.get("cheesecake")).toBe(0); } expect(Array.from(inventory.entries())).toHaveLength(1) });
#1 Causes the test to fail if it doesn’t execute two assertions.
Loose assertions
The goal of writing tests is for them to fail when your application doesn’t do what you want. When writing assertions, you want to ensure that they will be sensitive enough so that they can make tests fail whenever anything goes wrong.
Again, let’s use your addToInventory
function as an example. For this function, you could write an assertion which ensures that the result of addToInventory
is a Number
.
Listing 7. inventoryController.test.js
const { inventory, addToInventory } = require("./inventoryController"); beforeEach(() => inventory.clear()); #1 test("returned value", () => { const result = addToInventory("cheesecake", 2); #2 expect(typeof result).toBe("number"); });
#1 Empties the inventory
#2 Checks whether the result is a number.
Now think of how many possible results this assertion allows. Numbers in JavaScript can go from 5e-324
to precisely 1.7976931348623157e+308
. Given this enormous range, it’s clear that the set of possible results accepted by the assertion is too big. This assertion can guarantee that the addToInventory
function won’t return, for example, a String
or a boolean
, but it can’t guarantee that the number returned is correct. By the way, you know what else is considered a Number
in JavaScript? NaN
, which stands for not a number.
console.log(typeof NaN); // 'number'
Figure 1. The range of results accepted by the type assertion.
The more values an assertion accepts, the looser it is.
One way of making this assertion accept fewer values — make it “tighter” — is to expect the result to be bigger than a particular value.
Listing 8. inventoryController.test.js
const { inventory, addToInventory } = require("./inventoryController"); beforeEach(() => inventory.clear()); test("returned value", () => { const result = addToInventory("cheesecake", 2); expect(result).toBeGreaterThan(1); #1 });
#1 Expects the result to be greater than one.
Figure 2. The range of results accepted by the toBeGreaterThan
assertion
The toBeGreaterThan
assertion drastically reduces the number of accepted results, but it is still way looser than it should be.
The tighter and most valuable assertion you can write is an assertion which only allows a single result to pass.
Listing 9. inventoryController.test.js
const { inventory, addToInventory } = require("./inventoryController"); beforeEach(() => inventory.clear()); test("returned value", () => { const result = addToInventory("cheesecake", 2); expect(result).toBe(2); #1 });
#1 Expects the result to be greater exactly two.
Ideally, your assertions should accept a single result. If your assertions customarily allow many results, it can be a sign that your code is not deterministic or that you don’t know it well enough. Loose assertions make it easier for tests to pass, but they make those tests less valuable because they might not fail when the application produces invalid output. Writing tighter assertions makes it harder for your tests to pass when the application code has problems, making it easier to catch bugs.
Deterministic code A code is said to be deterministic when, given the same input, it always produces the same output.
Figure 3. The range of results accepted by the tight toBe
assertion.
An assertion that, for example, verifies whether an array includes a value, usually tells that you don’t know what the entire array should look like. Ideally, you should have written an assertion which checks the whole array.
Negated assertions — assertions which ensure an output does not match another value — also tend to generate loose assertions. When you, for example, assert that an output is not 2
, you accept an enormous range of values (all values, of all types, but 2
). Avoid writing negated assertions whenever possible.
Figure 4. The range of results accepted by a negated assertion.
Writing loose assertions is acceptable when you want tests not to be tied to factors you can’t control, like true randomness. Assume that you are testing a function which generates an array with random numbers, for example. When testing this function, you probably want to check the length of the array and the type of its items, but not the array’s exact content.
TIP: Even though Jest has a diverse set of assertions — which you can find at jestjs.io/docs/en/expect — I’d recommend readers to stick to toBe
and toEqual
whenever possible, as they are extremely strict.
To make it easier to control how loose your assertions are, Jest has asymmetric matchers. Asymmetric matchers allow you to determine which aspects of a particular output Jest should validate loosely and which ones it should validate tightly.
Assume you have a function which returns the content of your inventory indexed by their name. For auditing purposes, this function will also include the date at which the information was generated.
Listing 10. inventoryController.js
const inventory = new Map(); // ... const getInventory = () => { const contentArray = Array.from(inventory.entries()); const contents = contentArray.reduce( #1 (contents, [name, quantity]) => { return { ...contents, [name]: quantity }; }, {} ); return { ...contents, generatedAt: new Date() }; #2 }; module.exports = { inventory, addToInventory, getInventory };
#1 Creates an object whose keys are the inventory item’s names and whose values are each item’s respective quantities.
#2 Returns a new object including all the properties in contents and a date.
When testing this function, your date will change whenever the test runs. To avoid asserting on the exact time the inventory report was generated, you can use an asymmetric matcher to ensure that the generatedAt
field will contain a date. For the other properties, you can have tight assertions.
Listing 11. inventoryController.test.js
const { inventory, getInventory } = require("./inventoryController"); test("inventory contents", () => { inventory .set("cheesecake", 1) .set("macarroon", 3) .set("croissant", 3) .set("eclaire", 7); const result = getInventory(); expect(result).toEqual({ #1 cheesecake: 1, macarroon: 3, croissant: 3, eclaire: 7, generatedAt: expect.any(Date) #2 }); });
#1 Expects the result to match the object passed to the toEqual
method.
#2 Allows the generatedAt
property to be any Date.
Asymmetric matchers can do many different kinds of verifications. They can, for example, check whether a string matches a regular expression or whether an array contains a specific item. Check Jest’s documentation to see which matchers are available out-of-the-box.
Using custom matchers
In the previous section, we’ve seen that even though we want our assertions to be as strict as possible, in some instances, it’s still necessary to be flexible when it comes to verifying values.
Just like when you encapsulate behavior into functions, you can encapsulate your verifications into new matchers.
Let’s say, for example, that you are writing a test to ensure that the generatedAt
field in the getInventory
is not a date in the future. One of the ways you could do this is by manually comparing timestamps.
Listing 12. inventoryController.test.js
const { inventory, getInventory } = require("./inventoryController"); test("generatedAt in the past", () => { const result = getInventory(); const currentTime = Date.now() + 1; #1 const isPastTimestamp = result.generatedAt.getTime() < currentTime; #2 expect(isPastTimestamp).toBe(true); #3 });
#1 Adds one millisecond to the current timestamp to ensure that the timestamps compared won’t be the same. Alternatively, you could wait for one millisecond before calling Date.now
.
#2 Checks whether the result’s generatedAt
property is smaller than the one generated by the test and stores a boolean value.
#3 Checks whether the stored boolean value is true.
This test can be great when passing, but when failing its feedback may not be as clear as you’d expect. Try, for example, to set the year in the generatedAt
property to 3000 so that you can see what happens when the test fails.
Listing 13. inventoryController.js
const inventory = new Map(); // ... const getInventory = () => { const contentArray = Array.from(inventory.entries()); const contents = contentArray.reduce((contents, [name, quantity]) => { return { ...contents, [name]: quantity }; }, {}); return { ...contents, generatedAt: new Date(new Date().setYear(3000)) #1 }; }; module.exports = { inventory, addToInventory, getInventory };
#1 Creates a date in the year three thousand.
Running your tests should yield the following output:
FAIL ./inventoryController.test.js × generatedAt in the past (7ms) ● generatedAt in the past expect(received).toBe(expected) // Object.is equality Expected: true Received: false
As you can see, the diff generated by Jest doesn’t provide much information. It says that you expected true
to be false
, but it doesn’t tell you anything about what was the subject of your assertion. When a test fails with such a generic diff, you will need to re-read its code to determine what went wrong and what the exact difference between the actual and expected results was.
To get access to more precise assertions, we will use jest-extended
. The jest-extended
module extends Jest’s assertions, providing you with even better and more flexible checks.
NOTE: You can find the documentation for jest-extended
and its assertions at github.com/jest-community/jest-extended.
Go ahead and install jest-extended
as a dev-dependency.
To set up jest-extended
so that you can use its assertions, update your jest.config.js
and add jest-extended
to the list of files which should run after setting up the test environment.
Listing 14. jest.config.js
module.exports = { testEnvironment: "node", setupFilesAfterEnv: ["jest-extended"] #1 };
#1 Extends Jest with assertions from jest-extended.
Once you have done this, you will be able to use any of the assertions shipped with jest-extended
.
To make the test’s feedback clearer, we will use the toBeBefore
assertion, which checks whether a Date
is before another. Update your test so that it uses this new assertion.
Listing 15. inventoryController.test.js
const { getInventory } = require("./inventoryController"); test("generatedAt in the past", () => { const result = getInventory(); const currentTime = new Date(Date.now() + 1); #1 expect(result.generatedAt).toBeBefore(currentTime); #2 });
#1 Creates a date which is one millisecond ahead of the current time. Alternatively, you could wait for a millisecond before generating a Date.
#2 Expects the result’s generatedAt
property to be before the date generated in the line above.
Now, when this test fails, the feedback provided by Jest will be way more precise.
FAIL ./inventoryController.test.js × generatedAt in the past (11ms) ● generatedAt in the past expect(received).toBeBefore() Expected date to be before 2020-02-23T15:45:47.679Z but received: 3000-02-23T15:45:47.677Z
Now you know exactly what the test was checking and what the difference between the two dates was.
Using precise assertions enables you to improve the quality of your test’s feedback by indicating what kind of output Jest should produce.
Tests with precise assertions are way easier to read and take less time to fix since it’s easier to understand what went wrong.
Circular assertions
Circular assertions are assertions which compare your application’s code to itself. You should avoid circular assertions because when comparing your code’s results to themselves, your tests will never fail.
Let’s say, for example, that you create a route for returning the inventory’s content. This route uses the getInventory
function you already have.
Listing 16. server.js
// ... router.get("/inventory", ctx => (ctx.body = getInventory())); // ...
To facilitate testing this route, you may feel tempted to use getInventory
again within your test.
Listing 17. server.test.js
// ... test("fetching inventory", async () => { inventory.set("cheesecake", 1).set("macarroon", 2); const getInventoryResponse = await sendGetInventoryRequest("lucas"); // For the sake of this example, let's not compare the `generatedAt` field's value const expected = { #1 ...getInventory(), generatedAt: expect.anything() #2 }; // Because both the route and `expected` were generated based on `getInventory` // you are comparing two outputs which come from the exact same piece of code: // the unit under test! expect(await getInventoryResponse.json()).toEqual(expected); #3 }); // ...
#1 Copies to a new object the properties in the getInventory
function’s result and includes a generatedAt
property whose value is an asymmetric matcher.
#2 Allows the generatedAt
property to have any value.
#3 Compares the server’s response to the object created within the test.
The problem with this approach is that, because both the route and the test depend on the same piece of code (getInventory
), you end-up comparing the application to itself. If there’s a problem in the getInventory
route, it won’t cause this test to fail because the result you expect was also incorrect.
Try, for example, changing getInventory
so that it returns 1000
as the quantity for each item.
Listing 18. server.test.js
const inventory = new Map(); const getInventory = () => { const contentArray = Array.from(inventory.entries()); #1 const contents = contentArray.reduce((contents, [name]) => { #2 return { ...contents, [name]: 1000 }; }, {}); return { ...contents, generatedAt: new Date() }; #3 }; module.exports = { inventory, addToInventory, getInventory };
#1 Uses the inventory’s entries to create an array of key and value pairs.
#2 Creates an object whose keys are the inventory item’s names and whose values are always set to one thousand and represent each item’s respective quantities.
#3 Copies every property in contents to a new object which also contains a generatedAt
key whose value is a Date.
Even though the quantity of items in the inventory is now wrong, the test for your route will still pass.
Circular assertions don’t tend to be a big problem if you are already testing the different parts of your application separately. In the case above, for example, even though the route’s tests didn’t catch the bug, thorough tests for the inventoryController
itself would have.
Regardless of whether you could have caught that in a separate test, the tests for the route will pass even when they shouldn’t. This inaccurate feedback could cause confusion and, if you didn’t have rigorous tests for inventoryController
, let the bug slip into production.
A test which contains the expected result explicitly written into the assertion would have been far better. It would make the test more readable and facilitate debugging.
Listing 17. server.test.js
// ... test("fetching inventory", async () => { inventory.set("cheesecake", 1).set("macarroon", 2); const getInventoryResponse = await sendGetInventoryRequest("lucas"); const expected = { #1 cheesecake: 1, macarroon: 2, generatedAt: expect.anything() }; // Notice how both the `actual` and `expected` // outputs come from different places. expect(await getInventoryResponse.json()).toEqual(expected); #2 }); // ...
#1 Creates an object literal without using any dependencies.
#2 Expects the server’s response to match the object literal created within the test.
Whenever possible, create separate utility functions for your tests instead of just reusing the application’s code. It’s preferable to have a bit of duplication or hard-coded expected results than to have tests which never fail.
That’s all for this article.
If you want to learn more about the book, you can check it out on our browser-based liveBook platform here.