Recap
I left off session IV with a PR I said I'd come back to. It had 4 failing tests waiting on incoming tech debt to fix.
When I came back to it I just didn't like the idea of merging in a PR with failing tests... in real life no reviewer would let me, so let's continue session IV till we have all our tests passing.
Fixing our first failing test
Let's start with this case
{
name: 'one minute after, TO HAVE BEEN called',
dateString: '2022-02-19T16:31Z',
clockTick: 4000,
dayExpected: 6,
callsExpected: 4,
},
Here, if the app crashed and restarts one minute after the cron job should have run, we still want the cron job to run. We want it to do this by checking if it's scrapped all the available data on Leetcode, and to run a scrape for any dates that haven't been scraped.
We haven't hooked up a db yet to store Leetcode data which we'll check against, but we can pretend we have one with a mock.
Initial attempt at a mockDb
Here's my initial attempt to think about a mockDb:
export class TasksService {
private readonly logger = new Logger(TasksService.name);
_called = undefined;
_contestDate = undefined;
_mockDb = undefined;
constructor(contestDate: Date, mockD?: { string: boolean }) {
this._called = 0;
this._contestDate = contestDate;
this._mockDb = {
'26-02-2022': true,
};
}
@Cron('* 30 4 * * sun')
handleWeekly(): void {
this.scrapeContestData();
}
@Cron('* 30 16 * * sat')
handleBiWeekly(): void {
console.log('today', this._contestDate.toUTCString());
if (!this.checkDataExists(this._contestDate)) this.scrapeContestData();
}
scrapeContestData(): void {
this._called += 1;
}
checkDataExists(key: Date): boolean {
const ISOFomat = key.toISOString();
return this._mockDb[ISOFomat.substring(0, ISOFomat.indexOf('T'))];
}
}
The idea is that on instantiation of TasksService, I pass in my DB string and the index of the scraped data we want to find, so that TasksService can check whether a complete entry for that contest already exists.
Dealing with Dependency Injection
The first problem I'll face with this is injecting those constructor dependencies contestDate: Date, mockD?: { string: boolean }. If you've been following, you'd have noticed that NestJs has so far handled our dependency injection for our tests.
I need to figure out where to place these dependencies so that NestJs can automagically pick them up and inject them into the TasksService constructor when it's creating a new instance of TasksService.
The alternative is that I'll have to figure out how to do this imperatively, which is yeah, not a good idea, as that would mean potentially having to orchestrate the entire instantiation process manually, making sure that I instantiate classes needed further down the pipeline first, and so on and so forth. Better to leave this to an IoC container! If you want to learn more about dependency injection and IoC containers, I've written about dependency injection here.
Digging through NestJs Docs on Dependency Injection
Let's head back to the docs to figure out how NestJs does DI. There's an entry on how Dependeny Injection is done in NestJs which looks exactly like what we're looking for, so let's try that out.
I create a DataService class that will imperatively pull in the required info
@Injectable()
export class DataService {
contestDate = undefined;
mockDb = {
'26-02-2022': true,
};
}
I then use this class as a dependency in TasksService
...
import { DataService } from './data.service';
...
@Injectable()
export class TasksService {
private readonly logger = new Logger(TasksService.name);
_called = undefined;
_contestDate = undefined;
_mockDb = undefined;
constructor(private dataService: DataService) {
this._called = 0;
this._contestDate = this.dataService.contestDate;
this._mockDb = this.dataService.mockDb;
}
...
Now hopefully, we can mock out DataService when we need to for our TasksService tests. Let's try that!
Fixing issues with our mock approach
A moment later we're getting 15 failed tests, and when I try to start the app I get lovely red errors, so something's not right!
Looks like it's telling us to add DataService as a provider to our TasksModule
@Module({
providers: [TasksService, DataService],
})
export class TasksModule {}
That seems to do the trick.
So back to our tests.
We're still getting 15 failed tests but It looks like the exact same problem, so let's see what happens when we add DataService as a provider to our tests.
const module = await Test.createTestingModule({
imports: [
{
module: AppModule,
imports: [ScheduleModule.forRoot(), TasksModule],
providers: [TasksService, DataService], //<-- added DataService here
},
],
}).compile();
Note: I noticed later on that adding DataService in this way really didn't do anything. I'm still not sure why at this point it seemed to do something.
Only 5 failing tests now and a whole bunch of undefined errors! We started off with only 4 failing so we definitely broke something but we'll figure it out.
Very quickly we can see that the undefined errors are because contestDate is undefined in the DataService class, but we'll leave that as it is because we're going to mock it in the tests anyway.
@Injectable()
export class DataService {
contestDate = undefined;
mockDb = {
'26-02-2022': true,
};
}
Mocking a NestJs Service
Let's get around to mocking our DataService. I don't remember exactly where, but I recall seeing a useValue pattern in one of the test examples, so I'm just going to search for useValue in the NextJs docs, and... found it!
According to the docs:
The useValue syntax is useful for injecting a constant value, putting an external library into the Nest container, or replacing a real implementation with a mock object. Let's say you'd like to force Nest to use a mock CatsService for testing purposes.
And this sounds like just what we need, so let's try it out!
A few moments later I've wired up a mockDataService
const mockDataService = {
contestDate: new Date(),
mockDb: {
'26-02-2022': true,
},
};
beforeEach(async () => {
const module = await Test.createTestingModule({
imports: [
{
module: AppModule,
imports: [ScheduleModule.forRoot(), TasksModule],
providers: [
TasksService,
{
provide: DataService,
useValue: mockDataService,
},
],
},
],
}).compile();
app = module.createNestApplication();
});
But we're still getting undefined issues, although, it looks like these have reduced in number.
I'm not sure why this is happening, but I'm more interested in my next problem:
Using a Dynamic mockDate per Test
I want to dynamically set the contestDate (later renamed to mockDate) per test, but the way I've set up my tests limits me to setting only one mock before all tests. How can I fix this? I decide I'll focus on each test case one at a time to spot a pattern for how to do this.
Starting with the highlighted test that is now failing when it used to be passing. Recall 5 tests now fail instead of just 4 we began with.
Recalling our error messages with the undefined errors, it's clear that our mock isn't being applied in some cases
I initially have a hunch it has something to do with the way we're getting our TasksService instance:
const service = app.get(TasksService);
But it doesn't. The clue to what's wrong comes when I try to use the mock pattern in the TasksModule itself, and that works beautifully.
const mockDataService = {
contestDate: new Date(),
mockDb: {
'2022-02-26': true,
},
};
@Module({
providers: [
TasksService,
{
provide: DataService,
useValue: mockDataService,
},
],
})
export class TasksModule {}
If this is the case, then the pattern I've chosen for mocking in my tests is not the right one, so I first of all remove everything in my tests that does not make a difference, and that leaves me, surprisingly, with the realization that to create a TestingModule for this series of tests all I needed was my AppModule.
Using overrideProvider to correctly mock our DataService
Furthermore, after digging through the docs I find that when doing e2e testing (as we're doing here really) there's a pattern for overriding providers with mocks using the overrideProvider method.
beforeEach(async () => {
const module = await Test.createTestingModule({
imports: [
{
module: AppModule, //<-- Only need to import AppModule
},
],
})
.overrideProvider(DataService)
.useValue(mockDataService)
.compile();
Using this pattern finally allows me to mock the DataService module predictably, and clears all our undefined errors.
Getting rid of our beforeEach in favour of test.each
Once mocking was sorted, I pretty much went into the zone and stopped my real time documenting. During this period this question came up when I got back to solving the problem of dynamic mockDates:
How do we pass parameters to Jest's beforeEach hook?
I love the way it's answered in this stackoverflow question. We don't need to. The only reason I'm using a beforeEach hook is to keep things DRY (don't repeat yourself). So instead of complicating things, I move the beforeEach code into the test.each code block, which accomplishes the same DRY goal, but with the benefit of allowing me to pass dynamic mockDates.
And that's pretty much it. We end up with the code in this PR and 15 passing tests,
after implementing new methods in our TasksService to cater for the test cases that were failing.
import { Injectable, Logger } from '@nestjs/common';
import { DataService } from './data.service';
import { Cron } from '@nestjs/schedule';
@Injectable()
export class TasksService {
private readonly logger = new Logger(TasksService.name);
_called = 0;
constructor(private dataService: DataService) {
this.onStartUp(
this.dataService.leetcodeDb.keys,
this.dataService.mockDb.keys,
);
}
@Cron('* 30 4 * * sun')
handleWeekly(): void {
this.scrapeContestData();
}
@Cron('* 30 16 * * sat')
handleBiWeekly(): void {
if (
this.dataService.leetcodeDb.lastBiWeeklyKey === undefined ||
!this.dataExistsInDb(
this.dataService.leetcodeDb.lastBiWeeklyKey,
this.dataService.mockDb.keys,
)
)
this.scrapeContestData();
}
scrapeContestData(): void {
this._called += 1;
}
dataExistsInDb(key: Date, mockDb: string[]): boolean {
const ISOFomat = key.toISOString();
const mockDbKeySet = new Set();
mockDb.forEach((key) => mockDbKeySet.add(key));
return mockDbKeySet.has(ISOFomat.substring(0, ISOFomat.indexOf('T')));
}
onStartUp(leetcodeDb: string[], mockDb: string[]) {
const mockDbKeySet = new Set();
mockDb.forEach((key) => mockDbKeySet.add(key));
leetcodeDb.forEach((key) => {
if (!mockDbKeySet.has(key)) {
this.scrapeContestData();
console.log(key);
}
});
}
}
We also end up with a clunky looking DataService ... interface, which I'm sure will change as I learn from database patterns, such as the repository pattern, suggested in Nest's documentation.
Next Session
In the next session we'll hopefully start pulling in data from Leetcode and storing it in our DB of choice. I'm deciding to stick with my initial choice of a Postgres Db and to learn from whatever pain points I encounter if I've made the wrong choice.
Attributions
Thanks to Marco Santarossa for the cover image.
