When searching for a solution to handle both NextJS cron jobs without Vercel and queue jobs, I explored various options across the internet but couldn’t find one that fit my needs perfectly. I came across node-cron, which is easy to implement but didn’t quite meet my expectations. I needed something more robust, capable of efficiently managing both cron jobs and queue jobs.
After some research, I discovered BullMQ—a powerful tool that supports both job scheduling and queue management. In this article, I’ll walk you through best practices for setting up NextJS cron jobs without Vercel using BullMQ. Stay tuned for the next article where I’ll dive into implementing a robust queue system with BullMQ.
Table of contents
- Prerequisites
- Folder Structure
- Step 1: Set Up the Scheduler
- Step 2: Create the Jobs
- Usage
- Conclusion
Prerequisites
Before we dive into the implementation, ensure you have the following installed:
- BullMQ and ioredis: Add these packages to your project with the following command:
pnpm add bullmq ioredis- Redis: BullMQ requires Redis to function properly. Install Redis by running:
- macOS
brew install redis- Ubuntu
sudo apt install redis-serverMake sure Redis is running before you proceed.
Folder Structure
To keep things organized, we’ll follow this structure for the scheduler and jobs:
/server
/scheduler
index.ts
/jobs
reminderEmails.ts
logCleanup.tsStep 1: Set Up the Scheduler
This file will define the scheduled tasks, create a BullMQ queue, and set up a worker to process the jobs. We’ll also use a helper class, SchedulerHelper, to define cron patterns easily.
You can find the SchedulerHelper class here copy that. If you encounter any issues with the helper class or have suggestions, feel free to open a pull request (PR).
import { Queue, Worker } from 'bullmq';
import Redis from 'ioredis';
import { sendReminderEmails } from './jobs/reminderEmails';
import { logCleanup } from './jobs/logCleanup';
import SchedulerHelper from './SchedulerHelper';
type schedulerType = {
name: string;
cron: string;
handler: () => Promise<void>;
};
// Redis connection
export const connection = new Redis('redis://127.0.0.1:6379', {
maxRetriesPerRequest: null,
enableReadyCheck: false,
});
export const schedulerQueue = new Queue('scheduler', { connection });
/**
* Scheduler Array
* Defines scheduled tasks with name, cron, and handler.
*/
const scheduler: schedulerType[] = [
{
name: 'send-reminder-emails',
cron: SchedulerHelper.everyFiveSeconds(),
handler: sendReminderEmails,
},
{
name: 'log-cleanup',
cron: SchedulerHelper.everySecond(),
handler: logCleanup,
}
];
// Clean up old scheduled jobs
const cleanOldJobs = async () => {
const repeatableJobs = await schedulerQueue.getRepeatableJobs();
for (const job of repeatableJobs) {
console.log(`Removing old job: ${job.name} with key: ${job.key}`);
await schedulerQueue.removeRepeatableByKey(job.key);
}
};
// Main function to schedule jobs
const main = async () => {
await cleanOldJobs(); // Clean up previously set cron jobs
for (const task of scheduler) {
await schedulerQueue.add(task.name, {}, {
repeat: { pattern: task.cron },
});
}
// Worker to handle the scheduler queue
new Worker(
'scheduler',
async job => {
const task = scheduler.find(t => t.name === job.name);
if (task && task.handler) {
await task.handler();
}
},
{
connection,
removeOnComplete: { count: 10 },
removeOnFail: { count: 50 },
}
);
const repeatableJobs = await schedulerQueue.getRepeatableJobs();
console.table(repeatableJobs);
};
main().catch(err => console.error('Scheduler Error:', err));Step 2: Create the Jobs
Now let’s create two example jobs that will be run by the scheduler.
export const sendReminderEmails = async () => {
console.log('Sending reminder emails...');
};export const logCleanup = async () => {
console.log('Cleaning up logs...');
};Usage
Let’s consider an example where you want to send a weekly newsletter via email.
Step 1: Define the Newsletter Job
Create a file named newsletterJob.ts in the /jobs directory:
export const sendWeeklyNewsletter = async () => {
console.log('Sending weekly newsletter...');
// Add your email sending logic here
};Step 2: Update the Scheduler
Modify your index.ts file in the /scheduler directory to include this new job. Add it to the scheduler array:
const scheduler: schedulerType[] = [
{
name: 'send-reminder-emails',
cron: SchedulerHelper.everyFiveSeconds(),
handler: sendReminderEmails,
},
{
name: 'log-cleanup',
cron: SchedulerHelper.everySecond(),
handler: logCleanup,
},
{
name: 'send-weekly-newsletter',
cron: SchedulerHelper.weeklyOn(0, '9:00'), // Runs every Sunday at 9 AM
handler: sendWeeklyNewsletter,
}
];In this example:
- The sendWeeklyNewsletter job is scheduled to run every Sunday at 9 AM.
- You can adjust the cron pattern in SchedulerHelper.weeklyOn(0, '9:00') to match your desired schedule.
Step 3: Add the Command to package.json
To run the scheduler, add this script to your package.json:
"schedule:run": "tsx ./server/scheduler/index.ts"You can now run your scheduler using:
pnpm schedule:runConclusion
By using BullMQ, you can efficiently manage both cron jobs and queue jobs in a single solution for your Next.js projects. This approach not only replaces Vercel’s native cron job functionality but also offers greater flexibility for scheduling and job management.
If you’re looking to set up a queue job system for tasks like order processing or notifications, check out my related article: How to Set Up Queue Jobs in Next.js Using BullMQ.
If you run into any issues or have suggestions for improving the SchedulerHelper class, feel free to contribute via a pull request