Orchestrating Parallel LLM Workloads with batchTriggerAndWait

Your cron job shouldn't await 200 LLM calls. It should fan them out and wait once.

I learned that while building an automated outbound recruiting pipeline inside an existing hiring platform. Every twelve hours, a scheduled job discovers target companies, enriches them with jobs and founder data, ranks every applicant in the resume pool against each company, and queues personalized email campaigns.

The ranking layer is pure LLM work: structured GPT-4o calls with explainable match scores. Small batch in production (roughly five companies per run), but each company fans out across the entire applicant pool. Forty applicants × five companies is two hundred ranking calls per run. Twice a day.

My first instinct was the obvious one: loop through applicants, await each ranking call, pick the winner, move on. It worked in development with eight test resumes. In staging, with a real pool and real API latency, one company took four minutes. Five companies sequential meant twenty minutes just for ranking, before discovery, enrichment, or Instantly campaign setup. Scale the pool to eighty applicants and you're measuring ranking time in hours, inside a cron that also has to survive rate limits and partial enrichment failures.

The fix wasn't a bigger machine. It was splitting orchestration from ranking work and using Trigger.dev's batchTriggerAndWait to fan out child tasks while the parent stayed a readable control loop.

The shape of the pipeline

The outbound engine is four layers wired through a single orchestrator:

The rule I settled on: anything that calls OpenAI and doesn't need shared mutable state becomes a child task. The orchestrator owns sequencing, branching, and aggregation. Children own one LLM call, one schema validation, one retry policy.

Orchestrator vs. worker boundaries

Getting this boundary wrong is how cron jobs turn into unmaintainable scripts.

Keep in the orchestrator:

• Loading the applicant pool once per run (shared read-only input).
• Per-company enrichment and the runtime decision between ranking modes.
• Picking the top match from batch results and building lead payloads.
• Dedup writes and campaign activation: side effects that should happen exactly once.

Push to child tasks:

• The OpenAI call with structured output.
• Per-applicant token usage and latency metrics.
• Retries on transient provider errors without re-running the whole company loop.

// src/trigger/outbound-rank.ts
import { task } from "@trigger.dev/sdk/v3";
import { z } from "zod";

export const matchResultSchema = z.object({
  score: z.number().min(0).max(100),
  rationale: z.string().max(500),
  redFlags: z.array(z.string()).max(5),
});

export const rankByJobPayloadSchema = z.object({
  applicantId: z.string().uuid(),
  jobTitle: z.string(),
  jobDescription: z.string(),
});

export const outboundRankApplicantTask = task({
  id: "outbound-rank-applicant",
  retry: { maxAttempts: 3 },
  run: async (payload: z.infer<typeof rankByJobPayloadSchema>) => {
    const applicant = await loadApplicant(payload.applicantId);
    const result = await rankAgainstJobDescription({
      resume: applicant.resumeText,
      jobTitle: payload.jobTitle,
      jobDescription: payload.jobDescription,
    });
    return matchResultSchema.parse(result);
  },
});

The child task is boring on purpose. Input in, validated score out. No Instantly calls, no Fiber lookups, no "while we're here" scope creep.

The orchestrator stays readable:

// src/trigger/outbound.ts
import { schedules } from "@trigger.dev/sdk/v3";

export const outboundEngine = schedules.task({
  id: "outbound-engine",
  cron: { pattern: "0 */12 * * *" },
  run: async () => {
    const companies = await discoverTargetCompanies();
    const applicants = await loadApplicantPool();

    if (applicants.length === 0) {
      return { ok: true, skipped: "empty_pool" };
    }

    const leads: Lead[] = [];

    for (const company of companies) {
      const enriched = await enrichCompany(company);
      const topMatch = await rankApplicantPool(applicants, enriched);

      if (topMatch) {
        leads.push(buildLead(company, enriched, topMatch));
      }
    }

    await markCompaniesContacted(companies);
    await activateInstantlyCampaigns(leads);

    return { ok: true, leadsQueued: leads.length };
  },
});

One loop. One place to read the business logic. Ranking complexity is behind rankApplicantPool, which is where batchTriggerAndWait lives.

The fan-out: batchTriggerAndWait

For each enriched company, ranking mode depends on what enrichment returned. If there's a job title and description, rank against the job. If not but employee profiles exist, rank against the team they already hired. Same fan-out pattern, different child task.

// src/trigger/outbound.ts
async function rankApplicantPool(
  applicants: Applicant[],
  company: EnrichedCompany
): Promise<TopMatch | null> {
  const { jobTitle, jobDescription, employees, name: companyName } = company;
  const hasJobData = Boolean(jobTitle && jobDescription);

  let batchResult;

  if (hasJobData) {
    batchResult = await outboundRankApplicantTask.batchTriggerAndWait(
      applicants.map((applicant) => ({
        payload: {
          applicantId: applicant.id,
          jobDescription: jobDescription!,
          jobTitle: jobTitle!,
        },
      }))
    );
  } else if (employees.length > 0) {
    batchResult =
      await outboundRankApplicantByEmployeesTask.batchTriggerAndWait(
        applicants.map((applicant) => ({
          payload: {
            applicantId: applicant.id,
            employees,
            companyName,
          },
        }))
      );
  } else {
    return null;
  }

  return pickTopMatch(applicants, batchResult);
}

batchTriggerAndWait does two things that matter in production:

1. Triggers N child runs with isolated payloads and retry semantics.
2. Returns when all children settle (success, failure, or retry exhaustion) so the orchestrator can aggregate in one place.

You're not manually managing Promise.all over bare OpenAI calls inside a long-lived cron process. Children get Trigger.dev's task lifecycle: idempotency keys, run logs, per-child traces, and failure isolation.

Why sequential ranking blows the cron budget

Back-of-napkin math from staging:

That's ranking only. Add Fiber enrichment (multiple HTTP calls per company), Instantly campaign setup, and PostgreSQL writes, and a sequential design stops fitting comfortably inside a twelve-hour cadence once the pool grows, or once you increase companies per run.

Parallelism isn't free. It trades wall-clock time for concurrency management. Which is the next problem.

Backpressure: pool size, rate limits, and batch sizing

OpenAI rate limits don't care that your architecture is elegant. Fan out forty tasks at once and you'll hit 429s, not on one child but on many.

What worked in production:

1. Cap applicants at the pool query, not at fan-out time

Don't load five hundred resumes because the model can rank them. Load the active pool with a sane upper bound aligned to your tier's throughput. We kept the pool scoped to recently uploaded, fully parsed resumes, typically forty to sixty candidates.

2. Let child retries handle transient 429s

Child tasks retry independently. A rate-limited ranking call doesn't fail the orchestrator or block siblings that already succeeded. Trigger.dev's per-task retry config matters here; the orchestrator should not implement its own retry loop around OpenAI.

export const outboundRankApplicantTask = task({
  id: "outbound-rank-applicant",
  retry: {
    maxAttempts: 3,
    factor: 2,
    minTimeoutInMs: 1_000,
    maxTimeoutInMs: 30_000,
  },
  run: async (payload) => {
    // ...
  },
});

3. Keep companies sequential, applicants parallel

The outer loop over five companies stays sequential. That's intentional backpressure. You don't fan out 5 × 40 = 200 ranking tasks in a single batchTriggerAndWait call across the entire run. You fan out per company, aggregate, move on.

If you need more throughput later, batch companies too, but only after you've measured provider limits and cost per run. Premature global fan-out is how you learn your OpenAI tier's TPM cap at 2 AM.

Typed payloads across task boundaries

The orchestrator and child tasks are separate deployable units in Trigger.dev's model. Treat the payload as a public API between them.

Every payload gets a Zod schema. Parse on the way in, parse the LLM output on the way out. If a child returns malformed JSON (structured output drifts at the edges), you fail that child, not the run.

// src/trigger/outbound-rank.ts
function pickTopMatch(
  applicants: Applicant[],
  batchResult: BatchResult<typeof matchResultSchema>
): TopMatch | null {
  const scored: ScoredApplicant[] = [];

  for (const run of batchResult.runs) {
    if (!run.ok) {
      logger.warn("rank_child_failed", {
        applicantId: run.payload?.applicantId,
        error: run.error,
      });
      continue;
    }

    const applicant = applicants.find((a) => a.id === run.payload.applicantId);
    if (!applicant) continue;

    scored.push({
      applicant,
      score: run.output.score,
      rationale: run.output.rationale,
      redFlags: run.output.redFlags,
    });
  }

  if (scored.length === 0) return null;

  return scored.sort((a, b) => b.score - a.score)[0];
}

Shared types live in one module imported by both orchestrator and child definitions. Don't duplicate stringly-typed payload shapes across files; the compile-time safety is only as good as the runtime validation at the boundary.

What broke in production: partial batches

The failure mode I didn't simulate well enough in development: most children succeed, some return garbage, a few fail outright.

Early staging runs looked fine. Scores clustered between 55 and 85. Rationales were readable. Then we hit a company whose job description was mostly boilerplate ("join our fast-paced team," no stack, no seniority signal). About forty percent of children returned valid schema JSON with nonsense rationale: high scores for clearly mismatched candidates because the model latched onto generic keywords.

Valid JSON. Failed business logic.

The orchestrator still completed. pickTopMatch chose the least-bad option from the survivors. That's dangerous for outbound: you can email a founder about a backend candidate when they need mobile, and the score said 78.

Fixes that actually helped:

1. Minimum viable context gate before fan-out

If job description token count is below a threshold and employee profiles are thin, skip ranking entirely. Don't burn two hundred calls to learn nothing.

if (hasJobData && jobDescription!.length < 120 && employees.length === 0) {
  logger.info("rank_skipped_thin_context", { companyId: company.id });
  return null;
}

2. Red-flag filtering after aggregation

The match schema includes redFlags: string[]. If the top scorer has more than two red flags, or the gap between first and second place is under five points, downgrade to "no confident match" rather than sending a weak lead.

3. Log batch health, not just batch completion

logger.info("rank_batch_complete", {
  companyId: company.id,
  mode: hasJobData ? "job" : "employees",
  total: batchResult.runs.length,
  succeeded: batchResult.runs.filter((r) => r.ok).length,
  failed: batchResult.runs.filter((r) => !r.ok).length,
  topScore: topMatch?.score ?? null,
});

When ops asks "why zero leads this run," you need counts, not a green checkmark on the parent task.

Observability: parent runs vs. child runs

A common mistake with batch fan-out is logging only at the orchestrator. When ranking quality drifts, the parent log says leadsQueued: 3 and tells you nothing about why two companies were skipped.

What I look at in Trigger.dev's dashboard:

The orchestrator run should read like a summary line in a ops channel. Child runs are where you debug the LLM.

When to use batchTriggerAndWait (and when not to)

Use it when:

• One cron or parent task needs N independent LLM (or CPU) units of work with the same shape.
• Each unit has its own retry semantics and failure shouldn't poison siblings.
• You want aggregation logic in one place after all units settle.
• Work is embarrassingly parallel, with no shared mutable state between children.

Skip it when:

• N is tiny (three items) and latency doesn't matter; plain Promise.all inside a single task may be simpler.
• Children must run in strict order because each step depends on the previous output.
• You're fanning out to hide a design problem. If every child needs the full company context re-fetched, fix data loading in the orchestrator first.

This pipeline also has graceful degradation at the enrichment layer: missing job posts switch ranking mode, missing founder emails skip outreach, exclusion lists prevent duplicate contact. Fan-out handles throughput. Degradation handles incomplete upstream data. You need both; neither replaces the other.

The pattern in one paragraph

Load shared inputs once. Loop the business entities sequentially if you need predictable backpressure. For each entity, batchTriggerAndWait the expensive independent work. Aggregate with explicit handling for failed children and low-confidence winners. Side effects (emails sent, domains excluded) stay in the orchestrator after aggregation, not inside children.

If you're building a cron that ranks, classifies, or scores a collection with an LLM, start by drawing the boundary: what is the control loop, and what is the unit of parallel work? Get that line right, and batchTriggerAndWait is the difference between a script that works in dev and a pipeline that survives a growing pool at twice-daily cadence.

Related: Automated outbound recruiting case study · Two-step LLM pipelines · The Anatomy of a Production-Ready MVP