← Back to all posts
7 min read

Feeding Make.com from Python

Every Make.com scenario is a consumer. It waits for data to arrive — from an app, an API, or someone upstream who did the unglamorous work of collecting it. That upstream role is exactly where plain old Python shines.

job-scrapper is my producer: a small scraper that watches OnlineJobs.ph for remote gigs matching automation keywords (n8n, make.com, zapier) and pushes each hit as JSON into a Make.com webhook, which forwards it to Telegram. Here's what I learned building the kitchen that feeds the restaurant.

The moving parts

  • A keyword list at the top of main.py — currently four automation-related terms.
  • GitHub Actions running the script on a cron, every 4 hours, with the webhook URL injected as a repository secret. No server, no hosting bill.
  • A requests-only dependency list. No Selenium, no headless browser — the site renders server-side enough to parse.
  • The script exits immediately if WEBHOOK_URL isn't set. Fail loudly at startup, not mysteriously at runtime.

One job, one JSON object

The payload contract is deliberately flat:

{
  "jobTitle": "Senior Digital Marketing, AI Content & E-Commerce Manager",
  "company": "MadeEA",
  "salary": "$1200/month",
  "employmentType": "Full Time",
  "url": "https://www.onlinejobs.ph/jobseekers/job/12345",
  "datePosted": "2026-08-24",
  "description": "About the Role\nWe are an established and growing…"
}

Three decisions worth stealing:

  • Flat beats nested. Every key maps one-to-one onto a field in Make's mapping panel. No dot-path spelunking inside arrays when you're wiring up a filter at midnight.
  • One request per job. Each listing POSTs individually instead of batching the whole search. Downstream routing stays trivial — no iterator needed unless you want one.
  • Presentation stays out of transport. The description is capped in the scraper (600 characters, cut at a word boundary), because only the scraper knows about Telegram's 4096-character message limit once title, salary and button text are added. The payload arrives pre-sized for its destination.

Politeness is a feature

Scrapers that hammer sites get banned, so the delays are randomized like a human's: 5–12 seconds between searches, another 2–5 second pause before visiting each listing page for its full description.

When OnlineJobs.ph throws up a bot-check interstitial anyway, both request types retry several times with growing waits before giving up gracefully — sending the teaser text with a logged warning rather than failing the whole run. And because I parse raw HTML with regex (no fragile DOM tree dependency), a markup change means zero results, never crashes. The fallback path makes sure downstream still gets a single ping with a search link, so my notifications never stall silently.

Queue discipline

The most important constant in the file is MAX_JOBS_PER_KEYWORD = 1. Each keyword contributes at most one listing per run, deduplicated across keywords. Two reasons:

  1. Operations budget. Every job that hits the webhook costs operations downstream. Four keywords × every 4 hours is already a steady trickle; uncapped, a hot day on the job board would torch a free plan's monthly budget in a weekend.
  2. Notification fatigue. A Telegram channel that pings forty times a morning gets muted by Thursday. One strong signal per keyword per run keeps me actually reading them.

Treat the webhook like an API contract

The payload shape lives verbatim in the README, with types and a real example. That sounds obvious until you've rebuilt a downstream scenario and had to reverse-engineer your own fields from execution logs. Producer and consumer should be strangers who agree on a schema — then either side can be rebuilt without touching the other.

This design has one more trick up its sleeve: it's cheap. How cheap, in actual operations-per-month arithmetic? That's the next post: The free-plan ops math.