The Antivirus That Ate a News Source

At 06:00:54 on Sunday, August 23, the antivirus on our ingest workstation starts logging blocked URLs faster than one every four seconds. By 06:03:52 it has written 55 of them. Every one names the same news source, www.crewai.com. Every one carries the same label: Blocked URL / PUA blacklist. Three minutes of alarm, one domain.

At 06:04:00, eight seconds after the last blocked event, our own database writes its account of the same morning: feed-crewai, last_fetch_status = 'ok', no error recorded.

One of those two records is lying. It takes a 603-event log export to establish which one, and the answer is the reason this post exists: the failure was hiding inside a success.

What the two logs agreed on

We export the antivirus's full "Filtered websites" log, 603 events spanning 2026-05-27 to 2026-08-25, and line it up against the database.

Across the whole window the domain accounts for 71 of the 603 events, all PUA blacklist, spread over six different hosting IPs. The blocks are real. They held for twelve days without a gap, and every fetch path we have was intercepted. And for those same twelve days, our feed-status column insists the source is healthy.

The clue that pointed the wrong way

The tempting story assembles itself: the antivirus started eating CrewAI, and that's why we have no fresh CrewAI articles. It has the right villain and the right victim. It fails on the dates.

The newest CrewAI article in the corpus was ingested 2026-07-12. The first blocked event is 2026-08-11, a month later. Whatever opened the freshness gap, it was open for thirty days before the blocking began, which means the block explains none of it. (The gap's actual cause turns up later in this same investigation, and we'll get there, because the two bugs turn out to be feeding each other.) We give the theory its fair reading, then rule it out and narrow the question to the part the evidence does support: how does a source that is fully blocked at the network layer report ok with a straight face?

The reveal

When a feed has an RSS URL, smart_ingest.py runs a chain of fetchers until one returns content. For https://www.crewai.com/rss/ the chain is: Playwright's headless Chromium with a 30-second timeout, then plain requests with 15 seconds, then Botasaurus, which drives a real installed Chrome through undetected-chromedriver.

The first two fail honestly. The block happens at the connection stage, so Playwright gets a navigation error and returns nothing, and requests gets an exception and returns nothing. Each one falls through to the next.

The third fetcher is a real browser. A real browser pointed at a blocked page does not return a connection error to its caller, because from the browser's point of view nothing failed: the local security product intercepted the request and served a block page, and the browser rendered it, exactly as designed. The antivirus lowered a curtain in front of the site. Our third fetcher photographed the curtain and filed the photograph as the news.

Non-empty HTML came back. The pipeline's rule was: any fetcher returned content, so the fetch succeeded. ok.

That rule is the bug. Not the antivirus, which blocked a domain its reputation data told it to block, and not the browser, which rendered the page it was given. The defect was ours: we treated "some fetcher returned bytes" as proof that the source served real articles, and a block page is bytes.

The export even shows the chain's fingerprint. The 55-event storm resolves into repeating clusters: one headless-shell event, then two or three Python events, then one chrome.exe event, the exact fetcher order in the code, repeated about fifteen times in three minutes.

Which raises the second question. Why fifteen?

Why fifteen

One scheduled ingest run touching one feed should produce one cluster. We got roughly fifteen, and a process snapshot two days later confirms the cause directly: about sixteen concurrent smart_ingest.py runs, all children of one long-lived dev server.

Every app server here is also an ingestion daemon. The dev server registers a 30-minute refresh interval when src/hooks.server.ts loads, and Vite reloads that module every time a file it imports changes. Each reload registered another interval and reset the module-scope in-flight flag. The file lock that should have caught this excused any lock holder whose PID matched the current process, and every stacked run lived inside the same process. Leave a dev server up for a day of editing and the intervals accumulate quietly until sixteen ingest runs are contending for the same SQLite database.

So the two bugs compounded. The stacking opened the original freshness gap by saturating the scheduler, and when the blocking began on August 11, the stacking multiplied every blocked fetch by fifteen into an alert storm loud enough to get investigated. The bug that hid a failure was exposed by the bug that amplified it.

What held

One part of the pipeline behaved well under pressure, and it's worth naming, since it was luck-adjacent. Connection-stage blocks surface in our httpx fetcher as a ConnectError, which is re-raised without any fallback. The fallback that matters is Jina Reader, a third-party service that fetches URLs on our behalf, and the fail-closed path means a locally blocked URL never gets shipped to it. The export confirms this held: zero r.jina.ai events in all 603 records. A security block stayed local instead of becoming a disclosure.

The export also settled what kind of block this was. No named malware appears anywhere in the log, and the matching is string-based: even Google favicon lookups whose URLs merely contained crewai.com were flagged, while all six hosting IPs were blocked equally.

The fix

PR #172, merged 2026-08-25, in two commits.

Commit 80bd294 changed what ok means. Status is now decided by a pure helper, scripts/python/ingestion/status_decision.py, from what the cycle proved rather than what it fetched. Parsed entries or scraped articles, at least one of them: ok. Content fetched but zero items parsed: no_entries, with the note "possible local block page or site change". No content from any fetcher: fetch_failed. The error text records the whole trail per method, like playwright:no_content; requests:none; botasaurus:error:TimeoutError, so the next investigator sees which fetchers ran and how each one ended. The helpers are side-effect-free and pytest-covered without any network.

Commit 1f6365d closed the stacking. The refresh interval is now a singleton across Vite reloads, and the ingest lock is authoritative even when the holder PID is our own.

And the block itself went to the vendor: a false-positive submission with a sanitized 76-row log excerpt, sent 2026-08-25. The automated acknowledgement came back the same morning with a tracking number, and six hours later a person on the vendor's malware response team answered in one line: the site will be unblocked in the next update. It was. On 2026-08-31 the same workstation fetched feed-crewai clean and ingested three new posts, the first new rows for that source since July.

There is a postscript, and it is the kind this job specializes in. By the time this post was edited, the source had retired the RSS endpoint the feed points at: the old address now redirects to the apex domain and answers 404, and the blog page advertises no feed link. So the next failure of feed-crewai will be a page that parses to zero items, which is exactly the failure the status fix now records as an error instead of a success.

Where this bug hid

Every monitoring system draws a line between success and failure, and this bug lived exactly on the line: a state that failure could reach. Our ok was reachable by "a browser rendered something", and interstitial pages, consent walls, error pages with HTTP 200, and parked domains all render something. If a failure mode can produce your success signal, you don't have a success signal; you have a bytes-arrived signal wearing the wrong name.

The transferable test is cheap. For each status your pipeline can record, ask what the laziest possible failure would produce, and if the answer is that status, tie the status to evidence of the thing you actually wanted, such as the number of items parsed or rows written. Count the outcome, not the transport. The transport will happily photograph a curtain.