Pliny — News & Intelligence Aggregation

A pipeline that collects from several news and open-source intelligence sources on a schedule, normalizes everything into a single consistent format, and makes it available to standard feed readers. The interesting problems here are the ones shared by any ingestion system that has to behave itself against third-party APIs: respecting rate limits, not re-reporting the same item twice, and deciding what to keep and what to let go.