Building A Simple RSS Aggregator In PHP

An RSS aggregator gathers headlines and article summaries from several websites, then presents them in one readable stream. The idea is modest, yet it introduces useful software development habits: separating network access from display logic, validating external data, storing a stable local copy, and making sensible decisions when a remote service is unavailable.

A PHP feed reader is well suited to a small personal project or an internal dashboard. It can run on inexpensive shared hosting, suit a developer following technology sites during a Sydney commute, and remain understandable without a large framework. The finished application will fetch RSS or Atom documents, extract their entries, remove duplicates, order them by date, and display links in a clean interface.

Choosing Scope And Feed Format

Begin with a narrow definition of the application. A first version might monitor six feeds, show the ten newest entries from each source, and refresh the collection every thirty minutes. It does not need user accounts, full-text indexing, recommendations, or an elaborate administration panel. Keeping the scope small makes it easier to inspect each stage and determine whether a failure comes from PHP, the feed, the database, or the browser.

RSS is a family of XML-based formats, with RSS 2.0 still common among publishers. Atom is a related syndication format with different element names and namespaces. A practical aggregator should support both, even if the first test feed uses RSS. In Australia, a reader might combine technology publications with feeds from the ABC, a Melbourne developer group, or a local business site, so relying on one exact XML structure would make the application unnecessarily fragile.

The application can use a simple sequence: retrieve a document, parse it, convert each entry to a common internal shape, save it, and render the saved records. This separation matters because a feed is an external input rather than a trusted database. The display layer should receive predictable fields such as title, URL, summary, author, and publication time, regardless of whether the original source called an item an entry or an item.

Preparing The PHP Environment

A small PHP installation with SimpleXML and cURL is enough for the core project. SimpleXML provides a convenient interface to XML nodes, while cURL gives better control over redirects, timeouts, response headers, and user-agent identification than a basic file request. On an older shared host, check that both extensions are enabled before writing application code. A command such as phpinfo() can expose configuration details, but remove the diagnostic page once testing is complete.

Create a configuration file outside the public document directory when the hosting arrangement permits it. Database credentials, feed addresses, cache paths, and the application’s time zone should not be mixed into the template files. Australian sites may serve readers in several time zones, from Perth to Brisbane and Sydney, so store dates in UTC and convert them for presentation rather than treating server time as a reliable local clock.

The database schema can remain small. A feeds table needs a name, URL, last retrieval time, and status message. An entries table can contain the feed identifier, title, canonical URL, summary, author, source publication date, and a unique fingerprint. A unique constraint on the combination of feed and URL prevents repeated downloads from creating duplicate rows when a publisher changes the order of its items.

Fetching Feeds Reliably

Network requests deserve careful limits. Set a connection timeout and a total timeout, follow redirects selectively, and reject responses that are far larger than the application expects. A feed is usually small, but a broken endpoint can return an entire web page or a generated file several megabytes in size. Check the HTTP status code and content type where possible, then record a useful error instead of allowing the request to stall the entire refresh process.

A clear user agent helps feed publishers identify the application. Something like PliantReader/1.0 (+contact@example.com) is preferable to an empty or misleading browser identity, provided the contact address is real. Fetch each source independently so an unavailable server does not prevent healthy sources from updating. A temporary failure from an Adelaide-hosted site should leave its previous entries visible while the rest of the collection continues to refresh.

Caching reduces unnecessary traffic and makes the reader faster. Store the response body or the relevant HTTP headers, then use ETag and Last-Modified values on subsequent requests. If a server returns 304 Not Modified, retain the existing records and update the check time. A scheduled command-line job is usually better than refreshing all feeds during a visitor’s page request, especially on low-cost Australian shared hosting where execution time and outgoing connections may be restricted.

Parsing RSS And Atom

SimpleXML makes a basic RSS loop compact, but compact code should not become careless code. An RSS 2.0 document commonly places entries under channel/item, with title, link, description, guid, and pubDate fields. Atom generally stores entries under feed/entry, with title, summary, content, id, updated, and a link represented by an attribute. Namespaces mean that Atom elements may need to be accessed through children() with the correct namespace URI.

Convert both formats into one array before writing to the database. For every entry, identify a title, choose the best available link, preserve a stable identifier, and parse the publication date. If a source supplies no date, use the retrieval time as a fallback while marking the record as approximate. If a feed provides an invalid URL, skip the entry rather than creating a link that points to the aggregator itself or to an unsafe scheme.

Descriptions and summaries often contain HTML. That content should be treated as untrusted markup. Strip unsupported tags, allow a small set of harmless formatting elements if necessary, and escape the final value before placing it into an HTML document. Titles require the same treatment. htmlspecialchars() with the correct UTF-8 settings prevents a quotation mark, embedded tag, or malformed character sequence from changing the page structure.

Dates require special attention because publishers use several conventions. PHP’s date parser handles many RFC-style values, but the application should catch exceptions and retain the original text for diagnostics. Display a friendly local value such as “18 June, 9:40 am” while retaining an ISO timestamp for sorting. Australian daylight saving changes affect Sydney and Melbourne but not Brisbane or Perth, which is another reason to store absolute UTC values rather than manually adding a fixed offset.

Storing And Normalising Entries

Normalisation is the step that turns inconsistent publisher data into useful application data. Trim whitespace, decode XML entities once, remove control characters, and impose reasonable length limits on titles and summaries. Resolve relative URLs against the feed’s own address when required, and permit only http and https links. A canonical URL can be cleaned of harmless tracking parameters, though aggressive rewriting may accidentally merge two legitimate pages.

A URL is often a practical duplicate key, but it is not perfect. Some publishers change tracking parameters, while others reuse a URL for a page that has been updated. Combine the normalised URL with a feed-specific identifier when the source provides one. If neither value is stable, calculate a hash from the title, source name, and publication date. Store the original link as well, so debugging does not become guesswork when a publisher changes its feed format.

Use prepared statements for every insert and update. A transaction around one feed’s entries keeps its records consistent if parsing fails halfway through. An update-or-insert operation can refresh a changed title or summary without creating another copy. Keep a last_seen value so older entries can be removed according to a retention rule, such as keeping ninety days for a personal reader and thirty days for a lightweight office dashboard.

This local archive provides a useful advantage over reading feeds directly in the browser. The page remains available when a publisher is slow, and readers can browse a known set of records without exposing their IP address to every source on each visit. It also enables simple search and read-state features later without changing the retrieval process.

Rendering A Useful Reading View

The front end should make scanning easy. Present the source name, headline, publication time, and a short summary in a consistent card or list. Keep the original article link prominent, and distinguish unread items with a restrained visual change rather than a collection of bright labels. A calm layout suits a reading archive: generous spacing, a narrow text measure, and clear typography are more useful than decorative controls.

Use semantic HTML so the page works across desktop browsers, tablets, and older machines. A heading for each entry, an ordered or unordered list for the collection, and a real anchor for each source make the structure understandable without client-side scripting. A developer checking headlines between trains in Sydney may have a narrow mobile connection, so avoid loading large images or requiring JavaScript merely to reveal article titles.

Filtering can remain simple at first. A source selector, a date range, and a text search over stored titles are enough for many users. Paginate the results rather than sending the entire archive in one response. If the interface displays a summary derived from feed HTML, truncate by words after sanitising it, not by cutting arbitrary bytes, since UTF-8 characters can be damaged by careless string operations.

Accessibility belongs in the initial design rather than as a late repair. Maintain visible keyboard focus, provide meaningful link text, preserve adequate contrast, and avoid communicating state through colour alone. If a feed fails, show a short status beside its name and keep its last successful entries available. A clear “updated two hours ago” message is more informative than hiding the source and leaving readers to wonder whether the application is broken.

Protecting The Application

Feed retrieval creates a boundary between your server and arbitrary remote content. Disable dangerous XML features, reject unexpected protocols, and apply limits to document size and parsing time. Modern PHP libraries handle many XML concerns safely, but the application should still validate inputs and keep dependencies current. Avoid accepting a feed URL directly from a public form unless the request is protected against server-side request forgery.

When adding an administration page, require authentication and use a request token for actions such as deleting a feed or forcing a refresh. Escape output at the point where it is rendered, not only when it enters the database. Database escaping and HTML escaping solve different problems. The defensive habits are similar to those used in a small antivirus scanner: define the input boundary, inspect data before trusting it, and fail in a controlled manner.

Logging should record the feed address, response status, elapsed time, parser errors, and the number of entries accepted. Do not write complete response bodies or database passwords into ordinary logs. Rotate logs and provide a modest error page to visitors while keeping detailed diagnostics for the administrator. A recurring timeout from one host may indicate a remote problem, an incorrect DNS record, or a local hosting restriction, and these possibilities are easier to separate with consistent records.

Testing And Maintaining The Reader

Test with more than one ideal feed. Keep fixtures for RSS 2.0, Atom, an empty channel, missing dates, duplicate items, invalid XML, non-ASCII characters, and a feed containing HTML in its summary. Include titles with accents and characters used in Australian place names, such as “Wollongong” or “Kūmara”, to confirm that database and page encoding remain UTF-8 from retrieval to display.

A useful test run should verify that a failed source does not erase old entries, a 304 response does not create duplicates, and a changed summary updates the existing record. Check sorting around midnight UTC and during daylight saving transitions. A post published late in Perth should not unexpectedly appear ahead of a newer item from Melbourne because the application compared formatted display strings rather than stored timestamps.

Maintenance is mostly routine observation. Review the feed list when publishers retire endpoints, lower the refresh frequency for rarely changing sources, and remove entries that no longer serve a purpose. Respect publisher terms and reasonable request intervals. For a small Australian readership, half-hourly retrieval is generally sufficient; fetching every minute provides little benefit while increasing load on both the aggregator and the sources.

The finished PHP RSS aggregator is valuable because its parts remain visible. Retrieval, parsing, normalisation, persistence, and presentation can each be understood, tested, and replaced independently. That clarity makes the application a useful exercise in web programming and a dependable personal reading tool, whether it runs on a home server in Brisbane or a modest hosting account shared with several other projects.