Simplepie database caching
From Steeple
[edit] 1 simple pie database caching
When parsing an rss feed, simplepie only caches the whole feed on disk. This will work will work for smaller number of items (~ 50, 100, 1000?). After that it will get too slow (or, with increased load, performance becomes an issue). The idea is here is to improve simple pie by inserting a database caching mechanism, to buffer the feed, so that it doesn't need to be accessed via simplepie every time the page is loaded.
This has the advantage that the caching is 'transparently', as an extension to simplepie, so that feed data is deposited in a database, and then read from the database, unless the feed is refreshed. The feed access methods should be exactly the same as if accessing a feed directly.
Database caching is planned for simple pie 2, see 'Caching Module' on
- http://simplepie.org/wiki/sp2/goals,
- caching would use cache core http://code.google.com/p/cachecore/.
- http://uk3.php.net/manual/en/intro.pdo.php
- http://en.wikipedia.org/wiki/Create,_read,_update_and_delete
- http://en.wikipedia.org/wiki/Representational_State_Transfer
The basic question is: Suppose you have 10,000 items. How do you load these effciently? How do you efficiently query them? How do you display subsets, such as 20 items making up a podcast feed, or a single item?
Requirements:
- The above caching method would need to allow the extraction of a single item, without having to read the whole database.
- Single items need to be retrievable by guid ("Get the item where guid=something")
- Sets of items need to be retrievalbe by equality ("Get the list of items where category[domain=something]=something")
- Sets of items need to be retrievalbe by basic searching ("Get the list of items where
category[domain=something] contains something")
Desirable:
- Sets of items need to be retrievalbe by more advanced searching ("Get the list of items where
category[domain=something] contains (something or something) AND something")
Some of these would only be possible in simple pie by iteration, ie. loading all elements and then searching. If this is the only option after database caching it's probably too inefficient. I.e. access to the feed must leverage database features, such as searching.
[edit] 2 Steps
Set up LAMP with at lest PHP 5.1. Obtain http://simplepie.org/wiki/sp2/goals,
Obtain a large media rss feed (e.g. from Steeple or otherwise). Set up basic parsing with simplepie, and check performance, e.g. for displaying a single item out of that feed.
Implement db caching.
