UPDATE: OK, so there were a couple of minor teething troubles, mostly to do with me breaking the buy-now option. That’s now fixed, I hope. I did some more work on the parser, and it should work properly for Property, Motors, and kinda sorta works for Jobs. TMJobs doesn’t look like it was made by the same people as the rest of it, so the names and everything are different, causing it not to play nice with the rest of my system.
Original Post: Today I got annoyed at my local auction site Trademe for not offering RSS feeds of anything. What I wanted was feeds notifying me of new search results and listings in categories – yes, TM offer email notifications for both these cases, but I spend a lot more time in Google Reader than I do in my email client. That, and their email only comes once a day, whereas RSS can be updated a lot more frequently.
I also wanted to (optionally) only be notified when a buy-now was available below some amount. Basically I want to see cheap stuff before anyone else has a chance to notice it, but I don’t want to do any of the work normally associated with that (ie: hitting F5 a lot)
Trademe do have a fairly limited public API. However, the closest I could find to what I wanted was this post, which wasn’t quite what I was after.
So I wrote a small script that downloads a trademe URL and extract the relevant information into a simple computer-readable format. Then I wrote another one that reads that format and renders it to RSS. After that I made it into a CGI. Then I needed a simple way of building up the URLs to the script with the correct escaping and parameters so I ran up a quick webpage for it. Then people complained that the webpage looked awful and kept asking the same questions, so I made the page a bit more complicated. It was a bit of a slippery slope, if I’m to be honest…
Anyway, I’ve put up a copy of it on this server to act as a stable and theoretically usable version. If anyone wants to use it, it’s here:
Trademe RSS Generator
I’m quite happy with the end result. By using feeds like this one, I automatically get notified of cheap PS3 games, for example
If anyone has any other suggestions as to what I could do with live machine-readable data of TM listings, feel free to leave a comment below.