I just launched a new version of my Trademe stuff. It’s on its own domain now, with a shiny new look. Go check it out now: http://metatrade.co.nz
Screenshot below for those people too lazy to click links:
As you can see, it has a shiny new search interface. Behind the scenes, that set of filters is being turned into the query string ‘size > 40 and size < 70 and not (town like “%Hamilton%”) and bid < 1500 and bid > 1000 ‘. The query grammar is…fairly complete.
It also does various other interesting things. Go see.
Just a quick note to say that the site now works in most (all?) varieties of IE. So, if you tried it yesterday and it was horrifically broken, try it again now 
Last night I was messing around with my Trademe parser, and almost by accident managed to create something useful. By parsing data on past auctions using the ‘advanced’ search, I was able to compute the average closing price for a particular type of item (dependent on how useful the name is, of course). This is quite neat, as I can now filter on items below the average price – ie: items that are cheaper than average. Combined with a bit of CSS, this was quite easy to turn into a simple firstin.co.nz / 1day.co.nz clone, that automatically shows ‘good’ (defined as at least 10% below the average price) buy-now deals in various Trademe categories.
Because I know that people don’t read the text in these posts and instead only look at the images I embed, here’s an image of what the site looks like at the time of writing:
Click on the picture or here to go to the actual site. Despite the name up the top, it’s not hosted on a standalone domain yet. That site actually turned out to be unavailable, so I’m attempting to think of a better name. Suggestions welcome
So, a while back we spent a little while working on a system to generate and do some fairly advanced filtering on RSS feeds of Trademe search listings (available in user-friendly form here). It’s been pretty neglected since then, although we’ve fixed the odd bug and added a few tweaks here and there. Anyway, recently I had a reason to glance at the log files, and was suprised to see it was getting a fair number of hits – over 12k in April alone. Granted, a lot of this is due to the fact that some feed readers make requests far too often, but still
Because everyone likes graphs, here’s a more detailed breakdown:
Trademe RSS requests/day
(click the image or here for the raw data)
Note that these are only successful requests. If TM denied my request, or Apache decided to lock up out of spite, or (most likely) I was messing around with the script and broke it, it doesn’t get logged. I suspect that may account for the dips. Not entirely sure what happened with that big spike though.
Anyway, should anyone want it, the source is here. It’s fairly rough in places, be warned:
http://bieh.net/svn/tmscraper
Running ‘./scraper.py http://trademe.co.nz’ may be a good place to start.
