Last night I was messing around with my Trademe parser, and almost by accident managed to create something useful. By parsing data on past auctions using the ‘advanced’ search, I was able to compute the average closing price for a particular type of item (dependent on how useful the name is, of course). This is quite neat, as I can now filter on items below the average price – ie: items that are cheaper than average. Combined with a bit of CSS, this was quite easy to turn into a simple firstin.co.nz / 1day.co.nz clone, that automatically shows ‘good’ (defined as at least 10% below the average price) buy-now deals in various Trademe categories.
Because I know that people don’t read the text in these posts and instead only look at the images I embed, here’s an image of what the site looks like at the time of writing:
Click on the picture or here to go to the actual site. Despite the name up the top, it’s not hosted on a standalone domain yet. That site actually turned out to be unavailable, so I’m attempting to think of a better name. Suggestions welcome 
So, a while back we spent a little while working on a system to generate and do some fairly advanced filtering on RSS feeds of Trademe search listings (available in user-friendly form here). It’s been pretty neglected since then, although we’ve fixed the odd bug and added a few tweaks here and there. Anyway, recently I had a reason to glance at the log files, and was suprised to see it was getting a fair number of hits – over 12k in April alone. Granted, a lot of this is due to the fact that some feed readers make requests far too often, but still
Because everyone likes graphs, here’s a more detailed breakdown:
Trademe RSS requests/day
(click the image or here for the raw data)
Note that these are only successful requests. If TM denied my request, or Apache decided to lock up out of spite, or (most likely) I was messing around with the script and broke it, it doesn’t get logged. I suspect that may account for the dips. Not entirely sure what happened with that big spike though.
Anyway, should anyone want it, the source is here. It’s fairly rough in places, be warned:
http://bieh.net/svn/tmscraper
Running ‘./scraper.py http://trademe.co.nz’ may be a good place to start.
I spent a bit of time tweaking the sequence of libnoise modules that generate the terrain. It’s slowly getting to the point I’m happy with the base terrain mesh. The vegetation on top of it, that’s another story
So, stuff to fix (to remind me later):
- Artifacts in the water reflection
- Steal better art assets for trees.
- Still some obvious texture tiling
After some time spent working on other things than Universe, I finally find myself with some free time to devote to it again. For some time I’d been meaning to update the terrain texturing, mainly to add a bunch of new source textures and swap out some of the old ones for better looking ones. The previous pallete was getting a little gray for my liking.
The new textures do suffer slightly from the gray-ness of the old ones as well (a result of the fog and sunlight shaders that need tweaking), but at least they’re greener and more varied. For example, there was one type of grass texture before, now there’s five.
I also tweaked the height generation slightly to give some more varied environments. There’s more flatter areas (such as in the pics above), and the water level is lower meaning there’s more land.
I’m getting somewhat sick of terrain rendering nowdays though. It may soon be time to venture back into space again
Tonight I spent some time playing with the native bindings to OpenGL ES on my shiny new Android-based phone, with the eventual far-off goal of porting the Universe client to it. There’s plenty of information on Google about porting C/C++ command line apps and libraries, but nothing that I could see about 3D applications – the closest was this thread which talked about porting a software renderer. So, I figured I’d write this post up with some tips and perhaps spare someone else some pain
- First I followed the steps on this site to download and build the Android source. I used the agcc wrapper script linked off there to simplify compiling, which seems to work fairly well.
- My test application consists of a coloured triangle, that can be rotated with ‘w’ and ‘d’ on the keyboard. Not very exciting, but it’s enough to test with.
- ’src/androidgl.cpp’ – This is a quick little library of helper functions I put together to ease the pain of setting up a fullscreen GL context and getting keyboard events. It also contains implementations of some GLU functions, as GLES doesn’t provide them.
- It’s still rather buggy – I have the same problem as the software renderer guys have with the application ‘behind’ the GL context showing through. This seems to happen whenever an event is signalled – ie: every time I push a key. There’s also a problem with the screen not refreshing properly when the application exits. Both of these issues could probably be solved by using a minimal java program that would invoke the C++ binary, instead of just running it from the shell.
Anyway, the source is here, should anyone find it helpful:
http://bieh.net/svn/androidgl/
Assuming you have the Android toolchain, agcc script and adb set up properly, ‘make && make run-app‘ should compile, push the binary to the phone over adb and execute it. Note that /sdcard is unhelpfully mounted noexec, so I used /cache instead.
In a previous post, I mentioned that I was working on improving the water rendering, something that I feel has always been somewhat lacking in Universe. I picked it up again tonight and made the necessary improvements to the water reflection to deal properly with the curved reflection plane (remember, it’s a planet…).
Basically what you’re seeing there is:
- Terrain being inversely scaled back towards the center of the planet, then rendered to a low-res reflection texture
- Modulating the reflection texture with some noise to make refraction (which changes with time as I scroll the noise)
- A couple more layers of noise being added to make the ‘waves’. I’m not happy with this bit yet, but it looks better than nothing
It looks a lot nicer in motion as the waves and refraction move about. I should take a video at some point.
So, way back when I started this project, I was using the libnoise library to generate my heightmap, but got rid of it after I couldn’t see a decent way to generate precise enough noise with the noiseutils wrapper I was using. Tonight I looked a little deeper into it, and found out how to get the precision I needed, which actually turned out to be pretty simple – guess I should have looked a bit harder last time. Anyway, I now have a height generation backend that uses libnoise, and it’s producing some fairly nice results:
That’s using mostly ridged fractal noise to generate mountains, with some additional noise modules to add flat bits. I really must work on the ground combat gameplay stuff at some point soon – those mountains would make for a pretty decent Tribes-style battle
I recently did some more work on this old project. It’s now significantly faster and better, with a shiny new CEGUI-based interface. Hopefully this new client will make its way to the WAND project page at some point. In the mean time, here are some screenshots:
I also tried out this new-fangled Youtube thing, although the quality has suffered somewhat in the conversion.
httpv://www.youtube.com/watch?v=KXp2CuOd-Qc&fmt=18
Wordpress doesn’t seem to want to display the high quality version, but here’s a direct link to it.
Despite the long silence since the last post on this project, I’ve actually been doing a bit of work on it on and off. I completely reworked much of the the underlying systems, moving it to a proper engine. While it doesn’t look that much different, it’s much easier to code for. Pics:
I’m using atmosphere shaders based off those in GPU Gems, which still require a bunch of tweaking to look realistic. Even at this stage it looks a lot better than the old hacky atmosphere I used to have though. I’m also playing with water reflections in a couple of those shots, which has proved slightly annoying due to the curved planet surface messing up the standard methods to do reflection.
Behind the scenes there’s also a server that generates an endless procedural universe. It uses perlin noise to generate star and planet clusters and sends the relevent seed data for each planet to the client. As a result, all of the stars in the sky are actually stars with planets, and you can actually fly to them by pointing the camera at them and holding the up arrow for a while – if you have the patience, of course. I’m still working on implementing the different planet types on the client. In one of the screenshots above I’m orbiting a mars-like planet that I flew to from the earth-like one.
While it’s still a very long way off anything game-like, I’m not too far off having at least the ability to fly around the entire universe at will and look at stuff, which I think is kinda neat
I finally got around to upgrading this blog to WP2.7, only a month after it came out.
I also jumped on the stats bandwagon and set up Google Analytics. I’ve got a couple of posts saved up to make sometime when I get around to it, so now I’ll be able to see if anyone actually reads them
