link to this article

A Major Feature has been extended (GNU+Linux only, at the moment)

This is a frontline report. Frontline because I am telling you of a feature that is currently not portable and runs on *nix Operating Systems only, at the moment. Anyway, I am happy that the technique works, even if it's so slow for very large datasets that Path Tracing is not an option, yet. But I have to say I haven't really tested it with smaller heightmaps so far (that is for me heightmaps that are smaller than 16,000^2 pixels).

What I am talking about is the "Explicit Paging" technique mentioned in the last news. It means that only parts of the heightmaps are kept in actual memory, while most of it is stored on your harddisk. This keeps picogen's memory footprint small, which is a Good Thing if your render goes on in the background.

Additionally the new tec makes it possible to have more (depends on how much harddisk space you can afford) heightmap than is addressible on a 32bit-CPU, which is important because 32bit are sucked up fast if you render terrain.

So, here's the updated todo list (porting the tec to Windows is an implementation detail;)):

• Lazy Building: Build nodes in the hierarchy only if they are actually needed. Tests have shown that the size of the Quadtree can be reduced to 1/6th of the original size, and even less.
• (solved) Explicit Paging: Enable storage of rarely used nodes on harddisk (or similar) devices so that RAM pressure is reduced.
• Sub Pixel LOD: It is common that nodes in the Quadtree are much smaller than a Pixel, so determine the maximum (sane) level of detail for nodes. This will also help out the Explicit Paging above.

And now, ppl, the renditions!

32769² = 2147614722 ~= 2148 MTriangles ~= 2.1 GTriangles (next one will be 8 Gtris!)

grrr, 32k^2!

heat

Orodruin

link to this article

A Major Feature has been added to picogen

Last week I sat down and wrote yet another incarnation of a Quadtree Acceleration Structure. A quadtree is a hierarchy of nodes, which makes selecting those parts of the heightmap that actually contribute to the image very fast and effective.

A primitive like Quadtree enables the users of picogen to render heightmaps of high resolution in a relatively small amount of time. The biggest limit I discovered was actually the address space of 32bit CPUs, and as such CPUs are not uncommon even in these times with 64bit CPUs I have to add several features to the Quadtree:

• Lazy Building: Build nodes in the hierarchy only if they are actually needed. Tests have shown that the size of the Quadtree can be reduced to 1/6th of the original size, and even less.
• Explicit Paging: Enable storage of rarely used nodes on harddisk (or similar) devices so that RAM pressure is reduced.
• Sub Pixel LOD: It is common that nodes in the Quadtree are much smaller than a Pixel, so determine the maximum (sane) level of detail for nodes. This will also help out the Explicit Paging above.

8193² ~= 130 MTriangles

This image reminds of a place where there has been a glacier hundreds of years ago, malming this chaos into stone.

Rendered with picogen. This image is a presentation of the quad-tree implementation I have hacked this week. It was capable of rendering this image with 800 samples per pixel in around 7 hours, on one crappy Athlon XP singlecore (2400+). The resolution of the heightfield is 8193*8193, or around 130 Million Triangles. I believe in that I can render 16385*16385 on my box, with smaller nodes maybe even more.

Note that this quadtree has to do with the upcoming release of picogen 0.2, the first release that is capable of rendering userspace stuff; until now all images were pure programmers art, hardcoded into the sourcecode; what you see heree is fully defined in a so called SSDF-file (==static scene description format), exactly like this:

The actual scene definition file for this image is rather small:

list{
  brdf = lambertian(reflectance:0.8)
  hs-heightfield(resolution:8193; center:0,-5,0; size:50,5,50; code: (^ (- 1 (abs ([2 LayeredNoise frequency(15) layercount(12) seed(123) persistence((* 0.56 (-1 (abs x))))] (* x 0.5) (* y 0.5)))) 3) )
}
camera-yaw-pitch-roll(position:20,-4.0,-20.0; yaw:-0.7; pitch:0.0)
sunsky (turbidity:2.2)
sunsky (direction: 0.01,0.7,1.0)
sunsky (color-filter-rgb: 0.2, 0.2, 0.2 )
sunsky (sun-color-rgb: 900, 800, 700)
sunsky (fog-exponent: 0.05; fog-max-distance:100000)
    

16385² ~= 536 Million Triangles

interesting: i had to boot up my box in windoze because i had to scan some sheet of paper. so i did hibernate my running session, where 'session' includes the ongoing rendition of the above image. i do my hibernation via s2disk, which, unllike tuxonice, does not pump everything back into ram after un-hibernation. so the job of paging stuff from swap back into RAM is left to the kernel.

the point now is: the above render took 2.7 GiB of my 2.0 GiB RAM, what is above is paged out into swap, actually there was 1.8 GiB in RAM, before hibernation. after un-hibernation and relaxing back, picogen was running in only 470 MB. so what I conclude is that even in this scene, where i set the size to xyz(50.0, 5.0, 50.0), only a good 1/6 of the acceleration structure is actually needed (and so paged back into RAM). what i condlude from that is that it should be highly worthful to support some kind of explicit paging directly in the AS, as Thatcher Ulrich suggested it.

when you think of it, and that 1/6 ratio is common for such scenes (but i expect even lower ratios for larger terrain) good paging should enable me to render 32^2k-heightmaps and more, resulting in 2 GiTris and more .... *buff* :D

some stats:

• 666 samples, so this is an unholy rendition (pure accident, really)
• 8 hours of rendition (including 30 minutes of quad-tree creation (itself including the sampling of the heightfunction))
• one core (Athlon XP 2200+, uh oh)

Rendition vs. Heightmap

Here you can see a comparison of a top-down rendition of a heightmap, and the according heightmap.

Finis

If I made you at least a bit snoopy, have a look at the Roadmap.

Thanks for keeping up, ciao!

link to this article

I have put up a new version of the roadmap, a plain ascii paste, in the simplest possible format.

link to this article

Version 0.1 of picogen is out

picogen 0.1 is out. It is a strange release, as there are no rendering capabilites contained, but "only" the tool "mkheightmap". But believe me, it's worth looking at it, since mkheightmap is quite a powerful tool for the task of heightmap-creation.

Documentation, esp. for mkheightmap, is now available, too.

link to this article

The Galleries are now online

Finally the galleries are online[0]. Dive down into the different galleries and see yourself what picogen is about. If you want to comment on some of the images, please feel free to send them to me (seb at greenhybrid dot net), and if you want, they will be published here.