Sucking the elevation map out of filename.scn

[Romaq]

My 'filename.scn' has the string 'ZLIB$' in it, and I've seen it in other files. I surmise that some form of elevation map is stored digitally within the .scn file as a compressed object. Unfortunately for me I'm not clever enough yet to extract that data. I don't find a program for sucking a DEM out of http://caesar4.heavengames.com/downloads/lister.php?category=misc either. Has anyone been able to do this? I would really like to have access to that data for making better color and blend maps. The terrain editor in the game isn't the same as having a reference bump-map in GIMP I can toggle while painting landscape.

--Romaq

[Pecunia]

Yes, I know where the heightmap data is stored in the .scn file, but hadn't written a program to extract it yet. I've slapped together a small program, its output is a greyscale image representing the height of the terrain at any point on the map: dark = sea, light = mountains. I've attached an example image from Corinth ... does this come close to what you had in mind? If so, I can make this program ready for release; if not, let me know what you'd like to see different :)

[Romaq]

Oh deary yes! That's exactly what I had in mind, but my hope is to get something that does both extraction and insertion. My goal is to take Virunum's map with C4_Apron_NE_01.GR2 and redo the map. It turns out the Virunum is a perfect fit to work from, but I really want to make it 'my map' and not just a hack copy of Virunum.

To accomplish what I *hope*, I would like to use an image like yours not only as a reference, but as a means to fine-tune the height-map and reinsert it BACK into the .scm. I've done this for a SimCity 4 region on 16 bit greyscale, so-called 'Deep Grey'. So if the output data is in 8 or 16 bit numbers, that won't be a bother at all. I suspect it's 8-bit data, 0 (-15m below water) to 255 (240m above mean sea), and values above 240m are simply not possible within the heightmap data.

Anyway, I'm rather excited to know you can pull that data out, and I hope you are able to reinsert modified data as well. That would be quite the boon.

Thank you for looking into this!

EDIT: 16 bit data. I should have said 16 bit data because it *can* do 1/10th of a meter. -15.0 would be the floor, and going to 0 would be 150 steps total. Clearly too fine for 8-bit values. If you can do output to 16-bit PNG or other 16-bit format, that would be the trick for me. If your toolkit doesn't include 16-bit image output, PGM (http://netpbm.sourceforge.net/doc/pgm.html) is a way to shove the data out using a text file only. I'll convert to PNG and back to PGM. It's painful, but it's a step closer to the goal.

I hope this sort of export/ import business is built into the SC:S editor when it ships.

--Romaq

[Pecunia]

To accomplish what I *hope*, I would like to use an image like yours not only as a reference, but as a means to fine-tune the height-map and reinsert it BACK into the .scm.
I've given some thought to that myself a while ago: unpack the .scn, change some things, and assemble everything back again. I have not succeeded in doing this :( The resulting .scn gave fatal errors and shut down C4 when I attempted to load it. I suspect that there's some checksum stuff to check for cheats and other alterations for online play: the .c4s format is basically the same as the .scn, but with gameplay info added. So, I'm afraid importing data back into the .scn is not going to be possible :(

The height values are stored as floating point numbers (32-bit floats) in the file, though only the range from -15 to ~+16 is used. Currently I've mapped those to the 0-255 range of the 8-bit png image. I'll look into outputting 16-bit PNGs... since I'm using the PNG library directly, that shouldn't be a problem, but finding out how to do it is always a nice puzzle ;)

One question for you: would you be OK with working with a command-line tool or would you like to have a point-and-click user interface? The program I have now is command-line, but building a simple .NET interface on top of it won't be too much work :)

[Romaq]

A command-line tool would be just fine. 16-bit output doens't help for reference images, it only helps for reinsertion. Since reinsertion isn't isn't an issue, usefulness as a 'reference image' might be more a focus.

There are two important things to know for reference: is this under water? Is this buildable? If the program has an option create a simple 'waterline/ buildable' mask consisting of just white (buildable), blue (0 and below) and red (3 tile outline + slope filter) that would be quite useful for people painting according to those issues. I'm not sure if you can pull off the 'no build' detector, and if you can't don't worry about it, and I can do my own water mask from a height-map. But if you have edge detection/ gradient tools in the toolkit and it happens to match the no-build mask, that would be a Boon of Jupiter. If you don't have the time or interest in a 'buildable' mask, it's trivial to make the mask manually using flood-fill. There are usually only 1 or 2 levels worthy of proper building on most maps.

As to height-mapping, since proper reinsertion is not an issue, the height-map would be more useful taking the min-max 32-bit float values -15 to ~+16 and scaling them to cover all 0 to 255 8-bit int. Part of what I wanted to do was make a bump map of the height-map and use the bump map as a guide for painting draws and bluffs. 16-bit images don't help for this sort of rough reference, and I can not manipulate 16-bit images 'easily'. GIMP won't do it and CinePaint is somewhere between ugly and awful.

I'm just fine with command-line tools, and it would give me a chance to play with how it works and give you quick feedback. However, to be more generally useful, perhaps it should be part of the mini-mapper? You could have a click-button toggle off/ on the building references, toggle off-on buildable and water references, and toggle off-on between the 'color map' image and the height-map image. That would make the mini-mapper more generally useful. :)

I hope these issues catch the attention of the TM Devs for them to consider MAKING tools that do this SC:S, or better yet building this functionality into the SC:S editor.

Thank you for taking the time to do this. The simple -15m to ~+16m --> 0 to 255 8-bit PNG image command-line extractor would be a gem, and everything else I want I can do. But of course, if I don't ask for more I can't be told 'no'. :-)

--Romaq

[Pecunia]

Attached is a water/buildable/not buildable map of corinth... is that something like you had in mind? :D I've checked it against the "buildable tiles" button in the editor and it seems to be correct.

I'll attempt to port the two command-line programs (one for the heightmap, one for the buildable tiles) to windows tomorrow.. I do my primary development on Linux, so I'll have to do some work to make it compile in visual studio.

And yes, putting this into the minimapper is a good idea. However, I need to figure out how to make that work: the minimapper is written in managed C++/.NET, these little programs are written in plain C. Since the minimapper was the first .NET program I wrote, it's a little... badly structured and not easy to add features to.

[Romaq]

Excellent! Yes, that's what I was hoping could be done. I have a Linux 'headless' machine right beside the Windows one and I initially used it to look at the .scn file, but I didn't know enough of what I was looking at to pull the height map out.

Thank you for taking the time to do this.

--Romaq

[Pecunia]

I attached a zip file with the two command-line applications and a short readme. Let me know what you think of it :)

[Romaq]

It's excellent. With the reference material I'll be able to add 'deep shadow' hints to the 2048 color image, as well as do other masking and planning work.

Thank you for your time!

--Romaq

[Pecunia]

No problem... I like writing such little programs, and I like it even more if I know that some people actually find them useful :)

I'll look forward to your scenario(s)!

[Romaq]

I have several things going on with the Scenario I have to work through. It's a HARD core military one that takes place in a specific historical context, even though it's a made-up city. It's likely a scenario that *I* couldn't play, I don't have the skills to build up a city in the timeframe AND have a good defensive army. But it needs to be PLAYABLE, and for that I have to research other scenarios of that type and see what is reasonable where you expect people to play and win, and feel like they where challenged.

If anyone knows of a scenario that has a military build-up to a massive invasion in a very short timeframe, AND it's one where good players can win without cheating, I wouldn't mind borrowing from the template. And if anyone wouldn't mind talking with me over PM about it, that would be great too. But I don't want to give out too much in spoilers about it. Or does it matter? Should I just go ahead and start a thread about what I *WANT* to accomplish, and get help with just the right pace and resources for this thing? Not worry about spoilers?

--Romaq

[aramann]

PM Goonsquad, and check out CBC... http://citybuildingcontests.net/
Goonsquad's current scenario/contest has these things, along with some excellent scripting. Even if you don's submit for the contest, please do download it and play, and then PM goonsquad some feedback! (I tested for him and am highly involved with this particular contest)

[Hieronymus]

@Romaq - feel free to PM me if you need any help with scripting.

I like writing such little programs, and I like it even more if I know that some people actually find them useful :)
Your C4 scenario checker is not just useful - to me it's now indispensable!

[Romaq]

Thank you. It's not the scripting at this point, but the 'script', taking the story outline and figuring out the pace of events so that the scenario is tough, but not impossible. I could pace a 'Full Roman Assault' invasion one year after you start the city and forbid bribes, but that's not going to be particularly 'fun'.

I want to make the player sweat, bleed and cry within a two to three year period of starting the scenario, but be able to survive it. For various loose definitions of the word 'survive'. So I'm looking for as close a fit scenario as I can for paceing issues. The scenario has to have build-up attacks within that two year to three year period, and the player experienced with military scenarios needs to believe they can survive it. That isn't a 'scripting' issue anyone can help me out with. It's just 'the script'.

--Romaq