Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / Elevation data .dem, GTOPO30 dem...anyone know how to read?

Author
Message
Vandetta
23
Years of Service
User Offline
Joined: 17th Dec 2002
Location:
Posted: 10th Jun 2003 11:50
Link
Im trying to make a program that will generate 3d landscapes from digital elevation models, specifically the GTOPO30 dems, on the off chance does anyone know how to read these?

If you dont but want some of the gtopo30 elevation data for yourself and give it a shot, have a look at the link below. Do a search for 3dem and microdem for programs that will read the elevation data and generate 3d models.
http://edcdaac.usgs.gov/gtopo30/gtopo30.html
Back to top
Profile PM
Mentor
23
Years of Service
User Offline
Joined: 27th Aug 2002
Location: United Kingdom
Posted: 10th Jun 2003 13:24
Link
this site any help?

http://rockyweb.cr.usgs.gov/nmpstds/demstds.html

I think that gives you the file format for DEM, but you would have to write you own decoder, either that or some 3D CAD packages can handle DEM and you may be able to export as dxf or something then re-import into a modeler and export as 3ds or x, cheers.

Mentor.

Back to top
Profile PM
Vandetta
23
Years of Service
User Offline
Joined: 17th Dec 2002
Location:
Posted: 11th Jun 2003 01:19
Link
Thanks I figured there was a 9/10 chance no one would have a clue what I was talking about I read somewhere that DEMs are just ascii files but when I try reading it with word I get a whole bunch of scrambled junk. Is that what you mean when you say I would have to write my own decoder? I dont think any 3d modeler would handle the GTOPO30 DEMs as they have about 57 million faces when converted into a 3d model. I just need a way to be able to read the file in DBPro. If I read individual bytes into DBPro would it still be scrambled junk? Thanks for the help anyone have any other thoughts?

Back to top
Profile PM
IanM
Retired Moderator
23
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 11th Jun 2003 01:45
Link
I've just had a look at the web site, and don't think it would be too difficult to translate the data.

It just seems to be a standard 16 bit bitmap. The only difficulty is that the data is signed, and DBPro doesn't have an easy way of reading 16 bit signed data. You would have to read into a word, then if the number is above 32767, subtract 65536 to get the correct reading.

You'll need to read the text-format header to get the correct line length, but apart from that, it should be fairly simple.
Back to top
Profile PM Email Website
IanM
Retired Moderator
23
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 11th Jun 2003 01:54
Link
Actually, you may need to read byte-by-byte, as the data appears to be in big-endian format and would need converting to little-endian (read as the bytes are in the reverse order of what Intel processors need).
Back to top
Profile PM Email Website
Vandetta
23
Years of Service
User Offline
Joined: 17th Dec 2002
Location:
Posted: 11th Jun 2003 02:47
Link
IanM, I envy you how on earth were you able to deduce all that? Anyways, I'll give your reccomendations a shot. I dont know how much luck I'll have though, you dont have AIM do you, would you be interested in helping me write some code to read it?

Back to top
Profile PM
IanM
Retired Moderator
23
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 11th Jun 2003 04:03
Link
I don't use any messenger services - too intrusive. But feel free to mail me

I'll have a look and see what I can do with the file (I have broadband so it's easy to get a sample data file).

What are you planning to do with the data? It'd be nice to know so that I can tailor the code to what you want.
Back to top
Profile PM Email Website
Vandetta
23
Years of Service
User Offline
Joined: 17th Dec 2002
Location:
Posted: 11th Jun 2003 04:12
Link
Well what I want to do on the whole is make a matrix from the data. But in terms of short term goals I just want to be able to read the data and store the elevation data in an array. Thanks a lot for the help, I appreciate it.

Back to top
Profile PM
IanM
Retired Moderator
23
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 11th Jun 2003 05:02
Link
The data I used was the one containing the UK. Download it, extract the file W020N90.DEM then run the code I've provided.

I suggest that you *don't* try and put the data into an array, because that would quadruple the size of the data you have in memory (from 57MB). Just load the data file like I have, and use the functions I've provided.

Just run my code in any window mode to see the results. It's all commented, so you shouldn't have any trouble following it.

Now, the clockwork engine that I call my brain has wound down for the day - I'm off to bed
Back to top
Profile PM Email Website
Vandetta
23
Years of Service
User Offline
Joined: 17th Dec 2002
Location:
Posted: 11th Jun 2003 05:33
Link
Thank you so much, I'll have to give it a go. Why would reading it into an array quadruple how much memory it takes up? Also how did you figure out how to do this, how did you find out how the file was organized, etc?

Back to top
Profile PM
Vandetta
23
Years of Service
User Offline
Joined: 17th Dec 2002
Location:
Posted: 11th Jun 2003 09:01
Link
IanM, I ran your code. Is this how its supposed to look or am I missing something?



If the pic doesnt work for some reason heres the post I posted on realgametools.com

http://www.realgametools.net/forums/index.php?board=14;action=display;threadid=16058

Back to top
Profile PM
IanM
Retired Moderator
23
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 11th Jun 2003 12:41 Edited at: 11th Jun 2003 12:41
Link
Which file did you download? And did you adjust the rows/columns to the one you used?

The reason that memory usage would quadruple is due to the way the DBPro implements arrays.

The way I worked it out is a secret ... but may have had something to do with the README

Here is what you were supposed to see:



It looks mostly black, but it's actually a very dark green
Back to top
Profile PM Email Website
Vandetta
23
Years of Service
User Offline
Joined: 17th Dec 2002
Location:
Posted: 11th Jun 2003 15:52 Edited at: 11th Jun 2003 15:56
Link
I downloaded the w020n90.dem and checked the header file to make sure the header file. I wonder what went wrong, seems to be working beautifully for you.

....Thats odd, I just took out the sync, and I get the same results, if the sync was taken out shouldnt it just be black?

Back to top
Profile PM
IanM
Retired Moderator
23
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 11th Jun 2003 16:14
Link
I didn't switch to manual syncing, so it should make no difference. I'm suprised that taking out the sync did make a difference - ah, well. It doesn't matter.

All that matters is that you now know how to extract the data you want
Back to top
Profile PM Email Website
Vandetta
23
Years of Service
User Offline
Joined: 17th Dec 2002
Location:
Posted: 11th Jun 2003 20:36
Link
Actually it didnt, I didnt realize you didnt have sync on. I'll go look at the code later to see if I changed anything by accident. You didnt give me the wrong code by chance did you?

Back to top
Profile PM
IanM
Retired Moderator
23
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 11th Jun 2003 22:29
Link
It may be that winzip has corrupted the data while uncompressing it.

Start winzip, select the 'Options/Configuration' menu option. Switch to the miscellaneous tab, and switch off the 'TAR file smart CR/LF conversion' option. Now re-extract the data file.
Back to top
Profile PM Email Website
Vandetta
23
Years of Service
User Offline
Joined: 17th Dec 2002
Location:
Posted: 12th Jun 2003 04:28
Link
YA! that was it thank you thank you, your awesome, superb, stupendous! Please tell me how did you figure out how to do all this with the file?

Back to top
Profile PM
Vandetta
23
Years of Service
User Offline
Joined: 17th Dec 2002
Location:
Posted: 12th Jun 2003 04:29
Link
Ok, I just looked at your profile and think I figured it out, your a c++ analyst programmer. What should I read to be able to figure out how to understand files, etc like you did with this one?

Back to top
Profile PM
IanM
Retired Moderator
23
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 12th Jun 2003 13:00 Edited at: 12th Jun 2003 13:01
Link
It's all down to my vast knowledge of all things computer related ... either that, or the README on the site : http://edcdaac.usgs.gov/gtopo30/README.html

Plus a bit of multi-platform C thrown in for the low-level stuff.
Back to top
Profile PM Email Website
Vandetta
23
Years of Service
User Offline
Joined: 17th Dec 2002
Location:
Posted: 13th Jun 2003 02:36
Link
LOL, thanks. One more question. How come if you loaded it into an array it would quadruple the size taken up in memory. Thanks for all the help.

Back to top
Profile PM
IanM
Retired Moderator
23
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 13th Jun 2003 04:28
Link
Because arrays aren't stored as flat tables like they are in C.

What you actually have is a table of pointers to the actual data itself. So for each 16 bit/2 byte value you need to store, you also need a pointer of 4 bytes - that means a minimum of 6 bytes storage required for a 2 byte value.

So if you already have the data loaded in memory (56MB) you need an extra 112MB of pointers, and 56MB of array storage. Of course, you could eliminate a quarter of that by loading smaller chunks of data from the source file and populating the array piece by piece, but that would be terribly slow.

I'm still undecided whether the way DBPro implements arrays is a good idea or a bad idea, although I've had one or two ideas of hacking the array system for various purposes.
Back to top
Profile PM Email Website
indi
23
Years of Service
User Offline
Joined: 26th Aug 2002
Location: Earth, Brisbane, Australia
Posted: 13th Jun 2003 05:25
Link
I used to play with demview on the mac then create my favorite place into a bryce render fly by.

It looks like good work going on here boys.

http://www.lunarpixel.com

http://www.lunarpixel.com/dt/
Back to top
Profile PM Email
Vandetta
23
Years of Service
User Offline
Joined: 17th Dec 2002
Location:
Posted: 13th Jun 2003 09:03
Link
Ya Im really excited that I can use real elevation data for generating some landscapes now.

Back to top
Profile PM
Vandetta
23
Years of Service
User Offline
Joined: 17th Dec 2002
Location:
Posted: 13th Jun 2003 09:04
Link
IanM, do you care if I post your code in the code snippets section?

Back to top
Profile PM
IanM
Retired Moderator
23
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 13th Jun 2003 12:01
Link
Help yourself

Don't forget to add the advice on the winzip problem that you had - someone else is bound to get the same problem.
Back to top
Profile PM Email Website
Back to top

Login to post a reply

Server time is: 2026-07-18 07:31:10
Your offset time is: 2026-07-18 07:31:10