Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / darkdata problem.

Author
Message
OldPMan
TGC Store Seller
17
Years of Service
User Offline
Joined: 10th Aug 2008
Location:
Posted: 17th Nov 2009 19:39 Edited at: 30th Nov 2009 21:28
dark data not understand the file number is more than 1.000.000 - DFS Create 1000000, "items.dat", 1 why?

.....already beside..... for all
OldPMan
TGC Store Seller
17
Years of Service
User Offline
Joined: 10th Aug 2008
Location:
Posted: 30th Nov 2009 21:33 Edited at: 30th Nov 2009 21:34
it possible to increase the number of file numbers is understood in DARKDATA COMMAND DFS Create. I need At least 20 000 000 files.

.....already beside..... for all
Mobiius
Valued Member
23
Years of Service
User Offline
Joined: 27th Feb 2003
Location: The Cold North
Posted: 1st Dec 2009 11:46
Quote: "I need At least 20 000 000 files."

Thats a stupid amount of data! Twenty million files? Damn!

Your signature has been erased by a mod yet again
OldPMan
TGC Store Seller
17
Years of Service
User Offline
Joined: 10th Aug 2008
Location:
Posted: 1st Dec 2009 14:07
Well, let's say I have a play area broken into squares.
768x768 squares.
13 dfs and kfs files in each.
Each square has a number of databases which are described in this square contained the objects of their material and physical properties.
and to handle such a number of databases, I had to divide the playing area into several parts, so dark data to work with the desired number of files.

but this is due to the fact that at one point must be dynamically loaded dozens of databases and made them work

.....already beside..... for all
Chiblue
16
Years of Service
User Offline
Joined: 6th Nov 2009
Location: NE PA, USA
Posted: 20th Jan 2010 17:02
Why can't you use a singly file structure... have a data file to represent the specifics of a particular cell location.. i.e. using a key like "X0001_Y0001" to "X0768_Y0768" then have all controls specific to the co-ordinate in the file, not sure what you need but say.. material, gravity, etc... then if you need multiple entries for the same location allow duplicate keys and then loop through the records.. You could even use seperate file to break your area into zones.... and store the coordintes of the zone area. say zone 1 is a specific zone that is conttained by locations X and Y, if you had irregular zones use multiple records for each zone..

Zone X1 Y1 X2 Y2
1 20 130 40 200
1 40 130 60 150
2 120 230 150 280

Then have a second or more file using the zone number again that lists the properties of that zone. This will minimize the data reads and each zone could be cover each object you have on that zone...

Not sure about dark data as I have only been using it a short while, but almost all databases have the same performance issue which is when you open and close tables this is usually the bottle nexk, better to open once at the start of the program and then close at the end.
OldPMan
TGC Store Seller
17
Years of Service
User Offline
Joined: 10th Aug 2008
Location:
Posted: 20th Jan 2010 18:08 Edited at: 20th Jan 2010 18:09
Thanks for the tip Chiblue.
I have decided this issue and sorry that I left the topic open.
Use one file for the entire database is not possible because of the very large number of insertion data.
Assume only one square of 589824 will have a few thousand records, each of which has a pair of hundreds of additional records, and so I did not smile we used a single file size of a 30-50 Gigabytes

.....already beside..... for all

Login to post a reply

Server time is: 2026-06-11 13:12:02
Your offset time is: 2026-06-11 13:12:02