Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / Large Text File DB

Author
Message
Rknight
22
Years of Service
User Offline
Joined: 25th Sep 2003
Location: NJ
Posted: 31st Aug 2010 14:42
Link
I have a large, nearly 2MB and growing text file database.

It's a simple file for a random quote php machine, 'quote:' and 'author:' are the marker tags for each entry.

While I'd love to write some php myself to navigate and expand it and work with through a website, DBP is all I know at the moment. Might offer to pay someone to do it if I can find a cheap help,

I'd like to write a little front end to add/subtract/edit quotes in it, and generate statistics. But reading through this file takes forever.

What do I need to programming wise do to efficiently and quickly navigate this database?
Back to top
Profile PM
DVader
22
Years of Service
User Offline
Joined: 28th Jan 2004
Location:
Posted: 31st Aug 2010 15:01
Link
One large file not a array with individual quotes? If so, I would probably recommend that you read that file in once at the beginning and break it up into a more manageable array. Then after changes save the whole file again. You should be able to use DB ok as standard, but IanM's utils have some good commands for this and will help you considerably. Plus, unless things have changed, I think there is a max length of string$ size in DB, which caused me problems messing with a csv file a few years back. The plug in removes that problem.
Dark data is another possiblity, but not free and I have never used it, so cannot really give much info on it.

http://s6.bitefight.org/c.php?uid=103081
Back to top
Profile PM Website
thenerd
17
Years of Service
User Offline
Joined: 9th Mar 2009
Location: Boston, USA
Posted: 1st Sep 2010 04:35
Link
Don't use dark basic. I'd recommend you look at another language, DB isn't at all designed to handle things like this. I'd advise you take a look at Python, it's a very easy language to learn, and can do just what you want.


Back to top
Profile PM Email
Grog Grueslayer
Valued Member
21
Years of Service
User Offline
Joined: 30th May 2005
Playing: Green Hell
Posted: 1st Sep 2010 05:49
Link
Quote: "DB isn't at all designed to handle things like this. "


That does appear true when you look at the standard file commands that come with Darkbasic Pro but as DVader suggested IanMs Matrix 1 Utilities Plugin Darkbasic can and does handle files very well. His database commands make working with large files easy. You can read data from anywhere in the file and edit it, then close it whatever you want to do without having to resave the entire file.

IanMs Matrix 1 Utilities Plugin has much more than file commands and should be downloaded asap if you don't have it.

http://forum.thegamecreators.com/?m=forum_view&t=85209&b=18

Back to top
Profile PM Email
Rknight
22
Years of Service
User Offline
Joined: 25th Sep 2003
Location: NJ
Posted: 1st Sep 2010 06:08
Link
Hmm, yes, Ian's memory bank functions look like they might do the trick, I will test them out. Thanks!

I don't want to learn a new programming language to do a small job like this unless I have to. I'd be open to tinkering with some pre-existing program that is close to what I want, but not writing a whole new one in a new language. I'm not a fast thinking programmer, I'm a plodding one.
Back to top
Profile PM
Kevin Picone
23
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 1st Sep 2010 06:21
Link
Assuming there's peekstring (from mem block) command, it'd probably be quicker to load the whole text file into a bank, then parse it manually.


New PlayBasic Learning Edition Released 24th April 2010
Back to top
Profile PM Website
Rknight
22
Years of Service
User Offline
Joined: 25th Sep 2003
Location: NJ
Posted: 1st Sep 2010 14:00
Link
Well, trying the bank commands, but for some reason I can't get the bank size, so all my 'search bank' commands eventually crash when they the search position value goes over the bank size. 'bank position outside range error'

Odd. Neither 'file size' or 'get bank size' are useful. File Size always gives a zero, and the bank size doesn't matter in determining the error, it gives an oversize size, or the oversize size I set, I 'map file to bank' but the size is off compared to the file.

In fact 'file exist' gives a zero too, even though the file exists, the directory is set perfectly.. it's being read by the map bank command.. it's the app's directory.. I must be missing something..
Back to top
Profile PM
IanM
Retired Moderator
23
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 1st Sep 2010 15:03
Link
You certainly appear to be missing something somewhere.

The help file generation program that I use reads all of the source data by mapping the file into a bank and then processes the data by reading it as a string and parsing it. Writing is similar, map a file to a bank but giving the size of the data to pre-size the file.

So I'm pretty sure that plug-in doesn't have any obvious bugs in it

Perhaps a little code will help us to determine what the problem is.

Utility plug-ins (26-JUL-2010)
I'm applying terms of use that require you to wear a red nose and honk a horn whenever you use the Internet
Back to top
Profile PM Email Website
Back to top

Login to post a reply

Server time is: 2026-07-23 12:10:35
Your offset time is: 2026-07-23 12:10:35