Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / Advanced Lighting loading times?

Author
Message
CumQuaT
AGK Master
16
Years of Service
User Offline
Joined: 28th Apr 2010
Location: Tasmania, Australia
Posted: 17th Jun 2014 16:32 Edited at: 17th Jun 2014 16:33
Link
Hi all,

So I use the Evolved Advanced Lighting system in my game, and I have a strange thing going on...

During the course of a user playing the game, the advanced lighting system is loaded and cleared multiple times. The game is broken into 3 different area "types"; dungeons, towns and countryside.

When setting up Post Processing in dungeons and countryside areas, the whole process is done in about 4 seconds, but when setting it up in towns for some reason that blows out to about 35 seconds, and I'm not entirely sure why. The code used to set up the advanced lighting and post-processing is the same for each of the three areas - the only difference being the shaders loaded (which happens AFTER the post processing setup anyway).

I thought perhaps it was due to the game assets not being purged correctly beforehand, but that's not the case - I checked... Every memblock, object, image, sprite, effect, etc is being fully cleared before setting up the advanced lighting in each of the three scenarios, meaning the environment should be clear for it to be set up.

=== LONG STORY SHORT ===
Thinking this was odd, I went deeper into the Evolved code and stepped it out to see where it was slowing down, and I found out where the slow-down is happening... There's a part of the code where it opens up the shader files and edits the values in there based on the settings which are being passed in. It does this with a simple "OPEN TO WRITE" process. However, despite the shader file being only about 80 lines of text long, it is taking up to 6 seconds per file to open up and process. And yet, when I run the exact same code in other parts of my program, it doesn't take that long - in fact it takes a 10th of that time or less.

What could cause a file open/write to slow down so much? I'd post code examples, but there's about 94,000 lines lol so basically I'm looking for suggestions on where I should be looking to find the cause of this issue...

I'm aware that this is a bizarre problem, so if I have missed out some vital piece of info please tell me and I'll do my best to answer it. This issue has been driving me mental!


Back to top
Profile PM Email
SamKM
17
Years of Service
User Offline
Joined: 25th May 2009
Location:
Posted: 17th Jun 2014 16:42
Link
Probably not much help, sorry, but you could try replacing all the DBPro file handling commands in the code with the Matrix1Utils Datafile commands, and see if that eliminates the slowdown! If it doesn't, I guess you can rule out some obscure bug with the DBPro file commands, at least. Good luck

The code never bothered me anyway...
Back to top
Profile PM Email
CumQuaT
AGK Master
16
Years of Service
User Offline
Joined: 28th Apr 2010
Location: Tasmania, Australia
Posted: 17th Jun 2014 16:43
Link
It's certainly worth a shot! I've been playing around with the Matrix1Utils lately... Definitely worth a try and not something I'd thought of yet! Thanks SamKM!


Back to top
Profile PM Email
CumQuaT
AGK Master
16
Years of Service
User Offline
Joined: 28th Apr 2010
Location: Tasmania, Australia
Posted: 17th Jun 2014 17:16
Link
Hmm... No, that didn't do the trick, unfortunately...


Back to top
Profile PM Email
SamKM
17
Years of Service
User Offline
Joined: 25th May 2009
Location:
Posted: 17th Jun 2014 18:41 Edited at: 17th Jun 2014 18:43
Link
Oh well
Do you know which specific commands are causing the slowdown? Is it lagging on opening the file or on reading/writing?
When you're loading up the town area, is there any difference pre-post process set up, compared to the country or a dungeon? Is it just a few variables, or are the objects and other data initialised before the post process?
I'm curious about this one, it sounds like another example of how DBPro sometimes screws up for no reason, especially in large projects! Makes me think there must be some hidden memory leak somewhere :/

The code never bothered me anyway...
Back to top
Profile PM Email
TheComet
18
Years of Service
User Offline
Joined: 18th Oct 2007
Location: I`m under ur bridge eating ur goatz.
Posted: 17th Jun 2014 18:56 Edited at: 17th Jun 2014 19:00
Link
How thorough is your profiling code? Are you able to pin-point which routine specifically is causing the slowdown? If you've found said function, could you post it?

Back to top
Profile PM Email Website
CumQuaT
AGK Master
16
Years of Service
User Offline
Joined: 28th Apr 2010
Location: Tasmania, Australia
Posted: 18th Jun 2014 13:12
Link
This is the specific block of code which sometimes works in milliseconds and other times takes several seconds (even though it's accessing the same file)



As it happens, I worked out a work-around for the problem, though it doesn't mean that this isn't some sort of issue. A workaround may be a solution in this instance but not necessarily a fix, if you know what I mean.


Back to top
Profile PM Email
Rudolpho
20
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 19th Jun 2014 02:16
Link
Not really related to the problem at hand but won't that snippet always exclude the final line from file 1 from being written to file 2? That can probably cause some trouble further down the road...

As for the slowdowns, you're opening and writing to two different files simultaneously. On the off chance you have a mechanical disk where these files are in significantly different places that might take some time for the read/write head to move around reading the files line by line. You could try to just read the whole file into memory at once and read the individual lines from there when writing your output file, however I doubt it would be as slow as 6 seconds for 80 lines.


"Why do programmers get Halloween and Christmas mixed up?"
Back to top
Profile PM Email Website
CumQuaT
AGK Master
16
Years of Service
User Offline
Joined: 28th Apr 2010
Location: Tasmania, Australia
Posted: 20th Jun 2014 06:50
Link
Yeah, it does seem a bit odd...


Back to top
Profile PM Email
TheComet
18
Years of Service
User Offline
Joined: 18th Oct 2007
Location: I`m under ur bridge eating ur goatz.
Posted: 20th Jun 2014 11:12 Edited at: 20th Jun 2014 11:13
Link
How reproducible is this anomaly? Does it happen across multiple computers? Is it perhaps dependent on how long the game has been running?

Reading a lot of small files is generally not a fast thing to do.

Back to top
Profile PM Email Website
Kevin Picone
23
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 20th Jun 2014 14:37
Link
I'm probably missing something here, But rather than constantly read->writing the file, which doesn't seem like a very efficient model to me. You should be able remove the reading from disc process entirely and cache the files in memory.

So on load, we pull the original template version of the file(s) we need to write into from a string array/cache then when it's time to customize the text we pull from the cache in memory and drop in on disc in as big a fragments as possible.

Avoid streaming the file to disc where possible (chr by chr / row by row), You'll get much before performance make all the changes in memory first, then dumping the entire buffer out to disc one hit. You might need to the Matrix extensions to do this in Dbpro though..


Convert PlayBASIC To Machine Code
Back to top
Profile PM Website
TheComet
18
Years of Service
User Offline
Joined: 18th Oct 2007
Location: I`m under ur bridge eating ur goatz.
Posted: 20th Jun 2014 14:57 Edited at: 20th Jun 2014 14:58
Link
Quote: "Avoid streaming the file to disc where possible (chr by chr / row by row), You'll get much before performance make all the changes in memory first, then dumping the entire buffer out to disc one hit. You might need to the Matrix extensions to do this in Dbpro though.."


DBP handles that kind of stuff internally already. That's why you have to call close file to flush the rest of the buffer to disk. Managing a custom cached version is overkill is what I'm saying.

Now would really be a good time to have a function that can load a shader from memory.

"hands are ruined by repetitive stroking... those weeds are killers, literally" - Mr. V, 2014
Back to top
Profile PM Email Website
Back to top

Login to post a reply

Server time is: 2026-07-05 16:05:37
Your offset time is: 2026-07-05 16:05:37