Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / Automatic Code Optimization

Author
Message
Mr Kohlenstoff
20
Years of Service
User Offline
Joined: 7th Jun 2006
Location: Germany
Posted: 16th Mar 2012 20:14 Edited at: 16th Mar 2012 20:25
Hey guys.

I was thinking about code optimization the other day and I was wondering why there (seemingly?) is no application yet that automatically optimizes your DBPro-Code.

As far as I know (which is not that far, but still) most C++-compilers for instance tend to low-level-optimize code automatically (with stuff such as inline functions, replacing calculations of constant values by their result, unwrapping small fixed-size for-loop and so on), but if I am not mistaken the standard DBPro-compiler is not that intelligent (or did that change curing the last years?).
Oddly enough, with compiler V6.1 it goes as far that using the value "0xFF" inside your code instead of "255" runs much slower - which is, frankly, ridiculous.

So, basically what I thought was that it might be a good idea to develop a program that gets a DBPro-Code as input, optimizes it, and outputs another valid DBPro-Code which might be hard to work with but faster to execute. It should then only be used for release-versions where performance is of major importance I guess, and not always since it would increase compile time (significantly, at least when the optimization-program itself was written in DBPro, which is not necessary).

Maybe it is a good idea to write such a program, in DBP, keep it open source and let everybody contribute to the optimization process by providing single self-contained optimization functions.
To make the process easier it should start with bringing the code to an easy-to-read form (trim unnecessary spaces/tabs, make sure every line contains exactly one command, remove comments alltogether, make all commands, variable-names, function-calls etc. lower case...) to allow programmers to easily add new optimization-rules.
It might also be nice to track the line-numbers to make sure the output-code stays consistent, so that potential dbpro-error-messages still refer to the right lines - but this is not too important, as long as the optimization pays off, in my opinion.

As an example, an inline-optimization might look like this, given that there's already a framework that transforms the code into readable form and, in this case, proves information on user-defined functions:




What do you think? Is this a good idea? Has something similar already been done? Would you use such a code-optimizer? Would you contribute to it?

Diggsey
20
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 16th Mar 2012 20:40
All of this low level optimisation would need to be done inside the compiler, it's not going to be very effective to try to optimise DBPro code to more DBPro code.

[b]
Sergey K
22
Years of Service
User Offline
Joined: 4th Jan 2004
Location:
Posted: 16th Mar 2012 20:58
to optimize a code you have to get rid of all the code comments, and all unnessesery spaces. but doing that, you actually gonna make the code less readable.
so its not that good idea i guess...

more 3d models .x/.obj and more foramts here:
[href]https://www.turbosquid.com/Search/Index.cfm?keyword=gogetax1&x=0&y=0[href]
Mr Kohlenstoff
20
Years of Service
User Offline
Joined: 7th Jun 2006
Location: Germany
Posted: 16th Mar 2012 21:03
@Sergey: Indeed, that's why this program should not be used to permanently change your code, but just once to compile a fast version of your game/application. You are not supposed to read the optimized code in the first place.

@Diggsey: It is meant to be some kind of preprocessor, transforming usual DBP-code to faster DBP-code. Of course a compiler doing all that stuff natively would be much better, but I think this is all we can do (without actually writing a complete DBP-compiler from scratch... *cough*), and still better than nothing, right?

Diggsey
20
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 16th Mar 2012 21:13
Quote: "It is meant to be some kind of preprocessor, transforming usual DBP-code to faster DBP-code."


But DBPro code doesn't easily convert to faster DBPro-code. Removing comments has no effect on speed, reducing the number of lines of code will save a few instructions, but then you'll mess up line numbers. The best thing I can think of is writing a post-compiler, which converts the assembly generated by the DBPro compiler to faster assembly, as there's a lot you can do to optimise there.

[b]
Mr Kohlenstoff
20
Years of Service
User Offline
Joined: 7th Jun 2006
Location: Germany
Posted: 16th Mar 2012 21:25 Edited at: 16th Mar 2012 21:26
Quote: "But DBPro code doesn't easily convert to faster DBPro-code. Removing comments has no effect on speed, reducing the number of lines of code will save a few instructions, but then you'll mess up line numbers."


Removing comments is not meant to optimize the code, it's just one way to make the code easier to handle - so it will be done before actually optimising it.

Here's an example of how the application could transform code in order to make it run faster:

Original Code:



Code after Transformation:


Both codes are semantically equivalent, i.e. will behave identically. The difference is that the second code is worse to work with, but on the other hand will run faster, because e.g. the overhead caused by the for-loop and the function-call of 'Times4', as well as the calculation 36/4 is gone.

MrValentine
AGK Backer
15
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 16th Mar 2012 21:26
Diggsey beat me to mentioning error line numbers... imagine your program threw an error on line 2567 and you did not have a readable compiled version so in your commented code that line might actually be line 8766...

Nuff said lol

but yeah can you optimise processed machine code/assembly code?

Mr Kohlenstoff
20
Years of Service
User Offline
Joined: 7th Jun 2006
Location: Germany
Posted: 16th Mar 2012 21:29 Edited at: 16th Mar 2012 21:41
Quote: "Diggsey beat me to mentioning error line numbers... imagine your program threw an error on line 2567 and you did not have a readable compiled version so in your commented code that line might actually be line 8766..."


As indicated in the original post, this could be taken care of by saving the line number of each command and bringing the output code to the according form after optimization.
On top of that, a release-version should not contain any DBP-errors in the first place - I mean, don't you usually turn DBP-error-checking off for release versions? (so the program will run much faster, but errors, in case they still do exist (which is to avoid) result in a windows-error)

This kind of optimization is not meant to be used for Beta-Versions or debugging of any shape or form. It is just one rather "brutal" way to gain maximum performance.


Edit: To give you a more practical example what such a program could be used for. Consider the case that your code contains many coloring-operations and you always write them in HEX-form in your code (e.g. "ink 0x80FF0000,0" or just "myColor = 0xFFFFFFFF" or whatever). Maybe those color-HEX-values are used so extensively that they are responsible for 10% for the program's runtime.
Now, as said before, at least in my current version of the DBP compiler hex-values are much slower than integer-values when used as constant in the code. So the optimization-program would just replace "0xFFFFFFFF" by "4294967295" which is the decimal equivalent, resulting in the program running maybe 8% faster than before.

Another example would be getter and setter-functions when working with large chunks of data.
I came across a somewhat similar case: when manipulating images using memblocks, to access each pixel I usually do something like this:



This implementation would be slower than necessary due to the overhead generated by the GetMemPos()-function (putting parameters and result on a stack and all that, or however functions work internally), and a program that optimizes the code and tries to inline simple functions as this one would increase the program's performance significantly while still allowing you to keep this modular coding-style.

The output after running the optimization program could look like this:



The function call is removed and so is the variable p which generated overhead without actually doing anything useful.

Mr Kohlenstoff
20
Years of Service
User Offline
Joined: 7th Jun 2006
Location: Germany
Posted: 16th Mar 2012 21:52
Here's an example code prooving that such low-level optimization does make quite a difference in certain situations:




When I run this code, the second loop is between 25 and 50% faster than the first one.
Of course in games the optimization will not be that massive since stuff such as rendering a scene will not be optimized by the program, but even if it increases the frames per second by only 5 or even 2%, it's probably worth the effort (which is basically 0, since a DBPro-programmer will only need to open the optimization program, input the code, wait for the optimization and compile the output - a matter of a few minutes).

MrValentine
AGK Backer
15
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 16th Mar 2012 21:58
im walking off this debate

as a fundamental factor was completely ignored...

dont bother replying to this post as I am removing mailback for this thread... but best of luck

Mr Kohlenstoff
20
Years of Service
User Offline
Joined: 7th Jun 2006
Location: Germany
Posted: 16th Mar 2012 22:06 Edited at: 17th Mar 2012 03:30
What do you mean? I'm confused... are you still referring to the line number-problem? That's practically solved.

And I doubt that there is a practical way to optimize machine code - it is certainly possible somehow, but probably not as well and easy as compared to optimizing the given rather high-level code written in DBPro. Furthermore the optimization of machine code is not DBPro-exclusive to begin with, so there's no reason to discuss it on this board, I guess.

Kevin Picone
23
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 17th Mar 2012 02:08
Yeah, i've had crack at this in DB classic many many moons ago. Not too sure what that version supported now, but I know there's some user controlled loop unrolling, but mostly it was to evaluate constant functions from expressions. Which could be beneficial in DB, DBPRO and probably even AppGameKit BASIC

There's newer versions of sorts, but they're mostly for dealing with syntactical issues that occur when dragging old code between BASIC dialects now. Way too lazy to do that stuff all by hand.

WLGfx
18
Years of Service
User Offline
Joined: 1st Nov 2007
Location: NW United Kingdom
Posted: 17th Mar 2012 02:26
I ain't got a clue what compiler DBP uses but usually before even considering setting the optimisation options in a C compiler for example, it's normal to do simple things yourself such as un-rolling small loops, setting a macro instead of a function call (ie inlining a function), that kind of thing.

Often I re-write code nowadays, sometimes more than once, but the original code has only been written to get it working and it'll be used as a model for the optimised re-write. Re-writes are only needed when bottle-necking occurs in time critical areas of a program or if something is taking way too long to calculate.

Mental arithmetic? Me? (That's for computers) I can't subtract a fart from a plate of beans!
Warning! May contain Nuts!
Diggsey
20
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 17th Mar 2012 04:53
Your example of optimisation makes very little difference to the speed of the program, and it's also very limited in what it can do. A function call equates to a few extra assembly instructions, a single array access is tens of assembly instructions. It's just not worth going to this trouble to save the occasional instruction, you would be much better off doing higher level optimisation. There is a lot of optimisation you can do on the machine code though, since DBPro generates a LOT of extra instructions for error checking and the like that could and should be skipped in release versions.

[b]
WLGfx
18
Years of Service
User Offline
Joined: 1st Nov 2007
Location: NW United Kingdom
Posted: 17th Mar 2012 05:24
@Diggsey - One thing I forgot about DBPro is the extra debug error checking code.

So really unless array optimisations are worth it (xx% of optimisation problems) and using different variable types (ints, floats), DBPro is as fast as anything you'll get unless you move over to writing your own plugin using C++.

Mental arithmetic? Me? (That's for computers) I can't subtract a fart from a plate of beans!
Warning! May contain Nuts!
mr Handy
18
Years of Service
User Offline
Joined: 7th Sep 2007
Location: out of TGC
Posted: 17th Mar 2012 08:19
Better do a next patch
Mr Kohlenstoff
20
Years of Service
User Offline
Joined: 7th Jun 2006
Location: Germany
Posted: 17th Mar 2012 13:49
Quote: "A function call equates to a few extra assembly instructions, a single array access is tens of assembly instructions. It's just not worth going to this trouble to save the occasional instruction, you would be much better off doing higher level optimisation."


I often work on programs that somehow manipulate images using memblocks. The program handles a lot of data and calculates very little for each pixel, consequently the overhead from additional unnecessary function calls is responsible for a lot of the calculation time.
As I said, I don't know how big the difference is for actual games. It is certainly true that high-level optimization is usually more effective, but the case of image manipulation is one example where, even if any kind of high-level optimization is still possible, the low-level stuff may still be the bottle neck.

I guess the only way to find out whether such optimization pays off for games is to actually build the program and run some benchmarks. I will see what I can do. Even if it does not improve the performance significantly, such a program would allow other useful stuff, since it can be used as preprocessor, e.g. for advanced error output (manually returning line numbers), removing debugging-operations and so on.


Quote: "There is a lot of optimisation you can do on the machine code though, since DBPro generates a LOT of extra instructions for error checking and the like that could and should be skipped in release versions."


If I am not mistaken you can turn that off using the Compiler\setup.ini-file, there are some entries that seem to be handling that:

Quote: "
[DIRECTIVES]
RemoveSafetyCode=No
SafeArrays=Yes
"


I haven't run a speed test yet, but it is certainly worth it for release versions.

Diggsey
20
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 17th Mar 2012 18:07
Quote: "If I am not mistaken you can turn that off using the Compiler\setup.ini-file, there are some entries that seem to be handling that:"


That only gets rid of some of the extra instructions, there's still plenty of room for optimisation.

[b]

Login to post a reply

Server time is: 2026-07-11 02:29:19
Your offset time is: 2026-07-11 02:29:19