Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Classic Chat / Handles instead of object numbers

Author
Message
Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 25th Jan 2011 14:33 Edited at: 25th Jan 2011 14:36
I'm sure this one's been mentioned before, but I think it's worthy of it's own thread, simply due to the fact that asking the user to maintain the numbers of all the objects in a game only helps to slow development. In fact, that's putting it nicely. It really makes it difficult to make any application if you have to number all your resources.

That's why I think the solution to this should be implemented directly in the AppGameKit system. Instead of doing:


One would instead do:



I don't think this requires much explanation.

There can be some discussion though as to what the handles should be. Should they simply be integers representing objects? Maybe they should actually be a reference to that object in memory. Then we can do things like this, for example: myObject.texture = LoadImage("bleh.jpg");

"everyone forgets a semi-colon sometimes." - Phaelax
Van B
Moderator
22
Years of Service
User Offline
Joined: 8th Oct 2002
Location: Sunnyvale
Posted: 25th Jan 2011 15:01
It would be nice to base everything on entities - like a sprite might be created like this:

dudetexture = LoadImage("dude.png")
mydude = CreateSprite(dudetexture,16)

And a sound like this:

sound = LoadSound("hit.wav")

And maybe a particle emitter like this:

bloodspurt = CreateParticle(bloodtexture,8)

But then have them all use the same basic positioning vector, like X,Y position, angle, size, perhaps even velocity XS,YS. That way, a sound could be attached to a particle emitter, could be attached to a player sprite, using a specified 'parent' entity. That could really save us some coding . Imagine firing a missile, with a looping sound and flame particles attached - all being controlled by the parent missile entity - have child entities only update if their parent is alive.

Health, Ammo, and bacon and eggs!
Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 25th Jan 2011 16:50 Edited at: 25th Jan 2011 16:55
Yup. I discussed this in another thread. If there was some kind of inheritance, whereby types can extend other types, and by necessity, reference types are allowed, it would be quite easy to build scene graphs like the one you described - and it wouldn't even have to be OO.

Furthermore, it makes development easier, simpler, and less prone to bugs. If the programmer wants to create a large, elaborate game, with many different types of entities, you might suspect they do this:



However, using polymorphism, it's a lot easier to do this:



Proof that it's possible to harness these very powerful ideas, and bring them to BASIC pretty seamlessly.

I don't know why this wouldn't be an option for TGC, because it doesn't make it more difficult for anyone, it doesn't require more experience to get to grips with, and, perhaps most importantly, it allows people to write libraries for this language, in this language. With DBP, you can write some pretty clunky function collections, but it's hard to take them seriously. DBP is not well designed for large projects, and games are large projects.

"everyone forgets a semi-colon sometimes." - Phaelax
swissolo
14
Years of Service
User Offline
Joined: 9th Jan 2010
Location:
Posted: 25th Jan 2011 23:22
I personally like obj#s because they take the place of variables in this situation, but you could just use an array to replace obj#s, so it fits more people.

I really don't care. Let's see what others think...

swis
Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 26th Jan 2011 00:42
The problem with numbers is this.

If I have this variable, representing an object, "player", what do you think the purpose is of the object that it represents?

What if I have object 24? What object does that represent? You're going to forget eventually.

"everyone forgets a semi-colon sometimes." - Phaelax
DMXtra
22
Years of Service
User Offline
Joined: 28th Aug 2002
Location: United States
Posted: 26th Jan 2011 00:42
Quote: "
It would be nice to base everything on entities - like a sprite might be created like this:

dudetexture = LoadImage("dude.png")
mydude = CreateSprite(dudetexture,16)

And a sound like this:

sound = LoadSound("hit.wav")

And maybe a particle emitter like this:

bloodspurt = CreateParticle(bloodtexture,8)
"


YES PlEASE... Sorry about all caps, but this is really needed. No object numbers and organization of code!

This is what I vote for, so easy and yet so flexible

Dark Basic Pro - The Bedroom Coder's Language of choice for the 21st Century.
Fatal Berserker
14
Years of Service
User Offline
Joined: 2nd Jul 2010
Location:
Posted: 26th Jan 2011 01:39
Quote: "This is what I vote for, so easy and yet so flexible"

Why?
We already can do this..
Just make it into an integer variable... Its less effort and less code...

DMXtra
22
Years of Service
User Offline
Joined: 28th Aug 2002
Location: United States
Posted: 26th Jan 2011 03:30
Not at all, you have to keep track of those number and build extra code to do it.

It's really not necessary to use variables for object numbers using this method because the system takes care of everything for you and you don't have to do things manually.

I shouldn't have to keep track of object numbers manually (using a variable or not, it does not matter). I want to produce something quick and not have to look at the part of my code to make sure I didn't use that object number already and I don't want to build code to manage for it.

Dark Basic Pro - The Bedroom Coder's Language of choice for the 21st Century.
Kevin Picone
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 26th Jan 2011 04:21
You'd still have to keep track of what you're using. Which means storing the return values in something. Be that a Variable / array / UDT list.

DMXtra
22
Years of Service
User Offline
Joined: 28th Aug 2002
Location: United States
Posted: 26th Jan 2011 09:40
Quote: "
You'd still have to keep track of what you're using. Which means storing the return values in something. Be that a Variable / array / UDT list.
"


Yes, this is true. However, you don't have to manually store a number.

The more complex your program is, the easier it is too lose track of what number you were on.

That is why it is a very bad idea to have object numbers. Sure in a small program you just have a few variables with numbers stored in them and it's no big deal. But is everyone just going to write the smallest program?

It in other words it just doesn't scale well.

The entity way of doing it, you still use variables like you would with numbers, but I don't have to remember a number.

Why make programming harder and why make the programmer keep track of things that they shouldn't need to do.

It's like telling someone they don't need a database for something complicated, they can just write it all down on a notepad.

It doesn't make logical sense and neither do object numbers.

Dark Basic Pro - The Bedroom Coder's Language of choice for the 21st Century.
Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 26th Jan 2011 09:50
You also wouldn't need to keep track of every single resource. If entities can be attached to other parent entities (like Van said), then you can trust them to take care of all their subordinates.

Object numbers are global. Creating object 1 inside a function is still object 1 everywhere else.

However, if I write a function that creates an entity and attaches 5 sub-entities to it, I only need to return the handle of the parent entity. All the rest of the handles can be forgotten about and simply stored in local variables. If you still want access to the sub-entities of the parent, outside the function, you could always get them through a field in the parent.

In other words, using this method, one handle represents everything beneath it in the scene graph, as opposed to the more linear version DBP currently uses, where one number = one resource.

"everyone forgets a semi-colon sometimes." - Phaelax
Scraggle
Moderator
21
Years of Service
User Offline
Joined: 10th Jul 2003
Location: Yorkshire
Posted: 26th Jan 2011 13:53
Quote: "If I have this variable, representing an object, "player", what do you think the purpose is of the object that it represents?

What if I have object 24? What object does that represent? You're going to forget eventually."


#constant player 24

Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 26th Jan 2011 14:19
Sure, if you actually want to bare the responsibility of finding objects numbers for all the hundreds, if not thousands of objects in your game, and that's ignoring all other types of resources.

Another problem is that if you have a single, huge, global linear list of constants like that, things will get disorganised just as easily, if not as quickly. It doesn't imply any kinds of associations between objects. All objects are completely separate and distinct from each other, which isn't a good way of modelling a game, because generally that's just not the case.

The way you describe it is what many TGCers do, or at least similar (such as using functions such as FreeObject), because it's better than the default. So why should the default stay the way it is, when people aren't using it? It strikes me as rather strange to design a language with the intent that people find work-arounds to old problems. The only people that will actually use plain old object numbers are people who are inexperienced, so it's no good teaching them bad practices early on.

"everyone forgets a semi-colon sometimes." - Phaelax
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 26th Jan 2011 14:38
There are four different but entwined subjects being discussed here that are maybe not so obvious.

1. Whether you specify the id, or the id is given to you.


or


2. If the id is given to you, whether that id is an integer, or has it's own type.


or


3. Whether these handles, when given their own type, should have common commands.


4. Whether these handles with common commands should have some sort of special syntax.


5. Whether these handles have properties attached


My opinion:
1 - id given to you
2 - id has non-integer type
3 - no common commands - confusing for a simple language
4 & 5 - no special 'OO'-like syntax - confusing for a simple language

There have also been discussions on other posts about any special properties that handles should have - I believe now that they should be dumb and have no properties except for assignment of 0, and assignment from another handle of the same type.

If something like 3, 4 & 5 are really needed by someone, then you can either write a translator, or use tier 2 (being able to write a translator was one of the things I was thinking of when the suggestion was made to simplify and standardize the syntax).

dark coder
22
Years of Service
User Offline
Joined: 6th Oct 2002
Location: Japan
Posted: 26th Jan 2011 14:42 Edited at: 26th Jan 2011 14:44
I think we can unanimously agree that there's no purpose to making any of the resource creation functions require you pass an index to it, and instead they should return the index or handle.

The obvious reason for this is because it's impossible to write fully modular code using DBPro's system; completely unrelated systems are easily capable of ruining your day by altering the resources you've made and there's no way to prevent them doing this.

As for the nature of the handles, I see no point to using integers for them because it's not descriptive in the slightest. Making a unique handle type per resource adds code safety via additional type checking.

I also I think such handles should only be capable of pointing to either nothing(null), or a valid resource, and that deleting the resources will effectively null all references to it. This reason for this is because in languages like C++, deleting an object means all references to it are invalid, and dereferencing it results in a crash. In DBPro it's pretty similar, except that you can quite easily place another object at that memory(i.e. reuse the ID), so there isn't really a way to check if your resource has been removed, short of manually doing nulling all references to it(setting IDs to 0). (and yes, you can obviously check if a resource at a specific ID exists, but that isn't the same thing, as some other code could have deleted your resource and made another one)

This reference nulling system can be done with insignificant overhead as I discussed in one of the other threads.

IanM beat me to it!

IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 26th Jan 2011 14:53
... except we disagree on the handles.

I've written the code to keep handles valid or not, I've written the code to use reference counting, and I've written the code to do both (in-engine counted and user resources in a list). The more you do, the slower it gets, and the more complex the code gets too. The 'passing a handle' overhead is at least as heavy as the 'check it exists' required for a non-checked handle, and heavier in user code.

Benjamin
21
Years of Service
User Offline
Joined: 24th Nov 2002
Location: France
Posted: 26th Jan 2011 15:04
Quote: "I've written the code to keep handles valid or not, I've written the code to use reference counting, and I've written the code to do both (in-engine counted and user resources in a list). The more you do, the slower it gets, and the more complex the code gets too. The 'passing a handle' overhead is at least as heavy as the 'check it exists' required for a non-checked handle, and heavier in user code"


How much realistic impact is it going to have on performance though?
Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 26th Jan 2011 15:19 Edited at: 26th Jan 2011 15:20
Quote: "My opinion:
1 - id given to you
2 - id has non-integer type
3 - no common commands - confusing for a simple language
4 & 5 - no special 'OO'-like syntax - confusing for a simple language"


I disagree with 3 and 5.

3 - There's nothing confusing about keeping consistency between entity types. Also, if there was to be some simple inheritance (doesn't necessarily conflict with BASIC syntax, and users that don't care about inheritance don't even need to know it exists), it would make sense to be able to use the same command for that type and all subtypes.

5 - I agree there shouldn't be method calls, because then we really are getting into OO territory here. But as for simple fields, DBP already supports this with UDTs. I don't see how we're being too drastic by simply allowing users to directly access resources from a handle. Of course it's not that this is a big deal. mySprite.image can be replaced by GetSpriteImage(mySprite), though if you ask me, this will only mean that users have to learn a ton of new commands.

"everyone forgets a semi-colon sometimes." - Phaelax
dark coder
22
Years of Service
User Offline
Joined: 6th Oct 2002
Location: Japan
Posted: 26th Jan 2011 15:38 Edited at: 26th Jan 2011 15:50
Just so we're clear on what my idea would entail, here's some not quite working C++ code to demonstrate how you can (effectively)null handles without maintaining a list of references:

Resource class(i.e. Image, Object, etc)


Global functions to manipulate the class, i.e. LoadImage/DeleteImage/PasteImage/etc


Some DBPro-esque code showing how it would work with clear annotations of all calls made as a result of the assignments


I don't really think you can even call this overhead, it's just an additional if statement and some variable inc/decs that can be inlined.

Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 26th Jan 2011 15:52
This is essentially the way I'd like to see it, except I'm not sure about having to zero a pointer to properly free it.

Perhaps now would be a good time to consider some kind of mark and sweep garbage collector?

"everyone forgets a semi-colon sometimes." - Phaelax
dark coder
22
Years of Service
User Offline
Joined: 6th Oct 2002
Location: Japan
Posted: 26th Jan 2011 16:07
Quote: "This is essentially the way I'd like to see it, except I'm not sure about having to zero a pointer to properly free it."


I don't really see that as an issue, the worst case non-fictional scenario is that the user has a bunch of global handles to resources and doesn't assign 0 to them. But realistically, they wouldn't be in this bound state for long, if the user changes levels then I presume the handle would be reinitialised fairly quickly. Not to mention the memory footprint for the class sans-data would just be a few pointers.

As you say, a garbage collector is an alternative, but it's very different from DBPro, because there, all resources' lifetimes are managed by the user. So assuming that doesn't change, I think my suggestion gives an additional benefit over IDs or dumb handles with insignificant overhead.

Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 26th Jan 2011 16:28
Perhaps, but there are other reasons why a garbage collector would be useful.

Most notably, it would be the use of dynamic arrays in UDTs. This isn't necessarily a feature that will exist, but lets face it, everybody wants it.

To be a dynamic array, I assume it would have to be a pointer to somewhere in memory, rather than all the contents of the array to be stored in the same space as the rest of the object.

Consider this code:



The only solution I can think of is manual memory management or garbage collection, or, if we really wanna bite the bullet, just do away with dynamic arrays in types altogether.

"everyone forgets a semi-colon sometimes." - Phaelax
Diggsey
18
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 26th Jan 2011 16:41
You don't need garbage collection for that. Just free the array whenever a variable of type "myType" goes out of scope. If multiple arrays can refer to the same block of memory then you need a reference count as well, but garbage collection is definitely unnecessary.

[b]
dark coder
22
Years of Service
User Offline
Joined: 6th Oct 2002
Location: Japan
Posted: 26th Jan 2011 16:45
Quote: "The only solution I can think of is manual memory management or garbage collection, or, if we really wanna bite the bullet, just do away with dynamic arrays in types altogether."


Why would the array be stored as a pointer? The exact same code can be done in C++, and due to RAII there's no need to write any additional code, because the destruction of the container would delete its contents.

If reference types were introduced then you'd need reference counting, or a system like mine, but not for such an example.

Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 26th Jan 2011 16:46 Edited at: 26th Jan 2011 16:49
Reference counting breaks of course when you have circular references. I don't know what's more expensive: detecting cycles or garbage collection. Either way, it won't be possible to make all these operations in constant time unless you give the programmer manual memory management.

I don't mind which technique is used though, as long as it's optimised and automatic.


[edit]
Darkcoder, it would have to be stored as some kind of pointer or reference, because otherwise the arrays wouldn't really be dynamic. Once you've filled it's capacity, you can't reallocate with a different size, since it would be part of the object. And RAII wouldn't work well in this case because you might assign the array to some global array. If the variable went out of scope and it deleted the whole array with it, the global array wouldn't be valid.

"everyone forgets a semi-colon sometimes." - Phaelax
Diggsey
18
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 26th Jan 2011 17:16 Edited at: 26th Jan 2011 17:16
@Zotoaster
You can't get circular references without doing something like this:


But you can't use a type before it's been fully declared.

[b]
Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 26th Jan 2011 17:39 Edited at: 26th Jan 2011 17:41
You can get circular references when you do things like this:


(not an example of arrays - just showing the principle)


"everyone forgets a semi-colon sometimes." - Phaelax
dark coder
22
Years of Service
User Offline
Joined: 6th Oct 2002
Location: Japan
Posted: 26th Jan 2011 18:06
Quote: "Darkcoder, it would have to be stored as some kind of pointer or reference, because otherwise the arrays wouldn't really be dynamic. Once you've filled it's capacity, you can't reallocate with a different size, since it would be part of the object."


But your post made it appear as if arrays would be implemented like a pointer to an array class, rather than the more logical, array class within the UDT, how it internally handles the array data is irrelevant.


Quote: "And RAII wouldn't work well in this case because you might assign the array to some global array. If the variable went out of scope and it deleted the whole array with it, the global array wouldn't be valid."


This assumes you can store arrays as references, I can see the utility in passing arrays as references to functions, but not so much storing them.

Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 26th Jan 2011 18:34
Quote: "But your post made it appear as if arrays would be implemented like a pointer to an array class, rather than the more logical, array class within the UDT, how it internally handles the array data is irrelevant."


How it handles the array data is relevant. Fair enough having an array class inside the UDT, as long as it always stays the same size. But then the problem still exists, only in the array class. If it doesn't allocate the data somewhere on the heap, then it won't be dynamic.

"everyone forgets a semi-colon sometimes." - Phaelax
dark coder
22
Years of Service
User Offline
Joined: 6th Oct 2002
Location: Japan
Posted: 26th Jan 2011 18:58
It doesn't matter, example:



Internally std::Vector uses the heap for the array data, but its implementation is irrelevant because it gets deallocated with the struct when it leaves the scope due to RAII, that is, its destructor is called. I imagine any dynamic array within UDT implementation in AppGameKit would use this same system, so memory isn't an issue.

Diggsey
18
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 26th Jan 2011 19:01 Edited at: 26th Jan 2011 19:01
@Zotoaster
You've used mytype2 in mytype before it's been declared...

[b]
Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 26th Jan 2011 19:19
Darkcoder,

Ah I see what you're saying. In that case it would probably work fine. Copying the array class wouldn't necessarily be as expensive as copying the whole array.

Diggsey,

That works in DBP. Back in the day, I used to put all my UDTs at the end of my code.

"everyone forgets a semi-colon sometimes." - Phaelax
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 26th Jan 2011 20:19
@dark coder,
Would you like to try writing your handle code again, in such a way as to avoid dereferencing invalid pointers from different copies of the handle?

For handles and references, you need:
1. A handle class that refers to the resource.
2. A resource class.
3. Resource life-time control that is independent of the handle (but can be rolled into the resource class).

I have uploaded the actual working code I tested with - check out the readme that contains the speed comparison.

Also, consider how many times you access a 'handle' (a user-allocated integer) for such a simple thing as repositioning and rotating an object: 1 for reading each position, 1 for setting position, 1 for reading each angle, 1 for setting = 8 times the handle is copied (passed via the stack into the DBPro functions/commands).

Right now, I consider that anyone who can't back up performance arguments with code has nothing I want to hear on the subject.

@Zotoaster,
That code is accepted by DBPro, in error, and doesn't do what you think it does.

However, you are correct that reference counting can't deal with circular references, and that is a problem with what I and Diggsey have been discussing, if that code was allowed.

Attachments

Login to view attachments
Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 26th Jan 2011 20:30
IanM,

I simply meant it compiles, though I understand it doesn't do what you think I think it does

"everyone forgets a semi-colon sometimes." - Phaelax
Diggsey
18
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 26th Jan 2011 20:51 Edited at: 26th Jan 2011 20:55
Quote: "Also, consider how many times you access a 'handle' (a user-allocated integer) for such a simple thing as repositioning and rotating an object: 1 for reading each position, 1 for setting position, 1 for reading each angle, 1 for setting = 8 times the handle is copied (passed via the stack into the DBPro functions/commands)."


Unless it's a delete command, the handle cannot be invalidated inside a DBPro command, so you don't need to reference and dereference the resource each time. As for the handle itself being copied, that will be the same for whatever method you use...

Finally, there is another method noone has thought of yet
Use handles for everything, but don't bother actively keeping track of them, just search for handles when you actually delete the object. All existing handles must be in either a global variable or on the stack, or in an array, and the variables on the stack for a given function are defined at compile time.

Given that the stack is probably only going to be a few layers deep, (less than 10 in almost all cases) it will take very little time to traverse it when you delete a resource. All you need is a fast way to look-up the variables in a given function.

Arrays containing handles can either be traversed in the same way, or use the method of keeping track of handles on the fly. As long as you ignore all arrays not containing a handle type, it shouldn't be too slow.

[b]
dark coder
22
Years of Service
User Offline
Joined: 6th Oct 2002
Location: Japan
Posted: 26th Jan 2011 20:56
Quote: "Would you like to try writing your handle code again, in such a way as to avoid dereferencing invalid pointers from different copies of the handle?"


Can you point out which part you're referring to? Perhaps I didn't make it apparent, but in my DBPro code, the 'SomeHandle' type would be equivalent to 'SomeHandle*' in C++. Thus the handle itself would just be 4bytes and assignment would copy over the 4 bytes(same as existing IDs) as well as inc the ref count of the assigned handle and dec the previous one if it pointed to something not null. All of the asserts in my code would be runtime error messages that are already present in DBPro and my comments show the additional check and any ref inc/dec required.

IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 26th Jan 2011 21:23 Edited at: 26th Jan 2011 21:28
So you have a pointer, NOT a handle? Spend 30 minutes putting some compilable and executable code together, and we'll all be able to see what you mean.

@Diggsey,
Quote: "Unless it's a delete command, the handle cannot be invalidated inside a DBPro command, so you don't need to reference and dereference the resource each time. As for the handle itself being copied, that will be the same for whatever method you use..."


... now the sprite is referencing an texture that no longer exists, if the engine-code does not hold handles to the resource.

There are two versions of the 'list' type code. One where the list is all there is, and one where the engine only reference counts, and user-code handles are added to the list. The list is used to clear all handles related to the reference, as per your requirements. In the list code, engine-based handles are cleared, while in the other, only user-code based handles are cleared, and the resource is not deleted until the engine also clears all handles to the resource.

I can't comment on your garbage-collected scheme, as I'm not willing to spend the time to actually code and debug it. If I'm not likely to code and debug it, how likely is it that anyone at TGC will be willing to do so?

[edit]spelling

Diggsey
18
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 26th Jan 2011 21:30 Edited at: 26th Jan 2011 21:37
Quote: "... now the sprite is referencing an texture that no longer exists, if the engine-code does not hold handles to the resource."


OK, there are SOME cases where a handle is needed, but for the vast majority, and the example you gave of moving an object, it's not necessary. Also, there are all sorts of optimisations the compiler can do to reduce the number of times a handle needs to be added or remove from an array. Each function which uses a delete command from a particular command set, or calls a function which does would be marked. The compiler only needs to generate the code to keep track of handles of a particular type if the function is marked as being able to delete that type. Maybe there could be a way to disable this optimisation for particular functions for special cases?

Quote: "can't comment on your garbage-collected scheme, as I'm not willing to spend the time to actually code and debug it. If I'm not likely to code and debug it, how likely is it that anyone at TGC will be willing to do so?"


LOL, I see your point, but then again, they are writing an entire compiler and set of libraries anyway.

[b]
Zotoaster
19
Years of Service
User Offline
Joined: 20th Dec 2004
Location: Scotland
Posted: 26th Jan 2011 21:40
Quote: "I can't comment on your garbage-collected scheme, as I'm not willing to spend the time to actually code and debug it. If I'm not likely to code and debug it, how likely is it that anyone at TGC will be willing to do so?"


The basic idea isn't difficult, but optimising it is, and I believe there's a lot of theory involved when it comes to this kind of stuff.


But basically it boils down to this (untested):


"everyone forgets a semi-colon sometimes." - Phaelax
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 26th Jan 2011 22:39
It's not that easy. You're forgetting that references can be held in the heap too, not connected to the user-code, but to the engine-code.

You'd need a GC scheme where not only the user-code globals, and stack were monitored, but the entire heap was also monitored, which would get slower as the heap expands (unless you get into compacting the heap, which gives a whole new set of issues, such as resetting other handles to point to the new location of the resource when it moves).

@Diggsey,
Quote: "OK, there are SOME cases where a handle is needed, but for the vast majority, and the example you gave of moving an object, it's not necessary"

Dead right, as long as the same handle is used in all/most cases, but how does the optimiser 'know' that none of the references will change, especially in the presence of code where circular references are present? (I mean in the case of handle points to a resource, that resource holds pointers to many handles).

You and I know that we can probably make some well-documented assumptions/optimisations to the code, but how do you instruct the compiler-to-be-written that that is the case? Especially when there's a compilation barrier between the user-code and the engine-code?

Note that the handle_ref_handles.cpp code already carries this assumption out in a way, passing a slightly different handle to engine-code that is not held on the list (to be closer to what you say here, just comment out the code in the Resource class' AddRefCount and ReleaseCount to test speed - I get a time on my system of 1.341s, a bare improvement over the 1.357s of the original code).

Quote: "Also, there are all sorts of optimisations the compiler can do to reduce the number of times a handle needs to be added or remove from an array."

Better is to reduce the number of calls (which still doesn't fully counter the overhead of list-based handles, as you can do the same with other methods), by for example, returning the position/angle as a single vector3 with 1 function call, rather than individual floats with 3 function calls.

Also, id's and dumb handles can be optimised easily in the engine, as I did in several places to the DBPro code - you cache the last id (and it's pointer) or handle, and if the next one matches, you've validated without searching (even if it's only a O(logn) search). You need to remember to clear the id/handle cache if you delete it though.

(See the RemoveImage and UpdateImgPtr functions in CImageC.cpp in the image plug-in for an example).

The auto-cleared handles and the reference-counted handles could also be optimised in a similar way, but *still* fall behind because of their problems.

Diggsey
18
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 26th Jan 2011 23:58 Edited at: 26th Jan 2011 23:59
If we use IDs/dumb handles then, there is a feature which would be really useful: Each time a handle/ID is allocated by the system, it should be a new one (unless you get through 2^32-1 IDs ) The easiest way would be just to increment the value used each time. At least this way we get the main benefit of using smart handles, in that if we use a handle/ID which has been freed, it will error immediately, as it is very unlikely to have been reassigned to a new resource by then.

Also the typed IDs/handles feature would be useful, so the compiler detects if you try to use a handle of the wrong type.

Also, this is kind-of obvious, but make sure there is an ID reserved for representing no resource.

[b]
dark coder
22
Years of Service
User Offline
Joined: 6th Oct 2002
Location: Japan
Posted: 27th Jan 2011 00:09 Edited at: 27th Jan 2011 00:13
OK, here's my idea using your test layout:



In short, it's the same as your scoped_handles example, except it allows you to call DeleteResource on it. In doing so, any handles pointing to it are effectively nulled, that is, ResourceExists will return false, rather than crash as with dumb handles/straight pointers.

On my machine your ID example runs about 10% slower than this, but in a more realistic setting, that would be much more slower because of all the lookups required for the ID.

IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 27th Jan 2011 18:51
Ok, I think I understand what you are suggesting here - you keep the Resource class active until it has been deleted by the programmer, and until all StrongHandles referencing it are freed. You use the WeakHandle to avoid passing StrongHandles around to avoid unnecessary updates of the reference count.

The only problem with that is if the resource is a large one, it would tie up that memory unnecessarily.

However, a solution to that is to make your Resource class into a resource owner instead, with a pointer to the actual resource class (maybe replacing the bool member variable), and the resource owner forwarding all member function calls to that resource class. I'll play with your code to see what happens, but I suspect it's got a good chance of being the fastest method yet.

The only thing I'm confused about is why you decided to include the ResourceList set, as I can't see any need for it at all.

Diggsey
18
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 27th Jan 2011 18:55 Edited at: 27th Jan 2011 18:55
Quote: "The only thing I'm confused about is why you decided to include the ResourceList set, as I can't see any need for it at all."


The user/engine might need to iterate through all resources of a particular type.

[b]
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 27th Jan 2011 19:19
Ok, then that can be added later, but seeing as this hasn't been a focus of performance testing for all of the other methods, I don't think it's fair to give just this method that extra burden to carry.

kamac
13
Years of Service
User Offline
Joined: 30th Nov 2010
Location: Poland
Posted: 27th Jan 2011 20:26
Maybe i am the only one that thinks this way, but the numbers as for me are good. I don't think they should be changed, or even if, instead of putting integer as a number, it's i think easier to put string as a "number", so instead of:

Load Object "mupkidz.x",1

This:

Load Object "mupkidz.x",hero

I think it's much more simplier, ofcourse in AppGameKit there will be ( ) so it would be

LoadObject("mupkidz.x",hero)

I think it's easier to do this way ratcher than the idea mentioned above.

In progress of making Archery Simulator... Or maybe not simulator x]?

@There'll be a sig @
Diggsey
18
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 27th Jan 2011 20:56
@kamac
Why is this:
Quote: "LoadObject("mupkidz.x",hero)"


Any easier than this:
Quote: "hero = LoadObject("mupkidz.x")"


Surely the second one makes more sense?

[b]
kamac
13
Years of Service
User Offline
Joined: 30th Nov 2010
Location: Poland
Posted: 27th Jan 2011 21:29
Imagine now

Position Object

function. It's more natural, to use it like that:

PositionObject(hero,x,y,z)

after definition by LoadObject("mupkidz.x",hero). It's just less change, it's easier to implement(i think), and it's more natural. That's all

In progress of making Archery Simulator... Or maybe not simulator x]?

@There'll be a sig @
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 27th Jan 2011 21:45
Ok, I have two versions of what I believe dark coder was attempting, but with the playing-field levelled somewhat, and with the separation of Resource/ResourceOwner that I wrote above above.

The main changes are:
- reduce memory usage of deleted resources by separating resource and owner.
- Handles passed by copy, as they would be in the user code, not by reference.
- Pass StrongHandle to user-code and WeakHandle to engine-code (assuming that the compiler has some way to tell which is which).

The downside of this system, is that it is possible for the engine to still hold a WeakHandle to a resource that has been deleted ... but see at the end.

The first runs 100000 iterations in 0.561s, which surprised me by how slow that was.

So I rewrote it to avoid unnecessary conversions of StrongHandle to WeakHandle, by using inheritance instead. That ran in 0.467s instead - still not as fast as expected, but more reasonable.

First rewrite:


Second rewrite:


The inheritance-based version of the code has another thing going for it - it is possible to create a new StrongHandle from a WeakHandle (given an appropriate member function in WeakHandle), something that the engine can do so that it can hold StringHandles instead of WeakHandles to resources that may get deleted by the user.

Diggsey
18
Years of Service
User Offline
Joined: 24th Apr 2006
Location: On this web page.
Posted: 27th Jan 2011 21:48 Edited at: 27th Jan 2011 21:57
That's exactly how position object would be:
Quote: "PositionObject(hero,x,y,z)"


The variable "hero" needs to be set somewhere though, and it makes sense to use it like this:


Since LoadObject is giving you a new object, isn't it more natural to have it as a return value?

edit:
Just saw you post, Ian. It looks to me like the slow-down could be caused by the extra allocation and deallocation due to the resource and resource owner types being separated. This could be improved significantly by pooling resource owners together, and overriding the new/delete operators. All resource owners will be identical (or at least be the same size) regardless of the resource type, so pooling would be very effective.

I think the test would be more realistic if you passed the handles about more often than creating and deleting them. Say about 10-20 times between making and deleting them? You could also comment out the allocation and deletion of the resource itself, as that will be identical across all methods, so won't help the test.

[b]

Login to post a reply

Server time is: 2024-11-21 17:49:56
Your offset time is: 2024-11-21 17:49:56