Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Dark GDK / darkGDK and multithreading

Author
Message
axiagame
16
Years of Service
User Offline
Joined: 6th Dec 2007
Location: far far away...
Posted: 6th Dec 2007 11:06
Link
Hi everybody

I just downloaded a pthread library which let me create threads like on linux (eg. pthread_create(&my_pthread, NULL, my_function, NULL); )

And so I created two threads in my code...
well the problem is that my code don't stop to bug, and every time it doesn't bug at the same place (try to read unaccessible memory address). Once it is the function bdPointCamera which bugs, once the function dbSync, or even the while condition of my C++ code...


could someone help me???
Back to top
Profile PM Email
APEXnow
Retired Moderator
21
Years of Service
User Offline
Joined: 15th Apr 2003
Location: On a park bench
Posted: 6th Dec 2007 11:44
Link
Hi

Although the translation was particularly difficult to understand, it is impossible for us to determine whether this is a DGDK related problem with threading, or whether there is an issue with memory access via pointers in your code that could cause your bug. Can you possibly post an example of your code so that we may determine what the problen is.

Paul.


---- The Meridian Culmination ----
Back to top
Profile PM Email Website
axiagame
16
Years of Service
User Offline
Joined: 6th Dec 2007
Location: far far away...
Posted: 8th Dec 2007 20:49
Link
the error occurs in functions like dbPositionCamera(0.0, 0.0, 0.0); It can not be a pointer error, because I don't use any pointer while calling this function. I decided to do in an other way, so I can't post my code because I removed the multi_thread option.

I will create a small code using the pthread library in order to show you the problem and post it soon...


thanks paul
Back to top
Profile PM Email
axiagame
16
Years of Service
User Offline
Joined: 6th Dec 2007
Location: far far away...
Posted: 9th Dec 2007 11:50
Link
here is a piece of code which don't work...



In order the project to compile, you have to add the lib and include folder of the attachment as you did with darkGDK

(don't forget to put the DLL in the project folder)

Attachments

Login to view attachments
Back to top
Profile PM Email
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 10th Dec 2007 17:09
Link
Do somesearching - there has been some threading in GDK discussions before.

I haven't done it - but - I THINK I remember some mention about only one thread can access the "Pipeline" at a time - ...maybe it was objects - but - think about it - You need to be careful about calling routines that effect something that could be getting manipulated in another thread.

Also - You will want to add some SERIOUS debug "logging" of your own to dump text to a file for later analysis.

You can't expect VStudio to "step through" multiple threads - if it does that - WOW - but usually - you need to have various checks and balances.

Any/All multi-thread debugging I've done - required CAREFUL "what if" slow code reading sessions - and basically adding "tiny pieces at a time" and testing - adding more - testing. Because of what Multi-threaded is - Errors aren't what they seem - not easy to diagnose.

Jason


http://www.jasonpetersage.com/?PAGE=ironinfantry&SECTION=ironinfantry
Back to top
Profile PM Email Website
Benjamin
21
Years of Service
User Offline
Joined: 24th Nov 2002
Location: France
Posted: 10th Dec 2007 17:15
Link
DarkDGK isn't thread-safe, and thus you shouldn't try using it in multiple threads simultaneously otherwise you will get very strange behaviour.

Tempest (DBP/DBCe)
Multisync V1 (DBP/DBCe)
Back to top
Profile PM Email
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 10th Dec 2007 17:58
Link
Wow - that sums it up nicely Benjamin.

Then having threads would still be good for AI, and other things one might want running "alongside" I suppose.

From what you just said - anything to "DarkGDK" at all is out - even loading stuff "Behind the scenes"


http://www.jasonpetersage.com/?PAGE=ironinfantry&SECTION=ironinfantry
Back to top
Profile PM Email Website
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 10th Dec 2007 19:23
Link
... and that's precisely what we discussed at the convention. IIRC, the lightmapping and AI plug-ins for DBPro use a separate thread, and maybe even the physics does too.

Utility plugins collection and
http://www.matrix1.demon.co.uk for older plug-ins and example code
Back to top
Profile PM Email Website
axiagame
16
Years of Service
User Offline
Joined: 6th Dec 2007
Location: far far away...
Posted: 11th Dec 2007 02:32
Link
So, if I understand well, I can't use multithreading method while coding with darkGDK...?
Back to top
Profile PM Email
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 11th Dec 2007 06:52
Link
@axiagame - Not exactly. Care must be taken to separate GFX/DIRECTX stuff from other code you can have run along side - in another thread. Like IanM eluded - so - Physics - could be done in a separate thread - though - their would obviously need to be a communication back and forth between the threads - this is where things always get dicey with multi-threading.

Thread one - managing Game gfx/sound/userinput - and say a new Bad guy is spawned - a "Message" is sent to thread 2 - (Our AI thread) - and the little baddie starts moving, eating, fighting "other non player char's" while moving about - then - thread one checks if any baddies need to be drawn/rendered - Yes? It snags the position - and takes it from there until the baddie if "out of sight" again - where the AI might continue in thread 2 again.

silly example - but its that kind of "thing" I think you can make multi-threading work for you with DarkGDK etc.


http://www.jasonpetersage.com/?PAGE=ironinfantry&SECTION=ironinfantry
Back to top
Profile PM Email Website
axiagame
16
Years of Service
User Offline
Joined: 6th Dec 2007
Location: far far away...
Posted: 11th Dec 2007 12:49
Link
Ok, thank you for all of theeses details!

So, as an exemple if I want to use a thread forthe game and an other one for the sounds management, I have to use another library for the sounds (other than directX)
Back to top
Profile PM Email
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 11th Dec 2007 14:39
Link
That's probably best, as the dbSync function in particular touches pretty much all of the system, not just graphics.

If you divide your application into larger areas of responsibility for threading then you'll be safer - AI, Physics, Rendering/sound.

If you decide to use another sound system then that can probably be a separate thread even if it does use DirectX for the sounds - it's the combination of the sync updating sounds within the GDK engine while you're modifying them that'll cause you problems, but if you create your own sound system then you avoid that.

Utility plugins collection and
http://www.matrix1.demon.co.uk for older plug-ins and example code
Back to top
Profile PM Email Website
jasuk70
21
Years of Service
User Offline
Joined: 3rd Dec 2002
Location: Hemel Hempstead
Posted: 11th Dec 2007 15:29
Link
With my experiments in the DGDK.net area, you need to do the main game loop in your initial thread. Any sync's in the other threads will block until the main thread does it's sync.

Also to be safe, any calls to DGDK functions, I've put in a mutex (sync lock) to make sure that only one thread is trying to access a function at a time. So far I've had no random crashes or hangs etc.

I may find this has slowed things down considerably later on but will look back at that when I need to.

Jas

----
"What is this talk of 'release'? Klingons do not'release' software. It escapes leaving a bloody trail of developers and quality assurance people in its wake!"
Back to top
Profile PM Email Website
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 11th Dec 2007 17:50 Edited at: 12th Dec 2007 13:42
Link
I wonder if .Net makes that a bit easier than in C++. hmm. I rememebr you mentioning this in times past jasuk70.

There is another way - less "demanding" and a little more riske' for passing data back and forth between threads - its less intrusive - and you need to know if it works for you or not before hand - or it could let you down.

1: Why use? Can speed things up
2: Drawbacks? Timing is haphazard because no "Critical section"/mutex/sync logic
3: why it can work? because ONE memory location (32bit) can not be written too at the same moment another thread tries.

Make a UDT for Data going from thread1 to thread2, and another for any data going from thread2 to thread1. Make your code guarantee that One thread writes while the other only reads from a given UDT.

This allows info to be "shared" - but not necearily in sync - avoid dead locks/syncing etc. Use with care. [edit]spelling change[/edit]

This basically allows interprocess communication - without ever making either side WAIT. Though - this could be considered sloppy - but its not - it just doesn't care what either thread is doing. so your threads need to play nice - and it works

If you need DEAD ON - RELIABLE (from a timing standpoint) info - than mutex stuff is the way to go.

[edit] if you pass more than one field at a given time - like a record of data - go with mutex.[/edit]


http://www.jasonpetersage.com/?PAGE=ironinfantry&SECTION=ironinfantry
Back to top
Profile PM Email Website
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 11th Dec 2007 19:19
Link
Sorry jason, but that's only safe if you are using threading on a uniprocessor system - If you use a multiprocessor system then each processor can still access the data independently. The only safe way to access even an int/DWORD is to use a mutex or to use interlocked memory access (provided by MS by using the Interlocked* functions).

@jasuk, that's the safe way to do it, but as you said, pushing every GDK command through a mutex is going to slow your process down at some point.

Utility plugins collection and
http://www.matrix1.demon.co.uk for older plug-ins and example code
Back to top
Profile PM Email Website
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 11th Dec 2007 23:01 Edited at: 11th Dec 2007 23:04
Link
What is the drawback? The memory hardware won't allow a WRITE DURING a read to my knowledge. If you write a 32bit value - in a 32bit architecture - or a 64bit value in a 64bit architecture - that is native (meaning full width of mem bus - not "interlaced" - some use the term "Aligned" - then no matter what - if I write a value of say 128 - and another thread reads it - I'll get one of two things. The value BEFORE the 128 was there - or the 128. I shouldn't ever get a "wrong byte" because it was writing while reading. Even with prefetch - You either get the value before the 128 got stored - or the 128 - and it wound't surprise me if the prefetch got invalidated a reloaded (not great from a speed perspective...but wouldn't surprise me)

??? - Safe - not Safe - but works without Data Corruption? Sure. Should work fine.

I'm not bickering - I'm just feeling I'm pretty dead on here. I do agree it's not safe like a mutex would be - but if your pushing data - and timing isn't important - this does save on a the locking bit... - which unless absolutely neceaary for critical sections - completely "halts" the smooth nature of independant threads doing their own thing.

I also have done a lot of low level coding that supports this - so I have my reason for sounding "What? Are you Sure? hmm" I hope you understand and have the time to elaborate.

We might actually beleive the same thing and describe it differently or just see the value of my comments in a different light - I feel I'm stronger in this area than C++ for instance - which I would (coming from you) triple check before contesting - you have given me awesome advice in DBPro/DarkGDK and C++ - but I've done more assembly and OS coding than I care to recall at times - and this is my playground when I'm in MAJOR code mode. e.g. Task Segment Registers - Descriptors - int tables - jump tables - preemptive schedulars etc. Reading Schematics, manfacturer clock cycle stats, understand memory lowlevel hardware handshaking - how that relates to the clock and the CPUs etc.

I'm saying that you can make one way doors from thread to thread - and avoid sync lock (mutex etc) - there is also ways to make your own pretty easy - but that's another topic - just as easy to use what you know - It's just [edit] NOT [/edit]something for a realtime system where you need to KNOW FOR SURE when/what - etc about the value.

For instance - if a thread kept track of - say HEALTH - and just kept pushing the current health from one thread to another - my method works. One thread just writes - the other just reads.


http://www.jasonpetersage.com/?PAGE=ironinfantry&SECTION=ironinfantry
Back to top
Profile PM Email Website
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 11th Dec 2007 23:41
Link
From everything I've read and understand about the x86 platform, there is no safe way to access a shared variable (read or write) in a multithreaded environment - reads and writes above 1 byte in size are not atomic.

This is why the Intel instruction set includes an assembly instruction modifier called LOCK which literally locks the memory bus while the access is taking place, forcing existing outstanding memory transactions to complete and stopping any new memory transactions until the current one has completed. Standard variable assigns/accesses don't lock, the Interlocked* functions do.

Unfortunately, this makes the retrieval or update of the value held slightly slower, but that's the price to pay for accessing shared data.

Utility plugins collection and
http://www.matrix1.demon.co.uk for older plug-ins and example code
Back to top
Profile PM Email Website
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 12th Dec 2007 02:55
Link
Hmm...thanx for elaborating - I'm thinking, I'm thinking - I'm aware of the lock command - and it is basically part of the opcodes needed to do sync locking for shared stuff properly etc.

The part I'm struggling with your statement about atomic etc - is this: The memory bus architecture - the actual electronics - per hardware clock cycle - read or write - and they simply can not do both - additionally - the writes are the width of the address bus - so - truly aligned, bus width data (32bit,64, etc)- is not piece mealed - lo-quadqord, hi quadword - unless NOT aligned. So - I'm still a bit hestitant to jump on board here.

Additionally, the LOCK command is used to prevent a memory location from being written to UNTIL - the "CPU that Issued the LOCK" is "Done". Therefore - I'm convinced - (both from theory and tried and true code) that "pushing values one way" - atomically is safe. If passing information where only one thread writes - and the other reads - it will boild down to if thread 2 reads the old value - fine - if it reads a freshly updated value - fine. Definately Safe. I have working source to prove this actually. Multi-threaded web server I wrote - that is faster than LigHttp and apache by over 200% from little assembly tricks like what I'm describing.


http://www.jasonpetersage.com/?PAGE=ironinfantry&SECTION=ironinfantry
Back to top
Profile PM Email Website
Benjamin
21
Years of Service
User Offline
Joined: 24th Nov 2002
Location: France
Posted: 12th Dec 2007 06:11 Edited at: 12th Dec 2007 15:59
Link
To me it seems logical that on a 32-bit processor, a properly-aligned 32-bit read/write operation would be atomic (due to it happening in a single clock cycle), however without reading this from a reliable source I wouldn't trust it.

But, when searching around MSDN for detailed information about synchronization objects, I came across this page, and I quote:

Quote: "Simple reads and writes to properly-aligned 32-bit variables are atomic operations. In other words, you will not end up with only one portion of the variable updated; all bits are updated in an atomic fashion. However, access is not guaranteed to be synchronized. If two threads are reading and writing from the same variable, you cannot determine if one thread will perform its read operation before the other performs its write operation."


[edit] Fixed link to actually point where I got the quote from.

Tempest (DBP/DBCe)
Multisync V1 (DBP/DBCe)
Back to top
Profile PM Email
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 12th Dec 2007 13:38
Link
Exactly - that's why its ok for pushing data - but not good where you need to be certain "when" it was updated. In short - if you know the difference - and how this "trick" can be beneficial - then this is an optimization trick - because you don't bind up the gears where you don't have to.

Now - whenever I have a communication of sorts - where one thread needs to accurately pass info to the other - and the value being correctly managed is a requirement - then mutex/sync is the absolute ONLY way to do it. For example - Thread one gets a network connectin - and passes the info to thread 2 before allowing thread 2 to "open that connection". This requires both mutex/sync and both threads "working together" to make sure the data handoff is accurate and in sync.


http://www.jasonpetersage.com/?PAGE=ironinfantry&SECTION=ironinfantry
Back to top
Profile PM Email Website
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 12th Dec 2007 14:36
Link
I couldn't find that quote you posted Ben, but I've no reason to doubt it's there somewhere, and looking at the Interlocked* functions further, it's clear that they seem to all be of the Read-Modify-Write type, so Jason, I'll withhold any further comments, at least for 32 bit aligned access

Utility plugins collection and
http://www.matrix1.demon.co.uk for older plug-ins and example code
Back to top
Profile PM Email Website
Benjamin
21
Years of Service
User Offline
Joined: 24th Nov 2002
Location: France
Posted: 12th Dec 2007 15:58
Link
Oops, I posted the wrong link.

http://msdn2.microsoft.com/en-us/library/ms684122.aspx

Tempest (DBP/DBCe)
Multisync V1 (DBP/DBCe)
Back to top
Profile PM Email
IanM
Retired Moderator
22
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 12th Dec 2007 19:00
Link
Some of the wording of this worries me:
Quote: "Simple reads and writes to properly aligned 64-bit variables are atomic on 64-bit Windows. Reads and writes to 64-bit values are not guaranteed to be atomic on 32-bit Windows. Reads and writes to variables of other sizes are not guaranteed to be atomic on any platform."

What is 'properly aligned'?

Is it aligned to an address with a multiple of 4, or is it some other multiple due to cache line sizes (anywhere from 8 to 64 bytes IIRC, or maybe even more depending on CPU)?

Is it as simple as using __declspec(align(n)) on your variable declaration?

Utility plugins collection and
http://www.matrix1.demon.co.uk for older plug-ins and example code
Back to top
Profile PM Email Website
jason p sage
17
Years of Service
User Offline
Joined: 10th Jun 2007
Location: Ellington, CT USA
Posted: 12th Dec 2007 19:53 Edited at: 13th Dec 2007 00:20
Link
Aligned is all about the storage in RAM being whatever "Bits" Wide - as the CPU (typically in our x86 world) has the same bus size. So - On a 64bit machine - the Bus is 64bits. The CPU is simply snagging a value - stored at addresses that are actually multiples of ENDIAN/8bits - because Addressing - to my knowledge is still referenced at the byte level.

64/8(bits per byte) = Multiples of 8 for a 64 bit platform.

32/8(bits per byte) = Multiples of 4 for a 32 bit platform.

[edit] I mis-spoke a little here - fixed it to be more true[/edit]
Note - "Other Data" does benefit from being aligned sometimes - not a big enough deal to worry about usually - in fact good assembler's actually allow wasting "Bytes" as necessary to asure that every new "variable" starts at a multiple if possible - sacrificing bytes as necessary to keep things as aligned as possible. For example a "Word" would break up your aligned data - (if there wasn't two of them in a row in your "UDT" or "User Defined Type") so by just skipping the next two bytes so the next variable (a "32bit int for example) - is worth it.
[edit]done edit[/edit]

The Whole "Not Atomic on any platform" jargon basically means - Atomic - is Atomic - and if its not atomic - it's taking the system more than one clock cycle to store the value. This means for the multi-threading stuff - corruption! You might Read HALF of a value during its save - hence - BAD.

[edit]IanM - just saw your declspec question - I don't know the answer to that. In FreePAscal there are "compiler Directives" that tell the compiler to align the data - same in Assembler generally speaking - I don't know the C++/C Construct(s) to such data aligning - and I'm the first to admit - In Iron Infantry - I've been pretty pysched everytime I get something to work at all in C++ etc etc - and have not really even tried to align stuff etc. I'm secretly hoping the C++ compiler does some optimization on its own - I'm paying much more mind to - Class created? YEAH! Did it Destroy correctly? Yeah? etc etc.[/edit]


http://www.jasonpetersage.com/?PAGE=ironinfantry&SECTION=ironinfantry
Back to top
Profile PM Email Website
Back to top

Login to post a reply

Server time is: 2024-11-17 00:59:20
Your offset time is: 2024-11-17 00:59:20