Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit/AppGameKit Studio Showcase / Sound Buffering in Tier 1

Author
Message
adambiser
AGK Developer
3
Years of Service
User Offline
Joined: 16th Sep 2015
Location: US
Posted: 29th Dec 2018 20:43 Edited at: 4th Jan 2019 01:22
While considering another plugin for AppGameKit, I wondered whether sound buffering is possible and it turns out it is even though the documentation makes me believe that it shouldn't be. This code works because the documentation for CreateSoundFromMemblock seems to be incorrect when it says that "This will not affect any sound instances already playing, only future ones." CreateSoundFromMemblock does appear to alter playing instances, perhaps only when there's a single instance of the sound. I'm not sure.

This code creates a sound memblock that acts like two sound buffers. The sound is created from the memblock and then played in an infinite loop. One half of the sound loads while the other half is still playing.
Since there is no GetSoundInstancePosition like there is a GetMusicPosition, I use GetMilliseconds to determine when a buffer stops playing and can be reloaded. The timing is resynced each time the sound loops so hopefully it won't be too off.

The sound got choppy when I went below a sound buffer size of 4000. Higher numbers will be more stable, but can cause a noticeable lag when the sound should change (since the playing sound buffer has to finish before the chance is heard).

EDIT: Minimum buffer sizes before choppiness on my system
15 FPS = 8192
30 FPS = 4096
60 FPS = 2048

Anyway, something to toy with:
Virtual Nomad
13
Years of Service
User Offline
Joined: 14th Dec 2005
Location: SF Bay Area, USA
Posted: 30th Dec 2018 04:17
i have to up the buffer to 8192 to get rid of the pop @ 30 FPS but definitely something to play with.

thanks for sharing
adambiser
AGK Developer
3
Years of Service
User Offline
Joined: 16th Sep 2015
Location: US
Posted: 30th Dec 2018 05:07
Thanks for taking trying it out. The minimum buffer size is going to vary per system, but hopefully not a whole lot. 8192 already starts to have a noticeable lag when switching between sine and square wave.
adambiser
AGK Developer
3
Years of Service
User Offline
Joined: 16th Sep 2015
Location: US
Posted: 3rd Jan 2019 04:41
I made a change to the buffer load timing (updated code in first post). I figured it was better to maintain a constant speed and only adjust to GetSystemMilliseconds when the timing gets really off.

@Virtual Nomad: If you get the chance, would you see what minimum buffer size you can use for 30 FPS with this new timing code (updated in the first post)? The size isn't restricted to 2048, 4096, 8192, etc You can try 5000 or 4500, etc.
Thanks.
Virtual Nomad
13
Years of Service
User Offline
Joined: 14th Dec 2005
Location: SF Bay Area, USA
Posted: 3rd Jan 2019 20:27 Edited at: 3rd Jan 2019 20:54
adam,

it wasn't/isn't about the FPS for me where i maintain 30 FPS regardless of buffer size but that i had to increase the buffer to remove to any "POP!" from the sound generated.

with the current code, and sampling ~15-20 seconds at various buffer sizes, i needed a ~3.5K buffer to eliminate the POPs which, of course, are more frequent as the buffer size is decreased.

ie, @ 3k:

1.5k-4k samples attached.

Specs: WIN 8.1, Quad Core @ 1.8 Ghz, 6 Gb DDR3 @ 800 Mhz

add: broadcasting to my samsung j320 required 8k buffer to remove POPs and struggled to maintain 28-30 FPS.

Attachments

Login to view attachments
adambiser
AGK Developer
3
Years of Service
User Offline
Joined: 16th Sep 2015
Location: US
Posted: 3rd Jan 2019 21:00
@Virtual Nomad: So it seems the new timing code works a bit better since you 3500+ works without a pop now whereas you needed 8192 before, right? If so, this is great news.

On my system it takes about 2-3 ms for LoadSoundBuffer to run. It sounds like it takes longer on your system. The old timing code used GetMilliseconds() to calculate the next time to load the buffer, which meant that it would also include LoadSoundBuffer's run time into the calculation. Now it just keeps adding soundBufferLengthMS to the start time each load so it shouldn't matter how long it takes LoadSoundBuffer to run (as long as it doesn't take an entire frame's time).

I was just curious what buffer size you could use for 30 FPS, not because I suspected FPS as being an issue. FPS is only part of the equation. From how I look at things, the theoretical minimum buffer is: (SOUND_SAMPLE_RATE / FPS) * 2
(44100 / 30) * 2 = 2940 samples, but that amount would require exact precision which we can't have. I think 3500 to 4000 sounds good and changes quickly enough not to have any noticeable lag in transitions.

Thanks for checking this out again.
Virtual Nomad
13
Years of Service
User Offline
Joined: 14th Dec 2005
Location: SF Bay Area, USA
Posted: 3rd Jan 2019 21:11 Edited at: 3rd Jan 2019 21:12
a bit more: if i set sync rate to 0 on my phone, i still need an 8k buffer to be POP-free but i get 54-59 FPS. if i select play again, the fps steadily decreases (stopped it when it got around 40; it was losing a frame or 2 per second).

but, yeah. compared to the initial code, a definite improvement.
adambiser
AGK Developer
3
Years of Service
User Offline
Joined: 16th Sep 2015
Location: US
Posted: 3rd Jan 2019 23:05
If you comment out the "Print(status)" line, does your FPS stay stable? The length of time it takes to draw the status string increases as it gets longer. Not noticeable when using a set FPS.

I need to look into why it would pop though. Does yours still pop with that line commented out? What FPS do you get now?
Virtual Nomad
13
Years of Service
User Offline
Joined: 14th Dec 2005
Location: SF Bay Area, USA
Posted: 4th Jan 2019 00:00 Edited at: 4th Jan 2019 00:15
i'm running longer samples now (at least 200 loops) and i rebooted my phone and PC but now i'm also compiling/running more than once sometimes and noting performance fluctuation:

with Sync 0 on my PC = on first run i can go as low as a 750-800 buffer with no POPs & 450-500 FPS. 2nd run produced constant POPs at 800 buffer?
at Sync 30 1st run: i get POPs under 3k. 2nd run, NO pops @ 2k...?
at Sync 60 i get POPs under 2.2k, 2nd run, no pops @ 2k...

on my cell @ sync 0, i get a single POP @ 6k early on and then no pops for the rest of the 200+ loops with 55-65 FPS (twice/consistent results). dropped it to 5k on 3rd run and got ZERO pops in 300 loops with FPS actually spiking into the 70s. same @ 4k buffer for 4th run. @ 3k = consistent POPs
@ sync 30 i got 3 POPs between 115-150 iterations and no others @ 5k buffer. @ 6k buffer, i got 1 POP in the first few loops, then no pops for the remaining 200+.
@ 60 i got a single pop early on then none @ 6k w/ 55-57 FPS. same on 2nd run. 3rd run ZERO pops at 5k. 4th run, 1 pop around loop 48, then none for the next 200+ (still @ 5k).

i can't make heads nor tails out of it; hopefully you can

meanwhile, why are the channels backward?


i see this:

which i guess is "why" but actually looking at them they aren't actually mirrors of each other as i would expect (if i could explain why they were "reversed").

just curious
adambiser
AGK Developer
3
Years of Service
User Offline
Joined: 16th Sep 2015
Location: US
Posted: 4th Jan 2019 01:36
I made another adjustment, this time adding a little bit of time to loadBufferTime during PlaySoundStream in hopes of loading a little after the other buffer starts playing instead of trying to do so immediately. This gave me better results for sync 0 and buffer size 4000.

I also added "LoadSoundBuffer run time" to the printed text to better see how long LoadSoundBuffer takes to run. On my PC it's 2-3 ms, on my cell it's about 25 ms, which is a lot and why the cell will need a bigger buffer. Ideally, this should never pop. I can't get my cell to not pop even at really big sizes, 20K.

What you're seeing in that graphic are an ascending C scale. The low C is all in the left, the high C all in the right. Each bump is a note... They decrease in volume on the left, increase on the right. I calculate the rightBalance first because I'm going from left to right and it makes the math for leftBalance easier.

Many things for me to try to figure out. It could just be the fact that I'm using the sound in a way that AppGameKit isn't intending and there won't be any way for me to get it completely right.
adambiser
AGK Developer
3
Years of Service
User Offline
Joined: 16th Sep 2015
Location: US
Posted: 17th Jan 2019 04:09
Another bit of code. This one generates the notes on startup and stores the samples in a memblock for each note + wave type. I also added sawtooth and triangle wave types for fun.

The end result is that loading the buffer is much faster and shouldn't affect the looping as much as doing all the calculations each time. Since this idea is for a plugin, this seems acceptable for testing since the C++ code should be much faster than the tier 1 code.
Despite this, my phone still wouldn't play it without popping even with a buffer size of 44000.

EdzUp
16
Years of Service
Recently Online
Joined: 8th Sep 2002
Location: Citadel Cyberspace
Posted: 19th Jan 2019 04:14
Would this be able to get a internet radio stream?
-EdzUp
adambiser
AGK Developer
3
Years of Service
User Offline
Joined: 16th Sep 2015
Location: US
Posted: 19th Jan 2019 05:15
@EdzUp: Potentially, if there were a plugin that loaded the sound buffer while decoding the stream. I haven't tried this on Linux or Mac though and Android seems to be hit or miss since Virtual Nomad could get better results than I could on my phone.
This code would probably translate into Tier 2 code pretty easily.

Login to post a reply

Server time is: 2019-05-24 11:41:56
Your offset time is: 2019-05-24 11:41:56