It's probably the fact that it has to scan the whole image and built a frame around it with as few lines as possible. That's never gonna be quick.
My first thought is the same as yours, to have lots of pre-calculated sprites, but I have an alternative as well.
You can set the polygon shape yourself with raw data. Like - say you took note of the lines you need on the sprite, like a sequence of XY coordinates. Well, you could delete the physics shape when the sprite frame changes, then re-apply your own collision shape. That should be much much faster than letting AppGameKit do it itself - and it would lend more control as well - like, having a base line that doesn't move, for platform collision for example... just the sort of thing that might mess up if you change the shape automatically - like a shape could be created that already intersects other sprites, not ideal.
I've been meaning to write an editor for manually setting physics shapes - any sprites that need it will tend to be on a sprite sheet - so really it's the only way apart from sticking to box or circle collision. The way that AppGameKit specifies the shape is quite straightforward - I've even used it to make a 2D planet terrain with hundreds of segments, no issues at all. I did this with DBPro years ago, making a 2D shooter - I just had a big array full of the 2D coordinates and an index - so for a specific image number, I find it in the index which tells me where in the coordinates array to start building the shape.
This obviously takes a bit of planning, but I think it's well worth doing - it'd save you a lot of headaches and free up your games resources - no need to have a copy of every collision enabled frame in your project.
Also - only change the frame if it's actually changing. Like, store the current frame, then only change it if the new frame is different - that way you won't be updating frames that don't need to be updated.
I got a fever, and the only prescription, is more memes.