I think it's time for a deep down clean.
the idea here is we want to try to get your gun as close to new as possible and see if it shoots good again (instead of doing one area/issue at a time). If not then you have something mechanically bad.
remove the action carrier from the frame (tutorial is under maintenance).
clean using some kind of foaming cleaner or blasting type cleaner and a brush (though a brush won't reach some areas).
LIGHTLY oil everything in action carrier.
clean poly frame (i believe idpa steyr says he uses a dish soap 8O ), i just use a patch slightly dampened w/ hoppes 9, then a dry one to follow.
Slide i believe you know how to do extractor and fp channel (do not over lube, sorry thought the tutorials warned about overoiling the channel).
let it all sitt a day. Lightly reoil the rails and go shott the heck out of it.
if it gives you several hundred trouble free rounds then acts up again (or has a bunch more flakes) then we know that is the issue and will try to isolate it later.
EDIT: Clean mags too, and make sure your mags are marked in some way (i use silver nail polish i got from the wife, and 9mm mags are a,b,c, and .40 are 1,2,3,..)