Are you monitoring temps?
What you're experiencing is what reviewers will call the '0.1% Low' framerate. Essentially, if you're pulling a fairly consistent say, 100fps, and for one or two frame loads the rate drops to say, 40fps, you'll notice a big stutter, before it returns to 100fps and...