I'm actually a PC player and my specs will be below:
GTX 970
Windows 7 64-bit
16GB RAM
i5 4670k
I'm running everything on Ultra or more plus 2xMSAA, I recently overclocked to avoid this issue and I actually got some frames back which was an amazing feeling. I currently sit at around 45-60 around there, the drop in 15 frames doesn't make sense to me and is usually prevalent in the centre of the village. Getting such a large drop makes me think that it is an issue from the village itself.
There's something real wrong with your PC, my friend is playing with an i5 4440 and a GTX 970 and the game stills 60 Frames all the time at Ultra settings, but he doesn't use MSAA, on Redcliffe his frames always dropped from 60 to 35, when he turned off the MSAA and the set the Tessellation to High the frames returned to 60.
My PC had something like it, in Redcliffe my frames dropped from 60 to 58, my solution was to disable Origin overlay and any other programs while I was playing, like Raptr and etc.