Have found the issue, it is a bit complex to manage but basically a user worked with webshell to delete over 5000 files, and the way webshell operates it actually moved files deleted to a "trash can" and this process was using over 50% of cpu time and IO on the server to process each file move. The problem was only happening at the times they did these deletes in mass(over 500 files each time I have noted), so it is the reason for slowdowns happening randomly.