MongoDB: Odd write performance

I have been running into odd write performance as well using MongoDB 3.0.5 and mongo java driver 2.13.1. If the application has been idle for a given amount of time, the next write will take about 17 seconds, then any further writes are immediate. Preceding read is mere milliseconds. So, as observed, with exact same record count and size. There are only a few dozen records, and they are all rather small. Using default Write Concern, which is ACKNOWLEDGE.

Idle. Read ~10ms. Write 17 seconds. Read ~10ms. Write ~25ms.

Code sample is simple, with a rudimentary timer for above results:

Long timer = System.currentTimeMillis();
Stack<BasicDBObject> stack = new Stack<>();
DBCursor cloneCursor = cloneMongo.getCollection("ourCollection").find();
while (cloneCursor.hasNext())
{
    BasicDBObject cloneRec = (BasicDBObject) cloneCursor.next();
    if (cloneRec.containsField("name"))
    {
    if (cloneRec.getString("name").equals("project"))
    {
        cloneRec.put("default", destProject.getID());
    }
    }
    stack.add(cloneRec);
}
Util.print("Clone - Clone Read:"+(System.currentTimeMillis() - timer), 1);
DBCollection collection = cloneMongo.getCollection("sds_structure_" + destProject.getName());
BulkWriteOperation bulkOp = collection.initializeUnorderedBulkOperation();
Util.print("Clone - Clone DestConnect:"+(System.currentTimeMillis() - timer), 1);
while (!stack.isEmpty())
{
    bulkOp.insert(stack.pop());
}
bulkOp.execute();
Util.print("Clone - Clone Write:"+(System.currentTimeMillis() - timer), 1);

We have observed this exact pattern on Platter Disks, FusionIO drives, and 15kRPM SAS Drives, so I am dubious it is IO performance. If we set the write concern to JOURNALED, we consistently get 17 Second writes on all attempts.

The interim solution I found works is to periodically (~10 secs) write junk (even increment a value in an existing record) to a junk collection. This seems to fool the system, avoiding whatever is causing it to 'fall asleep'.

What settings in MongoDB or changes to our logic can avoid these abysmally slow initial writes?

from Newest questions tagged java - Stack Overflow http://ift.tt/1Lytn6n
via IFTTT