Thanks everyone for your replies.
Some more points into it:
- @Silvio - As you mentioned, yes sometimes OOM killer invokes jetty process to stop, but many of times it doesnt.
- @Greg - We are using cloud, and running on CentOS 7. We have stopped updates due to some reasons.
- @Joakim - Yes, checked the thread dump and some times OOM killer invokes jetty crash, sometimes I see below properties in dump file:
- Current thread (0x00007f18e01b9800): VMThread [stack: 0x00007f18d33f4000,0x00007f18d34f5000] [id=11584]
- Stack: [0x00007f18d33f4000,0x00007f18d34f5000], sp=0x00007f18d34f3140, free space=1020k
- Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
- V [libjvm.so+0x9a320a] VMError::report_and_die()+0x2ea
- V [libjvm.so+0x498d3b] report_vm_out_of_memory(char const*, int, unsigned long, char const*)+0x9b
- V [libjvm.so+0x82191e] os::Linux::commit_memory_impl(char*, unsigned long, bool)+0xfe
- V [libjvm.so+0x821e69] os::pd_commit_memory(char*, unsigned long, unsigned long, bool)+0x29
- V [libjvm.so+0x81bb6a] os::commit_memory(char*, unsigned long, unsigned long, bool)+0x2a
- V [libjvm.so+0x88d623] PSVirtualSpace::expand_by(unsigned long)+0x53
- V [libjvm.so+0x88e9f8] PSYoungGen::resize_generation(unsigned long, unsigned long)+0xf8
- V [libjvm.so+0x88db62] PSYoungGen::resize(unsigned long, unsigned long)+0x22
- V [libjvm.so+0x88af1d] PSScavenge::invoke_no_policy()+0xf3d
- V [libjvm.so+0x88b761] PSScavenge::invoke()+0x41
- V [libjvm.so+0x843f40] ParallelScavengeHeap::failed_mem_allocate(unsigned long)+0x70
- V [libjvm.so+0x9a4a97] VM_ParallelGCFailedAllocation::doit()+0x97
- V [libjvm.so+0x9abf35] VM_Operation::evaluate()+0x55
- V [libjvm.so+0x9aa2fa] VMThread::evaluate_operation(VM_Operation*)+0xba
- V [libjvm.so+0x9aa67e] VMThread::loop()+0x1ce
- V [libjvm.so+0x9aaaf0] VMThread::run()+0x70
- V [libjvm.so+0x8238c8] java_start(Thread*)+0x108
I do not have much of knowledge in to these configuration, need you guys help.
Is this because of my server goes OutOfMemory, if yes then why it doesnt show the memory occupied more in VisualVM when we check heap size