[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [jetty-users] Need help with finding logs when Jetty killed
|
Dear Toshimichi,
I would start with checking /var/log/syslog and look for a kernel panic.
Under extreme situations the kernel may decide it needs to kill one or
more resource hungry processes. I have seen many Java processes being
killed because of out of memory situations, especially if the maximum
heap size (-Xmx...) is too large considering the available physical
memory and possible memory demands from other applications.
Lacking that the only situations where a Java server process would
suddenly die on me was when I employed a third party library that
decided to call System#exit under some error conditions. That was quite
a puzzler...
Good luck,
Silvio
On 25-07-17 10:47, p-naitou@xxxxxxxx wrote:
Hi,
We are using Jetty 9.3.9.v20160517.
Sometimes, Jetty's process killed.
We checked following logs, but couldn't find the cause.
- /var/log/messages
-- can't find oom killer or signal messages.
- /var/jetty/logs/*.stderrout.log
-- no exception messages.
- /var/jetty/logs/*.request.log
-- everytime final request didn't make response, but no relation on each endpoints.
- application custom logs
-- when hundle uncaught exception, put stacktrace on log file. but no logs.
I tried some JAVA_OPTIONS settings and finally Jetty looks running stably.
Probably "CompressedClassSpaceSize" or "MetaspaceSize" was too small.
Because after increase two parameter's value, Jetty isn't killed.
I have a question.
- Is there no log when Jetty killed by "CompressedClassSpaceSize" and/or "MetaspaceSize" shortage?
- How can I find evidence that "CompressedClassSpaceSize" and/or "MetaspaceSize" is wrong?
- Is there a possibility that these guesses are wrong and this is Jetty's issue?
Environment is follows:
- aws ec2 t2.2xlarge(4vCPU, 16GiB), Amazon linux 2017.03
- jetty-distribution-9.3.9.v20160517
- OpenJDK Runtime Environment (build 1.8.0_131-b11)
- hosts 3 web application(all apps are using Spring FW 4.1.7)
seting files are follows:
----------------
# cat /etc/default/jetty
JETTY_HOME=/opt/jetty
JETTY_BASE=/var/jetty
JETTY_RUN=/var/run
JAVA_OPTIONS="\
-Xms16384m \
-Xmx16384m \
-XX:CompressedClassSpaceSize=80m \
-XX:MetaspaceSize=128m \
-XX:MaxMetaspaceSize=192m \
-verbose:gc \
-Xloggc:/tmp/jetty.gc.`date +%Y%m%d-%H%M%S`.log \
-XX:+UseGCLogFileRotation \
-XX:GCLogFileSize=10M \
-XX:NumberOfGCLogFiles=5 \
-XX:+PrintGC \
-XX:+PrintGCDetails \
-XX:+PrintGCDateStamps \
-XX:+PrintGCTimeStamps \
-XX:+PrintClassHistogram \
-XX:+HeapDumpOnOutOfMemoryError \
-XX:HeapDumpPath=/tmp \
-XX:+ExitOnOutOfMemoryError \
-XX:ErrorFile=/tmp/jetty.hs_err_pid%p.log \
-XX:+UseConcMarkSweepGC \
-XX:CMSInitiatingOccupancyFraction=70 \
-XX:+UseCMSInitiatingOccupancyOnly \
-Dspring.profiles.active=stg \
-Dfile.encoding=UTF-8 \
"
----------------
new CompressedClassSpaceSize is 200m
new MaxMetaspaceSize is 512m
----------------
# cat /var/jetty/start.ini
jetty.server.dumpBeforeStop=true
--module=home-base-warning
--module=ext
--module=resources
--module=server
jetty.threadPool.minThreads=10
jetty.threadPool.maxThreads=100
jetty.httpConfig.sendServerVersion=false
--module=http
jetty.http.host=0.0.0.0
--module=deploy
jetty.deploy.monitoredDir=webapps
jetty.deploy.scanInterval=0
jetty.deploy.extractWars=true
--module=jsp
--module=requestlog
jetty.logging.dir=${jetty.base}/logs
jetty.requestlog.dir=/logs
jetty.requestlog.filePath=${jetty.requestlog.dir}/yyyy_mm_dd.request.log
jetty.requestlog.filenameDateFormat=yyyy_MM_dd
jetty.requestlog.timezone=JST
--module=debug
jetty.debug.timezone=JST
--module=setuid
----------------
Thanks,
Toshimichi Naito
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
