Page 1 of 3
Server Shutdown error
Posted: Wed Jun 24, 2009 12:49 pm
by lishawj
If you want to receive support we need this info to help you properly.
» Find Revision
L2J Revision
Number:3225
L2JDP Revision
Number:6316
Does anyone know how I may resolve this shut down error? The server just hangs there unless an admin manually closes the dos window. When no admin or server admin is around, my server is pretty much dead. I have a weekly schedule for the server to auto restart so this is becoming a problem for me. Any help identifying the issue is much appreciated.
Someone suggestioned using the option -Djava.awt.headless=true but this did not work for me.
Re: Server Shutdown error
Posted: Mon Jun 29, 2009 1:27 pm
by lishawj
No one running L2J server on Windows Server 2003 R2 is having this issue? I had replaced all 4 GB of system RAM, re-install the OS, re-install Java runtime 6 U14, and I still can not shutdown the server normally. I had even gone as far as running my server on a completely different server and still coming up with the same error upon shutdown.
I started having this problem within the last week or so. I can no longer do automatic restart because the server now hangs upon shutdown/restart calls. The Java.exe thread go skyrocket high and uses 100% of the CPU time when the shutdown is initiated. I am getting many hits on Google when searching for "java AWT-Windows exception" but none that would help me to fix my issue.
I am running my server on the lastest nightly build (last night), Windows Server 2003 R2 (application server role), 4 GB RAM, 150GB HD, Java runtime 6 u14. I need some help here guys, please.
Re: Server Shutdown error
Posted: Mon Jun 29, 2009 1:34 pm
by _DS_
I use 2003R2 standard, 6u3 and does not have such issue.
Re: Server Shutdown error
Posted: Mon Jun 29, 2009 1:59 pm
by janiii
lishawj wrote:The Java.exe thread go skyrocket high and uses 100% of the CPU time when the shutdown is initiated.
this is from the shop item fix. the saving of shops could take 6 minutes on shutdown. just let it save all, wait that 6-10 minutes. like DS said, next time it should be already fine.
Re: Server Shutdown error
Posted: Mon Jun 29, 2009 2:11 pm
by lishawj
janiii wrote:lishawj wrote:The Java.exe thread go skyrocket high and uses 100% of the CPU time when the shutdown is initiated.
this is from the shop item fix. the saving of shops could take 6 minutes on shutdown. just let it save all, wait that 6-10 minutes. like DS said, next time it should be already fine.
Okay, now that I know it is because of the shop fix, should I expect corrupt databases etc if I am forcefully killing the Gameserver console? On the subsequent shutdown, the server will shutdown faster or will it now take 6-10 minutes to shutdown going forward? The reason I am asking is because I am doing maintenance on the server on a regular weekly schedule. What side-effects, if any, are going to be present? I haven't noticed any side effects for premature termination of the game server console but please let me know if there are.
Thank you.
Re: Server Shutdown error
Posted: Mon Jun 29, 2009 10:09 pm
by janiii
hi,
could you try with clean core and dp? and default startGameServer.bat file? maybe some changes or custom mods are creating that error.
Re: Server Shutdown error
Posted: Tue Jun 30, 2009 12:14 am
by MELERIX
Log with clean Core/Dp
Code: Select all
2009.06.29 19:40:28,796 INFO 14 net.sf.l2j.gameserver.ThreadPoolManager All ThreadPools are now stoped2009.06.29 19:40:29,156 INFO 14 net.sf.l2j.gameserver.Shutdown SIGTERM received. Shutting down NOW!2009.06.29 19:41:06,046 INFO 14 net.sf.l2j.gameserver.Shutdown RaidBossSpawnManager: All raidboss info saved!!2009.06.29 19:41:07,562 INFO 14 net.sf.l2j.gameserver.Shutdown GrandBossManager: All Grand Boss info saved!!2009.06.29 19:54:21,171 INFO 14 net.sf.l2j.gameserver.Shutdown TradeController: All count Item Saved2009.06.29 19:54:22,078 INFO 14 net.sf.l2j.gameserver.Shutdown Olympiad System: Data saved!!2009.06.29 19:54:22,437 INFO 14 net.sf.l2j.gameserver.Shutdown Data saved. All players disconnected, shutting down.
13 minutes to shutdown!!!!
according to _DS_ this Changeset is the trouble:
http://www.l2jserver.com/trac/changeset/3206
Re: Server Shutdown error
Posted: Tue Jun 30, 2009 6:04 am
by janiii
L2TradeList.L2TradeItem.restoreInitialCount() is making problems.
after debugging it, these numbers came out:
- called when looping through lists and items from getCurrentCount() method of a L2TradeItem
- L2TradeList.L2TradeItem.restoreInitialCount(): the while loop on line 267 loops each time 346205 times!
- mostly the RejectedExecutionException is thrown
- L2TradeList.L2TradeItem.saveDataTimer() updated for each time it was called (for each item in each list) 1488 items
-
the list has 3 items, 2 of them have count 5:
2009.06.30 07:47:58,489 TradeController: in the for-loop _lists.values(): npcId: 35383; listId: 135383
2009.06.30 07:47:58,489 TradeController: list has limited stock item
2009.06.30 07:47:58,489 TradeController: in the for-loop list.getItems()
2009.06.30 07:47:58,489 TradeController: in the for-loop list.getItems()
2009.06.30 07:47:58,598 L2TradeList.L2TradeItem.restoreInitialCount the while looped 346205 times.
2009.06.30 07:47:58,598 L2TradeList.L2TradeItem RejectedExecutionException!
2009.06.30 07:47:58,847 L2TradeList.L2TradeItem saveDataTimer updated 1488 items
2009.06.30 07:47:58,847 TradeController: in the for-loop list.getItems()
2009.06.30 07:47:58,879 L2TradeList.L2TradeItem.restoreInitialCount the while looped 346205 times.
2009.06.30 07:47:58,925 L2TradeList.L2TradeItem RejectedExecutionException!
2009.06.30 07:47:59,175 L2TradeList.L2TradeItem saveDataTimer updated 1488 items
Re: Server Shutdown error
Posted: Tue Jun 30, 2009 6:37 am
by _DS_
Yeah, i know about it and if today will have sufficient time - will try to get into the tradelists and optimise save.
Re: Server Shutdown error
Posted: Tue Jun 30, 2009 8:17 am
by janiii
it is just a try, just an idea, maybe DS can look at it though..
- calling the getCurrentCount method only once for the update
- changed the while loop to an if statement
- updating the savetimer only for the given item (not for all items with same restore delay)
http://www.pastebin.cz/20512
Code: Select all
Index: java/net/sf/l2j/gameserver/TradeController.java===================================================================--- java/net/sf/l2j/gameserver/TradeController.java (revision 3244)+++ java/net/sf/l2j/gameserver/TradeController.java (working copy)@@ -321,10 +321,11 @@ for (L2TradeItem item : list.getItems()) {- if (item.hasLimitedStock() && item.getCurrentCount() < item.getMaxCount()) //needed?+ long currentCount;+ if (item.hasLimitedStock() && (currentCount = item.getCurrentCount()) < item.getMaxCount()) { statement = con.prepareStatement("UPDATE merchant_buylists SET currentCount=? WHERE item_id=? AND shop_id=?");- statement.setLong(1, item.getCurrentCount());+ statement.setLong(1, currentCount); statement.setInt(2, item.getItemId()); statement.setInt(3, listId); statement.executeUpdate();Index: java/net/sf/l2j/gameserver/model/L2TradeList.java===================================================================--- java/net/sf/l2j/gameserver/model/L2TradeList.java (revision 3222)+++ java/net/sf/l2j/gameserver/model/L2TradeList.java (working copy)@@ -264,9 +264,9 @@ _nextRestoreTime = _nextRestoreTime + this.getRestoreDelay(); // consume until next update is on future- while (this.isPendingStockUpdate() && this.getRestoreDelay() > 0)+ if (this.isPendingStockUpdate() && this.getRestoreDelay() > 0) {- _nextRestoreTime = _nextRestoreTime + this.getRestoreDelay();+ _nextRestoreTime = System.currentTimeMillis() + this.getRestoreDelay(); } // exec asynchronously@@ -352,9 +352,10 @@ try { con = L2DatabaseFactory.getInstance().getConnection();- PreparedStatement statement = con.prepareStatement("UPDATE merchant_buylists SET savetimer =? WHERE time =?");+ PreparedStatement statement = con.prepareStatement("UPDATE merchant_buylists SET savetimer =? WHERE time =? and item_id =?"); statement.setLong(1, _nextRestoreTime); statement.setInt(2, (int) (this.getRestoreDelay()/60/60/1000));+ statement.setInt(3, this.getItemId()); statement.executeUpdate(); statement.close(); }
Re: Server Shutdown error
Posted: Tue Jun 30, 2009 11:24 am
by lishawj
I create two new multisell NPC around the time the shutdown error appeared. I did try a clean core and dp but I also imported the new custom NPC over. Today, after work I will try clean core and dp without the new custom NPC. Thank you guys!
Re: Server Shutdown error
Posted: Tue Jun 30, 2009 10:57 pm
by MELERIX
janiii wrote:it is just a try, just an idea, maybe DS can look at it though..
- calling the getCurrentCount method only once for the update
- changed the while loop to an if statement
- updating the savetimer only for the given item (not for all items with same restore delay)
http://www.pastebin.cz/20512
Code: Select all
Index: java/net/sf/l2j/gameserver/TradeController.java===================================================================--- java/net/sf/l2j/gameserver/TradeController.java (revision 3244)+++ java/net/sf/l2j/gameserver/TradeController.java (working copy)@@ -321,10 +321,11 @@ for (L2TradeItem item : list.getItems()) {- if (item.hasLimitedStock() && item.getCurrentCount() < item.getMaxCount()) //needed?+ long currentCount;+ if (item.hasLimitedStock() && (currentCount = item.getCurrentCount()) < item.getMaxCount()) { statement = con.prepareStatement("UPDATE merchant_buylists SET currentCount=? WHERE item_id=? AND shop_id=?");- statement.setLong(1, item.getCurrentCount());+ statement.setLong(1, currentCount); statement.setInt(2, item.getItemId()); statement.setInt(3, listId); statement.executeUpdate();Index: java/net/sf/l2j/gameserver/model/L2TradeList.java===================================================================--- java/net/sf/l2j/gameserver/model/L2TradeList.java (revision 3222)+++ java/net/sf/l2j/gameserver/model/L2TradeList.java (working copy)@@ -264,9 +264,9 @@ _nextRestoreTime = _nextRestoreTime + this.getRestoreDelay(); // consume until next update is on future- while (this.isPendingStockUpdate() && this.getRestoreDelay() > 0)+ if (this.isPendingStockUpdate() && this.getRestoreDelay() > 0) {- _nextRestoreTime = _nextRestoreTime + this.getRestoreDelay();+ _nextRestoreTime = System.currentTimeMillis() + this.getRestoreDelay(); } // exec asynchronously@@ -352,9 +352,10 @@ try { con = L2DatabaseFactory.getInstance().getConnection();- PreparedStatement statement = con.prepareStatement("UPDATE merchant_buylists SET savetimer =? WHERE time =?");+ PreparedStatement statement = con.prepareStatement("UPDATE merchant_buylists SET savetimer =? WHERE time =? and item_id =?"); statement.setLong(1, _nextRestoreTime); statement.setInt(2, (int) (this.getRestoreDelay()/60/60/1000));+ statement.setInt(3, this.getItemId()); statement.executeUpdate(); statement.close(); }
Tested.
Now shutdown is stucked in
SIGTERM received. Shutting down NOW! & java.exe 100% cpu usage XD
Re: Server Shutdown error
Posted: Wed Jul 01, 2009 1:40 am
by lishawj
Just applied the patch and now my server does not even reach the AWT-Windows exception error.
It's now 25 minutes and counting...
Re: Server Shutdown error
Posted: Wed Jul 01, 2009 5:07 am
by janiii
MELERIX wrote:Tested.
Now shutdown is stucked in SIGTERM received. Shutting down NOW! & java.exe 100% cpu usage XD
i tested it too. this only changes the trade list save. shouldnt change anything else. for me it decreased the saving time of the tradelist.
lishawj wrote:Just applied the patch and now my server does not even reach the AWT-Windows exception error.
It's now 25 minutes and counting...
the patch should only decrease the time of the high cpu usage on shutdown. the awt-error probably comes from some custom code, but not sure. would be good if someone could reproduce the awt-error too.
Re: Server Shutdown error
Posted: Wed Jul 01, 2009 11:54 am
by lishawj
janiii wrote:MELERIX wrote:Tested.
Now shutdown is stucked in SIGTERM received. Shutting down NOW! & java.exe 100% cpu usage XD
i tested it too. this only changes the trade list save. shouldnt change anything else. for me it decreased the saving time of the tradelist.
lishawj wrote:Just applied the patch and now my server does not even reach the AWT-Windows exception error.
It's now 25 minutes and counting...
the patch should only decrease the time of the high cpu usage on shutdown. the awt-error probably comes from some custom code, but not sure. would be good if someone could reproduce the awt-error too.
Okay, I will start with clean core and dp again today without patching it with any of the custom codes I used. On the last test, I used clean core and dp but I patched it with a few custom codes without the custom NPC.
With clean core and dp using only above patch, I was able to shutdown. Now I will have to slowly add my custom item and see what happens.