Understanding basics of L2J Threading

Find all FAQs, HOWTOs and guides here.
Forum rules
READ NOW: L2j Forums Rules of Conduct
Post Reply
User avatar
L2j Inner Circle
L2j Inner Circle
Posts: 912
Joined: Mon Jan 22, 2007 12:14 am
Location: Spain

Understanding basics of L2J Threading

Post by DrHouse » Tue Feb 09, 2010 1:12 am

This is copyrighted stuff to L2JServer project (http://www.l2jserver.com).
All rights reserved. Do not copy, translate, use or reproduce it without permission.

After several years involved in L2J world playing multiple roles like coder (from initiate to inner circle), server owner, professional server developer, server game master, server tech manager, etc, I have realized there are many critical aspects about L2JServer that very few users understand or even take care of them.

One of this "taboo" "cursed" subjects is "Thread Configuration" that you can easily find it on General.properties. I bet 2 against 1 that any administrator has wondered about its effects once at least and I bet 3 against 1 you have heard some guys recommending 'A' and some others recommending just the opposite. That is precisely the target of this tutorial: setting up a set of basic ideas to let you decide your best configuration by yourself

It is structured into three parts: firstly, I will try to write a brief description of how Threads are handled in java, followed by an introduction about L2J implementation and most common patterns; finally, you will be able to find a practical guide to configure Thread Pools in L2J properly. Also, I have tried to write it using simple language, without being too specific: this tutorial is for server administrators and unexperienced developers.

Java Threading introduction

According to Doug Lea, "A thread is a call sequence that executes independently of others, while at the same time possibly sharing underlying system resources such as files, as well as accessing other objects constructed within the same program. A java.lang.Thread object mantains bookkeeping and control for this activity".

As you probably know, java programs (not applets or 'special' stuff) are started by calling main method which takes some parameters within an String array; the thread in charge of this task is called 'Main Thread' by the Java Virtual Machine (JVM). Hence, every java program has, at least, one alive thread during its execution. java.lang.Thread is our OOP abstraction of a real "lightweight process" (also known as sub-process), since Thread objects run within a real OS process: the java virtual machine.

Most important method in a threading evironment is the method "run". It is inhereted from the interface "Runnable" (which is implemented by Thread class). This method is what the thread will actually run: its code will be executed when Thread is started and the JVM

There are, at least, three ways to initialize a new thread in Java, but we will focus just in a couple of them that will let us advance into more deeper knowledge:

Code: Select all

class MyThread extends Thread{   public void run()   {       while(true)           System.out.println("L2JServer rocks");   }}public static void main(String[] args){   MyThread mythread = new MyThread();   mythread.start();}

Code: Select all

 class MyTask implements Runnable{   public void run()   {       while(true)           System.out.println("L2JServer rocks");   }}public static void main(String[] args){   Thread taskthread = new Thread(new MyTask());   taskthread.start();}
Despite being different, both code blocks do provoke the same effect (not 100% but on first approach this would be more than enough). Without getting deeper into more technical specification, I would like to remark an important difference: in the first block we have an object of the class Thread in which we have overridden the method "run()" This thread will execute the code within that method; however, in the second block, we have passed in the constructor argument a new instance of a class which implements itself Runnable interface. This is one of the most important points of this part of the tutorial: thread and task concepts.

Threads execute task(s) within its method run(). We can provide a new thread with a task to execute in its consctructor method (block 2) but also we can define a thread with an specific task by overriding its run method. Task are associated to Runnable interface.

We could go further looking into a great bunch of interesing stuff abut java threading, but this is your own (Runnable :P) task. For now, I would like to end this chapter with four important ideas:
  • A single CPU (with one core, non hyper-threading or similar) can only be executing one thread at the same time
  • In an scenario of multiple threads ready to be executed and just 1 CPU for them, there is not a predictable execution order guaranteed, even considering the priority system implemented for java Threads. This is mostly to accomplish the best code portability between different operative systems.
  • There are several interesting ways to control status of a thread that wont be discussed here (wait, sleep, notify, notifyAll, yield, stop, interrupt, resume, join, ...)
  • Thread initialization has a quite big associated overhead due to different causes so "one-new-thread-per-task-to-execute" pattern (also known as "Thread-per-message") is usually a performance killer even a complete devastator. This is a very important point, actually is in which L2J thread management is based upon. We will talk about it later
L2J Server threading pattern

As any other java program, L2JServer is started by Java Virtual Machine (JVM) calling main method in the main thread (you can find this method in com.l2jserver.gameserver.GameServer class). Here is where everything begins: data is parsed from data sources like SQL/XML/... scripts are compiled, network services are started... and, as you guess, this involves many underlying concurrent lightweight processes: threads!

Since this is a MMO Game Server (MMOGS), almost every event is triggered by an incomming amount of bytes that server must process. Here we find our first 2 threads of the gameserver: 'LoginServerThread', in charge of gameserver<->loginserver IO and 'SelectorThread', performing server<->clients communications. Analyzing SelectorThread operations, specially clients->server will give us a good chance to accomplish the objective of this tutorial: understanding the ThreadPooledExecutor class.

Bytes received from clients are demultiplexed and organized into packets (L2ClientPacket) by SelectorThread. However, these packets bring information from every clientthat requires being processed by gameserver. In other words, they are like "events" sending information and singnaling the event manager, but the gameserver needs to execute those tasks triggered by the received data.

If our MMOGS were prepared to achive a little amount of online clients, we would be able to keep server running with fine performance by creating a new thread for each of this received packets. However, this is not our scenario: a great amount of nowadays servers manage hundred or even thousands of concurrent connections signaling tasks to be processed in gameserver. This why we are not using "thread-per-message" pattern described in previous chapter but a very different one.

On the contrary, we have a (semi)fixed amount of threads performing all received tasks from clients. These threads are usually called worker threads and they have a very interesting implementation, which allow them to execute tasks over and over without needing to initialize a new thread per task. Look at code block:

Code: Select all

class WorkerThread extends Thread{   TasksQueue _sharedQueueOfTasks; // a reference to a list of tasks that requires to be executed    public void run()   {       Runnable currentTask;        while (true) // infinite loop, will poll and execute tasks over and over again       {            currentTask = _sharedQueueOfTasks.pollATask(); // here is where the task is removed from the list             currentTask.run(); // here is where the new task is executed       }    }}
Once initialized and provided with a source of tasks, this thread will retreive and execute a task from the source indefinitly. This is the basics of how a worker thread work, how it eliminateds associated overhead of creating new thread and how it help us to avoid 'thread-per-message' pattern.

When SelectorThread ends with a block of bytes and initialize a packet, it offers that packet to a queue of tasks, which is shared by a set of worker threads that are infinitly feeded by tasks from the same list. The worker threads, plus the queue and a "RejectExecutionHandler" is what we could call "ThreadPoolExecutor".

There are many characteristics about ThreadPoolExecutors, even there are a couple of types, but let's focus on just some basic features:
  • A ThreadPoolExecutor follows producer-consumer pattern. In our scenario, the producer is the SelectorThread (actually every client connected to server, but strictly just this thread) and the consumers are the worker threads inside the pool. Both producers and consumers are connected by a Queue of tasks following FIFO: newer tasks are inserted in the tail and oldest tasks are taken from the head and then are executed by the worker threads.
  • A ThreadPoolExecutor has some paramters able to configure to maximize efficency on every scenario. For example, the min and max number of worker threads that are feeded with tasks to execute from the queue.
  • The queue changes the behaviour of how new worker threads (bounded by minimun and a maximun value) are initializated
  • The queue provide an important feature: when it is empty, the worker threads are not consuming CPU (they are not involved in any 'active' process, they are under state of TIMED_WAITING). This is why the queue must be a BlockingQueue)
  • Some sorts of thread pool executors allow you to execute tasks with delay or over and over again on a fixed rate
On L2JServer we are using Java API ThreadPoolExecutor and ScheduledThreadPoolExecutor to execute almost all the required tasks in gameserver. There are also some other autonomous threads performing tasks, like GameTimeController, ItemsAutoDestroy, etc. For now we are using LinkedBlockingQueue, which is also in Java API (actually we need to change it, thanks NB4L1).

L2J Threading configuration

Finally... here we go! I hope you have read all the previous stuff before getting here :) If not, please scroll up, and scroll up, and scroll up to get to the beginning and read it :P. After it, you should be able to guess a nice answer for your question.

I will finish the last chapter plus a list of recommended further readings during this week (hopefully...) so it's time for you think and guess!

PS: if you even dare to PM asking about a nice config for your server I will smash your head and kick your ass painfully, really really painfully

Leadership and management are not talk and talk, but talk and do

Proud of being a part of this project

Post Reply