Another Linux Compile problem

[Zoey76]

Hello :p
You said you had all written to work with SQL right?
Well I was thinking maybe you can write lil routines to parse XML's into tables in your database.
You could execute them in your local PC and then upload the data into your database.

Anyway, good luck

[RiZe]

RiZe ... could you help me understand this, "caching," thing please? I can't get it in my head as to where this data would be cached, if not in a database.

It might help me if I could understand this further.

For example stored as serialized arrays.

EDIT: Btw database is not cache, even systems like community boards uses cache files instead of asking database for same data in every request.

[msknight]

Ah, you mean store all the data in memory? Am I right?

[RiZe]

If you split the data you get from the XML into several parts/files then it will not hurt that much.

[msknight]

I've read up on caching in PHP.

It would mean a marked change from sql statements and results to array searching.

I'm not convinced that I can "search" an array for a match as easily as I can specify a condition in an SQL statement.

Take an item search, for example. I'd have to write the traversing code that searched the array and apply the result to all the name strings of the array; also probably including a case conversion to the searching variable and the array variables in order to make the comparison routine a bit quicker.

The beauty of using SQL is that while the SQL command is being carried out, the PHP timer is suspended; the returned results are already sanitised so it is a simple array traversing where I know I've got correct hits. If I did an array scan manually within PHP, then I'd likely run out of execution time on some of the larger issues.

It would also mean a major re-write of the system.

... and it would still make it impossible for any data stored as XML, to be modified via the dropcalc.

Here's a good question ... what is the purpose of taking the data to XML rather than the database?

I know the downsides of XML ...
*) lack of easy alteration either manually or automatically
*) Difficulty of access and searching by third party systems (such as a dropcalc)
*) Lack of standards enforcement when different developers enter data with different delimeters or different data orders
*) Not all data can be taken to XML, so an SQL engine is still required anyway.

So what is the reason for taking the data to XML?

[msknight]

RiZe,

I'm not with you. You're meaning instead of having to input and parse the data from the XML file in to a database ... you're suggesting that I input and parse the data in to a group of smaller files? That doesn't sound like an improvement over the option of parsing to the database ... to me. What am I missing here?

[poltomb]

...and another thing ... I have a chunk of customising SQL code that removes some of the mobs ... how am I going to manage that in an XML file? It was bad enough having to re-edit the skill for the healing potion, before you put the code back in and made it an option in the server config.

If I've got to edit XML files to alter raid boss respawn times, remove mobs and all the other stuff that I do ... that makes administering a server a whole load more difficult.

Previously, I just loaded up my custom sql file and ran it. Job done in a few seconds.

Now, you tell me I'm going to be consigned to editing loads of XML files ... probably by hand.

That sucks.

It wouldn't be so bad. You could, if you want, write a script to convert XML to SQL. I did that not too long ago, and with java, you can either use plain old wc3 dom or sax. I used dom, when trying to pull info out for recipes and it worked well.

Hmm, so somebody is actually thinking about doing it?
Interesting...

This idea is not a new one. DrLecter, Fulminus, TPB, Aikimaniac and others that I cannot remember discussed this in great length. Then, Lecter, Fulmi, and myself all became very busy (Fulmi had to withdraw from the project) and this idea kinda fizzled out. I would be surprised if this wasn't discussed even before I joined the team.

[msknight]

I meant that I changed the statistics of various mobs, etc. in the game. Easy enough with database, not with XML.

I'll wait to see if anyone comes up with the positives for changing from SQL to XML, but is is very, very likely that if any more of the datapack goes from SQL to XML, then I'll retire my dropcalc.

For the dropcalc to be at its best and also give good management, it needs to be working on live data. If there is a serious move to take a bulk of reference data over to xml, then it will just make it too much hard work as given all I've read so far about PHP memory caching and array searching, the only way forward would be to parse the XML back in to SQL. ... which might not be possible for the larger xml files given the 30 second processing rule ... and as I've said before, I don't think it reasonable to ask people to alter that figure and risk a php program overrunning their box.

[msknight]

If you folks can have a discussion on the direction you're going to take, and when you're going to take it (if any) I'd be very grateful for a heads up so i can plan what I'm doing with the dropcalc.

For the moment, I'll get Freya installed and bring the dropcalc up to match.

[nBd]

Imo, having Static data inside XML's makes things easier. I can understand not everything should be stored in XML, like droplists, npcs.. but Items, classes etc can be put there, since they are not updated that often anyways.

Well, there are easy ways to extract data to SQL. Since you don't need all values for your Dropcalc, a simple script inside the Datapack on the Server running the Dropcalc could install needed Values / Update them if changed in the database.. You understand what iam pointing on? I dont use any dropcalc on my Server, but well, this could be done easy, as iam using simular system for some other XML -> SQL conversion.

[_DS_]

Code: Select all

                if (file_exists("inc/cache/".$jump."_".$type.".htm"))                {                        include("inc/cache/".$jump."_".$type.".htm");                        $b_info = '<br>(оНЯКЕДМЕЕ НАМНБКЕМХЕ: '.date("F d Y H:i:s", filemtime("inc/cache/".$jump."_".$type.".htm")).')';                }                else                {                        ob_start();                        $fp = fopen("inc/cache/".$jump."_".$type.".htm",'w+');                        include("inc/$jump.php");                        $data = ob_get_contents();                        fputs($fp, $data);                        fclose($fp);                        $b_info = '<br>(CРПЮМХЖЮ АШКЮ ГЮМЕЯЕМЮ Б ЙЕЬ)';                        ob_end_flush();                } 

[jurchiks]

Jesus Christ msknight, SO MUCH complaining (10 posts?) and for what??
It's not like you change drop data every 30 minutes for months WTF man?
You do your changes in xml file, then parse it to your sql and that's it! One time batch job!
For YOU sql is better, average server owners wouldn't feel any difference!
Look at aionemu/unique/lightning, ALL their static data is in XML files with nice schemas, sql contains ONLY char/ally and similar DYNAMIC info, and the performance is really good.

[blacksea]

Jesus Christ msknight, SO MUCH complaining (10 posts?) and for what??
It's not like you change drop data every 30 minutes for months WTF man?
You do your changes in xml file, then parse it to your sql and that's it! One time batch job!
For YOU sql is better, average server owners wouldn't feel any difference!
Look at aionemu/unique/lightning, ALL their static data is in XML files with nice schemas, sql contains ONLY char/ally and similar DYNAMIC info, and the performance is really good.

girl:)

[poltomb]

I meant that I changed the statistics of various mobs, etc. in the game. Easy enough with database, not with XML.

There are ways to do this, but it would take a tool to be found/made to do it specifically for L2j. http://www.w3schools.com/xquery/default.asp

For the dropcalc to be at its best and also give good management, it needs to be working on live data. If there is a serious move to take a bulk of reference data over to xml, then it will just make it too much hard work as given all I've read so far about PHP memory caching and array searching, the only way forward would be to parse the XML back in to SQL. ... which might not be possible for the larger xml files given the 30 second processing rule ... and as I've said before, I don't think it reasonable to ask people to alter that figure and risk a php program overrunning their box.

You run your dropcalc on the same sql server that runs your server? I would say that isn't very good practice If not, then you could code a script in whatever language you want that can:
1) take in xml
2) parse it and export to sql query format
3) execute the scripts on the sql server you are using for dropcalc
Optional) to optimize this process, maybe you can just diff old xml and new xml files and parse the diff output into updates for your dropcalc tables.

In case you were wondering, I was against moving everything to XML. I wanted to do the opposite: move everything to sql, skills and all (I think so did Fulmi). The problem is that SQL is not flexible enough unless the tables are split into many. This kind of system could create some processing overhead in the server load (or when the data is reloaded), slowing load times.

@jurkis, do you only complain, disagree, or argue on the forum now? Lately that's all it's been from you.

@blacksea: please keep your sexist remarks out of the forum.

[jurchiks]

Actually I sometimes do help people, but only if it's not pointless (there are some that don't understand anything you say to them) or they're not aggressive like this guy (or if i'm interested).
And STOP writing my nickname wrong!! This is the second post where you write it wrong, it resembles "surskis" more then my nickname!

@blacksea - wrong guess. Not to mention out of place.