NetPro: Packet Analysis and Visualization tool

[rocknowx]

Some packetid not have their own xml file:
client:
CM_REQ_TELEPORT (prelude_pts-1A / the_kamael-33)
130 (c1_harbingers_of_war-7A / c2_age_of_splendor-69 / the_kamael-84)
276 (gracia_final-D0 59 00 / lindvior-D0 56 00)
CM_REQ_EX_EVENT_CAMPAIGN_INFO (infinite_odyssey-D0 D2 00)
456 (ertheia-D0 FF 00)
461 (ertheia_update_1-D0 04 01)
CM_REQ_VIP_ATTENDANCE_ITEM_LIST (infinite_odyssey-D0 05 01)
CM_REQ_VIP_ATTENDANCE_CHECK (infinite_odyssey-D0 06 01)
CM_REQ_VIP_PRODUCT_LIST (infinite_odyssey-D0 07 01)
CM_REQ_VIP_LUCKY_GAME_INFO (infinite_odyssey-D0 08 01)
CM_REQ_VIP_LUCKY_GAME_ITEM_LIST (infinite_odyssey-D0 09 01)
CM_REQ_VIP_LUCKY_GAME_BONUS (infinite_odyssey-D0 0A 01)
CM_EX_REQ_VIP_INFO (infinite_odyssey-D0 0B 01)
CM_REQ_CAPTCHA_ANSWER (infinite_odyssey-D0 0C 01)
CM_REQ_REFRESH_CAPTCHA_IMAGE (infinite_odyssey-D0 0D 01)

server:
306 (c3_rise_of_darkness_update_1-FE 11 00 / the_kamael-FE 0B 00)
355 (c5_oath_of_blood-FE 42 00 / the_kamael-FE 43 00 / tauti_update_1-FE 44 00)
409 (the_kamael-FE 71 00 / tauti_update_1-FE 72 00)
410 (the_kamael-FE 72 00 / tauti_update_1-FE 73 00)
411 (the_kamael-FE 73 00 / tauti_update_1-FE 74 00)
442 (gracia_final-FE 97 00 / tauti_update_1-FE 98 00)
443 (gracia_final-FE 98 00 / tauti_update_1-FE 99 00)
486 (gracia_epilogue_update_1-FE C3 00 / freya-FE D3 00 / high_five-FE DE 00 / tauti_update_1-FE DF 00)
571 (ertheia-FE 04 01)
676 (ertheia-FE 74 01)
677 (ertheia-FE 75 01)
678 (ertheia-FE 76 01)
680 (ertheia_update_1-FE 78 01)
SM_EX_VIP_ATTENDANCE_ITEM_LIST (infinite_odyssey-FE 7C 01)
SM_EX_CONFIRM_VIP_ATTENDANCE_CHECK (infinite_odyssey-FE 7D 01)
SM_RECIVE_VIP_PRODUCT_LIST (infinite_odyssey-FE 7E 01)
SM_RECIVE_VIP_LUCKY_GAME_INFO (infinite_odyssey-FE 7F 01)
SM_RECIVE_VIP_LUCKY_GAME_ITEM_LIST (infinite_odyssey-FE 80 01)
SM_RECIVE_VIP_LUCKY_GAME_RESULT (infinite_odyssey-FE 81 01)
SM_RECIVE_VIP_INFO (infinite_odyssey-FE 82 01)
SM_RECIVE_VIP_INFO_REMAIN_TIME (infinite_odyssey-FE 83 01)
SM_RECEIVE_VIP_BOT_CAPTCHA_IMAGE (infinite_odyssey-FE 84 01)
SM_RECEIVE_VIP_BOT_CAPTCHA_ANSWER_RESULT (infinite_odyssey-FE 85 01)
532 (ertheia-FF)

[SaveGame]

Obviously, that's because I either have no clue about their structure, or possibly because they are not yet used in NA, or have been deprecated, like union packets, or were only used in internal builds/alpha, or because certain old clients included L2N*C, Ca***D and A***D internal packets as well (e.g. LeaveWorld, AuthServerList, etc.)

So, let's review this list:
CM_REQ_TELEPORT – never sent by the client, and curiously removed from client at Kamael, along with known L2N*C packets, such as LeaveWorld.
130 (RequestReviveReply) – the associated SM was not used by the server, and this feature has been completely deprecated when ConfirmDlg arrived in C3
276 (RequestExCleftEnter) – I have just started on C5-GF packets, will be able to comment after I'm done
CM_REQ_EX_EVENT_CAMPAIGN_INFO – Most likely related to dimensional raids (new campaign system); the NPC was missing and exalted quest chain was broken in NA until very recently; plus, I don't have a qualifying character to test, so structure is unknown.
456 (RequestAlchemySkillList) – Sounds like one of those packets that were lost due to wrong opcodes in packet handlers when converting old, non-incremental packet XMLs. I'll have to revisit them.
461 (RequestExAutoFish) – Haven't yet tried the "new new new" fishing system, but I will; already turned in my "deprecated deprecated" fishing gear and then also turned in my "deprecated" gear. Now, I only have to buy current gear before it becomes deprecated again
CM_REQ_VIP_* – Not used in NA, as far as I understand. LUCKY_GAME stuff might be used in some future event.
Same goes for CAPTCHA packets: not used in NA at this time, so no verified structure yet.


And the server list…
306 (ExEventMatchLockResult) – not used by the server; plus, injecting it will not lock the result for viewing, as strange as that may sound.
355, 409, 410, 411, 442, 443 – I have just started on C5-GF packets, will be able to comment after I'm done
486 (ExBR_AgathionEnergyInfo) – structure is not known
571 (ExGetWebSessionID) – structure is not known, plus I don't think it was used in NA
676, 677, 678 – Sounds like one of those packets that were lost due to wrong opcodes in packet handlers when converting old, non-incremental packet XMLs. I'll have to revisit them.
680 (ExReceiveClientINI) – not used by the server, at least for now
VIP & CAPTCHA – not used in NA at this time
532 – Sounds like another packet handler conversion issue, though I may be wrong on this one. Not used by the server in any case.


I hope that clears some things up.

Regards,
SaveGame

[rocknowx]

v1.1 is Good
I try to compile develop version, but failure

Code: Select all

[INFO] Scanning for projects...
[INFO] 
[INFO] Using the builder org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder with a thread count of 1
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building l2emuproject-netpro-app-1.2-SNAPSHOT 1.2-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] Downloading: http://maven.revengineer.eu:8081/nexus/content/groups/public/net/l2emuproject/l2emuproject-netpro-commons-l2/1.2-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: http://maven.revengineer.eu:8081/nexus/content/groups/public/net/l2emuproject/l2emuproject-netpro-commons-l2/1.2-SNAPSHOT/l2emuproject-netpro-commons-l2-1.2-SNAPSHOT.pom
[WARNING] The POM for net.l2emuproject:l2emuproject-netpro-commons-l2:jar:1.2-SNAPSHOT is missing, no dependency information available
[INFO] Downloading: http://maven.revengineer.eu:8081/nexus/content/groups/public/net/l2emuproject/l2emuproject-netpro-commons-l2/1.2-SNAPSHOT/l2emuproject-netpro-commons-l2-1.2-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project l2emuproject-netpro-app: Could not resolve dependencies for project net.l2emuproject:l2emuproject-netpro-app:jar:1.2-SNAPSHOT: Could not find artifact net.l2emuproject:l2emuproject-netpro-commons-l2:jar:1.2-SNAPSHOT in revengineer (http://maven.revengineer.eu:8081/nexus/content/groups/public/) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

[SaveGame]

Yeah, since it's a snapshot (moving target), I do not deploy dependencies, since I might forget to update them later

Anyway. You should just do maven 'install' on l2emuproject-netpro-main, it will build the required dependencies and deploy to your local maven repository for use. I have also deployed those dependencies to the remote repository, so you should no longer face any issues in the future.

On the other hand, I recall 'Resolve Workspace Artifacts' having some side-effects, but I can't recall what they were. I'll have to investigate this again, since it would definitely be the easiest solution to this build 'failure'.

[rocknowx]

4 WARNING in log

Code: Select all

WARNING proxy.ui.savormix.io.EndpointPacketLoader.visitFile(): Opcode(s) are not mapped for packet CM_SEND_L2_PARAM_SETTING (SendL2ParamSetting)
WARNING proxy.ui.savormix.io.EndpointPacketLoader.visitFile(): Opcode(s) are not mapped for packet CM_SUPER_CMD_CHARACTER_INFO (SuperCmd:CharacterInfo)
WARNING proxy.ui.savormix.io.EndpointPacketLoader.visitFile(): Opcode(s) are not mapped for packet CM_SUPER_CMD_SERVER_STATUS (SuperCmd:ServerStatus)
WARNING proxy.ui.savormix.io.EndpointPacketLoader.visitFile(): Opcode(s) are not mapped for packet CM_SUPER_CMD_SUMMON_CMD (SuperCmd:SummonCmd)
WARNING proxy.ui.savormix.io.ProtocolDefinitions$ProtocolCollisionResolver.apply(): Collision! [557] C3 Rise Of Darkness Update 2 2005-05-12 -> [557] Lindvior Update 3 2013-07-24

[SaveGame]

4 WARNING in log

Missing opcode mappings are for 2-op client packets that use the deprecated 'cc' prefix. For performance reasons, current implementation only supports c/ch/chd. I didn't pay much attention to those packets, since by prelude beta (336) they were already superseded by Alt+G's CM RequestGMCommand. Only previous clients (and I don't think that includes L2_2002_Alpha, so NC-internal builds only) used them.

The protocol collision warning is unavoidable, since a certain C3 version and a certain Lindvior (GDL) version uses the same protocol revision number. If you want NP to use C3 definitions for clients that report version 557, just mark the associated GDL version with 'disabled="true"' in all_known_protocols.xml

[Zealar]

1) How to save capture data?
2) Logs get huge spam

Code: Select all

net.l2emuproject.proxy.network.meta.exception.InvalidFieldValueInterpreterException: null(ASCII) cannot be cast to net.l2emuproject.proxy.network.meta.FieldValueInterpreter
	at net.l2emuproject.proxy.network.meta.container.MetaclassRegistry.getInterpreter(MetaclassRegistry.java:186)
	at net.l2emuproject.proxy.ui.savormix.io.EndpointPacketLoader.toStructureElement(EndpointPacketLoader.java:235)
	at net.l2emuproject.proxy.ui.savormix.io.EndpointPacketLoader.childrenOf(EndpointPacketLoader.java:172)
	at net.l2emuproject.proxy.ui.savormix.io.EndpointPacketLoader.visitFile(EndpointPacketLoader.java:140)
	at net.l2emuproject.proxy.ui.savormix.io.EndpointPacketLoader.visitFile(EndpointPacketLoader.java:69)
	at java.nio.file.Files.walkFileTree(Unknown Source)
	at java.nio.file.Files.walkFileTree(Unknown Source)
	at net.l2emuproject.proxy.ui.savormix.io.VersionnedPacketTable.loadConfig(VersionnedPacketTable.java:442)
	at net.l2emuproject.proxy.ui.savormix.io.VersionnedPacketTable.<init>(VersionnedPacketTable.java:86)
	at net.l2emuproject.proxy.ui.savormix.io.VersionnedPacketTable$SingletonHolder.<clinit>(VersionnedPacketTable.java:792)
	at net.l2emuproject.proxy.ui.savormix.io.VersionnedPacketTable.getInstance(VersionnedPacketTable.java:787)
	at net.l2emuproject.proxy.L2Proxy.main(L2Proxy.java:91)
	at net.l2emuproject.proxy.ui.savormix.loader.Loader.main(Loader.java:265)

[SaveGame]

1) How to save capture data?
2) Logs get huge spam

Code: Select all

net.l2emuproject.proxy.network.meta.exception.InvalidFieldValueInterpreterException: null(ASCII) cannot be cast to net.l2emuproject.proxy.network.meta.FieldValueInterpreter
	at net.l2emuproject.proxy.network.meta.container.MetaclassRegistry.getInterpreter(MetaclassRegistry.java:186)
	at net.l2emuproject.proxy.ui.savormix.io.EndpointPacketLoader.toStructureElement(EndpointPacketLoader.java:235)
	at net.l2emuproject.proxy.ui.savormix.io.EndpointPacketLoader.childrenOf(EndpointPacketLoader.java:172)
	at net.l2emuproject.proxy.ui.savormix.io.EndpointPacketLoader.visitFile(EndpointPacketLoader.java:140)
	at net.l2emuproject.proxy.ui.savormix.io.EndpointPacketLoader.visitFile(EndpointPacketLoader.java:69)
	at java.nio.file.Files.walkFileTree(Unknown Source)
	at java.nio.file.Files.walkFileTree(Unknown Source)
	at net.l2emuproject.proxy.ui.savormix.io.VersionnedPacketTable.loadConfig(VersionnedPacketTable.java:442)
	at net.l2emuproject.proxy.ui.savormix.io.VersionnedPacketTable.<init>(VersionnedPacketTable.java:86)
	at net.l2emuproject.proxy.ui.savormix.io.VersionnedPacketTable$SingletonHolder.<clinit>(VersionnedPacketTable.java:792)
	at net.l2emuproject.proxy.ui.savormix.io.VersionnedPacketTable.getInstance(VersionnedPacketTable.java:787)
	at net.l2emuproject.proxy.L2Proxy.main(L2Proxy.java:91)
	at net.l2emuproject.proxy.ui.savormix.loader.Loader.main(Loader.java:265)

1. Saved automatically, with no exceptions.
2. Either run via a JDK executable, or download the correct script.cache (based on the version you use).

[Zealar]

Well that script.cache file help about errors.
But still cannot find where capture data is saved (file)

[SaveGame]

Well that script.cache file help about errors.
But still cannot find where capture data is saved (file)

user.home

[Zealar]

So "C:\Users\%user%\l2emu-unique-netpro"

Well how to extract that some possible to read text form? Like "l2phx" format "*.pLog"
Or in other words how to can parse that file?

[SaveGame]

So "C:\Users\%user%\l2emu-unique-netpro"

Well how to extract that some possible to read text form? Like "l2phx" format "*.pLog"
Or in other words how to can parse that file?

Sorry for the ultra short replies recently. Having a flu or something, not really feeling well.

It's true, the file does not contain a raw application layer transmitted data dump. Currently there is no way to convert it to any other format (mostly because nobody has yet asked for this). To preview via the NP GUI, use [menu bar] File -> Open.

I do understand that this is not very convenient when you want to find a specific value in the whole log.

If you describe what output formats you would like, I'd look into implementing the conversions. Just FYI, only app layer data is available in these logs - no TCP or lower level data.

[Zealar]

Just need some format possible to parse.
What i do now with l2phx is log in server and leave character to hit something. Then open log file parse all damage from message and calculate proper dmg without randoms also hit/miss/critical and stuff.

[SaveGame]

Consider implementing a log loading script, use the PPE variation. There's no guide, though. Parsing raw data will be very time consuming to adjust offsets each time something changes: PPE will only require you to mark fields in packet definitions.

I'll try to make at least a primitive example when I get better.

[Zealar]

My guess is best option is NetPro to support save in raw data and xml form what keep packet structure just fill the data.