EQ2Interface.com
Search Downloads


Go Back   EQ2Interface > Utilities

Category: UtilitiesEQ2 Log Processor
Interface Information
Download
How do I install this? (FAQ)
Name: EQ2 Log Processor   Popular!
Author:
Date: 05-06-2008 01:41 PM
Size: 6.91 MB
Version: 1.0.0.46
Rate Addon: 5 out of 5 with 2 votes  
Pictures
Click to enlarge 

Views: 0
Size: 114.71 Kb
Dimensions: 621 x 548
Main window
 
Click to enlarge 

Views: 0
Size: 53.79 Kb
Dimensions: 598 x 582
Database Options
 
Click to enlarge 

Views: 0
Size: 62.79 Kb
Dimensions: 597 x 582
File Options
 
Click to enlarge 

Views: 0
Size: 46.39 Kb
Dimensions: 597 x 580
Filtering Options
 
Click to enlarge 

Views: 0
Size: 72.38 Kb
Dimensions: 598 x 582
Chat Parsing Options
 
Description
Link to main download site: http://www.eq2tools.com/
Please visit the above link for detailed info on usage and updates. I'll try to update this as much as possible as well...

EQ2 Log Processor is an application designed to extract various bits of data from your EverQuest 2 log files. Currently it will separate chat, mob data (kill locations, names, zones), items, loot data (dropped by what mob/where), location commands (using /loc from in game), mail attachments, broker sales and system messages.

Future releases will include a combat parser.

It's a work in progress, it does have a help file (not completely finished) and a number of utilities built in, and everything it does can get dumped to MS Excel (if you have Excel installed). Just drag and drop your log file on it and off you go.

If anyone decides to use it and has any issues with it please feel free to let me know and I'll fix em for you. There will be some minor bugs here and there I'm sure, I'm still finding some and it's almost been a year in development.

The chat processing will let you filter the output based on various criteria and will also let you format the output in various ways. If you are lost with any part of it, just press F1 and if the help doesn't explain something sufficiently let me know and I'll answer your questions and make a note to update the help to address the lack of clarity.

There's an interface for executing SQL as well as looking at raw table data. Walkthrough creation tool (please see website for more info on this).

Right now this is mainly geared towards a tool to be used in setting up and maintaining data on fansites, but works quite well for everyday use.

It will also archive your logs and organize them in zip files based on the options you select for it.

The options for the program can be saved to file as well as set as default and stored in the registry. All options and registry keys used are documented in the help file. All of the data from you log files can be stored via various database platforms and is completely optional. All settings that store passwords for database servers are encrypted when stored to the registry or options file.

There is no installation necessary, just unzip where you want it and run the executable. I can't guarantee it will work under Vista, it should, but there is the possibility some of it won't.

It was written in Delphi 5 (object oriented pascal) and the local storage platform utilizes Paradox 7 tables. The functionality of SQL is quite limited when querying the local tables, so if you want expanded functionality I would recommend using MySQL or PostgreSQL at the moment. I built it for MySQL 5, I have not tested it on releases prior to that. PostgreSQL support is built for PostgreSQL 8, but will probably work on earlier versions as well.

-------------------------------------------------------------------------
November 16th, 2007 - build 1.0.0.37

1. Storage of mail data via MySQL finished.
2. Fixed some bugs with storing mob and loot data where item IDs and mob IDs would get misaligned, or just not store records all together.
3. Fixed mob IDs being assigned random large integers, rather than a properly generated ID.

-------------------------------------------------------------------------
November 18th, 2007 - build 1.0.0.38

1. A bug with storing log data to databases during auto archiving has been fixed.

-------------------------------------------------------------------------

November 21st, 2007 - build 1.0.0.39

1. MySQL storage of mail and sales data complete.
2. Bug with realigning item IDs on loot, mail and sales data in a new update fixed.

-------------------------------------------------------------------------

December 6th, 2007 - build 1.0.0.40

1. PostgreSQL storage of mail and sales data complete.
2. Bug with realigning item IDs on loot, mail and sales data in a new update fixed.
3. Fatal bug where connecting to a MySQL or PostgreSQL database that isn't available during the storing of data caused an access violation that terminated the program has been fixed.
4. Time window for archiving data has been implemented.

-------------------------------------------------------------------------

December 19th, 2007 - build 1.0.0.41:

1. Fixed a bug with the alignment of item and mob IDs on loot data when storing the data using local storage.

-------------------------------------------------------------------------

January 17th, 2008 - build 1.0.0.42:

1. Added a visual query builder with more export formats like HTML, Word and comma delimited.
2. Fixed a bug with the alignment of new items when storing the data using local storage.

-------------------------------------------------------------------------

January 18th, 2008 - build 1.0.0.43:

1. Fixed a bug with the BDE reporting insufficient disk space when your free space is divisible by 4.

-------------------------------------------------------------------------

February 13th, 2008 - build 1.0.0.44


1. Auto Archive time editors behaving badly with a clean set of options fixed.
2. Visual Query Builder taking an access violation when opening a query then switching datasource to something else fixed.
3. Temp tables created before opening the Visual Query Builder if there is a log loaded, otherwise the ability to select temporary tables as a datasource from the Visual Query Builder is disabled.
4. Made the "filter" button and chat filter menu item clear that it only affects chat data.
5. Fixed loot data for stuff that is looted and assigned by the group leader, rather than using lotto or free for all looting options.
6. Added the name of the person who looted an item to the loot data.
7. Fixed some oddities with the tray icon when running in archive mode.

-------------------------------------------------------------------------

March 24th, 2008 - build 1.0.0.45


Mob locator is now complete and ready for use. If you are going to use any other method for looking up stuff other than a freshly loaded log file you will have to load a new one and store the data in order to execute the procedures for calculating unique locations. When storing mob data to permanent databases, a new table is created called "uniquekill". I decided to actually store them instead of recalculate the unique locations each time you open the search. The longer you have been storing the data in your log files to databases, the longer the routine would take, so in order for people to be able to use it without waiting for a long involved process of picking out the unique locations for mobs, it's done when you store the data from a log file to any one of the database platforms the program supports. During the storage routine, it will only update the zones that are present in the current log file, unless it is the first time it's being run. In order to build the initial set of data for usage with the program, you will either have to store a log file or just let the next archival process take care of it for you (if you chose to store the data from logs when archiving).

I've also added the ability to change the database options from most of the windows that are used for accessing data stored in any of the database platforms supported in the program.

I found one limitation with the local storage method at the moment. It seems the configuration for the BDE I've been distributing has the block size on the tables set to 2048. This means the max size of a table is 256mb. I will be updating the configuration to be able to utilize 1gig tables, but I need to write something to go through and restructure your existing tables with a 16,384 byte block size. I might even just include it as a separate little utility, as well as a table repair/packing utility. MySQL and PostgreSQL users are not affected by any of this.

There have been a few changes to the structure of some tables. MySQL floating-point fields have been changed from a floating point decimal to a fixed decimal with 2 places. This was done on coordinate storage since some mathematical functions in MySQL will treat a float as a double instead, which wasn't working out so well when calculating the distance between coordinates. A few character fields have been widened, mostly just zone names. More uniformity across tables that store the same types of information has been looked at. Primarily something that stores a zone name will have the same length across all tables instead of varying between 40 and 50 characters.

I've found some random bugs here and there and squished them. I also improved the speed of the PostgreSQL moving of text files to the server. I wasn't able to do it previously using the methods in their API so that the server did not have to have physical access to the file. That has been taken care of, now instead of executing a number of insert statements, it's loaded and transmitted across the database session. The insert statements would actually hang the program if you got in excess of 5,000 insert statements due to the way the query components handle string lists.

Next thing coming up is an easy raid stat generator. It basically will format out some information like raid participants listed with /whoraid and loot for a particular zone. It will output a bunch of text for making it easy to post on your guild website etc.

-------------------------------------------------------------------------

May 6th, 2008 - build 1.0.0.46

1. Raid Stats - you will need to enable logging of where a mob dies inside eq2 for the start time of the raid to populate correctly.
2. Auto save all options to registry except chat filters
3. Fixed playername on sales data
4. Default database selection
5. Archive time window should actually function correctly now.
6. Fixed couple misc errors
7. If you restore the app from the tray while in auto archive mode it shouldn't minimize the window to the tray anymore on it's interval for checking to see if it should archive your logs.
Archive List (Old Versions)
File Name
Version
Size
Author
Date
1.0.0.45
6.90 MB
g0thic_ice_crea
03-24-2008 06:27 PM
1.0.0.44
6.88 MB
g0thic_ice_crea
02-15-2008 02:01 PM
1.0.0.44
6.80 MB
g0thic_ice_crea
01-18-2008 10:56 AM
1.0.0.42
6.84 MB
g0thic_ice_crea
01-17-2008 04:48 PM
1.0.0.41
6.67 MB
g0thic_ice_crea
12-19-2007 12:02 PM
1.0.0.40
6.67 MB
g0thic_ice_crea
12-07-2007 09:25 AM
1.0.0.39
6.66 MB
g0thic_ice_crea
11-21-2007 05:55 PM
1.0.0.38
6.66 MB
g0thic_ice_crea
11-18-2007 08:36 PM
1.0.0.37
6.66 MB
g0thic_ice_crea
11-16-2007 12:42 PM
1.0.0.36
6.66 MB
g0thic_ice_crea
11-12-2007 11:39 AM
File Statistics
User Rating:
5 out of 5 with 2 votes
Downloads: 16056
Views: 53248
Favorites: 18
Uploaded By:
 
Last Modified: 05-06-2008 01:51 PM by g0thic_ice_crea    

View Pictures. Download Now! Post A Comment


Post A Reply
Author Comments Comment Options
Unread 04-30-2014, 09:01 AM  
g0thic_ice_crea
A Young Mystail Rat
 
g0thic_ice_crea's Avatar
Interface Author - Click to view interfaces

Server: Kithicor
Forum posts: 7
File comments: 13
Uploads: 1
Re: Parser not working

I have not tried this on windows 7 64. Most likely I will have to recompile and test. Shouldn't be a big deal to get it working. Sometime this weekend I'll see if I can get it "compatible" with win 7 64.


Quote:
Originally posted by txheart0711
I run Windows 7 x64bit. I click on the program, it installs, I set up the parameters (all I really want is sales) and click clean, or parse, or whatever it is. Immediately I get the windows popup saying this program has stopped working. I've tried running in admin mode, tried using it in compatibility mode, nothing helps. Any ideas?
g0thic_ice_crea is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 04-29-2014, 10:11 PM  
txheart0711
A Young Mystail Rat

Server: Unkown
Forum posts: 2
File comments: 1
Uploads: 0
Parser not working

I run Windows 7 x64bit. I click on the program, it installs, I set up the parameters (all I really want is sales) and click clean, or parse, or whatever it is. Immediately I get the windows popup saying this program has stopped working. I've tried running in admin mode, tried using it in compatibility mode, nothing helps. Any ideas?
txheart0711 is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 03-03-2010, 09:36 PM  
g0thic_ice_crea
A Young Mystail Rat
 
g0thic_ice_crea's Avatar
Interface Author - Click to view interfaces

Server: Kithicor
Forum posts: 7
File comments: 13
Uploads: 1
Well, after a long hiatus from EQ2, I'm back, and eventually going to start developing this again.

If anyone has any bugs for me to look at, requests or questions feel free to let me know

I seriously need to document all the stuff that got added before I stopped developing the program, so keep an eye out for that too. I probably won't update the help file right away b/c it's a very tedious task to do it, but I'll be posting more usage guides on the main website (eq2tools.com)
g0thic_ice_crea is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 05-21-2008, 12:05 AM  
g0thic_ice_crea
A Young Mystail Rat
 
g0thic_ice_crea's Avatar
Interface Author - Click to view interfaces

Server: Kithicor
Forum posts: 7
File comments: 13
Uploads: 1
Possible bug and additional features...

So it seems I got a new computer and am experiencing some access violation when it's running in auto archive mode...not sure where it's occuring. I put in some code a few revisions ago to pick up unhandled exceptions and log them, but I'm not seeing it log it, and of course, it doesn't happen from the debugger...so if you have it happen please let me know and send me any error logs that are written to the program's directory

I also found an interesting bug, well more or less a small issue with the way the zip files are created. Seems it will create an empty zip file if you time/date stamp the zip file filenames and there are no files to zip up lol. I'll be fixing that soon and releasing a small patch.

I have some code put in for checking for updates from the main site, but I haven't really implemented it on my webserver yet, so that's something else coming as well.

The main website will be having some of the features of the program like mob locator, and some other stuff in the future based on data I've collected from my own logs. I may let people submit their item and kill data as well through the program. I'm just starting on the project so it will be quite a bit of work, I wouldn't expect to see it anytime in the next few months

I will also be adding a feature to have the program go through all of your archived log files and reprocess the data, as well as migration tools for migrating your data from the different storage methods provided, just in case you change your mind on database platforms

I still need to include a new BDE configuration for larger tables and a little bit of code to restructure your existing paradox tables to handle up to 1 gig worth of data. Currently all tables are maxed at 256mb of data, due to a block size of 2048 bytes. This ONLY affects local storage (paradox tables) the other platforms are pretty much unaffected by this. I found that just running the storage to paradox for chat data will blow out the limit for a 2048 blocksize within a month LOL.
g0thic_ice_crea is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 05-06-2008, 06:16 PM  
g0thic_ice_crea
A Young Mystail Rat
 
g0thic_ice_crea's Avatar
Interface Author - Click to view interfaces

Server: Kithicor
Forum posts: 7
File comments: 13
Uploads: 1
Quote:
Originally posted by Landiin
Nice work, only thing I would suggest is using XML files, instead of having to deal distributing Borlands BDE. But thats just me Nice work in any case.
Thanks, I did actually consider it, but due to the massive amounts of data that can accumulate, I felt for local storage that paradox tables would be a bit more efficient when your data reaches over a few hundred thousand records, or even upwards around a million records + (when storing chat data and system messages)...I did a lot of indexing on the tables so that queries etc. would be a bit more efficient. I may include an XML export at some point though...I need to upgrade the project to at least Delphi 7 before I embark on that since both Delphi 5 and Delphi 6 have poor implementation of XML. I generally still develop in Delphi 5 since that's what I'm using at work. Delphi 6 came out about the time XML entered the scene, so it's implementation is rather lacking.

I'm just happy I found a way to be able to use the BDE without actually forcing the user to install it, and overcome the freespace bug where if the drive the BDE is operating on (operating on the tables) is evenly divisible by 4, you get "table full" errors LOL. It's b/c the BDE is using an old implementation of the window freespace API call. Bascially ended up overloading the call to the more current version of the function.

If you have the know-how, I would recommend installing PostgreSQL or MySQL for data storage PostgreSQL for more flexibility, MySQL for better performance
g0thic_ice_crea is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 05-06-2008, 05:06 PM  
Landiin
Slayer of clock cycles
Featured Artist

Server: Oasis
Forum posts: 3464
File comments: 21
Uploads: 24
Nice work, only thing I would suggest is using XML files, instead of having to deal distributing Borlands BDE. But thats just me Nice work in any case.
Landiin is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 05-06-2008, 01:56 PM  
g0thic_ice_crea
A Young Mystail Rat
 
g0thic_ice_crea's Avatar
Interface Author - Click to view interfaces

Server: Kithicor
Forum posts: 7
File comments: 13
Uploads: 1
Next additions coming...

I'm planning on adding an auto updater, actually have it mostly coded now so you won't have to keep checking the main site or here for updates, it'll just tell you...

Item information editor is coming one of these days, it's a lot of work to put together some sort of data entry dealie for inputting information about items...still working it out...

Other than that, no real major changes/additions planned...mostly just going to work out some bugs as people find them (that I haven't caught already), and update the horrbily out of date help file...
g0thic_ice_crea is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 03-24-2008, 06:47 PM  
g0thic_ice_crea
A Young Mystail Rat
 
g0thic_ice_crea's Avatar
Interface Author - Click to view interfaces

Server: Kithicor
Forum posts: 7
File comments: 13
Uploads: 1
BTW: After I get the raid stat stuff done, I will probably be putting the project on hold for a month or two in order to get the help file caught up/updated.

I've been putting new stuff into it since last Spring so fast that I never did get the help file updated past the initial release. I'll also be posting some more screenshots of some of the windows up here shortly just for fun :P
g0thic_ice_crea is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 03-04-2008, 05:35 PM  
g0thic_ice_crea
A Young Mystail Rat
 
g0thic_ice_crea's Avatar
Interface Author - Click to view interfaces

Server: Kithicor
Forum posts: 7
File comments: 13
Uploads: 1
New enhancements a week or two away...

A little detail on what I have coming up that should be pretty kewl.

I've added a "mob locator". This will get the unique locations within 30 meters (as far as I can tell 1 unit = 1 meter with eq2's coordinate system). It will work on a freshly loaded log file, or it will create a new table called uniquekill wherever it is you store your data. Originally I was going to have it just calculate it when you opened the mob locator, but then realized it would take forever to get all the unique locations from the kill logs when you get quite a number of them stored. In order to make it a little less time consuming, when you store your data from your logs to whatever database platform you choose, it will create this table with unique kills and update it from the stored kill logs every time you store mob data.

With the mob locator you can filter it down to specific zones or mobs, input your current location and have it update the distances from you for any of the mobs displayed. It will sort it by the closest locations to you and allow you to copy a /waypoint command to your clipboard for easy pasting into the game. Basically, when you double click or right click on one of the listed mob locations it sends "/waypoint xxx, xxx, xxx" to your clipboard, then you just hit <ctrl> V in your chat window and get your nice glowy trail to your mob.

If you have a log file loaded before opening the mob locator window it will attempt to process your log and calculate the unique locations from that log only, so if you want the locations you've already stored plus what is in the currently loaded log you will have to switch the datasource to either local, mysql or postgresql.

Anyways, it's in testing at the moment, so another week or so before it's ready :P

Last edited by g0thic_ice_crea : 03-04-2008 at 05:36 PM.
g0thic_ice_crea is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 02-15-2008, 07:34 PM  
g0thic_ice_crea
A Young Mystail Rat
 
g0thic_ice_crea's Avatar
Interface Author - Click to view interfaces

Server: Kithicor
Forum posts: 7
File comments: 13
Uploads: 1
For some reason the upload I did the other day didn't take and 1.0.0.43 was still up here for the past couple days...It's now the right version...
g0thic_ice_crea is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 02-13-2008, 10:26 PM  
g0thic_ice_crea
A Young Mystail Rat
 
g0thic_ice_crea's Avatar
Interface Author - Click to view interfaces

Server: Kithicor
Forum posts: 7
File comments: 13
Uploads: 1
I figured out what was going on with the latest update. I had part of the code for an auto updater in there and some of it was partially active.
g0thic_ice_crea is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 02-12-2008, 09:50 PM  
g0thic_ice_crea
A Young Mystail Rat
 
g0thic_ice_crea's Avatar
Interface Author - Click to view interfaces

Server: Kithicor
Forum posts: 7
File comments: 13
Uploads: 1
Yeah, I'm still working on the update. All the issues listed have been taken care of, but I've run into another problem with the tray icon. With the release of Windows XP, Microsoft introduced a bug in the call for adding icons to the system tray that didn't exist in Windows 2000. When I first got XP, I went around and around with Microsoft on the issue because I used to have the task manager start up minimized to the system tray on startup. Well, they never fixed it, and for some reason, even though I didn't touch that portion of the code, the tray icon component I'm using is now taking an error if it tries to add itself to the system tray upon booting up Windows. I've tried a few techniques that have been sucessful in the past for getting around it, but it seems it's not working, so I'm thinking I probably won't have the update out until the end of the week. The biggest problem with tracking down what to patch is the fact that it works perfectly from the debugger and if you load it after Windows has completely booted, so we'll see what I end up coming up with. Simply delaying the addition of the tray icon isn't working out.
g0thic_ice_crea is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 01-21-2008, 01:08 PM  
g0thic_ice_crea
A Young Mystail Rat
 
g0thic_ice_crea's Avatar
Interface Author - Click to view interfaces

Server: Kithicor
Forum posts: 7
File comments: 13
Uploads: 1
OK, I've gotten some feedback and have some things to work out on the interface to make it more easy to figure out what you're doing, and found a couple bugs with the new visual query builder...hopefully have those worked out in the next week or so...

After that I'm going to get the help file updated with some examples of how to do common tasks...

Here's a list of things that need to be taken care of:
1. Auto Archive time editors are behaving badly with a clean set of options. I don't think the time window for the auto archive is working properly either.
2. Visual Query Builder is taking an access violation when opening a query then switching datasource to something else.
3. I need to do something to get the temp tables created before opening the Visual Query Builder if there is a log loaded, otherwise disable the ability to select temporary tables as a datasource from the Visual Query Builder.
4. Make the "filter" button and chat filter menu item more clear that it only affects chat data.
5. The program is missing some loot data for stuff that is looted and assigned by the group leader, rather than using lotto or free for all looting options.
6. Need to add the name of the person who looted an item to the loot data.

-----------------

Please email me with any other things you might have encountered and need fixed/clarified.

I will be adding a quick mob search with the ability to send to the windows clipboard /waypoint xxx,xxx,xxx for the mob you select so it's easy to paste into EQ2 to track down mobs. This will use the same logic that the walkthrough creation part of the program uses for finding all unique locations within 30 meters of each other so that you don't have a million locations to pick from when you're looking for a particular mob. Eventually I'll add the same thing for a quick loot search. It will display mobs that drop a particular item and copy the waypoints for the mobs to the windows clipboard for pasting into the game. I will probably be adding the ability to send an item link to the clipboard as well for any items you may have stored.

Last edited by g0thic_ice_crea : 01-21-2008 at 02:32 PM.
g0thic_ice_crea is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 01-07-2008, 01:47 PM  
g0thic_ice_crea
A Young Mystail Rat
 
g0thic_ice_crea's Avatar
Interface Author - Click to view interfaces

Server: Kithicor
Forum posts: 7
File comments: 13
Uploads: 1
I'm currently working on a visual query builder for those who aren't familiar with SQL, and a bit of a shortcut from writing your own SQL. It will also include exporting data to CSV, HTML, MS Word and MS Excel.

It's almost finished, I expect another week or two for ironing out the bugs that I can find.
g0thic_ice_crea is offline Report comment to moderator   Reply With Quote Reply With Quote
Unread 12-30-2007, 05:10 PM  
foozlesprite
A Crazed Gnoll
 
foozlesprite's Avatar

Server: Everfrost
Forum posts: 23
File comments: 21
Uploads: 0
Helpful program but it's almost as hard on my processor as opening the log in Firefox or Notepad. Problem is those other programs don't do nearly as much. Thank you very much
foozlesprite is offline Report comment to moderator   Reply With Quote Reply With Quote
Post A Reply

 
Category Jump:
Search this Category:
 

All times are GMT -5. The time now is 12:15 AM.


Our Network
EQInterface | EQ2Interface | WoWInterface | LoTROInterface | ESOUI | MMOUI