Obviously with any "third party software" its up to you if you want to take the risk of using such software, not this sites, or bg's or ffxievo's , backup stuff before use!
Methodology and Parser:
Crabby Thread, by golfandsurf
DOWNLOAD CRABBY HERE
-- D/L at own risk -- etc. etc.
1) Download Crabby.rar from the link above and extract the EXE to your own directory.
2) Start FFXIV and log in using windowed mode.
3) Run Crabby.exe. Check if it has synced up by looking at the bottom of the window. It will say "Attached". If you need to resync, just double click the icon at the lower left of the window.
4) You can create separate window feeds for each chat type. Click "Settings > Chat Tabs" and select one or more types. They correspond to what you "Show" or "Hide" In-Game under "Configuration > Chat Log > Settings". Best of all, there is no limit to the history or number of log entries Crabby remembers.
5) Execute your test with the guidelines above, and copy out the timestamped entries into an external text editor. Please be aware that logging out of the game may crash Crabby and prevent you from copying out the logs. Be on the lookout for bugs and report in the Crabby Thread. Try a more stable version of Crabby if you experience problems. The newer features are not necessary for our testing purposes.
6) Parse these text files to determine your test conclusions. I have completed customized perl scripts which automatically evaluate your average non-critical damage, average critical damage, etc.
Example of what the parser does
Original post of where the above image came from and links found (If you'd like more info on what the above parsed information is about it was a PIE test on CNJ which you can read in detail below)