# Microsoft LogParser

**URL:** <https://discourse.codinghorror.com/t/microsoft-logparser/1031>\
**Category:** blog\
**Created:** [August 23, 2005, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031 "2005-08-23T00:00:00Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![codinghorror](https://discourse.codinghorror.com/user_avatar/discourse.codinghorror.com/codinghorror/32/36371_2.png) [@codinghorror](https://discourse.codinghorror.com/u/codinghorror)\
**Post date:** [August 23, 2005, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/1 "2005-08-23T00:00:00Z")

</div>

Ask yourself this question: **what if everything could be queried with SQL?** [Microsoft's LogParser](http://www.microsoft.com/downloads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en) does just that. It lets you slice and dice a variety of log file types using a common SQL-like syntax. It's an incredibly powerful concept, and the LogParser implementation doesn't disappoint. This architecture diagram from the LogParser documentation explains it better than I could:

* * *
This is a companion discussion topic for the original blog entry at: [http://www.codinghorror.com/blog/2005/08/microsoft-logparser.html](http://www.codinghorror.com/blog/2005/08/microsoft-logparser.html)

---

<div class="post-metadata">

**Author:** ![benjaminF1](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/b/76d3ee/32.png) [@benjaminF1](https://discourse.codinghorror.com/u/benjaminF1)\
**Post date:** [August 25, 2005, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/2 "2005-08-25T00:00:00Z")

</div>

About logs, I often find sad that few .NET developers know about the wonderful Log4Net Framework ([http://logging.apache.org/log4net)](http://logging.apache.org/log4net))

One cool example is the ability to receive your logs (error level form example) by mail.  
You instantly know when there is a production problem!

There are many other ways to configure log4net, one can do almost everything!

Just look at the features page:  
[http://logging.apache.org/log4net/release/features.html](http://logging.apache.org/log4net/release/features.html)

---

<div class="post-metadata">

**Author:** ![benjaminF2](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/b/da6949/32.png) [@benjaminF2](https://discourse.codinghorror.com/u/benjaminF2)\
**Post date:** [August 25, 2005, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/3 "2005-08-25T00:00:00Z")

</div>

And Log4Net is free!

---

<div class="post-metadata">

**Author:** ![robert12](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/r/a698b9/32.png) [@robert12](https://discourse.codinghorror.com/u/robert12)\
**Post date:** [August 25, 2005, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/4 "2005-08-25T00:00:00Z")

</div>

gee, finally, the stake in the heart of XML/XQuery/XFoolishness. and it’s from Uncle Bill. ah the irony, the sweet irony.

---

<div class="post-metadata">

**Author:** ![codinghorror](https://discourse.codinghorror.com/user_avatar/discourse.codinghorror.com/codinghorror/32/36371_2.png) [@codinghorror](https://discourse.codinghorror.com/u/codinghorror)\
**Post date:** [August 25, 2005, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/5 "2005-08-25T00:00:00Z")

</div>

A bit off topic, but I agree: the more I work with XML the less I like it.

It’s like someone took a text file, glorified it, then sainted and canonized it as the One True Data Format.

---

<div class="post-metadata">

**Author:** ![codinghorror](https://discourse.codinghorror.com/user_avatar/discourse.codinghorror.com/codinghorror/32/36371_2.png) [@codinghorror](https://discourse.codinghorror.com/u/codinghorror)\
**Post date:** [August 25, 2005, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/6 "2005-08-25T00:00:00Z")

</div>

Yes, but MS LogParser has nothing to do with writing logs; it’s all about processing and/or converting them.

---

<div class="post-metadata">

**Author:** ![codinghorror](https://discourse.codinghorror.com/user_avatar/discourse.codinghorror.com/codinghorror/32/36371_2.png) [@codinghorror](https://discourse.codinghorror.com/u/codinghorror)\
**Post date:** [August 26, 2005, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/7 "2005-08-26T00:00:00Z")

</div>

Aha..

[http://swigartconsulting.blogs.com/tech\_blender/2005/08/net\_language\_in.html](http://swigartconsulting.blogs.com/tech_blender/2005/08/net_language_in.html)

---

<div class="post-metadata">

**Author:** ![Stuart](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/s/cab0a1/32.png) [@Stuart](https://discourse.codinghorror.com/u/Stuart)\
**Post date:** [January 19, 2006, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/8 "2006-01-19T00:00:00Z")

</div>

I’ve been playing with this tool again recently to analyse IIS log files.

Something that I miss is the ability to do sub-queries. The work-around is to just run another query on the output of the first query, but it would be much nicer to have this feature supported.

Maybe I should write up a blog post myself and share my queries with everyone else.

Cheers,  
Stuart.

---

<div class="post-metadata">

**Author:** ![Jackal](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/j/858c86/32.png) [@Jackal](https://discourse.codinghorror.com/u/Jackal)\
**Post date:** [May 4, 2006, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/9 "2006-05-04T00:00:00Z")

</div>

Yes but Log4Net is over kill , like most of the Enterprise Library .

I like things simple.

---

<div class="post-metadata">

**Author:** ![Pradeep](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/p/f0a364/32.png) [@Pradeep](https://discourse.codinghorror.com/u/Pradeep)\
**Post date:** [April 19, 2007, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/10 "2007-04-19T00:00:00Z")

</div>

The post was really helpful. Thank you

---

<div class="post-metadata">

**Author:** ![Koen](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/k/a9a28c/32.png) [@Koen](https://discourse.codinghorror.com/u/Koen)\
**Post date:** [August 2, 2007, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/11 "2007-08-02T00:00:00Z")

</div>

Hi All,

LogParser is a good tool. It comes with a straight-forward dll that you can easily use with your custom .net (or other) applications.

I only have 1 problem. I use it to parse windows media streaming logs. It sees the sc-bytes as integer. But it happens that I parse more data that an integer can hold. The LogParser parses it as a negatif number :(

grtz

---

<div class="post-metadata">

**Author:** ![SteveT](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/s/858c86/32.png) [@SteveT](https://discourse.codinghorror.com/u/SteveT)\
**Post date:** [August 7, 2007, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/12 "2007-08-07T00:00:00Z")

</div>

Thanks Jeff, great article. I digging into log files now and this was a pleasant find.

---

<div class="post-metadata">

**Author:** ![Moshe\_Eshel](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/m/f4b2a3/32.png) [@Moshe\_Eshel](https://discourse.codinghorror.com/u/Moshe_Eshel)\
**Post date:** [September 11, 2007, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/13 "2007-09-11T00:00:00Z")

</div>

Sorry for chiming in so late but I think I might have a solution for one of the problems you raised with LogParser, personally I’ve only used it a few times and liked it very much.

You raise the issue of automated detection of column type, I encountered that in the past and I believe it is caused by the MS Text Driver implementation, the same one that does the job for Excel or SQL server when reading from a file. The solution I used back then was using the schema.ini file (read more here: [http://msdn2.microsoft.com/en-us/library/ms709353.aspx)](http://msdn2.microsoft.com/en-us/library/ms709353.aspx)) it allows you to define exactly what you want the driver to do explictly - and leave the implicit stuff well alone)

Hope this helps  
Moshe

---

<div class="post-metadata">

**Author:** ![Pete109](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/p/b5e925/32.png) [@Pete109](https://discourse.codinghorror.com/u/Pete109)\
**Post date:** [December 12, 2007, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/14 "2007-12-12T00:00:00Z")

</div>

I am trying to set up LogParser to output Netmon data (.cap file) to a SQL DB on a daily basis. In order to do this, I need to be able to set up the .sql file to read the .cap file using a wildcard.

So far, I have tried using all the variants that I could think of, find or otherwise attempt to use in my testbed.

Basically, my task will be to do:

Select frame,dateTime,SrcIp,SrcPort,DstIp,DstPort,PayloadBytes,Connection from ‘filename.cap’ to myTable  
where DstPort = x  
or DstPort = y

The difficulty I am having is finding the ‘right’ wildcard to replace filename, since the file will be changing daily.

Suggestion(s)???

Thanks in Advance!

---

<div class="post-metadata">

**Author:** ![codinghorror](https://discourse.codinghorror.com/user_avatar/discourse.codinghorror.com/codinghorror/32/36371_2.png) [@codinghorror](https://discourse.codinghorror.com/u/codinghorror)\
**Post date:** [May 5, 2008, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/15 "2008-05-05T00:00:00Z")

</div>

nice GUI for using LogParser:

[http://www.codeplex.com/visuallogparser](http://www.codeplex.com/visuallogparser)

---

<div class="post-metadata">

**Author:** ![lizard](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/l/dfb087/32.png) [@lizard](https://discourse.codinghorror.com/u/lizard)\
**Post date:** [January 23, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/16 "2009-01-23T00:00:00Z")

</div>

Log Parser GUI, Log Parser Lizard ([www.lizardl.com](http://www.lizardl.com))

---

<div class="post-metadata">

**Author:** ![GeorgeC](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/g/c89c15/32.png) [@GeorgeC](https://discourse.codinghorror.com/u/GeorgeC)\
**Post date:** [January 23, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/17 "2009-01-23T00:00:00Z")

</div>

I for one would love to find the source code to microsoft log parser some place, and then give it to some kid to do a nice C# version using interfaces and AppDomain.CurrentDomain.CreateInstanceAndUnwrap instead of COM.

Sure it has got its limitations, but it has proven to be just good enough to do the job and a huge time saver when compared with other more enterprise solutions.

On the other hand I wonder if Google Mapreduce might be a better  
way to go, to avoid the confusion that people can have with  
limited SQL subsets.

Or is LOG PARSE to go the way of DEC DATATRIEVE product?  
Another cute query system that worked over all sorts of file formats.

---

<div class="post-metadata">

**Author:** ![mbourgon](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/m/a4c791/32.png) [@mbourgon](https://discourse.codinghorror.com/u/mbourgon)\
**Post date:** [February 6, 2010, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/18 "2010-02-06T00:00:00Z")

</div>

Pete, one idea (haven’t tested) is to use SQL to dynamically generate the command to execute. There’s probably a better way to do it from a batch file or whatnot, but this should work. Mind the syntax.

declare @sqlstatement varchar(500)  
select @sqlstatement = ‘Select frame,dateTime,SrcIp,SrcPort,DstIp,DstPort,PayloadBytes,Connection from ‘’’ + convert(char(8),getdate(),112) + ‘.cap’’ to myTable where DstPort = x or DstPort = y’  
exec master..xp\_cmdshell (@sqlstatement)

---

<div class="post-metadata">

**Author:** ![deathwagon](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/d/9fc348/32.png) [@deathwagon](https://discourse.codinghorror.com/u/deathwagon)\
**Post date:** [February 6, 2010, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/19 "2010-02-06T00:00:00Z")

</div>

Regarding Standard Deviation in LogParser - I found this LogParser query a while back (forgot who the original author was, sorry). I checked it against SQL’s built in STDEV function and returned the same data so I am confident it is accurate - enough so for my needs anyway.

SELECT DIV ( MUL(1.0, SUM(time-taken)), Hits ) as AvgTime, SQRROOT ( SUB ( DIV ( MUL(1.0, SUM(SQR(time-taken)) ), Hits ) , SQR(AvgTime) ) ) AS StDev

---

<div class="post-metadata">

**Author:** ![TheUniGuide](https://discourse.codinghorror.com/letter_avatar_proxy/v2/letter/t/a88e4f/32.png) [@TheUniGuide](https://discourse.codinghorror.com/u/TheUniGuide)\
**Post date:** [July 25, 2010, 12:00am UTC](https://discourse.codinghorror.com/t/microsoft-logparser/1031/20 "2010-07-25T00:00:00Z")

</div>

The GUI for using LogParser over at codebase is pretty good.  
I did find a good book/site for brushing up on your xml for those that don’t like working with it too well!  
[http://www.lanw.com/books/xml4dum/default.htm](http://www.lanw.com/books/xml4dum/default.htm) and  
[http://www.lanw.com/books/xml4dum.htm](http://www.lanw.com/books/xml4dum.htm)

not saying that only dummies should use it, lol!

[Next page](https://discourse.codinghorror.com/t/microsoft-logparser/1031.md?page=2)