Interesting Tech Projects
Fast Reading of XML Files
I recently had the need to parse 780MB XML files, and quickly. My preferred language for PC development is C# because I can work quickly and efficiently with it, however I was a bit wary of the speed of XML file processing. Thankfully Mike Taulty has worked out a solution using LINQ and XmlReader, providing the user friendliness of LINQ with the speed of XmlReader.
An initial test showed that a 39MB XML file is processed in 4.9 seconds using Mono on a quad core server running Ubuntu. This is about 8MB/second which means that my 780MB file should be processed in around 98 seconds. Nice!
Print article | This entry was posted by Andy on July 14, 2011 at 3:47 pm, and is filed under Software Engineering. Follow any responses to this post through RSS 2.0. Both comments and pings are currently closed. |
Comments are closed.