ASP.NET 2.0 Speed Tests: Where's the Fire?

By
I read a very nicely done piece on MSDN about how the ASP.NET 2.0 pipeline, even though it is more complex than the ASP.NET 1.1 pipeline, has been optimized to be 30 percent faster.

"Great!", I thought. "Let's do a couple of tests."

So I whipped up a page in ASP.NET 2.0 that just outputs "Testing...", and removed all extraneous HTML from the ASPX portion except the FORM Tags. Did the same in ASP.NET 1.1. Pages were substantially identical, being designed primarily to test the pipeline itself - get a request, set up and process the Page, send it out.

I did two sets of tests on each using "Homer", 1 with 100 threads, no delay. and another with 100 threads, a 10X multiplier (the equivalent of 1000 simultaneous requests), and no delay between requests.

In both cases, the ASP.NET 2.0 pipeline was SLOWER, not faster. Both sets were compiled and run in Release, not Debug mode.

I suppose I should consider the fact that it is still BETA, but - SLOWER? Guess we'll be running more tests. . .

Anybody want to comment on this? I have plenty of experience doing web stress testing, having several weeks experience at the MS Testing Lab in Charlotte, NC. This is with IIS 6.0 on Windows Server 2003 64-Bit running on an AMD Athlon x64 3400+ with 2 G RAM.

NOTE: I did have a brief interchange with MS about this and after talking with them, I believe that the discrepancy has a lot to do with ASP.NET 2.0 running under 64-bit WOW. Unfortunately as of right now, the only other machines I've been able to test this on are Windows XP with a 10 connection limit, where it appears that there is less of a discrepancy between the 2 frameworks when both are running under 32 bit IIS. However, I certainly did not notice a "30%" faster pipeline. Having said that, I suppose that anybody really interested in pursuing this should do 2 things:

1) Run different kinds of tests - with both simple and complex pages. Run them on 32 bit only, and then if possible, 64-bit under wow64 and compare the results.
2) Ultimately, wait until RTM since that's really where the "HTTP Request meets the Browser" so to speak.

Peter has been active on the C# and other MS newsgroups, has had samples at gotdotnet.com (now codegallery.com) that have been downloaded over 56,000 times, and has written numerous articles for eggheadcafe.com, for which he is a co-founder, and MSDNAA sites. He has also been a technical reviewer on books from Prentice Hall, Manning and other publishers. Peter was a Microsoft MVP for 10 years. Peter was born and raised in New York City, and after a stint as a jazz musician playing contrabass with the Robert Hunt Trio, migrated to sunny Florida, where he was hired as a rookie Financial Consultant at Merrill Lynch in Orlando, specializing in Corporate Services. It was at Merrill that Peter realized computers and technology were his forte. He spent his last few years at Merrill developing real – time computerized trading and and stock charting program using advanced technical analysis indicators, some of which are his own inventions. In 1997, he moved on to architect and serve as lead developer on a number of innovative web-based and middleware solutions for the healthcare and financial services industry.

Comments

  1. Anonymous12:02 AM

    Hi Peter,

    Can you send me an email (scottgu@microsoft.com) with a copy of the test you ran? I can then investigate to try and better understand why you were seeing slower numbers.

    Thanks,

    Scott

    ReplyDelete
  2. Anonymous2:43 PM

    I'm glad I'm not the only person that needs to create their own tests before they believe marketing hype. You get 2 points from a fellow obsessive compulsive benchmarker.

    Back in my PHP days I ran many tests to disprove all the myths of speed and the preg_ functions (Perl inspride) vs the standard, less featured PHP functions.

    It's my tests along with ranting to the manual writers, that I believe led to the manuals recommending the improved functions.

    I'd like to also point out that most benchmarks show Windows 64bit to be slower at running 32bit code than normal 32bit Windows. But you already hinted at the fact that the 64bit server was affecting your results.

    ReplyDelete
  3. Anonymous10:50 AM

    Peter,
    did you manage to do any further tests with 32 bit. Also did you get a reply from Scott at Microsoft?
    I ask because I have just done some simple speed tests myself with a near-real-world site and the speed from 2.0 was disappointing. I gave it a good chance by making sure all the pages were pre-compiled (using aspnet_compiler) in advanceand it was still slower than 1.1 on initial page hits! The only place it did outperform was when there was a slow DB response, so not really a good metric.
    I have been putting the speed issue down to the framework being beta, but I would be interested to hear if you have got any further
    Cheers
    Richard

    ReplyDelete
  4. peterbromberg11:54 AM

    I did correspond with Scott, and we more or less decided that is was due to the 64 bit running 32 bit mode. Unfortunately I haven't been able to do any real 32 bit testing except on Windows Xp and there we have the 10 connection limit. I wouldn't be concerned about it at this point.

    ReplyDelete
  5. Anonymous2:33 PM

    have you enabled tracing in the page to determine if lag is occurring at a specific stage?

    ReplyDelete

Post a Comment

Popular posts from this blog

Some observations on Script Callbacks, "AJAX", "ATLAS" "AHAB" and where it's all going.

By
I've taken a more than cursory interest in the whole Remote Scripting (.NET species) vs. AJAX and now ATLAS discussion, mostly because I started using Remote Scripting since Microsoft first released it, and because I continued to refine it after seeing Brent Ashley's excellent work with JSRS, which was one of the first "real" cross-browser solutions back in 2000 (that's the turn of the Century for you history buffs). Now Bertrand Leroy, a Microsoft guy whose work I like , has authored some very interesting ASP.NET 2.0 script callback stuff that he points to on his blog, and he is obviously (at least, to me he is) involved quite heavily in the development of this new ATLAS infrastructure that they'll preview at PDC. Leroy's most recent post brings to the surface what I agree are the major differences between the "AJAX a - la .NET" approach as popularized by Michael Swartz with his AJAX.NET library, and Leroy's RefreshPanel, the ASP.NET 2.0 b...
Read more

IE7 - Vista: "Internet Explorer has stopped Working"

By
Image
This one was a bitch. All of a sudden for no reason at all, out of the blue, I get this dialog "internet Explorer has stopped working". I didn't install any new software, crap- I didn't do anything! So here I am using FIREFART to go on the internet and find out what to do! Damn! Good thing I've got the sucker on the same machine (I'm not "anti-Firefox", i just don't use it that much except to check my page renderings). The Fix The fix (at least for me): 1) Go into Control Panel, and choose "Internet Options". 2) Under the "Advanced" tab, press the "RESET" button at the lower right: Don't ask me why this happens, or why the fix works. That's a BUG, D00D - I don't care how you slice it!
Read more

FIREFOX / IE Word-Wrap, Word-Break, TABLES FIX

By
One of the most annoying things a developer (who would normally delight in writing code, not futzing with markup) can have is getting rendering issues between different browsers. The two biggest players are of course Internet Exploder and Firefart (as I lovingly like to refer to each). In most cases that's going to take care of 98 percent of your total site traffic. IE has had a proprietary "word-break" style attribute for a long time, and this made it into the CSS 3.0 spec. But that doesn't necessarily help you with Firefox right now. Firefox literally - (and I consider this completely idiotic, since it's been in their bug database for FIVE YEARS) does not have a reliable CSS style element to force table cell content to break in the middle of a word in order to stop the content from expanding your table / div off the page or over other content on the page. Here's a partial fix, which will work for most tables and browsers. The style declaration (I call it m...
Read more

System.Web.Caching.Cache, HttpRuntime.Cache, and IIS Recycles

By
Recently I chimed in on a newsgroup thread to an OP (Original Poster) who was questioning why he was losing Cache items. This individual would load a dataset from the Cache object into a local object to use to perform processing. He had been using httpcontext.current.cache and noticed that sometimes he was not able to get the dataset out of the cache. He tried switching to HttpRuntime.Cache and it seemed to help. The OP claimed that the application is under heavy hits, and once or twice every month he would see this error. He wanted to know why does the HttpContext.Current.Cache call work sometimes, while some other times does not. A couple of posters replied, one of whom is a respected MVP who usually makes very astute, high-quality posts. He posted," The cache is only valid during the life of the request. how is you[sic] code using the cache?" Somebody else (also pretty astute, usually) responded, "Cache is valid during lifetime of application not request.....". W...
Read more

FIX: Requested Registry Access is not allowed (Visual Studio 2008)

By
Symptom: in Visual Studio 2008, you attempt to add a new WebContentForm and associate it with a MasterPage. You receive an error dialog "Requested Reqistry Access is not allowed". Don't ask me what caused this; I'm guessing that the registry keys involved had their access denied due to removal of a previous version of Visual Studio, leaving the new guy (Visual Studio 2008) in the lurch. Here's the fix: Download "subinacl" here . This is a command-line tool that enables administrators to obtain security information about files, registry keys, and services, and transfer this information from user to user, from local or global group to group, and from domain to domain. After installing subinacl.exe (it will be in the C:\Program Files\Windows Resource Kits\Tools folder.), drop to a command prompt and run the following batch file, which you have saved in the same folder with the filename "FIX.bat": subinacl /subkeyreg HKEY_CLASSES_ROOT\VisualStudi...
Read more