Location: Home > Resources > REMOTINGVS.ASP.NETWEBSERV

Remoting vs. ASP.NET performance

2002
Ingo Rammer

In March 2002, I annouced that I'm working on some performance tests which will compare .NET Remoting to ASP.NET Web Services. Here you can find the first results:

I compared the following three technologies over a range of usage scenarios

• .NET Remoting with TCPChannel and Binary Formatter
• .NET Remoting hosted in IIS with Binary Formatter
• ASP.NET Web Services hosted in IIS using SOAP

I also did some comparisons between ASP.NET Web Services and .NET Remoting components hosted in IIS using the SoapFormatter. The short story: don't do it! ASP.NET is always faster (by a magnitude) with SOAP - this is due to the reliance on a completely different serialization architecture with a enhanced feature set (like serialization of private members).

But this doesn't prohibit me from comparing Binary Remoting with ASP.NET Web Services as a foundation for conventional distributed applications based on the .NET platform. The short story: TCP Remoting is fastest. IIS-hosted .NET Remoting with BinaryFormatter is in nearly all cases (see below for detail) faster than ASP.NET as well and has only been slightly slower in one single test scenario. This still does not mean that ASP.NET is bad - in fact, when you want to go cross platform or have/want to use SOAP, you should definitely go for ASP.NET web services!

The tests

For each of the technologies mentioned in the previous list I did the following tests:

• Plain Method Call: 1000 simple calls of a void DoSomething() method which did no processing.
• Small object: Calling a method which returns a "small" object. After this [serializable] object is returned to the client, it's passed back to the server and echoed back to the client again. For each test, the object is therefore serialized and de-serialized three times. A "small" object in this test has three properties (String Name, String FirstName, ArrayList orders) whereas the last one is passed as null. This method call is placed 500 times, resulting in 1500 object serializations and de-serializations.
• Medium sized object: Same as above but the ArrayList "orders" is filled with 10 [serializable] objects which contain the following properties: DateTime time, Double amount, Double price, String article (~ 48 characters in the test). This method call is placed 300 times resulting in 900 serializations/de-serializations of the top-level object as it's again reflected back and forth between client and server.
• Large object: Same as above, but this time an ArrayList containing 25 "Customer" objects, each with 25 orders is returned and reflected. This call is placed 20 times for a total of 60 serializations and de-serializations of the top-level ArrayList object.
• Small DataSet: An untyped DataSet is generated at the server side, passed to the client, passed back to the server and once again back to the client (3 passes, same as above). The "small dataset" contains 5 columns of type String and consists of 50 rows. This test is conducted 100 times for a total of 300 serializations/de-serializations of the DataSet.
• Medium sized DataSet: Same as above but with 250 rows and 30 method calls for a total of 90 serializations/de-serializations.
• Large DataSet: Again the same, but this time with 500 rows and 20 method calls. The total number of serializations/de-serializations is 60.

All programs are either C# Console Applications or Libraries, compiled in "Release"-mode and started outside of VS.NET!

Note: For each test an additional "first call" has been placed outside the timing loop to initialize the systems!

Hardware/Software

The tests have been conducted on two machines. The server has been a 850 Mhz P3 with 512 MB RAM running W2K Professional SP2. The client is a Sony Vaio with a 600 Mhz P3 CPU and 192 MB RAM also running W2K Professional SP2.

The machines were connected by a switched 100 Mbit network. No other applications have been running during the tests.

Disclaimer/Copyright

The following results are only valid on my machine in my exact configuration - always test for yourself! The tests have been conducted using only a single client and therefore do not account for any scalability issues. All tests have been conducted 6 times and the numbers below are the averages. You can get the complete numbers for each test in the excel sheet referenced in the downloads section below.

The test programs, the results, this article and all other documentation for these tests are copyright 2002 by Ingo Rammer. You may use them freely for your own purposes. Re-Publications and citations in any form and medium (web pages, magazines, books, ...) has to state the origin and requires permission of the author. You can contact me here.

Downloads

• Complete test (sources and binaries): AspNet_vs_Remoting.zip (138 KB)
• Results and charts (Excel): RemotingPerformance.xls (50 KB)

Results

(all charts show the average duration of the execution of the test, not the system's performance. Shorter bars are better!)

Plain Method Call

For simple method calls without any additional payload data, ASP.NET is nearly as fast as IIS/Binary remoting.

Small Objects

The transfer of small objects is the only test that shows a slightly better overal performance for ASP.NET web services. In this case, IIS/Binary remoting is slowest.

Medium Sized Objects

Working with medium sized objects shows that ASP.NET's performance starts to degrade depending on the size of the [Serializable] objects. In this case, IIS/Binary remoting takes only about 75% of the time as when using ASP.NET web services.

Large Objects

The "Large Objects"-test shows that the relative performance of .NET Remoting is increasing with the size of the objects. In this case, the Remoting components hosted in IIS (using BinaryFormatter) only take about 50% execution time of ASP.NET web services.

Small DataSets

Small DataSets show about the same behavior as the "Medium Sized Objects"-test did when looking at ASP.NET vs. HTTP-based Remoting. The noteable point is that TCPChannel-Remoting isn't much faster than HTTP-based Remoting here. This seems to stem from the serialization-patter of a DataSet which is always stored as an XML string (even when transferred using a BinaryFormatter).

Medium Sized DataSets

Medium Sized DataSets shown an interesting behavior. I really had to redo the complete test series several times because I could not believe this result as it's against the tendency here ... ASP.NET web services and IIS/Binary Remoting show nearly the same performance results - TCPChannel Remoting still as a lot better.

Large DataSets

This last test shows the expected behavior: TCP Remoting is fastest, IIS/Binary Remoting is close second and ASP.NET takes a little longer.

Conclusion 

What the heck does this mean? The clear advantage of TCPChannel-Remoting should make you think about using this channel whenever you can afford to do so. If you don't need any standard means of authentication and authorization and if you can create direct TCP connections from your clients to your server and if you need to support only the .NET platform, you should go for this channel. If you have to use HTTP as the underlying protocol but are going to stay with the .NET framework for both, your clients and your server, you should go for Remoting components hosted in IIS. This way you also have access to standard authentication, authorization and encryption (SSL!) features of Internet Information Server.If you are going to go cross-platform or you have the requirment of supporting SOAP via HTTP, you should definitely go for ASP.NET web services as .NET Remoting's SOAP performance isn't quite the best (not shown here, but it is a little slower than ASP.NET).

This shows you that each and every technology shown here has a place in this world - you only have to decide based on your requirements.