Is DotNet 8 Faster than DotNet 7?
Taking a pause from my series on becoming a manager(latest post), I took a morning to update my Expressur basic arithmetic library so I can test claims that .NET 8 had significant performance improvements over .NET 7.
Short answer? Add me to the people who say “apparently yes”, at least for pure C# code.
The real reason I built Expressur is so that I would have a meaningful but straightforward set of code that can be ported to almost any other language so that the languages can be compared. It does this by taking a normal problem — arithmetic — and using string manipulation, iteration, and primitive operations. It does not use much in the way of additional libraries, it’s pure C# code. That makes it a good candidate for doing an apples-to-apples comparison between two different versions of the framework. There’s no possibility of a bug or newly added features of a third party library causing issues1, no network in the way, and not even any reading or writing from input/output files.
Expressur features a test harness to facilitate performance comparisons (I mentioned it in my writeup on Rust), so performing the comparison between the two versions of dotnet was a piece of cake: run the test, then update the version, build, and run again.
This is done using Expressur.LoadTest. Expressur.LoadTest is a
simple executable that runs a very simple load test for Expressur. In
five minutes, it evaluates the following three expressions as many times
as it can:
cplusaplusb = c + aplusb
aplusb = a + b
extraindirection = (aplusb/ cplusaplusb)
It uses these values as initial seed values:
a = 1
b = 2
c = 3
On each iteration, it will encounter cpluasplusb before it knows the
answer to aplusb and thus also not know how to resolve
extraindirection, so that it fully exercises Expressur's deferred
evaluation on each pass.
"As many iterations as it can" is somewhat qualified. To both simplify the code and prevent the load test code from adding too much overhead that would skew the results, it runs in batches of 1 million. After every million batches, it checks if the total run time is greater than 5 minutes. If so, it ends. i.e.:
while (time < 5 minutes>) {
run 1 million batches
}
As a result, the run time will always be slightly greater than 5 minutes except in that rare case when the last batch was completed at exactly 5 minutes.
Running the same code built for dotnet 7 and 8, I saw these results:
| Metric | dotnet7 | dotnet8 |
|---|---|---|
| Number of iterations | 69 million | 79 million |
| Total time (minutes: seconds) | 5:04.702880 | 5:00.267134 |
| Time per million iterations (seconds) | 4.415984 | 3.800850 |
| Improvement in .Net 8 | 13.93% |
An almost 14% improvement is impressive when no code in Expressur changed at all.
While these results are encouraging, most apps I have worked on end up throttled by inefficient code (Expressur is by no means optimized!), network, database, or other computing environment issues instead of raw performance of the underlying code engine. Given my results porting Expressur to Rust didn’t seem to make it any faster — I suspect I would have to revisit all the string manipulations to get better Rust performance — seeing C# going even faster makes me stop and think if pursuing a language that is hard to master (Rust) is worth the effort for most of us. I guess the next more technical article will be updating the Rust version of Expressur to see if that performs any better.
Footnotes
-
The only Nuget libraries used are in used by the test projects, not Expressur itself. ↩