.Net Runtime core improvements

Looking through the slides for this presentation.  I found a few things interesting.

‘Server GC’ in .Net 4.5 now supports background garbage collection, so it is now more useful for desktop apps as well as pure-throughput apps.  It also supports better distributing the mark phase of mark-sweep across multiple cores allowing garbage collection to scale better.

Profile guided optimization can now be done with managed code, and the resultant information annotated into IL for improved startup-time and reduced memory usage.  Apparently.

On Windows 8 only, .Net 4.5 will ‘auto-NGen’.  No need to think about whether you should NGen or not, its all automatically done in the background based on observed assembly usage.

And finally quite possibly my favourite.  Profiler hooks can now rewrite methods and request a rejit.  So you can inject instrumentation to methods which have already been jitted, or disable instrumentation without a process re-start.

WinRT and the return of the memory leak…

WinRT, so much new and interesting… so much old and broken?  I’ve read a few articles now, watched a couple of talk recordings – nothing great and in-depth but I think I’ve picked up a few details.

WinRT is effectively the combination of ‘COM done better’ and ‘OS API done using the result’.

On the COM done better side we get, nicer integration into more languages.  Along with that is a metadata format which is shipped with the code.  Metadata which includes versioning information to try and avoid some of the COM interface versioning hell.  You get better runtime reflection – every WinRT type can tell you what interfaces it implements and provide some kind of name.  Apparently there is type inheritance (although apparently only the WinRT core is allowed to use it…), and generics and better handling of events and asynchronous callbacks.  A new better type of string, simple structs and collection types built into the core that everyone can consistently use as part of their own APIs.  Each app apparently even has its own COM(I mean WinRT) registry, so no conflict issues.

OS API done using the result gives opportunities to do things the way they should be done.  It also allows for namespacing and componentization, which when mixed with the metadata mean much improved discoverability and general developer experience.  No need to call clean-up methods, because they are automatically called when the associated type is released…

Which brings us to the ‘old and broken’ part of this story.  WinRT shows its COM roots, everything is IUnknown, the same IUnknown from COM.  This means AddRef and Release.  The real bonus here however is that this is all hidden away from c#/VB/javascript and even c++ if you use the c++/cx extension approach to dealing with WinRT.  I say bonus somewhat dubiously as fully-automatic reference counting results in fully-automatic memory leaks.

Take for example the PropertySet in WinRT.  Using this from c# in VS11 the only vaguely apparent suggestion that this isn’t a .Net type is its namespace.  I can easily see developers thinking that this was a replacement for StringDictionary (which has conveniently been removed from the .Net 4.5 core runtime).  But you will see the difference if you create a circle of references between PropertySets.  You have to manually break the circle or the PropertySet memory will never be released!

As a bonus to this post, I’ll end by mentioning that WinRT apparently does not support exceptions!  This may seem strange to anyone who has used it, since calling WinRT APIs certainly throws exceptions, but this is entirely language integration, under the hood an error is limited to a 32bit integer – a HRESULT.  This means WinRT components cannot provide useful error details directly through their APIs without mangling those APIs to provide a bunch of output parameters that are normally never used.

I was watching one of the WinRT talks where they decided to describe a ‘feature’ that if an error occurs in the WinRT core libraries, it can directly tell the attached debugger additional details, since they cannot be returned to the calling code! (One can only hope that they provide a way for custom written components to do this too, it is going to be bad enough that when ComponentArt 19 for Metro comes out every error returned to your code is ‘COMException’ at runtime, not being able to find out useful details while debugging would be a nightmare.)  Almost makes one wish for the return of GetLastError…

.Net 4.5 – Changes I find interesting

So I went through the 17meg file looking for the changes I found most interesting.  So here they are… in no particular order.

  • SQL Server connection string now supports an option called ‘application intention’ – lets you specify whether you intend on performing any modifications.  All the documentation I could find on this topic talks about primary/secondary replicas and denying access to secondary replicas if the intention isn’t read-only – but it seems like a great opportunity for a simple approach to reducing attack-surface.
  • MethodImpl attribute has gained a new option ‘AggressiveInlining’ – documentation says the method will be inlined if possible.
  • New assembly mode Preferred32bit – documentation seems to suggest that this is the new Require32bit, only it won’t break when Microsoft releases an OS with no WOW64 mode… or maybe that if you invoke it from a 64bit context it will still load…
  • Reflection.TypeInfo – this is a bit of a strange one.  In .Net 4.5 Core, it is practically ‘the’ place to go to get all your type reflection information needs, it has most of the APIs they removed from Type, and some simpler methods and properties for working via reflection.  In .Net 4.5 Full, it still has those simpler methods/properties, but all the APIs are still back on Type instead.
  • Regular expressions now have an option for ‘match timeout’ to stop runaway regular expressions with infinite recursive backtracking or similar.
  • Threading.Tasks.DataFlow – a higher-level approach to defining how threads send data to each other, rather than writing your own queue or thread-pool.  Apparently it has been released as a CTP before and is based off of an existing part of the c++ TPL.
  • Threading.Volatile.Read/Write – Documentation here is that this provides volatile semantics to visual basic, but apparently also for array elements in c#.  I think there might be some use for it in crazy corner cases to implement ‘partial’ volatile as well to improve performance.
  • WeakReference<T> – typed weak references.  Does not support trying to take a reference to a struct (oddly enough)!  But it does provide a combined test and retrieve method to avoid calling IsAlive only to receive nothing from GetValue.
  • Windows.Shell.WindowChrome – WPF gets access to custom Aero UI features.  Now I too can create really annoying glass windows…
  • Runtime.ProfileOptimization.StartProfile – this could be my new favourite.  Putting this at the start of application causes method JITs to be recorded, and if the application has been run more than once, it also causes a background thread to spin up and JIT methods from the previous recording, speeding up the application start-up so long as the application behaviour is relatively predictable.
  • IReadOnlyList – a read only interface for List<T> (but not IList<T>…).  This interface is covariant so it can be cast, but it also cannot have the Contains or IndexOf methods. (The CopyTo method is missing for no apparent reason, except that maybe it might as well be an extension method.)
  • IReadOnlyDictionary – a read only interface for Dictionary<TKey, TValue> (but not IDictionary<TKey,TValue>).  This interface is not covariant or contravariant in any way, despite only ever taking the key as an input parameter and the value as an output parameter.  I wonder if that is an oversight.  There is also an IDictionary wrapper which presents this interface – presumably to ensure someone doesn’t just cast away your read-only-ness.
  • CultureInfo.DefaultThreadCurrentCulture – appears you can change which culture new threads get given – very nice in some areas I can imagine.
  • Zip file support through Compression.ZipArchive, and supposedly gzip compression which is significantly improved.
  • IProgress<T> – little interface with just one method Report(T progress). Progress<T>, a default implementation which raises an event which report is called. Odd little thing, not sure what it is doing…
  • New interop types – IInspectable and HString.  These are apparently windows runtime types…
  • Monitor.IsEntered – just in case you forgot? (Guess threading code can get complicated at times…)
  • Threading.Timeout.InfiniteTimeSpan – finally, no more converting -1ms and hoping it works?
  • Improved virtualization support in WPF. (Pixel level scrolling and pre-post caching.) (Much more interesting than the ribbon!)
  • Async support is everywhere… not just streams/web requests/dns queries/database access – it is also in xml parsing (presumably to support the scenario where the underlying stream is a file).

Edit: watching some talks I discover that something that didn’t seem interesting at the time actually is.  ThreadLocal<T> had an IEnumerable<T> GetValues method added to it.  This actually lets you see all of the values currently allocated across all threads.  Not sure how often I’ll use it, but it is cool.

.Net 4.5 Developer Preview

So I, like apparently many others, downloaded the windows 8 developer preview.

Plenty of interesting stuff under the hood, but I decided to jump into .Net 4.5.  In the dev preview there is a VS 11 Express edition, for Metro apps. I quickly discovered that .Net 4.5 ‘Core’ Edition, which is what you can use in Metro, is quite a bit different. (How many hours did I spend trying to write the word hello to a file…)

So I decided to do a version comparison. Enter RuntimeVersionCompare.html.  Note this is a 17MB html document, so it may take a little while to download – and I may have to find somewhere else to host it if it becomes popular…

There are 4 colours used.  Red means ‘Only in .Net 4’, Green means ‘Only in .Net 4.5’, Yellow means ‘Missing from .Net 4.5 Core’ and white is whatever else is left over.  With 5 flags (since I included the client profiles), there are potentially 31 scenarios for colouring, but I was lazy.  Each entry does contain numbers mapping to which version contains the specific API, so if you don’t like the colours as they stand, a simple re-parse and output should fix that.  (You could even risk it and try and do such a change using regular expressions…)

You will see a lot of yellow – in part this is because of the absence of WinRT API in the output which Metro can use in addition to .Net 4.5 Core.  WinRT API is missing mostly because it is described using winmd files, which I haven’t yet attempted to reverse-engineer. (Hopefully they are actually .Net metadata-only dll files in disguise and ildasm will process them just fine…)

Also note that this analysis was all done using custom code which parses the output of IlDasm – as such it almost certainly contains errors, strange syntax in the output and many other flaws (not the least of which is that I didn’t use good CSS style in my output HTML).

Feel free to do whatever you like with this document, although attribution would be nice, and please try to avoid downloading it more than once…