Why were magic methods implemented in C#?

Question

In C#, I started seeing all these magic methods popping up, without being backed up by an interface. Why was this chosen?

Let me explain.

Previously in C#, if an object implemented the IEnumerable interface, it would automatically be iterable by a foreach loop. That makes sense to me, since it's backed up by an interface, and if I were to have my own Iterator function inside the class being iterated through, I could do that without worrying that it would magically mean something else.

Now, apparently, (not sure when), these interfaces are no longer required. It just needs to have the right naming conversions.

Another example is making any object awaitable by having a method named exactly GetAwaiter which has a few specific properties.

Why not make an interface like they did with IEnumerable or INotifyPropertyChanged to back this "magic" up statically?

More details on what I mean here:

http://blog.nem.ec/2014/01/01/magic-methods-c-sharp/

What are the pros and cons of magic methods, and is there anywhere online where I can find anything on why these decisions were made?

• DougM: You should edit out your personal opinion as to why "magic is bad" if you don't want to get closed.
• Jörg W Mittag: If you want to know why Anders Hejlsberg did that, you'll have to ask him. I only can tell you why I would have done that, and it is for forwards-compatibility. Extension methods allow you to "fake" adding methods to existing types, but there are no extension interfaces. If you require an interface for, say, async/await, then that will only work with code that was written after .NET 4.5 became wide-spread enough to be a viable target ... which is basically now. But a purely syntactic translation into method calls allows me to add await functionality to existing types after the fact.
• Jörg W Mittag: Note that this is basically applied object-orientation: as long as an object responds to the appropriate messages, it is considered to be of the correct type.
• Jesse C. Slicer: Your "previously" and "now" are backward - the magic method was implemented as the base functionality for a foreach loop back in the beginning. There never has been a requirement for the object to implement IEnumerable for foreach to work. It's just been convention to do so.
• Jesse C. Slicer: @gbulmer this is where I recall getting the idea from: blogs.msdn.com/b/ericlippert/archive/2011/06/30/… (and to a lesser extent ericlippert.com/2013/07/22/…)

Answer 1

In general "magic methods" are used when it's not possible to create an interface that would work the same way.

When foreach was first introduced in C# 1.0 (that behavior is certainly nothing recent), it had to use magic methods, because there were no generics. The options basically were:

1. Use the non-generic IEnumerable and IEnumerator that work with objects, which means boxing value types. Since iterating something like a list of ints should be very fast and certainly shouldn't create lots of garbage boxed values, this is not a good choice.

2. Wait for generics. This would probably mean delaying .Net 1.0 (or at least foreach) by more than 3 years.

3. Use magic methods.

So, they chose option #3 and it stayed with us for backwards compatibility reasons, even though since .Net 2.0, requiring IEnumerable<T> would have worked too.

---

Collection initializers can look different on each collection type. Compare List<T>:

public void Add(T item)

and Dictionary<TKey, TValue>:

public void Add(TKey key, TValue value)

You can't have a single interface that would support only the first form for List<T> and only the second for Dictionary<TKey, TValue>.

---

LINQ methods are usually implemented as extension methods (so that there can be just a single implementation of e.g. LINQ to Object for all types that implement IEnumerable<T>), which means it's not possible to use an interface.

---

For await, The GetResult() method can return either void or some type T. Again, you can't have a single interface that can handle both. Though await is partially interface-based: the awaiter has to implement INotifyCompletion and can also implement ICriticalNotifyCompletion.

• gbulmer: Are you certain "they chose option #3" for C# 1.0? My recollection is it was option 1. My memory is, C# claimed significant superiority over Java because the C# compiler did 'auto-boxing' and 'auto-unboxing'. It was claimed, C# made code like that foreach work much better than Java partly because of that.
• svick: The documentation for foreach in C# 1.2 (there is nothing on MSDN for C# 1.0) says that the expression in foreach "Evaluates to a type that implements IEnumerable or a type that declares a GetEnumerator method." And I'm not sure what you mean by that, unboxing in C# is always explicit.
• svick: @gbulmer And the ECMA specification for C# from December 2001 (which has to mean it's for C# 1.0) also says that (§15.8.4): "A type C is said to be a collection type if it implements the System.IEnumerable interface or implements the collection pattern by meeting all of the following criteria [...]".
• CodesInChaos: @svick foreach(T x in sequence) applies explicit casts to T on the elements of the sequence. So if sequence is a plain IEnumerable and T is a value type it will unbox without any explicit cast being written in your code. One of the uglier parts of C#.
• svick: @CodesInChaos "Auto-unboxing" sounds pretty general, I didn't realize it referenced this specific feature. (I guess I should have, since we were talking about foreach.)