How Microsoft Rewrote Its C# Compiler in C# and Made It Open Source (medium.com) 85
Mads Torgersen, the lead designer of C# at Microsoft, remembers "Project Roslyn," which built an open-source, cross-platform compiler for C# and Visual Basic.NET "in the deepest darkness of last decade's corporate Microsoft:
We would build a language engine! A unified, public API to C# code: We would redefine the meaning of "compiler". Of course, once you are building an API for the broad C# community, it is kind of a slam-dunk that it should be a .NET API, implemented in C#. So, the old dream of "bootstrapping" C# in C# was fulfilled almost as an accidental side benefit. Roslyn was thus born out of an openness mindset: sharing the inner workings of the C# language for the world to programmatically consume.
This in and of itself was a bit of a bold proposition in what was still a pervasively closed culture at Microsoft: We would share this intellectual property for free? We would empower tool builders that weren't us to better compete with us? The arguments that won the day for us here were about strengthening the ecosystem and becoming the best tooled language on the planet. They were about long-term growth of C# and .NET, versus short term monetization and protection of assets for Microsoft. So even without having mentioned open source, signing up for the cost and risk of the Roslyn project was a big and bold step for Microsoft....
F# released already in 2010 with an open source license and its own foundation -- the F# Software Foundation. The vibrant community that grew up around it soon became the envy of us all. Our team pushed strongly to have an open source production license for Roslyn, and finally a company-wide infrastructure emerged to make it real. By 2012, Microsoft had created Microsoft Open Tech; an organization specifically focused on open source projects. Roslyn moved under Microsoft Open Tech and officially became open source... C# language design and compiler implementation are now completely open processes, with lots of non-Microsoft participation, including whole language features being built by external contributors.
Torgersen's article says C# now enjoys "the scaling of effort via contribution of features and bug fixes, but also the insight and course correction we get through the instant, daily feedback loop that open source provides.
"It's been a long and wild journey, and one that to me is symbolic of the massive changes that Microsoft has undergone over the last decade."
This in and of itself was a bit of a bold proposition in what was still a pervasively closed culture at Microsoft: We would share this intellectual property for free? We would empower tool builders that weren't us to better compete with us? The arguments that won the day for us here were about strengthening the ecosystem and becoming the best tooled language on the planet. They were about long-term growth of C# and .NET, versus short term monetization and protection of assets for Microsoft. So even without having mentioned open source, signing up for the cost and risk of the Roslyn project was a big and bold step for Microsoft....
F# released already in 2010 with an open source license and its own foundation -- the F# Software Foundation. The vibrant community that grew up around it soon became the envy of us all. Our team pushed strongly to have an open source production license for Roslyn, and finally a company-wide infrastructure emerged to make it real. By 2012, Microsoft had created Microsoft Open Tech; an organization specifically focused on open source projects. Roslyn moved under Microsoft Open Tech and officially became open source... C# language design and compiler implementation are now completely open processes, with lots of non-Microsoft participation, including whole language features being built by external contributors.
Torgersen's article says C# now enjoys "the scaling of effort via contribution of features and bug fixes, but also the insight and course correction we get through the instant, daily feedback loop that open source provides.
"It's been a long and wild journey, and one that to me is symbolic of the massive changes that Microsoft has undergone over the last decade."
Re: (Score:1)
I remember that. It was a while ago. I think it was the same year as "the year of Linux on the desktop".
Re: (Score:1)
The first conversations about what would become Roslyn were already ongoing when I joined Microsoft in 2005 - just before .NET 2.0 would ship. That conversation was about rewriting C# in C#. This is a normal practice for programming languages; a proof point of the maturity of the language.
Re: (Score:2)
It's self-hosting but it's not in any way portable because of it.
I would be more impressed if they rewrote the C# compiler in plain vanilla C and released it to the world as source.
Bootstrapping is from bare metal, or as close as possible.
Re: (Score:2)
Re: (Score:2)
'Portable' would mean all it needs is a C compiler. For example, to compile it on NetBSD.
Re: (Score:2)
Re: (Score:1)
I can develop a .Net Core console app on my macbook, copy the bin folder over onto a PC and run it there. What more do you want?
Re: First selfhosting copiler EVER! (Score:2)
You mean like this? https://docs.microsoft.com/en-... [microsoft.com]
Re: (Score:2)
Microsoft providing binaries that can be used to compile it on various flavors of Linux is not good portability. Portability would mean it could be compiled on NetBSD, Irix, maybe BeOS. Anything that has a compliant C compiler.
Re: First selfhosting copiler EVER! (Score:2)
Ok so you mean this? https://github.com/dotnet/core [github.com]
Re: (Score:1)
This [phoronix.com] would have been a more helpful link.
Re: (Score:2)
This is even more interesting. Running .NET on nearly bare metal [ycombinator.com] using a NetBSD rump kernel.
Re: (Score:2)
It's self-hoisting but it's not in any way portable because of it.
FTFY (you forgot the "i" in "hosting").
Re: (Score:2)
I dunno, the GCC project has called it bootstrapping since day one. I used the SPARCworks C compiler to build GCC, then the built GCC to build another GCC, and using that GCC to build the final build.
Same deal on HP/UX, AIX, OSF/1, Amiga UNIX, etc.
Re: (Score:2, Interesting)
Letting your paranoia and politicalization of technology shows a person who has thrown away the concept of "using the best tool for the job". A good programmer is one who delivers the best solution not the one who willingly and reflexively ignores anything connected to MS. MS is their historically closed ecosystem with a more open and cross platform ecosystem. The closed ecosystem created one of the most successful corporations in history. The IT world has changed over the years and MS is changing what need
.NET is still dangerous to depend on (Score:3, Informative)
Re: (Score:1)
The GPL is pushed by the biggest copyright advocate in the industry: the Free Software Foundation.
The direction of the biggest software vendors is providing software as a service and/or tight coupling of hardware and software so abolishing copyright on software actually has relatively little impact on them now and going forward. But abolishing copyright on software is a big problem for the FSF who would stand to lose all their power and relevance if that were to happen.
The FSF play the same game as Hollywoo
Re: (Score:1)
Speaking as an active enemy of Microsoft
haha no. while you can make microsoft your enemy if you like they have absolutely no interest in you whatsoever no matter what your sense of self-importance tells you.
Re: (Score:2)
Re:Opportunity wasted (Score:5, Interesting)
That's really dumb, whilst I agree there are too many languages out there, and people don't always need to make new ones when existing ones exist, we also don't want a monoculture where tech goes down one singular path.
The fact is C# and .NET were already well established, before LLVM even came into existence, and have a massive ecosystem around them with millions of devs and companies using it.
Thus, making it able to work for more people in more places is a good thing, and given it's strong base, like Java, it's a reasonable piece of alternative tech that should stay alternative and not be converged into a great LLVM monolith where progress will never happen because there's no competition anymore. That would be a terrible outcome for everyone, it's what happened when IE "won" the browser wars and we were all stuck with IE6 ruling the browser world for half a decade.
As for your comment on bugs, I'm not sure what your point is. You're basically arguing against open source altogether at that point because you're saying if a community finds and logs bugs it's a terrible thing? How fucking stupid are you? The whole point in open source is precisely so that many eyes can spot problems and collaborate to fix them. There are plenty of major open source projects that are nothing to do with Microsoft in the exact same position. If you criticise an organisation for properly tracking all it's bugs and doing so publicly and for allowing people to contribute more bug reports and also therefore allowing open source devs to also chip in and fix them, then you're basically arguing companies shouldn't open source because you'll shame them for letting other people work on the project.
People like you really need to get over your irrational Microsoft hate and take a leaf out of Torvald's book and go and get some professional help. You're exactly the sort of toxic open source community member he was talking about, because even when you've got literally everything you wanted from a formerly proprietary company - i.e. a complete open sourcing of one of their key products, you STILL bitch and moan at them and imply they shouldn't have done it. You're exactly the sort of person the OSS community doesn't need because you're exactly the sort of no-life vermin that pushes organisations and individuals away from contributing to it.
Re: (Score:1)
> By the way. the reason people don't trust MS is most likely because they are a decade or 2 older than you, and actually remember what MS has done in the past first-hand, and also observe how most 'changes' from MS are actually double-bottomed party tricks.
And there we have it, speculation, bullshit, and paranoia, with no ability to look objectively at the fact that companies are made of the people who staff them, and that the people that staffed the Microsoft you remember from 20 years ago aren't even
Re: (Score:1)
And who is it that has to gain your trust back exactly? The artificial inanimate construct that is the trademark "Microsoft"?
Or the people who work at Microsoft NOW and are responsible for turning it into the biggest open source contributor on the planet and making it an open source power house, that are driving the company's open source strategy ever forward? You want them to somehow gain your trust back, even though they're the ones who are doing exactly as you think they should do?
Or are you expecting th
Re: (Score:1)
The irony is its called "open source" when most of the people who believe it are "closed" to most things outside their little community.
Re: (Score:3)
Do you know where those people work now? How do you feel about those companies?
Any long term /. reader will know that most big companies are frown upon. How Apple went from nerd's favorite to fanboys. How Google is almost synonym for Big Bro. How IBM went from markt leader to small boy.
Any big company should be observed with skepticism and reservation, not only the tech sector, but the tech sector simply has a big influence on our daily lives and the future to come. That's why they are under a microscope. It's also why they are not expected to change fast.
But when it comes to Microsof
Re:Opportunity wasted (Score:4, Interesting)
I thought the mindset of open source was that YOU did things how YOU wanted to do them, and provide the source so that people can fork if they disagree?
That is after all why there are over a thousand text editors for linux.
That is why you have X.org and X386 if I recall correctly.
VIM and emacs.
etc.
My guess is that no matter what Microsoft does, you will complain about it.
Re:Opportunity wasted (Score:5, Funny)
That is why you have X.org and X386
The project was named XFree86. You were so very, very close :)
Yaz
Re: (Score:1)
Indeed. MS is always very proud of doing it far worse than all those that have done something similar before...
Re:Opportunity wasted (Score:5, Interesting)
If they wanted to actually contribute to the community then they would have made an LLVM frontend
They did. [github.com] LLVM is not currently a great fit for garbage collected languages, though the LLILC team worked with some of the Azure folks to improve it in this regard. LLVM also suffered from longer compile times (important for a JIT). The Roslyn architecture makes it easy to incrementally replace the JIT, because it supports trying to compile individual functions with a new JIT and falling back to the old one if it doesn't support all of the functionality required. The LLILC team made use of this when trying to bootstrap.
Re: Opportunity wasted (Score:4, Insightful)
They have submitted code. Infact clang is an option in visual studio community as an option. .net core is available for Linux and so is visual studio code editor. Redhat is ditching Mono and even including Microsoft's Rosslyn and let's not forget WSL for Windows10 nor Android sdk including Android emulators with Visual studio either or the fact that Bill Gates himself owns a Samsung phone.
I am not saying to be a MS fanboy. What I am saying is MS is changing due to the world changing around them. It isn't 1999 anymore. Apple is much more powerful and popular than back then. Also IOS, Android, HTML 5 browsers, cloud computing, and a plethora of free languages and apis like python, R, rust, node.js, and others changed the landscape. Gone are VB, COM, vc++, Internet Explorer, and WinCE.
MS has a new CEO who realized they no longer set the pace of the industry and if they want to remain relevant they and not loose mellinial developers they need to include, not exclude.
So that is the argument. Remember Apple was cool here and open source too back in 1999 on slashdot. They turned assholish FAST and more aggressive than MS when they got power. All companies are the same once they corner a market
How? (Score:1)
The question I'm asking is why?
Re: How? (Score:4, Interesting)
Re: (Score:2)
Is it a patent trap if it is under the Apache licence? I'm not sure where one stands legally if they patent something and then release it under the Apache licence, which AFAICS is very similar to the BSD licence.
Commenting to undo moderation (Score:2)
Re: (Score:2)
So, Microsoft rewrote it's C-pound compiler in C-pound and made it
C-flat, I think.
But yeah, what's the point when openjdk is there, why bother, meh. Pretty much the same language, pretty much the same paradigm. Why bother? If you want to help the world, work on rust or nim. They look promising.
Give me break. (Score:1)