A history of question migration in tables and a graph

Question 1

[Note: Most of this post was written before the name change and I haven't changed to Software Engineering. I think that's appropriate, since I'm looking backward to when this site was Programmers.]

[Note: This in response to a comment by gnat on Meta Stack Exchange asking for information regarding migrations to and from this site. It's being shared here since it may be of interest to the community.]

I wrote a proposed philosophy of question migration. My thesis is that migration ought to be thought of as a convenience for the OP when they ask on the wrong site rather than a tool to manage content. Given there exists a chatbot to detect mentions of programmers.SE in order to deflect potential migration/crossposting, it seems to me that this site is particularly concerned about migration.

As a baseline, here are some statistics for the entire history of the site going back to the very first question asked:

Total Questions 
--------------- 
98768 
Migrated Direction 
-------- --------- 
 8900 out 
14484 in 
Exported to: count good 
------------ ----- ---- 
stackoverflow.com 6482 6030 
workplace.stackexchange.com 297 273 
codereview.stackexchange.com 264 228 
superuser.com 216 203 
security.stackexchange.com 173 171 
ux.stackexchange.com 165 162 
meta.programmers.stackexchange.com 125 111 
dba.stackexchange.com 118 115 
pm.stackexchange.com 103 97 
cs.stackexchange.com 87 81 
Imported from: count good 
-------------- ----- ----- 
stackoverflow.com 13828 11630 
codereview.stackexchange.com 300 272 
superuser.com 45 34 
serverfault.com 40 32 
pm.stackexchange.com 31 28 
cs.stackexchange.com 31 19 
webmasters.stackexchange.com 27 16 
workplace.stackexchange.com 26 22 
english.stackexchange.com 14 12 
ux.stackexchange.com 14 12

I'm just including at the top ten sites as there's a noticeable drop off in quantity after the top slot. As you can see, Programmers has received 5584 more questions from other sites (mostly Stack Overflow) than it has migrated away. A full 14% of the questions that have ever existed on this site began life on Stack Overflow.

But those all-time numbers hide an important story. Almost all of the migrations occurred before 2012:

Question migrations by year

In recent years, Programmers has been a net exporter of questions:

Total Questions since 
--------------- ------------- 
76426 2012年01月01日 00:14:59
Migrated Direction 
-------- ---------
6629 out 
2307 in 
Exported to: count good 
------------ ----- ---- 
stackoverflow.com 3547 3116 
codereview.stackexchange.com 151 118 
workplace.stackexchange.com 128 104
superuser.com 68 59 
meta.programmers.stackexchange.com 63 51 
cs.stackexchange.com 56 50 
ux.stackexchange.com 53 52 
dba.stackexchange.com 48 45 
security.stackexchange.com 41 39 
unix.stackexchange.com 32 28 
Imported from: count good 
-------------- ----- ---- 
stackoverflow.com 870 584 
codereview.stackexchange.com 179 158 
serverfault.com 23 16 
cs.stackexchange.com 22 10 
webmasters.stackexchange.com 8 5 
workplace.stackexchange.com 8 4 
pm.stackexchange.com 7 6 
ux.stackexchange.com 6 6 
superuser.com 6 4 
webapps.stackexchange.com 4 3

I picked January 1, 2012 because it's a handy cut off, not because of any particular event on the site. The quality of incoming questions has been respectable:

Migrated here Avg score Avg answers Avg comments Closure % Deletion % Avg age (days)
------------- --------- ----------- ------------ --------- ---------- --------------
 2307 6.0 2.6 3.5 14.3 16.0 32.3

Do remember votes from the source site are reflected in the score. For instance, questions migrated from Stack Overflow to this site had an average score of 5.8 and 1.1 answers. So score might be less useful a measure of quality than the fact that these questions got new answers after migration and are relatively less likely to get closed than organic questions.

That said, fully a third of the questions migrated from Stack Overflow have been closed or deleted. We don't have an easy way to quantify crossposted questions or questions asked at the prompting of misinformed commenters on another site.

Question 2

I don't think that there's anything surprising in this data.

The Importing of Questions from Stack Overflow

The high number of imported questions from Stack Overflow makes sense. Software Engineering was originally Programmers, which was originally proposed as Not Programming Related. The proposal for Not Programming Related was for a safe place to ask the questions that were too centered on opinions or too broad for Stack Overflow. That didn't last very long, and the quality of questions led to a blog post on the Stack Overflow blog. This site was one of the main reasons for the Good Subjective, Bad Subjective and Real Questions Have Answers posts, and numerous Meta discussions about question quality.

This transition from Not Programming Related to something pretty close to what we are today spanned from the proposal on 3 June 2010 through the private beta that started on 1 September 2010 through a lot of the public beta which began on 8 September 2010. However, by our full launch in early December 2010, we had changed our scope to something closer to what it is today. This turmoil in definition was about 6 months, but wasn't clear to anyone who didn't participate on the Meta site.

One big problem with all of this turmoil was that, until a few days ago, we continued to have the same name and branding. To anyone outside of active participants on our community, they may not have been aware of the drastic change in expected question quality or scope. This is why Duga exists - to help us find people (primarily on Stack Overflow) who are blindly suggesting our site without understanding our scope. The things that should have made our scope change blindingly obvious to anyone - new subdomain, new logo - were flat-out rejected in June 2012.

Until June 2012, Stack Overflow users could vote to migrate questions here. When that Meta post was written, we had a 26% rejection rate for migration. The history of migration from Stack Overflow to Programmers was so bad that, in December 2010, we had to ask Stack Overflow users to stop using Programmers as their toilet bowl. Even with us seeing the influx of garbage sent from Stack Overflow, it still took a year and a half to stop individual users from migrating questions.

Also in October 2012, migrations for questions older than 60 days were disabled.

2012 was a big year for stopping incoming migrations. It stopped users from sending old (and likely low quality) SO questions to us by taking away their ability to migrate to us and even stopped SO mods (who may not have understood our site) from migrating old questions to us.

The Exporting of Questions to Other Sites

The high number of exported questions also makes sense. A general trend in increase of migrations also makes sense.

Since 2010, our scope has been stable, for the most part. When we went live in December 2010 and were introduced to the world in a blog post, the things that were on topic here included software engineering, developer testing, algorithms and data structures, design patterns, architecture, development methodologies, quality assurance, software law, and business concerns.

Really, what has always been on-scope here is software engineering. Most of the things in the original list of things on-topic are part of software engineering. Today, we have a more succinct list of things that are on-topic, but it's still essentially all of software engineering.

However, some aspects of software engineering are better covered on other sites. Information Security, Database Administrators, Writers, Project Management, Computer Science, The Workplace, Law, and Open Source were all started after Programmers was, and these are all things that someone working in software engineering would learn, either as part of a curriculum or on-the-job. It's not that database design questions or data structure and algorithm questions or licensing questions are off-topic here, it's just that there are communities who can give better answers to these types of questions.

I think that part of this exporting culture may have been on us, as a community. When we shifted from the original Not Programming Related concept to Programmers, I think that we kind of normalized around the idea that we were actually a community of software engineers. Being primarily professionals, we wanted a site where the questions we had as professionals would be accepted without being lost in a sea of "fix my code plz" questions on Stack Overflow. There were already sites for writing and debugging code, professional-level system administration, using various computer software tools, mathematics, statistics, user experience, and a few other things that we cared about. So we include a variety of our professional concerns in the list of things that were on-topic. Until The Workplace came around, we had more tolerance of education and career questions. Before Computer Science, we took in some of the more theoretical/mathematical algorithmic questions. Before Law and Open Source, we took in a lot of law and licensing questions (but began to constrain that since we realized we weren't lawyers and couldn't answer all of these questions).

There are two good examples for migrations. First, we did a little experiment where the same question was asked on multiple sites. The answers were dramatically different. Here, they were very practical, focused on the quality attributes of the software (testability, readability, performance), and written in more plain English. On CS, they were more mathematical and focused on the theoretical behavior of the algorithm. Another example is Open Source and Law - the people there have learned more about and worked with copyright and licensing and the quality in answer is so much higher - just look at the answers here vs there in software license questions.

Ideally, we should be seeing this trend downwards. It may rise slightly if a new related site goes into a public beta or launches, but shouldn't really spike. At this point, we have a well-defined scope that focuses on the core aspects of software engineering without the tangential things. The main reason for this is that a lot of these tangential things now have their own communities in various states of development.

The Purpose of Migration

I disagree with the thesis that "migration ought to be thought of as a convenience for the OP when they ask on the wrong site rather than a tool to manage content".

Migration is a tool to manage content, just like deletion. However, unlike closure or deletion, it preserves the content and signals to the user "yes, you asked a good question, but although we can't give you a great answer here, you can get a great answer over there". By moving the good question, we help the asker get a solution to their problem and introduce them to a community of experts if they have a similar problem in the future. We help people who have the same question in the future have access to higher quality answers. We also add one more good question to another site that will help boost its discoverability by searching.

As long as we continue to uphold the rule of "don't migrate crap", we will probably be an exporter of questions. Our new name gives people a good impression of what we are about, but they may not be aware of niche communities that would be better for very specific questions. They may also not be aware of nuances between some communities, such as Software Engineering and Computer Science and the differences in answers.

Personally, I wouldn't mind seeing migration go away. A question is either on-topic or not and it doesn't need to be our concern if they get the best answers or not. We can use comments and other tools to let people know about other communities that exist. However, as it stands now, I have a tool that helps me ensure that the user gets the best possible answers for their questions, so I use it.

Question 3

wrt experiment with Computer Science, did you mean croissants? If yes, it also has been posted to Code Golf and Stack Overflow (at the latter it turned out an epic fail, meta drama followed by deletion)

Question 4

@gnat Thanks - edited in. I knew it had to do with breakfast, so I was looking for pancakes and bagels. I couldn't remember croissants.

Thomas Owens Mod 85.9k3 gold badges52 silver badges131 bronze badges · Answer 1 · 2016-10-22 02:09:39Z

I don't think that there's anything surprising in this data.

The Importing of Questions from Stack Overflow

The high number of imported questions from Stack Overflow makes sense. Software Engineering was originally Programmers, which was originally proposed as Not Programming Related. The proposal for Not Programming Related was for a safe place to ask the questions that were too centered on opinions or too broad for Stack Overflow. That didn't last very long, and the quality of questions led to a blog post on the Stack Overflow blog. This site was one of the main reasons for the Good Subjective, Bad Subjective and Real Questions Have Answers posts, and numerous Meta discussions about question quality.

This transition from Not Programming Related to something pretty close to what we are today spanned from the proposal on 3 June 2010 through the private beta that started on 1 September 2010 through a lot of the public beta which began on 8 September 2010. However, by our full launch in early December 2010, we had changed our scope to something closer to what it is today. This turmoil in definition was about 6 months, but wasn't clear to anyone who didn't participate on the Meta site.

One big problem with all of this turmoil was that, until a few days ago, we continued to have the same name and branding. To anyone outside of active participants on our community, they may not have been aware of the drastic change in expected question quality or scope. This is why Duga exists - to help us find people (primarily on Stack Overflow) who are blindly suggesting our site without understanding our scope. The things that should have made our scope change blindingly obvious to anyone - new subdomain, new logo - were flat-out rejected in June 2012.

Until June 2012, Stack Overflow users could vote to migrate questions here. When that Meta post was written, we had a 26% rejection rate for migration. The history of migration from Stack Overflow to Programmers was so bad that, in December 2010, we had to ask Stack Overflow users to stop using Programmers as their toilet bowl. Even with us seeing the influx of garbage sent from Stack Overflow, it still took a year and a half to stop individual users from migrating questions.

Also in October 2012, migrations for questions older than 60 days were disabled.

2012 was a big year for stopping incoming migrations. It stopped users from sending old (and likely low quality) SO questions to us by taking away their ability to migrate to us and even stopped SO mods (who may not have understood our site) from migrating old questions to us.

The Exporting of Questions to Other Sites

The high number of exported questions also makes sense. A general trend in increase of migrations also makes sense.

Since 2010, our scope has been stable, for the most part. When we went live in December 2010 and were introduced to the world in a blog post, the things that were on topic here included software engineering, developer testing, algorithms and data structures, design patterns, architecture, development methodologies, quality assurance, software law, and business concerns.

Really, what has always been on-scope here is software engineering. Most of the things in the original list of things on-topic are part of software engineering. Today, we have a more succinct list of things that are on-topic, but it's still essentially all of software engineering.

However, some aspects of software engineering are better covered on other sites. Information Security, Database Administrators, Writers, Project Management, Computer Science, The Workplace, Law, and Open Source were all started after Programmers was, and these are all things that someone working in software engineering would learn, either as part of a curriculum or on-the-job. It's not that database design questions or data structure and algorithm questions or licensing questions are off-topic here, it's just that there are communities who can give better answers to these types of questions.

I think that part of this exporting culture may have been on us, as a community. When we shifted from the original Not Programming Related concept to Programmers, I think that we kind of normalized around the idea that we were actually a community of software engineers. Being primarily professionals, we wanted a site where the questions we had as professionals would be accepted without being lost in a sea of "fix my code plz" questions on Stack Overflow. There were already sites for writing and debugging code, professional-level system administration, using various computer software tools, mathematics, statistics, user experience, and a few other things that we cared about. So we include a variety of our professional concerns in the list of things that were on-topic. Until The Workplace came around, we had more tolerance of education and career questions. Before Computer Science, we took in some of the more theoretical/mathematical algorithmic questions. Before Law and Open Source, we took in a lot of law and licensing questions (but began to constrain that since we realized we weren't lawyers and couldn't answer all of these questions).

There are two good examples for migrations. First, we did a little experiment where the same question was asked on multiple sites. The answers were dramatically different. Here, they were very practical, focused on the quality attributes of the software (testability, readability, performance), and written in more plain English. On CS, they were more mathematical and focused on the theoretical behavior of the algorithm. Another example is Open Source and Law - the people there have learned more about and worked with copyright and licensing and the quality in answer is so much higher - just look at the answers here vs there in software license questions.

Ideally, we should be seeing this trend downwards. It may rise slightly if a new related site goes into a public beta or launches, but shouldn't really spike. At this point, we have a well-defined scope that focuses on the core aspects of software engineering without the tangential things. The main reason for this is that a lot of these tangential things now have their own communities in various states of development.

The Purpose of Migration

I disagree with the thesis that "migration ought to be thought of as a convenience for the OP when they ask on the wrong site rather than a tool to manage content".

Migration is a tool to manage content, just like deletion. However, unlike closure or deletion, it preserves the content and signals to the user "yes, you asked a good question, but although we can't give you a great answer here, you can get a great answer over there". By moving the good question, we help the asker get a solution to their problem and introduce them to a community of experts if they have a similar problem in the future. We help people who have the same question in the future have access to higher quality answers. We also add one more good question to another site that will help boost its discoverability by searching.

As long as we continue to uphold the rule of "don't migrate crap", we will probably be an exporter of questions. Our new name gives people a good impression of what we are about, but they may not be aware of niche communities that would be better for very specific questions. They may also not be aware of nuances between some communities, such as Software Engineering and Computer Science and the differences in answers.

Personally, I wouldn't mind seeing migration go away. A question is either on-topic or not and it doesn't need to be our concern if they get the best answers or not. We can use comments and other tools to let people know about other communities that exist. However, as it stands now, I have a tool that helps me ensure that the user gets the best possible answers for their questions, so I use it.

wrt experiment with Computer Science, did you mean croissants? If yes, it also has been posted to Code Golf and Stack Overflow (at the latter it turned out an epic fail, meta drama followed by deletion)
@gnat Thanks - edited in. I knew it had to do with breakfast, so I was looking for pancakes and bagels. I couldn't remember croissants.

Stack Exchange Network

A history of question migration in tables and a graph

1 Answer 1

The Importing of Questions from Stack Overflow

The Exporting of Questions to Other Sites

The Purpose of Migration

You must log in to answer this question.

Linked

Hot Network Questions