Success Metrics and Survey

Events happening in the community are now at Drupal community events on www.drupal.org.
You are viewing a wiki page. You are welcome to join the group and then edit it. Be bold!
Posted by leisareichelt on April 25, 2011 at 12:48pm
Last updated by tvn on Mon, 2012年02月06日 15:39

As previously discussed, it's important that we monitor whether or not our work here is having a positive, negative or any impact at all.

There are two ways we will do this:
1 - site metrics. We'll grab a snapshot of certain site metrics at regular intervals (quarterly) and compare
2 - survey. We'll take a regular survey to capture attitudes to contribution to see if these change over time.

Survey:

I've created a survey here.
Can you please take the survey and give me any feedback on the questions/answers before we distribute this more widely.

Site Metrics

Drupal.org metrics can be seen at http://drupal.org/metrics.

Here are some metrics that would be great to capture for the six months ending 31 March 2011 (in anticipation of quarterly snapshots)

Participation

Total no. Drupal.org members who have logged in at least once (this is the count the Drupal.org front page uses):

Today (May 23, 2011): 592521

SELECT COUNT(uid) FROM users WHERE status = 1 AND login > 0;

March 31, 2011: 564507

SELECT COUNT(uid) FROM users WHERE status = 1 AND login > 0 AND created < UNIX_TIMESTAMP('2011-03-31 00:00:00');

(28,014 new members since March 31.)

Total no. of Drupal.org members who have been active in the last 6 months (active = posted to issue queue, discussion forum, updated documentation, posted to GDO, committed code, etc.

This one's a bit trickier, because that information is kinda scattered all over the place.

This is the number of users who authored either a comment or a node. This would catch discussion forum and issue queue activity. It would not catch documentation revisions, GDO, or code commits.

Total: 16,332

SELECT COUNT(DISTINCT u.uid)
FROM users u
LEFT JOIN node n ON u.uid = n.uid
LEFT JOIN comments c ON c.uid = u.uid
WHERE u.login > 0
AND (n.uid = u.uid OR c.uid = u.uid)
AND u.status = 1
AND (n.status = 1 OR c.status = 0)
AND (n.created BETWEEN UNIX_TIMESTAMP(
DATE_SUB(NOW(), INTERVAL 6 MONTH)
)
AND UNIX_TIMESTAMP(
NOW()
))
AND (c.timestamp BETWEEN UNIX_TIMESTAMP(
DATE_SUB(NOW(), INTERVAL 6 MONTH)
)
AND UNIX_TIMESTAMP(
NOW()
))
;

(This query also takes like 2 minutes to complete, and I welcome a more performant/accurate one. :P)

Collaboration

Of issues posted in the past six months:

Total: 61019

SELECT COUNT(n.nid) FROM node n WHERE n.type = 'project_issue' AND n.created > UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 6 MONTH));
  • Total open issues

Total: 18558

SELECT COUNT(n.nid)
FROM node n
INNER JOIN project_issues pi ON pi.nid = n.nid
WHERE type = 'project_issue'
AND n.created > UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 6 MONTH))
/* 1 = active, 4 = postponed, 8 = needs review, 13 = needs work, 14 = rtbc, 15 = needs porting */
AND pi.sid IN (1, 4, 8, 13, 14, 15);
  • Total issues resolved in past six months
  • Average no. people contributing per issue
  • Average comments per issue

This (thanks, mikey_p!) will get you a big "spreadsheet" of issue numbers + number of comments. Need to parse it down into averages, and filter by date:

SELECT pi.nid as nid, count(c.cid) as count FROM project_issues pi LEFT JOIN comments c ON c.nid = pi.nid GROUP BY pi.nid ORDER BY count DESC;
  • Average duration (time from date opened to date closed)
Categories:

Comments

Unless I'm doing it wrong,

Posted by davidhernandez on April 25, 2011 at 1:15pm

Unless I'm doing it wrong, the last part of the survey seems to not be working right. It only lets me answer 4. The bullets are tied together.

thanks - good pick up

Posted by leisareichelt on April 25, 2011 at 1:43pm

thanks David, that's exactly the kind of thing I was hoping someone would pick up for me.
fixed that (crazy default) setting now :)

thanks!

leisa reichelt - disambiguity.com
@leisa

The survey looks good

Posted by eliza411 on April 25, 2011 at 3:43pm

The survey looks good overall. What I see are all minor things:

  • I had to look carefully at Question 2 to find the training/support option. I might separate them just because I did support for a long, long time before I ever did any training, but they're probably fine as they are as well.
  • There are a couple typos in #5, third answer: "can moreo easily see when thinkgs,"
  • It's too bad the URL on the finished page isn't clickable.

Thanks for all the work you're doing here!

thanks!

Posted by leisareichelt on April 25, 2011 at 4:03pm

  • I was tossing up whether to separate training and support and decided not to because I wasn't sure how much 'contrib' training there is... I've separated them now tho, so we'll see

  • fixed those typos thanks - and yes, it's a pain but I can't put a clickable link in there... imperfect survey software...

leisa reichelt - disambiguity.com
@leisa

Added some metrics + queries...

Posted by webchick on May 24, 2011 at 6:29am

I went digging around in the Drupal.org database today, and started filling in some stats and queries you're asking for. Apologies for the rawness; I want to putter away on some kind of sweet dashboard-y thing like Mozilla is doing, but baby steps first. :) I'm going to ask around about how to fill in some of the stuff that is still missing.

Only a minor wording point, I

Posted by naught101 on May 24, 2011 at 6:42am

Only a minor wording point, I consider myself an active (or at least, intermittently active) member of the developer community, but I wouldn't really call myself established. That seems a bit pretentious for someone like me..

retrospective stats

Posted by catch on May 24, 2011 at 6:51am

I had a look at this for another community site, but I don't think I kept the actual queries anywhere useful, were quite simple though - was a quick afternoon's exercise.

It was interesting to compare a few things year by year - it's easy to do that retrospectively - in this case it was for 2005 to 2010:

  • number of non-blocked users created in each year.
  • number of comments posted in each year.
  • unique users posting comments in each year
  • unique users posting nodes in each year

You can also compare the unique users in each year to the unique users for the same query over the five year period - that tells you roughly how much turnover there is year on year (i.e. if 1,000 people post a comment on average for five years, that is going to be somewhere between 1,000 and 5,000 unique people over the whole period which is a massive difference).

Another thing we did was

Posted by catch on May 24, 2011 at 6:52am

Another thing we did was segment posting by node type - i.e. how many unique users post forum topics vs. issues vs. handbook pages.

There's some previous posts

Posted by greggles on June 10, 2011 at 1:14pm

There's some previous posts on this topic in the marketing group, specifically this post might have some helpful ideas.

Metrics thoughts

Posted by jhodgdon on October 24, 2011 at 7:37pm

At the GSoC mentor summit this past weekend, there was a discussion on metrics... which didn't seem to be going anywhere, so I went elsewhere. But one thing may be useful - a blog post that lists several projects that have done some interesting metrics things (and there may be more in the comments):
http://iquaid.org/2011/08/12/working-group-on-community-metrics/

Survey results analysed and published

Posted by lisarex on January 24, 2012 at 5:32am

Dharmesh Mistry and I did the analysis and written summaries of Leisa's Prairie survey, along with the Getting Involved survey we published earlier this fall. The results of both:

Executive summary:
http://drupal.org/node/1399056

Detailed analysis and findings from the Prairie 2011 survey
http://drupal.org/node/1399258

Detailed analysis and findings from the Getting Involved 2011 survey
http://drupal.org/node/1407270

==================================
http://about.me/lisarex

Drupal.org metrics is now

Posted by tvn on February 6, 2012 at 3:38pm

Drupal.org metrics is now live at http://drupal.org/metrics
as a result of this issue: http://drupal.org/node/1182998

Adding Metrics

Posted by mgifford on March 6, 2014 at 3:41am

Prairie Initiative

Group organizers

Group categories

Prairie tags

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds:

AltStyle によって変換されたページ (->オリジナル) /