Sustainability of Open Source software communities beyond a fork: How and why has the LibreOffice project evolved?
Tagged: community Toggle Comment Threads | Keyboard Shortcuts
The OpenStack Foundation has always tried to increase genders diversity in our community: we joined the Outreach Program for Women, established clear policies for our summits and in general we’ve been actively promoting good behavior across the board. A few weeks ago I realized that we were lacking of a decent way to track our progress in gender diversity in the OpenStack developer community and decided to improve the situation.
When Anne asked me for the number of women or non-male developers in OpenStack, I realized that we don’t have that number. When people register to become OpenStack developers they need to become members of OpenStack Foundation and have an account on Launchpad. Neither of them track gender information. The best number I had to offer was the t-shirt kind requested at our Summits. If you don’t measure it you can’t improve it, the saying goes. So we thought we need to offer the option for Foundation’s members to tell us their gender, if they want to. How do we ask for gender without being disrepectful to non-binary genders?
We did some research, discussed them on the bug and adapted the best practices to our case. We wanted to keep the choice short and not to offend anybody by leaving out options. An open entry form leads to hard to use data (people have very creative ways to spell just about anything), using pronouns would have made translations a nightmare. We need a way to restrict options so we can easily count them in the database so the binary options male/female were set first and ‘Prefer not to say’ was soon added as another option, since it’s really not mandatory to disclose that information. For non-binary genders using “Non-binary” sounded too geeky to me; using ‘other’ sounds weird, borderline offensive. We came to a consensus on my suggestion to have an open entry prefixed by “Let me tell you”. I liked this phrase because I feel like “Let me tell you” empowers the member to own their own gender definition. The new form is now live so you can now register or edit your OpenStack Member profile and add your gender (if you want to).
A few days ago, I asked 55 OpenStack developers why they decided to submit one patch to OpenStack and what prevented them from contributing more. The sample polled people who contributed only once in the past 12 months, looking for anecdotal evidence for what we can do to improve the life of the occasional contributor. To me, occasional contributors are as important as the core contributors to sustain the growth of OpenStack in the medium/long term. Since it’s possible that the community has already already hired the best candidates among Python developers and system operators, to sustain the growth of OpenStack we need to tap into the wider Python community and experts of large distributed systems coming from other languages (Java, for starters). Knowing what hurdles new contributors face when they first join OpenStack is a first step to make onboarding easier and multiple contributions more frequent.
People who replied are for the most parts users of OpenStack, that is developers and dev-ops people working on deployments of OpenStack for internal usage and system integrators who develop and deploy OpenStack for their customers. A third group of occasional contributors are developers of projects that use or are used by OpenStack, like Docker or Ceph.
The top reasons to submit a patch went from the practical convenience of having patches maintained upstream, to fixing bugs (discovered during a proof of concept or in more advanced phases of a deployment) or to add/fix funcionalities to better integrate OpenStack with other projects. Finally, people contribute patches for personal growth and to enrich resumes.
Why don’t they submit more? There are three main stumbling blocks. Legal hurdles delayed contributions, and even prevented (at least) one. The reasons for the delays range from NDAs with customers to incompatibility with corporate policies to release software under open source license and sign a Corporate Contributor License Agreement. Another group of reasons for the delay is the long time it takes for contributions to travel the review queue. The time it takes for patches to get in OpenStack repositories was mentioned more than once. Finally, the lack of simple issues to fix in the spare time: apparently the community fixes simple bugs too rapidly.
The results cannot be considered representative but they seem to confirm that legal hurdles and speed of the reviews are preventing more casual contributions. While there are already many discussions on how to improve reviewing times, changing the way we handle contributions legally requires a massive endeavour to change OpenStack Foundation’s bylaws. The fact that simple issues to fix are hard to find is new to me: it may indicate that there are lots of people joining the community fixing these ‘low hanging fruits’ or something else. I think more analysis is needed in this area.
No doubt that OpenStack is one of the largest open source development efforts existing at the moment, with Linux, Debian and few others. What can be improved to make the OpenStack community grow faster? How can we help new contributors become productive more rapidly?
Whatever dimension you look at, the growth of the OpenStack developers community is astonishing. I’ve looked at the reasons for this growth and I’ve identified four main causes summarized in my presentation at Linux Conf North America and Open World Forum in Paris last year:
- One clear mission to solve a hard problem
- One initial core team with strong motivation to solve that problem
- One team dedicated to recruiting new team members
- One clear path to onboard new members in the team
The first two points are pretty clear already and don’t seem to need much tweaking: those ain’t broken, so no need to fix’em. The other two are not broken either but they’re the kind of stuff you want to stay on top of because once you notice they’re broken, the cost to fix them will be higher. Recently I started to ask myself what margins of improvements there are in recruiting and onboarding new members in our communities.
I quickly realized that while we know a lot about the top contributors to OpenStack, we don’t know much about the ‘long tail’ of contributors. It’s fairly easy to understand why the people responsible for 80% of OpenStack code and documentation work on the project. But who are the occasional contributors and what are their motivations to join the community? I embarked on a quest to get to know this long tail in an effort to understand how we can improve recruiting and onboarding new developers.
After a conversation with Asheesh Laroia, of OpenHatch fame, I selected 55 random developers from the list of those who submitted a single change to the integrated OpenStack programs during the past 12 months. These people went through quite a huge chunk of trouble only to submit one single patch: besides learning enough of the quite complex OpenStack code, they had to sign the Individual CLA (maybe the Corporate CLA, too), learn about git review and the other tools, go through the code review process. I would expect that whoever goes through that process, does it with the intention to stick around. So I asked the sample two simple questions:
- What made you decide to submit a patch to OpenStack?
- What prevented you to submit more patches?
I’ll wait a few days for their answers and share more details. Feel free to answer the questions leaving a comment here, too.
Why people contribute to OpenStack | ][ stefano maffulli, stefano, Kevin Fox, and 1 other are discussing. Toggle Comments
Back from busy days in one of the most exciting cities I’ve ever visited, I needed some time to put thougths back in order, recover from jet lag and deal with my irremediably broken WordPress installation (will probably blog about this later, too). Hong Kong was a blast in many aspects. The Summit itself started with lions dancing at the sound of drums in front of over 3,000 people:
And then we saw that Bejing is the first city by total number of contributors to OpenStack in the whole world:
There has been an incredible growth inside and around OpenStack, the project is growing fast. Growth is good, it’s what we wanted so we have plenty of reasons to celebrate. The second edition of the User Survey brought us more insights about usage of OpenStack around the world. We announced our first OpenStack Ambassadors, people who will help the OpenStack Community team get closer to many communities around the world. It was great to meet personally more women from the Outreach Program for Women of OpenStack. we’ve been doing this for over a year now, it’s a thing.
Growth also brings challenges and some of them were evident in some of the conversations had at the Summit, around it and after. A few signals I caught during and around the Design Summit sessions highlighted that we may need to start taking new steps to reinforce the culture of collaboration inside the project. The challenges highlighted go from lack of reviewers (not core reviewers, just developers who pay attention and help others), PTLs getting overloaded, the high traffic on the Development mailing list (which leads to loss of information), the increasing number of questions on Ask OpenStack with no interactions (no up/down votes, comments, etc) and little engagement in its Chinese version, the challenges inside the Internationalization Team with processes and tools. We’ve also heard of a very few Design sessions where it was too hard to have a productive discussion because of one or two uncollaborative people.
Since we’re getting so many new developers in the project we’re probably getting to the point where we can’t assume they are accustomed to contributing upstream first. The founders and first members of OpenStack all had a brilliant pedigree of open source contributions and collaboration. New members of the OpenStack Foundation may need some help to succeed. I enjoyed the session Getting Your Blueprint Accepted Quicker: the VPNaaS Use Case so much that I’m proposing the Upstream University training as an official program at the OpenStack Foundation to help new members. I’ll write more about this in the future.
The next six months will continue to be super exciting and full of things to do. If you have missed Hong Kong go watch the recordings of the sessions keep watching this space for more news.
Linux kernel community has long been considered the greatest of all the open source communities. Linus Torvalds and his team has set the ground for open source development, defined processes and tools adopted and shared by other successful projects. The Linux kernel mailing list with its public review of patches and git, the tool to manage the incredible flow of code among thousands of people in tens of different branches laid the ground for many other open source projects. Even OpenStack doesn’t get close to those number (although OpenStack is only a three years old toddler and the kernel is old enough to vote and drink.) Linus has built an incredible community and an impressive culture around it. A culture where technology rules everything and also profanity and insults are common. And the results are clear: it worked for Linux kernel.
This doesn’t mean that any community can live and prosper like the Linux kernel with the same culture of harsh criticisms, middle fingers or what Linus calls management by perkele. In fact, I think nobody else can afford managing open source communities the way Linus does. Torvalds can get away saying things like “trying to come up with some ‘code of conduct’ that says that people should be ‘respectful’ and ‘polite’ is just so much crap and bullshit”. I certainly can’t and chances are you can’t either.
Now if you ask me if Torvalds should change his attitude my answer is: no, he is what he is and he’s made what he’s made because (or despite, who cares: results matter) of what he is. Should his lieutenants be assholes too? Of course not, and that’s why the kernel is still one of the most successful open source projects out there.
Episode two of my experiment with the Raspberry Pi I have received at Pycon 2013. On the way back from Santa Clara I stopped by Fry’s and bought the only USB sound card they had in store, a weird looking Creative and a powered USB hub. I ordered also a case from Adafruit industries (not convinced of the case though).
Bus 001 Device 006: ID 147a:e03d Formosa Industrial Computing, Inc.
Hooked them all together, installed avahi-daemon on the Pi so I can ssh into it easily from any lan (ssh raspberrypi.local, although I should change its name to something more unique). Tested arecord locally first. It took me a while to figure out how to use arecord, it’s old stuff that I’m not very used to. You need to specify the hardware device. If you get this sort of error:
arecord: main:682: audio open error: No such file or directory
probably you haven’t specified that you want to record from the card that actually has an input device
pi@raspberrypi ~ $ arecord -l
**** List of CAPTURE Hardware Devices ****
card 1: Audio [2 Channel USB Audio], device 0: USB Audio [USB Audio]
Subdevice #0: subdevice #0
I hooked my phone’s audio player to the mic input of the USB card so that there would be constantly audio coming in the Pi, then started recording
pi@raspberrypi ~ $ arecord -D hw:1,0 -f s16_le -c 2 > test.wav
I have specified the hardware device I want to use, the format and the number of channels. Playing back that file worked.
pi@raspberrypi ~ $ aplay -D hw:1,0 test.wav
Next step was to capture live audio from the line input of the USB card, transcode that to OGG Vorbis and ship the bits to the remote Icecast server I setup last week. I quickly gave up on ezstream and started using ices2, the Icecast client: it seems easier to manage as it takes care of the encoding. This is the input module I used for the Pi:
<!– Read metadata (from stdin by default, or –>
<!– filename defined below (if the latter, only on SIGUSR1) –>
The USB soundcard I’m using sends 16000hz samples. I chose not to resample that, only to downmix stereo to mono to save bandwidth.
<– stereo->mono downmixing, enabled by setting this to 1 –>
And all seems to work: the Pi sends a clear signal up to the streaming server and it’s been doing that for a while. Big success so far. Next step for me will be to write a script that grabs data from the OpenStack Summit schedule for one room and adds that information as metadata for the streaming: this way the listeners will have an idea of who is speaking or what the session is about.
Update: the stream was having a wide latency, around 20 seconds so I decided to play a little bit with the sampling rates. The latency went down to around 1 second using <param name=”rate”>48000</param> in the input module and <samplerate>48000</samplerate> in the encode module, with no resampling. Unfortunately the USB card dies every now and then when not connected through a powered USB hub. Too bad, because the USB hub looks ugly.
Any time a person takes its own life, anger is around with sadness. I’m angry too and I share Lawrence Lessig’s thoughts:
the question this government needs to answer is why it was so necessary that Aaron Swartz be labeled a “felon.”
This morning I was listening to a show in Italy a volunteer that talks to prisoners incarcerated into the horrible Italian jails. She mentioned a conversation with a man sentenced to six years for selling counterfeited CDs and DVDs on the streets. His words: “Six years! That’s too long”. We’re accepting that selling a song with no rights to do so is a crime worst than hitting a singer with a baseball bat. Our politicians should be ashamed.
My experience at the OpenStack Summit in San Diego has been really good. I have received lots of positive comments about this configuration, merging the Design Summit with the Conference. Despite the high amount of people it seems that things went well. I’m waiting to gather more details about the remote participation to the Design Summit with WebEx, I’ll report more about that soon.
I presented one session at the conference, lead two sessions on the Design Summit track and facilitated the meeting of the APEC group. Especially on the last day I missed the integration of the summit’s agenda from sched.org into my personal calendar applications (phone and desktop) because I ended up overbooking myself a couple of times. Thankfully Monty Taylor covered for me.
The ‘Community Dashboard‘ that I presented with zAgile was received with enthusiastic comments: the crowd cheered ‘ship it’ when I asked them what they thought of the demo. I’ve talked with Sanjiva and Andrew after the presentation, we should have an early beta out there by the end of the year.
More needs to be done in order to improve the community resources: IRC channels are not owned by the Foundation, some services depend by one person only (the main website and etherpad service, just to make an example). The forums need some love and probably we should have a Q&A system in place. During the sessions Atul Jha from India showed an askbot-powered system that he volunteered to run. In the next weeks I’ll help him go live with it. We discussed also the migration of the General mailing list out of Launchpad: unfortunately I have no news since my last update. The planet needs a better look, if nothing else. And the OpenStack blog needs a better content policy: some people in the room raised some concerns over the abuse of corporate posts on it.
During the discussion on how to track OpenStack’s adoption I was suggested to focus on users’ survey instead of proposing to add some ‘telephone home’ capability to OpenStack’s code (like Mozilla Foundation does with Firefox). I think this is a good idea and I’ll make sure this will become a project of the Foundation in the next months, once we’ll be fully staffed (we’re hiring).
Monty lead the session I proposed about an Integrated Identity System among all OpenStack tools. We went through the improvements we’re working on regarding the CLA workflow and its integration into Gerrit and the Foundation’s membership database. Todd Morey was in the session and we had more ideas on how to make things progress a bit faster (we agreed that hanging out on IRC is a prerequisite for make things happen faster). The future of Launchpad as the main ID system for OpenStack will be decided after Ubuntu Developer Summit: Thierry will spend time with Canonical’s folks there to understand if it will still be able to serve our purposes in the future.
My list of things to do has increased, as it’s expected after the Summit. Thanks everybody for joining.
This week was my first week as an employee of the OpenStack Foundation. I’m Technical Community Manager, still helping the OpenStack project succeed by helping the technical contributions. The difference is that my salary is now being paid by the newly formed OpenStack Foundation instead of Rackspace. Most of the people I worked with more closely at Rackspace are also at the OpenStack Foundation: Lauren, Mark, Jonathan and Thierry and we’re also hiring more people.
I have high hopes for the projects I will present at the Grizzly Summit: Achieving Visibility and Insight Across OpenStack Projects with Dashboards, Traceability, and Faceted Search, Integrated identity system for OpenStack and Tracking OpenStack adoption.