Updates from October, 2014 Toggle Comment Threads | Keyboard Shortcuts

  • stefano 8:21 am on October 7, 2014 Permalink | Reply  

    Next steps for ‘Hidden Influencers’ 

    With Paris only weeks away it’s time to announce that we have a time and place to meet people whose job is to decide what OpenStack means for their company. The OpenStack Foundation has offered a room to meet in Paris on Monday, November 3rd, in the afternoon: please add the meeting to your schedule.

    There have been discussions in the past weeks about the hidden influencers, there was a recent call by Randy Bias to identify people, functions, groups, anything who can own the product OpenStack. The time is ripe to get to know who directly control the engineers’ priorities and complete the circle of end users, operators, product owners, developers.

    Rob, Sean, Allison and I have the idea of creating a working group modelled after the Operators group: get people in the same room, some time mid-cycle and sync up on the development effort. It’s just an idea, mid-cycle is when the development roadmap gets more realistic, it’s more clear which blueprints are in good shape and which are less likely to make it into the release. We believe it would be valuable to have a moment when product decision makers can coordinate and get a chance to share their priorities.

    The room is available for 4 hours, but of course we can use for less time if needed. It makes sense to start hacking some of the topics in the next weeks, before Paris. Please join the mailing list and introduce yourself while the agenda will be drafted on the etherpad. For naming the group you can join the async brainstorming session (drop names at random and don’t judge yourself nor others: everything is fair game).

     

     
    • Debra 11:08 am on October 8, 2014 Permalink | Reply

      So glad you are organizing this group! Wish I was going to be at the Summit this time to participate in person.

    • Albert 8:30 am on October 23, 2014 Permalink | Reply

      Looking forward to the discussion Stef. See you in Paris.

  • stefano 3:40 pm on October 1, 2014 Permalink | Reply
    Tags: , , wiki   

    Improving the content of OpenStack wiki 

    The pages on wiki.openstack.org have been growing at a fast pace and it’s  time to give the wiki more shape: new contributors, end users and operators are having a hard time finding documentation since over time it spreads across many places. The wiki can have a role at directing readers where most appropriate. Luckily we have a team ready to help give the ~350 pages a more solid navigation and fixing content while at it:

    • Katherine Cranford (a trained taxonomist) volunteered to get through the wiki pages and propose a taxonomy for the wiki.
    • Shari Mahrdt, a recent hire by the Foundation, has volunteered a few hours per week to implement the taxonomy in the wiki pages, setup templates and write documentation to maintain the wiki.
    • I am overseeing the implementation and looking more carefully at content for contributors.

    We are keeping track of things to do on the etherpad: Action_Items_OpenStack_Wiki. Shari and I started implementing Katherine’s proposed taxonomy: it’s visible as a navigable tree on https://wiki.openstack.org/wiki/Category:Home.

    As an example of how the taxonomy works, let’s look at the tree of OpenStack Programs. One can think of Programs as teams of people using tools (code repository, bug tracker, etc) and coordinated processes to deliver one or more project to achieve a clearly stated objective. For  example, the Telemetry Program has a team of core reviewers responsible to drive development in code repositories for the Ceilometer project and the Ceilometer client, and each has pages for blueprints and specs, meeting notes and more. Programs contain projects, so the tree of categories under Programs will look like:
    • Programs
      • Telemetry
        • Ceilometer
          • Client
          • Blueprint
      • Block Storage
        • Cinder
          • Client
          • Blueprints
      • Compute
        • Nova
          • Client
          • Blueprints

    You can see this live on the wiki on https://wiki.openstack.org/wiki/Category:Programs. Fairly straightforward except that over the years some of the pages have started with describing a project and have now been repurposed to illustrate a Program. Look at Nova page for example: the name of the page is “Nova” but the title is OpenStack Compute. We’ll definitely have to shuffle content around.  For example, the Category:Programs page can be considered a duplicate of the wiki https://wiki.openstack.org/wiki/Programs page: since everything on mediawiki is a page, the Category pages can be edited and can be redirected to/from like all other pages. In this case, it would make sense to make high level content like Programs more of a dynamic page, like Category:Programs. The cool thing of this approach is that we can probably create new category page for new programs automatically when modifications to the programs.yaml are approved via jenkins.

    Adding a taxonomy and templates (more on this later) will help newcomers discover relevant content and find information more easily. While we implement the changes to the wiki we’ll also be reviewing the content on the pages, delete or mark as obsolete the old stuff and make things more readable for all. You can keep up with the progress by looking at RecentChanges.

    If you’d like to help us out or find out more please feel free to contact stefano@openstack.org and shari@openstack.org

     
  • stefano 3:19 pm on September 25, 2014 Permalink | Reply
    Tags: , contributions, ,   

    How is your OpenStack team organized? 

    I’ve been collecting a lot of good insights talking to directors and managers about how their companies are organized to contribute to OpenStack. For geographic reasons I have mostly gathered comments from people between San Francisco and Silicon Valley and I’d like to expand the research.

    I’m especially interested in learning about internal processes, system of incentives, things that impact performance evaluation for engineers contributing to OpenStack.

    To expand the research I’m asking the OpenStack community to fill in this brief survey or contact me directly (stefano openstack.org) for a quick interview.

     
  • stefano 5:03 pm on September 18, 2014 Permalink | Reply  

    Improving Documentation for new OpenStack Contributors 

    I think we’re starting to see too much friction and pain from contributors to OpenStack plugins, drivers, extensions: a lot is going on and joining the community feels like boarding a bullet train while it’s running at full speed. We’ve been doing a good job pulling people in the OpenStack community and educate them with our tools, processes especially for contributors to the Compute, Storage, Networking pieces of OpenStack; there are margins to improve life for “casual” contributors and newcomers in general.

    The main problem I think is that we’ve been treating all developers like they all have the same amount of dedication to the collaboration effort. We’ve been asking every person contributing to OpenStack to act the same way, whether they were fixing a small bug in Keystone or refactoring Neutron’s code: same gerrit review workflow, same process for blueprints/specs. We assumed always that anybody at any given time had the same level of knowledge of deadlines and recent changes to the processes. We are requiring people with radically different objectives and set of incentives to know and do the same things. I think this is starting to become too much of a problem for the community. We are already having conversations about this (split drivers, Neutron incubator, refocus summit, modify the legal requirements) and will continue discussing in Paris.

    The one thing that we should start improving immediately is the documentation for new contributors. New contributors have lots of resources available like the Welcome Guide, docs.openstack.org/developer, lots of wiki pages of varying degree of correctness and quality, tips and tricks on blog posts, upcoming video series, IRC support and Upstream Training. Based on what I hear from some, they struggle because these are in different places and owned by different people. A place where this content is shown in a more coherent way would be a great improvement.

    There are two efforts to improve various pieces of documentation: the Documentation program wants to redesign the Docs portal, and a new portal is being designed for developers of applications based on OpenStack. The Documentation program is not responsible for developers’ content on the Docs portal, it simply hosts the materials produced directly by the programs. While we wait for a redesign of the Documentation portal, I think we can start by fixing the wiki pages. With the help of a taxonomy produced by volunteer Katherine Cranford and Foundation’s staff Shari Mahrdt, we’ll give a bit more structure to the OpenStack wiki. Stay tuned for more details.

     

     
  • stefano 3:22 pm on August 4, 2014 Permalink | Reply
    Tags: , , contribute, , , open innovation,   

    How do companies do OpenStack? 

    I’m cruising Silicon Valley these days asking “how does your company do OpenStack?”  to collect best practices and notable mistakes from various leaders of OpenStack corporate community. I’m hoping to build a ‘how to’ manual to help managers build better dev teams, more effective at collaborating while shipping products to their customers. This is an effort that goes hand-in-hand with training new developers with Upstream Training and other initiatives aimed at sustaining OpenStack growth.

    I often have to explain to managers of OpenStack corporate members how and why to contribute to the project. The level of complexity reached has now exceeded the simple answers, like ‘fix simple bugs’ or ‘read the wiki pages’. For example, the participants to the Upstream Training in Atlanta had difficulties finding low-hanging-fruit bugs to work on: by the time they were ready to commit, someone else had already fixed it. Too much is going on at the same time, newcomers are confused at many different levels.

    While we will keep training developers through Upstream Training (and other initiatives), we’re going to start talking to managers too.  I’d like to know how development teams are organized, how they balance shipping products to customers with contributing upstream, what incentives are in place that help or prevent contributions to go upstream, how career paths are influenced by contributions to the open source collaboration, etc.

    If you’re managing a team of developers or devops, you’re a CTO, a HR executive or are managing in any way a team of engineers contributing code to OpenStack, I would like to talk to you: email me and vote for my talk proposal “How to maximize effectiveness of your developers contributing to OpenStack”.

     
    • miko matsumura 12:19 pm on August 5, 2014 Permalink | Reply

      I would like to talk with you about Hazelcast an Apache licensed open source In-Memory elastic data and processing grid. I am wondering if there might be a way for us to contribute/collaborate with open stack

      • stefano 9:11 am on August 6, 2014 Permalink | Reply

        Hello Miko, I’m sure there is a way to collaborate :) Hop on IRC freenode.net #openstack-community and we can have a chat. Or send me an email.

    • Jean-Daniel Bonnetot 8:34 am on August 12, 2014 Permalink | Reply

      “email me”
      Yes, but where? :)

      • stefano 8:48 am on August 12, 2014 Permalink | Reply

        I’m stefano at this domain or at openstack.org :)

  • stefano 11:17 am on July 14, 2014 Permalink | Reply
    Tags: blueprints, bugs, , , , operators, users   

    Only developers should file specifications and blueprints 

    If you try to solve a problem with the wrong tool you’re likely going to have a frustrating experience. OpenStack developers have been using the same workflow to plan and manage development of OpenStack software over time, they chose a set of tools appropriate for the project’s software development. Developers use blueprints define the roadmap for the various projects, the specifications attached to a blueprint are used to discuss the implementation details before code is submitted for review.

    These are the tools for the developers and this doesn’t mean that blueprints and specifications are the only way to interact with developers. Operators and users in general don’t need to dive in the details of how OpenStack developers organize their work and definitely should never be asked to use tools designed for and by developers. When I read ‘s post the place of a non-developer in the openstack community I immediately felt her pain: she was asked to use the wrong tool to solve a problem. I think this case is a major misunderstanding but the comments on the post signal that this is not an isolated case.

    The most common way for a non-developer to highlight a flaw in OpenStack is to file a new bug report. Bugs can be defects that need to be fixed  or they can be requests for enhancement. In this case, Dafna filed a bug report and interacting with the triager, they agreed that the reported bug was not a defect per se but more of a request for enhancement, a wishlist.

    In order to fix a defect or implement a wishlist item, developers need to file a blueprint (and a specification) before they can start writing code. If the person who filed the original bug report is also a developer then s/he can go ahead and continue carrying on the process, file a blueprint to solve a bug # and write specifications (when needed) to describe the details of the implementation. Users interested in the bug can chime in adding comments to the bug and to the specs, provide input to developers.

    The process above works in similar way when the person filing the bug is not a developer, like in Dafna’s case. The proper flow of bug #1323578 would have not required Dafna to file a blueprint and specs, but to have a developer do that. Users are required to interact closely with the developers to discuss the implementation details, and that’s where the new specifications process helps. Gerrit may not have the best UI but it’s definitely better than holding discussions on a mailing list. Adding comments and holding conversations with the developer assigned to resolve the issue, on the bug report itself is also a valid option.

    While I think that as a community we have plenty of ways to improve how we include users and operators in our conversations, in this particular case, I think frustration came from being pulled in the wrong place from the beginning.

    PS: To be super-clear, in this post I’m using the terms developer and operator to describe a role, not a person. One person can be at the same time a developer and an operator, and act at times as a developer writing blueprints, submitting code and as an operator filing bug, giving comments to specifications.

     

     
    • Stephen Balukoff 2:38 pm on July 18, 2014 Permalink | Reply

      So, while an in-depth reading of your article here makes it clear that your comments are meant for developers– that you are saying developers shouldn’t push non-developers to use tools only meant for developers, both the title and several parts of your article can be interpreted to say “stay out of my kitchen” to operators and users. I would suggest altering the title to say something less exclusionary to these other 2/3rds of the OpenStack community. (Something like “Developers should not send non-developers to launchpad or gerrit.”)

      Having said this, I really feel like your article somewhat misses the point of Dafna’s blog post: In my reading she was frustrated because she was not able to contribute to the discussion, and her opinion was considered value-less unless she was willing to go through the (for her, arduous) task of learning two new systems which are neither user-friendly, nor particularly suited to the kind of discussion she wanted to have.

      Your post reads to me somewhat like “there are tools non-developers should be using to communicate with developers, and these are not the same tools developers will use to communicate with each other.” To me this is akin to the cooks getting angry at the waiters and customers for coming into their kitchen whey they have problems or suggestions. That’s all fine and dandy– so long as the cooks are actually providing good tools the waiters and customers can use to communicate to them, and using those tools themselves. I read Dafna’s point being that this obviously wasn’t happening, and instead what was effectively red-tape was being used to make her go away.

      If the OpenStack community wants to change this reputation for being exclusionary to operators and users, then this is exactly this sort of thing that needs to be worked on! Unfortunately, I don’t think the way you worded your article is helping much here. :/

      Also one point in particular in your article I feel I need to take issue with: “Gerrit may not have the best UI but it’s definitely better than holding discussions on a mailing list.” Actually, the mailing list is the perfect place to discuss ideas among a larger crowd of people when a major new idea or significant change is being contemplated. You’ve already said that Gerrit shouldn’t be used by non-developers, so if you limit these kinds of discussions to Gerrit only, then you are thereby ignoring any input that 2/3rds of the OpenStack community might have on the subject. This can lead to solutions which don’t actually meet operator or user needs at all, can lead to really poor design which then is a pain in the patootie to correct (given OpenStacks backward-compatibility policies), and certainly leads to OpenStack not being as good as it could be in the same amount of time.

      Also, I don’t think any search engine is indexing Gerrit (from what I can tell), and it’s far too easy for comments on early revisions of a patch to get lost forever when later revisions of the patch are uploaded for review (since Gerrit doesn’t carry comments forward– this needs to be done manually by attentive reviewers).

      That’s not to say Gerrit isn’t a good tool. In fact, as consensus among the community is neared on a mailing list discussion, it’s important for the details of that consensus to be translated into blueprints and specifications. Gerrit is actually probably the best tool we could be using when it comes to things mostly concerning developers (comments on specific lines of code, or specific details within a specification). But it is wholly the wrong place to be having design or feature-improvement discussions, especially with non-developer contributors like operators and users.

      • stefano 10:19 pm on July 19, 2014 Permalink | Reply

        Hi Stephen, I appreciate your comment. In fact, I think you have caught my intention quite well. I worded the title in such way exactly to get this sort of comments: I wanted the article to be challenged.

        I think that the *tools* that Dafna was told to use confused her and that’s wrong. She was put in a position to fail and that is not fair to any user. Her thoughts and comments could have stayed on the bug report or migrated to a mailing list, as you suggested, and eventually become a blueprint+specs lead by a developer.

        I think that OpenStack has done an incredible amount of work to include operators and users in the development loop. In the past 12 months I think we’ve visibly included operators into the decision making process, starting from the user survey to the work on personas and the UX team, the Operators Summits, the working groups within the User Committee and there is even more coming.

        As for the debate about Gerrit… Let me just hope that storyboard accelerates and becomes the tool we can use to discuss specifications. I voiced my concerns publicly against the specs repositories but at the same time, I have a hard time coming up with a radically better alternative.

    • Maish Saidel-Keesing 7:25 am on July 20, 2014 Permalink | Reply

      Hi Stefano.

      I have to agree with Stephen here. I too (not being a developer) find it very difficult to navigate the multiple tools and methods used by the OpenStack project to drive directions, changes, fixes and improvements.

      Let me count.

      There is Gerrit.
      There is Launchpad
      There is IRC
      There is the mailing list.
      There is Etherpad.
      There is the Wiki.
      There is the documentation.
      Oh yeah – there is even a support helpdesk (whatever that is used for)

      Perhaps I missed something here – but those are the ones that come to mind.

      Yes they all have their use cases, and one does not replace the other – nor should it, but the amount of information that flies across all of these – is substancial.

      And as a user/operator – it took me a decent amount of time to understand how to submit a patch – something as simple as an error in a config file.

      I still have problems getting my head around how the whole patch and voting system works, who has to approve, when, how, what is the difference between a +1, -1, -2, +2. What are the intricacies and clicks that need to be handled in order to get a patch submitted or re-submitted or changed. What happens when one core reviewer disagrees with another on my patch?

      What will happen if you are new – will someone work with you to get the change you wanted to submit (and are doing so – only because you want to help out) and approved?

      I have the feeling (and evidently I am no the only one) that the developers community think this is second nature to them – it probably is – and that those who do not know the language – the talk – the culture – are outsiders – and that they do not have the time / patience / incentive to teach them – hold their hand or walk them through it.

      I can personally tell you an experience of where I asked a question in IRC about features in LBaaS and if they existed or were planned. The question that was presented to me – so why don’t you start writing the code to implement these changes? My answer was I am not a developer – so my contribution on this part would not be helpful, the answer that I was given thereafter was “Forget it – we have enough architects and conceptual designers – we need people to make it happen”

      Developers are a different breed, for better or for worse. The feeling Operators/users have is that we are not involved – and therefore should stay out the way of the way the “magic happens”

      I for one would love to see that change. There is room for improvement in every single project – and developers could benefit immensely from the input of those who are actually deploying and using the product, in the real world.

      • stefano 7:11 pm on July 20, 2014 Permalink | Reply

        Thanks Maish. Let me make it straight: I, too agree with Stephen that the tools and processes we use are complex to navigate and such complexity may discourage *casual* contributors. I think this topic deserves a longer conversation than just a reply to a comment, I’ll try to follow up in the next days.

        I substantially disagree with the idea that OpenStack developers don’t want to listen to users: the truth is that things are a lot better in OpenStack than in most other software development projects and they also keep improving. Feedback from users and operators is constantly being fed back into the development process, via the user survey, the operator summits, the design sessions, the conversations on the mailing lists, the comments on bug reports, specs and blueprints. Sure, a lot more can be done, but I don’t accept general criticisms that OpenStack developers don’t listen to users.

        I would not generalize from one single comment on an IRC channel, you should consider a random conversation on IRC exactly like a chat around a pint at a pub.

        • Maish Saidel-Keesing 11:34 pm on July 20, 2014 Permalink | Reply

          Hi Stefano – If it came over that all developers are not interested in hearing what we have to say – then that was not my intention. Really not. I just wanted to re-iterate the comments and feelings that were voiced before. Of course I see improvement – exactly as you said – with the Operators conference – and the joint sessions- but there is still quite a way to go. I do still feel that the developers could make more of an effort to “embrace” the newbies – it will be more beneficial to us all in the long run term.

          I hope that the Foundation will continue to invest time, money and effort into this – it will make the product better for us all.

          I would be happy to assist in any way I can to contribute to the conversation.

  • stefano 9:16 am on July 7, 2014 Permalink | Reply
    Tags:   

    Useful tips for multicultural communities 

    Learning to Speak Up When You’re from a Culture of Deference http://zite.to/1sn7YjI

     
  • stefano 3:49 pm on June 20, 2014 Permalink | Reply  

    Engage in technical discussions keeping in mind the OpenStack promise 

    When OpenStack launched, it made four clear promises to its community: Open Source, Open Design, Open Development and Open Community. Open Source for OpenStack founders has been clearly explained:

    We are committed to creating truly open source software that is usable and scalable. Truly open source software is not feature or performance limited and is not crippled. There will be no “Enterprise Edition”.

    I find it worth restating that promise because I believe the four ‘Open’ promises that largely contributed in boosting OpenStack growth. There is a conversation about what is Cinder itself and what’s the role of its drivers. It’s a highly technical debate and a very important one, where I think this promise needs to be reminded: There will be no “Enterprise Edition”.

    I won’t enter into technical details because it’s not my role.  John Griffith and Ken Hui have a lot of interesting things to say about this and I suggest you to read their posts.  My personal opinion based on what I heard in Atlanta and read afterwards is that one approach puts Cinder in danger of becoming only useful if you pay a license to someone offering “value added” (better, freedom removed) pieces of hardware and software. That would be very shortsighted and that’s not what OpenStack is.  I think any leader of OpenStack has to be very careful when implementing changes that may put OpenStack’s promises in danger. The promises we made as a community are the first things to keep in mind during any conversation, technical ones included.

     
  • stefano 12:54 pm on June 3, 2014 Permalink | Reply
    Tags: directory, , openstack. community, people   

    Towards a Directory for all OpenStack People 

    One of the challenges at large projects like OpenStack is to know people, understand who is doing what, how to reach out to them. There are a lot of different classes of people involved in OpenStack: developers, users, speakers and participants to conferences, voters for OpenStack Foundation’s official governing bodies, maintainers of company’s presence on Marketplace and more. At the moment we’re using different tools to collect this information and carefully provide access to it. Most of the data is in the database behind OpenStack.org website: that’s where users register to respond to User Survey, Foundation Individual Members create their identity to be later verified by Gerrit, company administrators manage the content for Marketplace. Coupled with Launchpad OpenIDs, the .org database contains the vast majority of identities of people involved in OpenStack. Since we’ll soon be able our own OpenID provider, we should start building a directory for all OpenStack People based on the database we have now.

    We’re finalizing the design of the new openid provider portal so that it will be the ultimate place to hold the profiles of all people involved in openstack, from users, developers, people editing copy on the marketplace, etc. The basic plan is to merge pages like Foundation’s member profile with the OpenID profile and get ready to add more data to those page, make them more interesting and ‘alive’ adding things like achievements, badges and more.

    We have also the chance of reducing the pain that it is now to manage Individual and Corporate CLAs. While the discussion to switch to DCO goes on, I think we should simplify what we have now since it’s very easy to do. Currently new developers need to create an account on openstack.org, go to launchpad.net and create an account there, then go to review.openstack.org and create an account there and click-sign an Individual CLA making sure they use the same email address in gerrit as in openstack.org. Gerrit then queries the .org database and if it finds an email match, gerrit stores the fact ‘user signed the icla’ in its database. It’s not over for developers who work for corporations: those should also have their managers approve/list them in the Corporate CLA (if/when needed) which is a totally different thing, managed ‘manually’ with an echosign form.

    We want to change all of this, since we already have most of the UI and infrastructure in place to make things simpler. There is a blueprint filed for this effort and specs, also check the discussion on the infra mailing list to gather feedback and move to implementation.

     
  • stefano 10:04 am on May 28, 2014 Permalink | Reply
    Tags: , , velocity   

    Treat Open Source Like a Startup 

    This experience by Velocity.JS confirms the sense I had in OpenStack:

    I treated Velocity like I treated my businesses: First, there’s development. That’s 10%. Then, there’s marketing. That’s 90%.

    Treat Open Source Like a Startup ✩ Mozilla Hacks – the Web developer blog.

     
c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel