We at logz.io are 100% behind the project in terms of our engineers and I agree there will be individuals such as myself and others here contributing and we hope to be part of the team making decisions and helping run the maintainership of the codebase. I do not know when this will happen, but for now, I want a good stable distribution and one we can adopt.
From our business, we have made a commitment to switch to these distributions in the coming months for our services. I have been in touch with over 80 individuals who are going to be adopting and contributing. I’m sending them this reply today:
Thanks for getting in touch with us atlogz.iowhen we announced that we’ll be working towards a new project to ensure that ElasticSearch and Kibana remain Apache 2.0 licensed open source projects. This initiative has been led by AWS, but many organizations have been working with themleading up to the launch of OpenSearch this week. If you have a look at theGitHub reposyou will find the community picking up. Here atlogz.iowe have put forward several proposals for features and capabilities we will contribute to the new community. I hope the governance and organization for the maintainers come together in the next 3-6 months, for now, AWS is leading the work, but they are open to all collaborations and users. The team has been easy to work with and accepting of our contributions, so the signs are encouraging.
I hope to see you on the discussions on GitHub or you can participate in the community through the newly launched website forOpenSearch
This actually those are some great questions and many answers are not yet defined. I’d love to know opinions about how to define ‘community’ - it’s a tricky thing to answer. And you’re right - we’ve got to define these things before we even think about a foundation. The bottom line is that we’re open to opinions and feedback. From there I think we (interested folks) can work together to figure it out.
(I can answer definitively on one: there is no steering group currently.)
I wouldn’t pick apart the wording of the golem article. To my knowledge, no one at AWS had any role in that piece. There is quite a few of non-English news organizations that take AWS blog posts and translate/paraphrase them into other languages (to varying levels of accuracy from what I can tell).
The AWS post (which I helped draft) uses the word ‘support’ which has a different meaning entirely than ‘cooperate’ in English. I won’t get into the nuances of German/English translation as I’m 100% not qualified though
The companies mentioned in the AWS post are those who connected with the team at AWS and voiced support in a quote. If you want to see those contributing to the software, I’d look at the github pulse for OpenSearch or OpenSearch Dashboards.
Yeah - I think I agree you here: contributors = individuals. Having contributors from more organization is great.
And for clarity since this thread is mentioning both terms: partners != contributors. It would totally be fine for a partner to provide OpenSearch-as-a-service and/or services for OpenSearch and never contribute a single line of code.
since the website is already on github it’s probably easy enough to just say that people can create a PR to add themselves to it (once the page itself has been created and a structure has been given for it)?
FWIW, this is a goal of Amazon as well (to have OpenSearch be a healthy & diverse community that is not just maintained by Amazon). There seems to be some loose consensus (in this thread) that a small steering committee to propose some specific structure around governance would be a good next step.
I say small here, not to limit outside interest, but to keep it manageable.
In my mind, the key questions are (thanks to janhoy):
When and how does someone get commit access to the OpenSearch repo?
How are community decisions made?
Who in the community gets to make those decisions?
In the interest of not reinventing the wheel, would it make sense to shamelessly borrow/copy e.g. ASF’s structure/rules as the starting point and then make adjustments only where really necessary? These things are not necessarily set in stone forever. Plus, they ought to be needed only for a little while.
From my personal perspective (again, I am not speaking on behalf of my employer), I think the structure and rules used by ASF is intended to help the community gathered to build software in ways that are in keeping with the philosophy of the foundation as a whole (The Apache Way). I personally like this philosophy, but it may not be 100% shared with the community that is forming here.
I think there is risk in adopting rules that were carefully crafted to work within, and promote, a very specific philosophical and social framework. Expectations may be set implicitly that don’t hold long term. And when expectations are not met, even implicit expectations, trust can be lost.
If we say “contributors = individuals” (as practiced at the ASF) and then adopt (even temporarily) ASF policies and rules that are designed to only invite individuals to participate in project governance, some may assume that the overall behavioral norms should be like an ASF project. And I would predict that the for-profit partners (if we adopt that term) in this effort are going to want to participate, and be part of some decision-making, that is excluded by the Apache Way.
I think you and I might share some core philosophy . Let me try to expand so you can agree or disagree.
I believe that initiatives that mindfully empower individuals to autonomously make high quality decisions are more functional and productive in practice. I am a big believer in self-organized, commons-based peer production of public goods (like open source software). This means that the individuals that are doing the work largely govern themselves, and find the best working methods organically over time.
This approach can feel chaotic, and there can be a natural hesitancy to lean into that. Instead, some want the safety of knowing who has power to make top-down, unilateral decisions. That is a practice that we have come to expect in today’s world of open source when companies, and for-profit corporate interests, are involved. But centralized decision-making is not part of the practices that have emerged around peer-production in the meshed networked world from which Linux and Open Source emerged—even when there is a “benevolent dictator” who is leading an effort. See Coase’s Penguin, or, Linux and The Nature of the Firm.
I think the majority of the feedback about governance, and a move to a foundation, seems to be about folks that work for companies getting comfortable investing their limited and valuable resources (people). I think this is an understandable concern when organizing production given the nature of the firm. It can be challenging to work through the “impedience mismatch” of organizing a team within the structure of a firm (i.e., hiring employees and assigning work) that then participates in virtual organization in a peer-production model. But I also might be reading something into posts like @dawnfoster’s that is not really there.
Personally, I believe that peer production of software is a powerful trend. If you can successfully set the conditions for peer production, it can be a significant tailwind for the effort, with benefits for everyone. But there are also corporate concerns having to do with bringing goods and services to market (e.g., trademark policy and enforcement), and handling operational aspects that are “downstream” of software production. There are matters that involve handling information that cannot responsibly be made public (embargoed security vulnerability coordination, legal disputes, code of conduct violation, and so on). Given the stakes involved, I imagine there will be reluctance to delegate the responsibility for these matters to others (or a committee) at present.
So, I think the thing to do may be to separate these concerns the best you can, so you can embrace as many of the benefits of peer production as possible.
I would like to participate in a discussion about this, but it’s unlikely that I would be able participate in a bootstrap committee if I were to be invited to be a part of it due to my other obligations. And I think it would be good to recognize the information availability gradients that can form when you organize a group to have a discussion in a forum that is not an open forum.
At VMware, we have employees who contribute to hundreds of open source projects as part of their day jobs, so participating in a peer-production model is something we do all the time, and it’s something we encourage our employees to do. However, this project isn’t a peer-production model. So far, it’s fully controlled by Amazon, and we’re a lot more careful about how we participate in projects that are controlled by a single company. We’ve been burned by these single company open source projects in the past, so I’m here to get a better sense for when / if / how this project becomes more like a peer-production model before we start using / contributing to it.
And that’s a good thing IMHO. As soon as you have “for-profit partners” whatever benefits their business is prioritized over benefits to the project itself.
Brrrr. That is safety only for those who are at the top making the decision. For everyone else it’s anxiety that the top will press some form of big red button one day because that makes more business sense for them.
I know I talk a lot about companies, but I agree wholeheartedly with this. I have a strong preference for leadership positions in projects to be held by individuals who keep those positions regardless of employer. I’ve seen too many issues with leadership seats allocated to companies who swap their people in and out like cogs, which is hugely disruptive for the project.
What I would like to see are individuals in a variety of roles, including leadership roles, who work for a mix of different employers. I would like to see those roles defined with clear criteria for moving into those roles with that criteria being applied equally to individuals working at Amazon and those of us who are working at other employers.
I brought up this exact thing yesterday during the community meetup/webinar thing, but the answer was super brief. There is a lot of talk about “community”, but you can’t have a true community if that “community” consists only of people who contribute via PRs. That’s exactly what Elasticsearch and Kibana had. You have to have people who own the project and make decisions together. And not because they are just puppets of their employers, but because they put the interests of the project and its various users first. Remember years ago, when there was a version of Kibana that people were contributing their PRs to and then one day the message - from the top/center - came that there is a change of direction, there is a new Kibana that had been worked on, that the old Kibana and all the old PRs for the then current Kibana and all the time and efforts that people put into those PRs will be going to trash? That’s what happens when business priorities are put first. You can’t really have that in a truly OSS project. Maybe that is why ASF pushes “community over code” so much. Yes, the decisions will take longer, things will be hairier, etc., but that is exactly how nature works.
Yes, we all have. That is why we are here. I think it may be helpful to acknowledge that has been the direct past experience of the majority of the people who are interested in this effort. It should inform the kind of community we want to cultivate now, and in the future.
It is my goal to try to shine a light on the things that matter the most in achieving a model that is more like peer-production, rather than a single-company producer that aggregates unequal rights (e.g., via a CLA), and just happens to release software under an open source license that works for all stakeholders… today. But no guarantees about the future.
Rather than waiting and seeing, why not participate in a way that assumes we will find our way with working methods, governance, and so on?
Really, for any well functioning team, whether it is a team of people working in the structure of a company, or a virtual team of peers collaborating on the Internet, you need some continuity and understanding of roles. You need an understanding of how performance is evaluated, and how additional responsibilities are taken on as people grow. You also need to enable people to change their role, if they desire something different. And an ability to quickly fill a role in the case of a vacancy.
I’ve also seen troubling dynamics when there are more strict rules about employer affiliation limits. I’ve seen cases where people are asked to step down from their role because of who they work for, and that troubles me. I understand why some communities adopt these kind of rules. But I would rather that some human judgement could be applied to determine what’s best for both the individual, and the community/project.
That is what I think everyone is seeking. But it is going to take time to get there. I think there will be some challenges based on the amount of investment that AWS is planning to put into the effort, and a cultural desire to move fast. But we know you can’t move fast without building a the right team, and team dynamics. I think that building a virtual team in the open, spanning affiliations and backgrounds, can take considerable time and effort. But I’ve seen the benefits that can come from it.
I think that these are helpful. And also your interest and engagement is helpful, and very welcome. I think you’re one of the most knowledgeable people in the community on the subject of how communities and companies interplay, and how company involvement can influence community health. I’d like to take some time to read your dissertation one day.
When I get philosophical and cite references in my posts it’s not my intent to try to explain any of this to you, or to others. Writing posts helps me lineralize my random thoughts on a subject. When I’m thinking, I’ll often go back to things that I’ve written and read in the past that influenced a thought so I can reflect on them. I’m including those references for future me, and hope they might be helpful for others as well.