Governance for the Forks

We at logz.io are 100% behind the project in terms of our engineers and I agree there will be individuals such as myself and others here contributing and we hope to be part of the team making decisions and helping run the maintainership of the codebase. I do not know when this will happen, but for now, I want a good stable distribution and one we can adopt.

From our business, we have made a commitment to switch to these distributions in the coming months for our services. I have been in touch with over 80 individuals who are going to be adopting and contributing. I’m sending them this reply today:

Thanks for getting in touch with us at logz.io when we announced that we’ll be working towards a new project to ensure that ElasticSearch and Kibana remain Apache 2.0 licensed open source projects. This initiative has been led by AWS, but many organizations have been working with them leading up to the launch of OpenSearch this week . If you have a look at the GitHub repos you will find the community picking up. Here at logz.io we have put forward several proposals for features and capabilities we will contribute to the new community. I hope the governance and organization for the maintainers come together in the next 3-6 months, for now, AWS is leading the work, but they are open to all collaborations and users. The team has been easy to work with and accepting of our contributions, so the signs are encouraging.

I hope to see you on the discussions on GitHub or you can participate in the community through the newly launched website for OpenSearch

Please contact me with any questions or opinions.

Thank you!

2 Likes

This actually those are some great questions and many answers are not yet defined. I’d love to know opinions about how to define ‘community’ - it’s a tricky thing to answer. And you’re right - we’ve got to define these things before we even think about a foundation. The bottom line is that we’re open to opinions and feedback. From there I think we (interested folks) can work together to figure it out.

(I can answer definitively on one: there is no steering group currently.)

(Also welcome!)

I wouldn’t pick apart the wording of the golem article. To my knowledge, no one at AWS had any role in that piece. There is quite a few of non-English news organizations that take AWS blog posts and translate/paraphrase them into other languages (to varying levels of accuracy from what I can tell).

The AWS post (which I helped draft) uses the word ‘support’ which has a different meaning entirely than ‘cooperate’ in English. I won’t get into the nuances of German/English translation as I’m 100% not qualified though :smiley:

The companies mentioned in the AWS post are those who connected with the team at AWS and voiced support in a quote. If you want to see those contributing to the software, I’d look at the github pulse for OpenSearch or OpenSearch Dashboards.

That’s right. A partners page will come along soon to opensearch.org, it’s just not done.

If anyone wants to be included in the page, let me know.

Also the website is open source, if anyone wants to help out (I’m doing it mostly myself and it’s pretty lonely, tbh).

Yeah - I think I agree you here: contributors = individuals. Having contributors from more organization is great.

And for clarity since this thread is mentioning both terms: partners != contributors. It would totally be fine for a partner to provide OpenSearch-as-a-service and/or services for OpenSearch and never contribute a single line of code.

sorry, i missed the AWS blog post. i lost the overview between the opensearch blog post, the opendistro blog post, the forum posts, news articles in my news feed, etc.

it’s a lot clearer now, thanks.

and i agree that it should be about individuals. thanks to the companies which allow their employees to contribute here during their working hours and thanks to anyone contributing in general :slight_smile:

1 Like

At Aiven we are also entirely behind the OpenSearch project and have been public and vocal about that https://aiven.io/blog/aiven-joins-the-opensearch-community, happy to be listed as partners (our individuals intend to be contributors too but as you say that’s a different thing).

I’d recommend that the best way to herd help for the website project is to open some issues for things that need doing - I know I’d be happy to lend a hand, and I expect others would too.

2 Likes

@lornajane That’s a great idea! I’ll add a few issues for the website shortly.

Hopefully we can get the partners page to be easy enough to be self-service (like uploading a yaml file or something).

since the website is already on github it’s probably easy enough to just say that people can create a PR to add themselves to it (once the page itself has been created and a structure has been given for it)?

@ralph Yeah - that’s the idea - just do a PR. Although I think accepting the PR would need manual intervention of course (preventing spamming, etc.)

Actually, I’m going to start a new thread for the website - I don’t want to detract from the current topic.

FWIW, this is a goal of Amazon as well (to have OpenSearch be a healthy & diverse community that is not just maintained by Amazon). There seems to be some loose consensus (in this thread) that a small steering committee to propose some specific structure around governance would be a good next step.

I say small here, not to limit outside interest, but to keep it manageable.

In my mind, the key questions are (thanks to janhoy):

When and how does someone get commit access to the OpenSearch repo?
How are community decisions made?
Who in the community gets to make those decisions?

1 Like

In the interest of not reinventing the wheel, would it make sense to shamelessly borrow/copy e.g. ASF’s structure/rules as the starting point and then make adjustments only where really necessary? These things are not necessarily set in stone forever. Plus, they ought to be needed only for a little while.

Otis

1 Like

From my personal perspective (again, I am not speaking on behalf of my employer), I think the structure and rules used by ASF is intended to help the community gathered to build software in ways that are in keeping with the philosophy of the foundation as a whole (The Apache Way). I personally like this philosophy, but it may not be 100% shared with the community that is forming here.

In particular, there are a number of rules and policies that are intended to reinforce independence for the community of developers that come together to the ASF to build and release software as a public good. For example the ASF does not allow corporations to participate directly in Apache project management or other governance activities at the ASF; only individuals.

I think there is risk in adopting rules that were carefully crafted to work within, and promote, a very specific philosophical and social framework. Expectations may be set implicitly that don’t hold long term. And when expectations are not met, even implicit expectations, trust can be lost.

If we say “contributors = individuals” (as practiced at the ASF) and then adopt (even temporarily) ASF policies and rules that are designed to only invite individuals to participate in project governance, some may assume that the overall behavioral norms should be like an ASF project. And I would predict that the for-profit partners (if we adopt that term) in this effort are going to want to participate, and be part of some decision-making, that is excluded by the Apache Way.

I think you and I might share some core philosophy :smile:. Let me try to expand so you can agree or disagree.

I believe that initiatives that mindfully empower individuals to autonomously make high quality decisions are more functional and productive in practice. I am a big believer in self-organized, commons-based peer production of public goods (like open source software). This means that the individuals that are doing the work largely govern themselves, and find the best working methods organically over time.

This approach can feel chaotic, and there can be a natural hesitancy to lean into that. Instead, some want the safety of knowing who has power to make top-down, unilateral decisions. That is a practice that we have come to expect in today’s world of open source when companies, and for-profit corporate interests, are involved. But centralized decision-making is not part of the practices that have emerged around peer-production in the meshed networked world from which Linux and Open Source emerged—even when there is a “benevolent dictator” who is leading an effort. See Coase’s Penguin, or, Linux and The Nature of the Firm.

I think the majority of the feedback about governance, and a move to a foundation, seems to be about folks that work for companies getting comfortable investing their limited and valuable resources (people). I think this is an understandable concern when organizing production given the nature of the firm. It can be challenging to work through the “impedience mismatch” of organizing a team within the structure of a firm (i.e., hiring employees and assigning work) that then participates in virtual organization in a peer-production model. But I also might be reading something into posts like @dawnfoster’s that is not really there.

Personally, I believe that peer production of software is a powerful trend. If you can successfully set the conditions for peer production, it can be a significant tailwind for the effort, with benefits for everyone. But there are also corporate concerns having to do with bringing goods and services to market (e.g., trademark policy and enforcement), and handling operational aspects that are “downstream” of software production. There are matters that involve handling information that cannot responsibly be made public (embargoed security vulnerability coordination, legal disputes, code of conduct violation, and so on). Given the stakes involved, I imagine there will be reluctance to delegate the responsibility for these matters to others (or a committee) at present.

So, I think the thing to do may be to separate these concerns the best you can, so you can embrace as many of the benefits of peer production as possible.

I would like to participate in a discussion about this, but it’s unlikely that I would be able participate in a bootstrap committee if I were to be invited to be a part of it due to my other obligations. And I think it would be good to recognize the information availability gradients that can form when you organize a group to have a discussion in a forum that is not an open forum.

1 Like

At VMware, we have employees who contribute to hundreds of open source projects as part of their day jobs, so participating in a peer-production model is something we do all the time, and it’s something we encourage our employees to do. However, this project isn’t a peer-production model. So far, it’s fully controlled by Amazon, and we’re a lot more careful about how we participate in projects that are controlled by a single company. We’ve been burned by these single company open source projects in the past, so I’m here to get a better sense for when / if / how this project becomes more like a peer-production model before we start using / contributing to it.

3 Likes

We’ve created some governance templates for CNCF projects that might be easier to adapt to this project. This one should be easy to adapt: project-template/GOVERNANCE-maintainer.md at main · cncf/project-template · GitHub

I’d prefer this one, but I’m not sure this project is ready to accept something like this: project-template/GOVERNANCE-elections.md at main · cncf/project-template · GitHub

Another possibility might be the subproject template if we wanted to consider OpenSearch, OpenSearch Dashboards, Plugins, and other components as sub-projects: project-template/GOVERNANCE-subprojects.md at main · cncf/project-template · GitHub

This draft Contributor Ladder template might also be useful: sig-contributor-strategy/contributor-ladder.md at master · cncf/sig-contributor-strategy · GitHub

I hope some of this helps.

3 Likes

And that’s a good thing IMHO. As soon as you have “for-profit partners” whatever benefits their business is prioritized over benefits to the project itself.

Agree.

Brrrr. That is safety only for those who are at the top making the decision. For everyone else it’s anxiety that the top will press some form of big red button one day because that makes more business sense for them.

Otis

I know I talk a lot about companies, but I agree wholeheartedly with this. I have a strong preference for leadership positions in projects to be held by individuals who keep those positions regardless of employer. I’ve seen too many issues with leadership seats allocated to companies who swap their people in and out like cogs, which is hugely disruptive for the project.

What I would like to see are individuals in a variety of roles, including leadership roles, who work for a mix of different employers. I would like to see those roles defined with clear criteria for moving into those roles with that criteria being applied equally to individuals working at Amazon and those of us who are working at other employers.

I brought up this exact thing yesterday during the community meetup/webinar thing, but the answer was super brief. There is a lot of talk about “community”, but you can’t have a true community if that “community” consists only of people who contribute via PRs. That’s exactly what Elasticsearch and Kibana had. You have to have people who own the project and make decisions together. And not because they are just puppets of their employers, but because they put the interests of the project and its various users first. Remember years ago, when there was a version of Kibana that people were contributing their PRs to and then one day the message - from the top/center - came that there is a change of direction, there is a new Kibana that had been worked on, that the old Kibana and all the old PRs for the then current Kibana and all the time and efforts that people put into those PRs will be going to trash? That’s what happens when business priorities are put first. You can’t really have that in a truly OSS project. Maybe that is why ASF pushes “community over code” so much. Yes, the decisions will take longer, things will be hairier, etc., but that is exactly how nature works.

Otis

4 Likes

Yes, we all have. That is why we are here. I think it may be helpful to acknowledge that has been the direct past experience of the majority of the people who are interested in this effort. It should inform the kind of community we want to cultivate now, and in the future.

It is my goal to try to shine a light on the things that matter the most in achieving a model that is more like peer-production, rather than a single-company producer that aggregates unequal rights (e.g., via a CLA), and just happens to release software under an open source license that works for all stakeholders… today. But no guarantees about the future.

Rather than waiting and seeing, why not participate in a way that assumes we will find our way with working methods, governance, and so on?

(speaking only for myself)

This is my personal position as well.

Really, for any well functioning team, whether it is a team of people working in the structure of a company, or a virtual team of peers collaborating on the Internet, you need some continuity and understanding of roles. You need an understanding of how performance is evaluated, and how additional responsibilities are taken on as people grow. You also need to enable people to change their role, if they desire something different. And an ability to quickly fill a role in the case of a vacancy.

When an individual has a role that is independent of their employer, it can increase mobility for employees. See Exit and Voice in Free and Open Source Software Licensing: Moderating the Rein over Software Users, PDF page 3. I think that is a good thing, and empowers individuals.

I’ve also seen troubling dynamics when there are more strict rules about employer affiliation limits. I’ve seen cases where people are asked to step down from their role because of who they work for, and that troubles me. I understand why some communities adopt these kind of rules. But I would rather that some human judgement could be applied to determine what’s best for both the individual, and the community/project.

That is what I think everyone is seeking. But it is going to take time to get there. I think there will be some challenges based on the amount of investment that AWS is planning to put into the effort, and a cultural desire to move fast. But we know you can’t move fast without building a the right team, and team dynamics. I think that building a virtual team in the open, spanning affiliations and backgrounds, can take considerable time and effort. But I’ve seen the benefits that can come from it.

I think that these are helpful. And also your interest and engagement is helpful, and very welcome. I think you’re one of the most knowledgeable people in the community on the subject of how communities and companies interplay, and how company involvement can influence community health. I’d like to take some time to read your dissertation one day.

When I get philosophical and cite references in my posts it’s not my intent to try to explain any of this to you, or to others. Writing posts helps me lineralize my random thoughts on a subject. When I’m thinking, I’ll often go back to things that I’ve written and read in the past that influenced a thought so I can reflect on them. I’m including those references for future me, and hope they might be helpful for others as well.

–msw

3 Likes