Guiding principles for the forks of Elasticsearch and Kibana

Thanks, Dawn!

100% agree with these comments. The “community” so far is AWS and people who decided to follow their lead. AWS must change the discussion in the community and have shared ownership. Launching this project without clarifying the governance is a problem.

I’ve been pushing for shared docs, open zoom meetings, recording the sessions, and other standard practices the OSS community uses. I have been pushing this and trying to use open docs with the AWS team for the last 2 months. This is not been adopted or accepted at all.

@searchymcsearchface how can we make progress on these issues in an open manner?

2 Likes

Hey Dawn

Thanks for surfacing this. I do see the principles and governance to be separate things - when I said sidetracked it wasn’t to lessen any discussion of the governance, just two things being perhaps parallel.

On “we”… You wouldn’t believe how much the team agonized on this particular word. The team tried constructing these without the word ‘we’ and it became hard to understand and more lengthy, ultimately ‘we’ was selected as the most clear, concise word although, admittedly, open to more interpretation that perhaps the team intended.

In this case “we” should be interpreted more broadly and applicable to the entire community of contributors (Amazon folks included). I think the most important (but easiest to ignore) line is “When we are successful, the to-be-named project will be” - defining ‘we’ to the project. That’s the intent at least; words are hard.

With regards to the community meetings - I hear you on this and personally like the open meeting format vs the webinar format. (Also, thanks for the email on the subject yesterday) We actually used this format successfully in the community meetings until the fork announcement. The possibility of the interaction is just deeper and organic questions come up that are healthy for the community. Open Zoom does leave us open to Zoom bombing, which was one of the considerations on switching formats, but the risk/reward here may land on the side of just taking the risk of bombing. I’ve resurfaced the issue.

I’m open to a contributory agenda - let me see what we can do for the next meeting. You make great points about recording - and they’re not unique. I’d like to see these points addressed in some way - all I can say is I’m working on it.

1 Like

I’d also like a more collaborative meeting style. It isn’t possible to know who is there, and I’d recommend that at least the presenters enable their video by default. Yesterday I did wonder if I was the only attendee but I did see one other person post in the chat who wasn’t a presenter. The format of lecture-only, no interaction, doesn’t fit well with what I see expressed here on the boards - the enthusiasm and the sense of being part of something that’s really got some momentum.

3 Likes

I think this is where our opinions differ. I actually see the principles as being a part of the project governance, which is how many of the CNCF projects do this (K8s, for example).

As part of my work within the CNCF Governance WG, we’re planning to start suggesting that CNCF projects include a statement of principles within their governance.md files.

I think the principles and governance go hand in hand in a way that makes thinking about them separately very difficult.

1 Like

@lornajane That’s good feedback - especially regarding what you see vs what we see. I wasn’t aware that you couldn’t see even the numbers of attendees - for someone running the meeting the UI is exactly the same as the regular zoom format.

@lornajane / @dawnfoster / @jkowall: All this being said, I think there is a lot of valid issues here regarding meeting format. Given this isn’t unique to the forks, I’d like to move the community meeting feedback to a thread in the general category for higher visibility

Hi. I’ll be direct here. It feels like a lot of dancing around the core issue. “We” is not about “We” or some other word, it is about the fact that “we” today really is “Amazon” and AFAICT there is nothing firm about opening up of the fork.

That “Thank you for surfacing this” makes it sound like a new thing has been brought up to the surface. But N people here have been trying to point this out for M weeks. I think they/we would all stop if “we” stopped being just Amazon or if the clear path to that was shared. But since that is not happening people keep expressing their concerns, fears, and hopes. :slight_smile:

Please note I am not trying to be harsh or ungrateful. I’m just trying to point out as directly as possible what (I think) is needed. When that happens then words will stop being as hard because it will all be out in the open. I imagine that, despite the desire to give up 100% control of the fork there are N other things Amazon has to think through and go through before that can actually happen…

Otis

2 Likes

To me, it is difficult to talk about governance (more explicit rules and practices for separating work, making decisions, and resolving disputes) without having a shared understanding of common beliefs and principles. I believe that a common understanding of the vision is most important, and then being explicitly flexible in the details of how we work toward that vision together.

Or, to put it a different way, making decisions becomes much more easy when there is general agreement on the “rules of thumb” of a shared vision. The exact manner in which decisions are made matter less, since there is a good likelihood that anyone who understands the shared tenets would act similarly.

As an advocate for being intentional about writing down core principles, and using that as a tool in building a community of like-minded people to collaborate together in building (see my closing thoughts in this OSCON 2019 panel), I welcome this. Thank you for pushing for it. For me, these are the things to debate robustly, to the point where you reach a general consensus and satisfaction that they are going to be relatively durable through time, yet not chiseled in stone.

Meanwhile, I think the specific practices that can be used to organize a project, and make decisions, might necessarily need to change over time. During the early phases of a project, it may make sense to have a designated initial project leader. As a project grows, the work put on a single leader may exceed what any one person can do, and it then makes sense to find a way to divide up the work. (These phases may not directly map to the situation at hand, since this software project is already meaningfully large in size, with many people interested in joining in.)

If you are able to look at it that way, @dawnfoster, what decisions about governance (the specific norms and practices used to make decisions and resolve disputes) would you want to be able to generally answer with the assistance of written-down principles?

I do think that your feedback on the use of “we” in the draft (which, for background, I wasn’t involved in drafting or reviewing) is helpful, since it raises the question about who is “we” and who is “other.” I think that partially boils down to the question about roles, responsibilities, authority, role assignment, and so on. I think that it is very important for people of diverse backgrounds and affiliations to be able to assume roles that have defined responsibilities to be stakeholders in the effort. “We” should be each person involved in the overall effort.

So, let me propose my own set of tenets that might could be used when faced with decisions about governance. These are based on things that I’ve observed to work well in practice with other open source community-focused projects in many sizes. I am writing only as an individual here, and I have not passed these around internally for any discussion or feedback, which I think needs to happen here in the open.

  1. Trust is fundamental. Lasting communities are built on trust, which builds slowly over time and is easy to lose. Trust is earned through establishing a shared vision, clearly setting expectations, being transparent, making promises to one another, and then keeping those promises. We mindfully reject proxies and shortcuts that may be substitutions for building trusting relationships between all stakeholders in this effort.
  2. Shared stewardship. Each participant in this effort is a stakeholder, and partial owner. We mindfully establish policies and practices that re-enforce the continued growth of a diverse group of stakeholders in the project, such as adopting the Developer Certificate of Origin rather than a Contributor License Agreement that aggregates rights to a single party that are not available to everyone.
  3. Permissionless Innovation. Through the use of open source licenses, well understood core principles, and shared norms, permissionless innovation conditions are supported by this effort. It is expected that some stakeholders that collaborate in the effort will also compete with one another. Permission is not required to innovate or to compete. All of the necessary permissions to do so are provided ahead of time through open source licenses and open, non-discriminatory policies.
  4. Designed for speed and scale in making decisions. Federation of responsibility and decision-making authority scales better than coordination. Modularity and extensibility in the design of the software, and the community that produces it, reduces the need for coordinated decision making. Through the use of written principles and clear areas of responsibility, rapid high quality decisions are made locally by those who are most familiar with a situation. Only decisions that are difficult or impossible to reverse require more extensive deliberation.
  5. Roles and responsibilities need not be equal. When federating responsibility, different layers of responsibility form with increasing scope. On points of the timeline of a project, some responsibilities may not be easily delegated (for example, trademark enforcement when a company is acting as the steward), or performed in the public (like handling reports of Code of Conduct violations, or addressing embargoed security vulnerabilities). Each stakeholder needs to act as a steward within their area of responsibility, making decisions that are in the best interest of all stakeholders. Disputes can often be resolved cooperatively by engaging with individuals that have a different role in the project.
  6. Growing a community requires welcoming all. Investments made by some stakeholders will be variable through the long-term timeline of this effort. Mindfully creating leadership opportunities for newcomers that show interest in the effort means that historic contribution metrics should not be a primary component of a formula to allocate areas of ownership and responsibility, or to establish a stake as a stakeholder. Stakeholders come in all shapes and sizes, and every newcomer starts small.
  7. We all wear multiple hats. Stakeholders in the project have different interests, and those interests are valuable to consider when making decisions, as it can provide multiple perspective. When participating in discussions, it is helpful to declare if you are trying to represent the interest of your employer or sponsor, stakeholders that have similar interests, or if you are providing an opinion as an individual. When an individual holds a role with responsibility to the community at large, they may find it helpfully to declare that they are wearing their “upstream hat” to avoid misunderstanding about who’s interest they are trying to represent. Some hats come with real or perceived authority, and wearing small hats is recommended.
  8. Forks and Distributions are welcome. While a goal of this project is to produce high quality software that can be directly useful for a wide range of users, it is also a goal to produce one shared component of a diverse supply chain of software used to build products and services. Innovation and building value that is “downstream” of this effort is encouraged, as is “upstream first” collaboration to enable innovation that is generally useful. We adopt open, non-discriminatory policies that enable downstream consumption of the software produced by this effort in a “distribution” model to correctly identify its origin, and to also market products using a common name. We ask forks to rename their effort to avoid confusion, and we avoid introducing artificial impediments to do so. An exit should always be an option for someone that fundamentally disagrees with a decision.
  9. Special hooks are not welcome. Maintaining significant special functionality that does not generally benefit multiple stakeholders in the project causes undue collective burden on the community as a whole. Stewards and stakeholders of the project should not ask for the community to take on that work.

Hopefully these provide some raw material for further discussion. I’m not particularly attached to these, but I think they’re relatively sound.

–msw

1 Like

I’m a bit of an outsider on this, but it’s been a very interesting discussion to watch. Setting up principles and governance for a new open collaboration generally happens organically over time, but here since the software is well-established and the potential ecosystem is huge, the stakes are higher and clear answers are more urgent. The uniqueness of the situation encourages me to throw my 2 cents into the thoughts bucket.

I agree with @msw that writing down base principles, basically defining the common project that various stakeholders would join, is necessary before writing down governance. The details of the governance of the project evolves over time, while principles (the project itself, in a large sense) ideally should not.

That said, principles can include open governance principles, like for example the idea that every contributor should be on a level playing field from a decision-making perspective. The governance implementation of that (anyone can get elected to the top governance body, one contributor = one vote…) can be defined later. The governance can be bootstrapped by an initial named committee, or an initial “project leader”. It can take time before truly open governance is fully set up. But contributors can trust that it is the direction this is going, since it’s baked in the principles. Here it seems some of the tension is coming from the absence of a clearly stated open governance goal in the principles (“a level playing field” or “shared stewardship” might not be enough).

Now, the project can totally be successful without open governance. The principles that you just listed are great, and are what were cruelly missing from the original Elasticsearch project. The guarantees around building a common base, rather than free labor toward a specific company product, are what ultimately matters… and I think your list captures it very well.

Finally, open governance is not as necessary when there is trust. Building trust takes time. Here one of the issues here is that this is being discussed without the benefit of past experience and building on existing trust.

Note that this is orthogonal to who owns the trademark (the association between the chosen name and the “project” itself in a large sense). The nice thing here is that there is no name yet, so who owns it is not distracting from thinking about what really matters at this stage: defining what your common project is.

2 Likes

Thanks for sharing your thoughts, I agree this is a unique situation. Your last comment… Amazon owns the new name and the associated trademarks, it will be announced when the repos are public.

There is a major concern with trust, and building that will be a challenge especially without governance that prevents a single company from controlling the projects and the decisions which may be more difficult to make.

2 Likes

As @msw and @ttx mentioned, we can include governance as part of some written principles.

Here’s a shot at what I think is important, stated in the form of a principle:

  • Neutral governance and leadership. Project decisions are made by people who work at a diverse set of companies with no single company having a majority say in decisions that impact the project as a whole.

Based on what I’ve seen in this community so far, this is what quite a few of us are asking for. I also know that this may or may not be something that Amazon is willing or able to agree to do. I’ve worked for lots of companies where the company wanted to maintain control of the leadership, decision-making, and governance for the project, and there is nothing wrong with this approach. The important thing is for all of us to be honest about how this will work, and putting this decision off until later only creates frustration, hard feelings, and usually some negative PR (she says with experience having been in this situation).

@searchymcsearchface - My recommendation for Amazon is to be clear with us now about whether you plan to commit to neutral governance for this project or whether you will be keeping leadership, decision-making, and governance within Amazon. This will likely involve tough discussions internally with your execs, but doing this now will make things a lot easier for you and all of the rest of us in the long run.

2 Likes

thanks, @dawnfoster, this sounds exactly like a point which i’d like to see in the list of principles and which i’d like to see clarified sooner than later.

this might be splitting hairs, but i’m just wondering about the wording (note however that i’m not a native english speaker): would it make sense to explicitly state that nobody has a veto power? having a minority vote while still having a veto right still gives you more power (see e.g. United Nations Security Council veto power - Wikipedia - and no, i don’t want to start a political discussion about this here, it’s just the most prominent veto power and it shows how much controversy this can cause).

Hi @ralph, veto power is a bit tricky, and I wouldn’t recommend it in this situation, but it’s good to talk about it. I agree that if one person or company has veto power, then they have more power than the others, which gives them a majority say in the decision even if they aren’t a majority of whatever leadership body is set up, so I would prefer not to give a single company veto power.

It’s also worth mentioning what we did in the Knative community when one company wanted veto power over trademark decisions is that we gave every committee member an equal power to veto decisions.

1 Like

Exactly. @searchymcsearchface is there an ETA for this?

Thanks,
Otis

I agree that’s ideal to form a truly diverse community. But it can be hard to get to if the project is bootstrapped from employees at a single company.

Personally I don’t mind that much a single company ending up with a majority say if it is a reflection of their contribution to the project. The critical thing with open governance is that no company should be special-cased in governance rules (like have reserved seats, keep a veto right), and the governance should focus on individuals. If you regularly elect leadership based on one contributor = one vote, and the result is that the elected individuals happen to be mostly employed by a single company, well that’s still open governance in my book.

So I would settle for:

  • Company-neutral governance and leadership. The project is governed by a representation of the active contributors to the project. No company gets special rights on the project governance, and leaders should wear their project hat rather than their company hat when making decisions affecting the project.

I agree with this position. May I suggest that it should be easier to add a representative that makes the committee more diverse? Likewise, it should be harder, in my opinion, to add someone from an overrepresented group.

1 Like

Let me point to the original blog post:

We’re in this for the long haul, and will work in a way that fosters healthy and sustainable open source practices—including implementing shared project governance with a community of contributors.

I know people up and down the chain at Amazon are committed to this. The real key thing here is a “community of contributors” - contributors means folks, not companies (similar to what @ttx is saying). Underlying ‘shared project governance with a community of contributors’ also is that contribution comes before getting decision making ability.

Companies introduces a whole host of complexity (I have a list!) that is probably not something that can be adequately addressed in a 1-3 sentence principle. I understand how lopsided company influence can be problematic, evident by countless other OSS projects that are single-company, and I don’t think anyone wants this (see principle “A level playing field”).

However, let’s be clear here: when the repos are released, they will be 100% Amazon folks who have contributed past the fork point. Like @ttx mentions, this is bootstrapping. Once there are other folks that are making contributions, I think that’s when the contributors can start having the discussions about how decisions are made, given the problems that are faced by that group. Then I think more specific guidelines about company influence can be crystallized as problems present themselves.

4 Likes

I totally get this (I’ve been working in open source for 20+ years), but most of us folks do indeed work for companies who pay us to participate in open source projects, and as a company, we need to make hard decisions about which open source projects we pay our staff to participate in as part of their job. Sure, anyone can participate in these forks, but you’ll get more people participating who have more time to devote to the project if they are being paid to do it as part of their full time job.

Companies like to know upfront that our employees will be able to participate on a level playing field with no one company in control of the project. Or, if the project will be controlled by one company, we want to know this upfront so that we can make a decision about whether or not to devote the precious time of our employees to this project.

Let me ask another direct question. Do you have plans to donate these forks to a neutral foundation, and if so, when? Or do you plan to keep them owned by Amazon?

3 Likes

it’s a chicken and egg here: if you start as a 100% amazon (while not letting others in), then you want non-Amazon folks to contribute so that you “can start having the discussions about how decisions are made”, and without any governance showing how contributors get chosen, promoted, vetted etc., this will deter non-Amazon people and companies (echoing @dawnforster on companies’ considerations) from contributing. Especially after the community has been burnt with Elastic.

2 Likes

This topic has generated quite a few responses and I wanted to consolidate a change as the project moves to the next phase, as per today’s announcement.

A key clarification is needed to the the meaning of ‘we’ in the principles. As I mentioned earlier in the thread, the original wasn’t precise as to who ‘we’ referred to. The original intent was that ‘we’ would be all those contributing to the project. To rectify the preamble will be changed:

Original
“When we are successful, the to-be-named project will be:”

New
“When we (the contributors) are successful, the OpenSearch project will be:”

With this update, we have posted the principals to the opensearch.org site.

As the project grows, I hope the number of people and diversity of contributors multiplies.
Nothing is etched into stone with these principles. I suspect as folks start developing on and interacting as a community of contributors, these principles will need to evolve and be changed/augmented/adjusted to fit the real-life interactions that development brings. When you have anything additional to add, continue the conversation in this topic or start a new one for a specific principle.

2 Likes

Speaking personally, only for myself.

Yes, I think that open governance is a deeply discussed concern, especially when a community is working from a trust deficit. To me, one case study is Moby, where many in the community had concerns with the BDFL model used by Docker. I was an outsider there, too. But I think that introducing the TSC was a step too late because there was little trust left between some of the participants in Docker.

Yes, I fully agree with this. I think that it is an unforced error to not be extremely mindful about what expectations you are setting, and promises you are making, when inviting others to join in a new effort. If you set expectations that something will happen in the future (like, for example, that Istio assets would be transferred to a foundation, but then reneging on that promise), the damage to trust can be significant and require much more investment to rebuild trust after such an event than was ever needed at the start of a new venture.

These mistakes can spill over into adjacent efforts, like Knative and the trademark committee rules, largely due to the association with one entity and their past behavior.

As Open Source professionals we know that there is a great diversity in day-to-day interpersonal relationships and practices from community to community, and software package to software package. Sometimes corporate interests get in the way from Getting Stuff Done. I think we’ve all seen patches that are submitted to an upstream maintainer slow walked, or rejected outright with no explanation. But, to me, the truth is you can’t really know that employees will have a good experience collaborating and working in the open without trying it. And I think we have also experienced gatekeeping and frustrating decisions in projects that are part of a “neutral foundation” just the same as projects that have a primary corporate steward.

To me, there are no guarantees here. It is far more important to build resiliency as a community, and as stakeholders, that expects and allows for changes over time.

Speaking personally, I have a lot of concerns about the signaling involved in “donating” things to a “neutral foundation.” If I apply my own RFC governance tenets, I worry that sometimes industry consortia are formed because there is mutual distrust between competitors that are collaborating. To me, that is why there is such a focus on “neutral”. And also why the notably industry consortium–the Linux Foundation–has “Built on trust” in the headline of their website.

I think if you aren’t careful, these decisions–ones that are not easy to reverse (see my tenet #4)–may be a proxy for doing the hard work to earn trust among stakeholders (see my tenet #1).

I also think that using the word “donate” is part of the trust signaling, since it is a virtuous thing that good, trustworthy, selfless people do.

For any big decision that is not reversible, I think it is a good practice to have a principled debate on pros and cons, taking into account feedback from multiple stakeholders of varying types. What would the community get from moving assets to a different holding entity? What might the cost be to certain classes of stakeholders? Would we we want to raise a pool of funds through vendor membership levels? What would we do with the funds? What might that do to the community dynamics, speed in decision making, and ability to innovate? Are some practices within foundations becoming the new cathedral? Back when the GNOME Foundation was being established, its founders wrote some poignant words in their charter:

The GNOME foundation must not stifle the interest of outsiders. An ill-conceived foundation could discourage outsider participation directly, by establishing rules which limit the ability of potential contributors to make their mark, or indirectly, by engendering an alienating sense of elitism. The stained glass of the cathedral creates a colorful spectacle for those inside, but from the outside, the building is just a hulking grey edifice, intimidating and impenetrable.

No matter who is acting as a steward for important assets of an Open Source effort, you have to do the hard work to build an inclusive community (if it is a goal to have one—and I think that having one is an essential and valuable feature). I don’t see how committees with membership based on “substantial contribution” that vote on things, with veto powers, helps to build an inclusive community. I think it builds a set of rules so that one competitor feels like they are not going to be wronged by another competitor due to conflicting corporate interests.

Personally, I would rather see a diverse community of stakeholders around OpenSearch (users, developers, software vendors, cloud providers, upstream projects like Apache Lucene, etc.) that is laser focused on specific common interests, rather than a consortium of vendors trying to work out the rules that protect each in their own interests up front before deciding to commit to participate. I think that if Amazon acts as a steward for the trademark, or a non-profit is the steward for the trademark, is a distraction from the hard work that is needed in community building.

I believe that the things that make a community work well are uncovered over time when you work together in the open, working mindfully to address specific concerns and needs of all stakeholders.