I was opening some proposals (basically an RFC or idea we want to build). I noticed we do not have a “type” for these issues, nor do we have a template. I was talking to @elifish today about it and wanted to share what we’ve been using. I took this off Jaeger where I also contribute.
OpenSearch and OpenSearch Dashboards Proposal Template
Requirements - What kind of business use case are you trying to solve?
Fill in
Problem - What blocks you from solving this today?
Fill in
Proposal - what do you suggest to solve the problem or improve the existing situation?
Fill in
Assumptions
Fill in
Any open questions to address
Fill in
You’ll see a couple of these in both OpenSearch and OpenSearch Dashboard repos which I have opened in the last few weeks.
Useful for proposals from the community. Light weight and lean.
What about major changes, new APIs and architectural changes? Elastic does this (I suppose) in an in-house process with product owners, roadmaps and planning in various teams.
How will the opensearch project handle this? What will replace the single-organization processes? One extreme is that one company does all of this. The opposite is that there is no roadmap and everyone scratch their own itch.
For the Open Distro project, anybody proposing to do the work to add a new feature generally started with an RFC, which was indeed a really useful practice. OpenSeach’s development principles require public input (#5) on features and a predictable foundation (#8), which I interpret to mean that either of those extremes (all-in-house decision making and no roadmap) are non-starters.
I like @jkowall’s template here a lot: I think it provides good consistency from proposal to proposal, making sure they all address the important questions, while keeping things appropriately simple for the early days of the OpenSearch.
Thanks for pointing to the KIP process. What do you like about it? Are there elements you recommend folding into what @jkowall proposed here? I’d love to start with a single process for proposals that could scale up to support architectural changes that call for a lot of deliberation… and also be useable for small, non-breaking feature proposals where the developers can safely move pretty fast.
Normally the way we do it (for Jaeger/OpenTelemetry) is that we have PMs who get involved internally and we work on the roadmap with the community, interjecting things we want to do which align with the vision for the projects. We figure out where the best place to implement things is. The community agrees upon the right way to do those things. We have a public roadmap for the projects, and we have roadmap meetings every quarter or so for Jaeger.
A good example of what I might have suggested is the “data prepper” for the trace plugin should be an OpenTelemetry processor or exporter for example versus another component. I would have suggested this be done in that project to make things cleaner.
my personal experience (mostly from commercial software in this case) is that it’s not the best idea to mix requirements & solution approaches in the same post.
often the person with the requirements is not intimately familiar with (or, worse: is currently on “Mt. Stupid” of the Dunning-Kruger effect (we’ve all been there!)) the inner workings of the software (and probably the same applies to hardware, processes and anything else in life as well!) and it then often happens that instead of stating the requirements the solution proposal is being stated as a requirement - even though it might be a sub-optimal approach for the actual requirement.
the good thing about the template is that it already tries to address this by having it as two separate blocks, but i’d write it every explicitly that the solution proposal section is purely optional. nobody should feel forced to fill this in. on the contrary, i could even imagine leaving this out - if somebody has a proposal they should just add this as a 2nd post directly underneath the initial one. this lets everyone (both the original author of the post and the readers) first focus on the problem/requirement and only afterwards will they start looking at possible solutions. if you read the solution suggestion directly you don’t have time to formulate your own and from then on you might well be biased towards the first proposal you read.
(and now somebody who actually studied psychology can hop in here and tell my that i’m currently on “Mt. Stupid” of the Dunning-Kruger effect for this and that i’m totally wrong )
also: the title should of course read “OpenSearch and OpenSearch Dashboards Proposal Template” - not “ElasticSearch and Kibana”
All of the templates are optional in Github, but they help the submitter understand what to think about. This also makes it approachable for those who haven’t built complex software systems to also have a leg in to build something in open source which could add value to the community of users.
Oh yeah sorry about the name, I started using it before the name was announced
I really like this template. It is light weight and starts with the problem and works backwards from there. What foundation do we have to build on? Why can’t this be solved with what we have? What do we need to build on our foundation to solve the problem?
My instinct is that we add this as an .md file to OpenSearch so that it is a readily available template. What does everyone think of adding this as an issue template type? We could add one and label it RFC or feature proposal or something else. Thoughts?
This is a nice template to start from. Can we open as an issue on the .github repo along w/ a PR to merge in the .github/ISSUE_TEMPLATE directory? I’ll also suggest a new proposal label on the repos so we can easily filter issues by proposals.
These “governance” contributions are new exciting territory so I think we’ll certainly need to work through the mechanisms. I was thinking you could open the issue and initial PR in the .github repo (which is a well intended, but not perfect, repo for these high level “governance” contributions) and then cherry pick the commit to the OpenSearch repo and open a PR there as well?
I know it feels like extra work to open the two PRs, but I’m all for this in the OpenSearch repo so if you don’t cherry-pick it, I will :). Then we can let the plugin communities decide if they want to adopt as well and have them open their own PRs (saving you the work)?
@nknize do you know if there is a way we could set templates org wide for proposal issues? I was trying to dig into this on GitHub’s docs and couldn’t figure it out. It would be nice if we could make this an org wide template so that each repo in the org has access to the template for consistency.
What i like about the KIP (and PEP, RFC) is that they are examples of successful “Design documents” from complex Open Source projects. GitHub issues and PRs are good for isolated well defined, changes, but they may prove lightweight for larger design work and formal decision making / voting.
A KIP allows for a consistent, thorough design, making sure all options are considered, that public APIs are not changed without a consistent plan, that security and performance is always a topic etc. And it provides a public record of important project decisions. A single approved KIP may lead to multiple PRs across several repos. If you look at Python, PEPs are the main decision making and recording vehicle in the project, where PEP-1 defines the PEP process itself, PEP-13 even defines the governance of Python etc.
Anyway, I lifted it here mostly for inspiration and to start thinking about how to solve the roadmap/design problem in a true, distributed and open fashion.
)
